Skip to content

surject changes: classify secondary and supplementary alignments - #5051

Merged
adamnovak merged 4 commits into
masterfrom
surject-stack-02-interval-classification
Nov 9, 2026
Merged

adamnovak merged 4 commits into
masterfrom
surject-stack-02-interval-classification

Conversation

@Sagorikanag

Copy link
Copy Markdown
Contributor

Changelog Entry

  • vg surject now distinguishes secondary alternative placements from supplementary read segments using overlap on the read.

Description

This change updates how multiple surjections on the same path and strand are classified.

The highest-scoring candidate is selected locally. Other candidates that substantially overlap it in read coordinates are retained as secondary alternatives, while candidates covering different portions of the read are treated as supplementary when supplementary output is enabled.

Secondary alternatives are excluded from supplementary_cover() and from choose_primary_strand() scoring, so repeated placements of the same read bases do not artificially increase or decrease a strand’s score. Genuine supplementary pieces still contribute together.

The selected local candidate is kept first so existing SA-tag behavior links it with its supplementary pieces, while locally classified secondaries are kept out of that SA group.

Tests cover secondary/supplementary classification, inherited secondary status, SA-tag membership, and repeat-count effects on strand selection.

Limitations

This layer does not fully resolve the broader problem of explicitly associating supplementary alignments with a particular primary/secondary placement and its mate. It also retains the current large-gap behavior change caused by splitting separated reference regions before supplementary reporting is considered.


Stack created with GitHub Stacks CLI • Give Feedback 💬

@Sagorikanag
Sagorikanag added this pull request to stack #5053 October 2, 2026 03:48
@Sagorikanag
Sagorikanag force-pushed the surject-stack-02-interval-classification branch from 2ce6896 to e55b935 Compare October 2, 2026 17:26

@adamnovak adamnovak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good, but I would adjust some comments, and in particular I would have a high-level overview in a doc comment somewhere of what the general idea is, with the idea that secondary status is something that ought to be inherited in particular ways and that something called a "supplementary group" exists.

Also, while it's pretty slick to write the test cases with SECTIONs that just set a flag, or that set up some of the input and set a flag, combining all the verification logic into one copy with a bunch of conditionals and ternaries makes it hard to see at a glance exactly what we expect to be the output for each input. This in turn makes it hard to tell if we are expecting the right output for each input.

I would favor writing things out more explicitly, even if that means we end up duplicating more of the check logic and the actual calls to the code under test. If we want to deduplicate check logic, we might consider turning it into helper functions that check that particular complex conditions are true. We want to set things up so the checks we have for an input read as much like a list of true facts about the expected output in English as we can manage.

Comment thread src/surjector.hpp Outdated
/// report_supplementary is enabled and are otherwise omitted.
///
/// Classifies secondary status relative to the current input placement;
/// the caller restores inherited secondary status when emitting results.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We seem to be introducing a concept of "inherited secondary status" here. I'm guessing it means the case where we're surjecting a secondary graph alignment.

But we refer to the concept a couple places in this PR without really explaining it.

I would expect to see something explaining conceptual system, either here or earlier on in the header. We should set out that reads coming in can be primary or secondary, and reads going out can be primary or secondary or supplementary, and how all those are expected to relate, which would include the idea that an output derived from a secondary input ought to inherit secondary-ness from that input.

Also, we write "restore" here, but we don't have an annotate_primary, so we can't be clearing secondary status. What exactly are we requiring the caller to do? Is there a way where we could explain this as us not doing something the reader might have expected us to do (i.e. saying that we only deal with secondary status coming from within a graph placement, not with inherited secondary status), rather than us imposing a condition on what the caller must do after calling us?

Comment thread src/surjector.hpp Outdated
Comment on lines +993 to +997
// Classify alternatives relative to this input placement. Its inherited
// secondary status is restored when surject_internal emits the results.
for (auto& surjection : surjections) {
set_is_secondary(surjection.first, false);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't removing any secondary status, so I think the comment is mostly about something that happens elsewhere that isn't really connected to what happens here.

Suggested change
// Classify alternatives relative to this input placement. Its inherited
// secondary status is restored when surject_internal emits the results.
for (auto& surjection : surjections) {
set_is_secondary(surjection.first, false);
}
// Classify alternatives relative to this input placement.
for (auto& surjection : surjections) {
set_is_secondary(surjection.first, false);
}

Comment thread src/surjector.hpp Outdated
Comment on lines +1009 to +1012
// Downstream path ranking and SA tags use the first non-supplementary
// candidate. Put the local primary first, preserving alternative order.
std::rotate(surjections.begin(), surjections.begin() + opt_idx,
surjections.begin() + opt_idx + 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should explain this in terms of the function's own guarantee that the local primary alignment will be first, not in terms of what other code elsewhere might or might not rely on that guarantee for.

Suggested change
// Downstream path ranking and SA tags use the first non-supplementary
// candidate. Put the local primary first, preserving alternative order.
std::rotate(surjections.begin(), surjections.begin() + opt_idx,
surjections.begin() + opt_idx + 1);
// Put the local primary first, preserving alternative order.
std::rotate(surjections.begin(), surjections.begin() + opt_idx,
surjections.begin() + opt_idx + 1);

Also, if for some reason it's important that we preserve the order, so we shouldn't just swap whatever was first and whatever we want to be primary, then it needs to be in the function doc comment that we guarantee that.

Comment thread src/unittest/surject.cpp Outdated
Comment on lines +87 to +99
for (const auto& candidate : candidates) {
const auto& aln = candidate.first;
const bool supplementary = has_annotation(aln, "supplementary")
&& get_annotation<bool>(aln, "supplementary");
CHECK(aln.is_secondary() == (aln.name() == "A"));
CHECK(supplementary == (aln.name() == "C"));
if (!aln.is_secondary() && !supplementary) {
CHECK(aln.name() == "B");
CHECK(aln.score() == 60);
++primary_count;
}
}
CHECK(primary_count == 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might make more sense to index the candidates by name in a map, so we can write "B is not secondary" and "B is not supplementary" and "B has sore 60" instead of "The alignment that is neither secondary nor supplementary is named B and has score 60".

Someone shouldn't have to be that smart to read the test code, because the test code needs to be correct by inspection, because the test code does not itself have any tests.

Comment thread src/unittest/surject.cpp Outdated
Comment on lines +108 to +121
surjector.add_SA_tag(output, positions, pos_graph, false);
for (const auto& aln : output) {
if (aln.name() == "A") {
CHECK_FALSE(has_annotation(aln, "tags"));
} else {
CHECK(has_annotation(aln, "tags"));
if (has_annotation(aln, "tags")) {
const string expected = aln.name() == "B"
? "SA:Z:ref,161,+,60S40M,0,0;"
: "SA:Z:ref,101,+,60M40S,0,0;";
CHECK(get_annotation<string>(aln, "tags") == expected);
}
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this is maybe a different test, or at least deserves a comment explaining that we're changing subjects. Are we now testing that the SA tag is generated by add_SA_tag() and has the correct value?

Also, do we have a way to parse the tags handy? I'm not sure it's a good idea to rely on SA being the only tag.

Comment thread src/unittest/surject.cpp Outdated
Comment on lines +170 to +183
SECTION("Secondary alternatives cannot increase the strand score") {
add_candidate(path_a, step_a, 0, 100, 110, 0);
add_candidate(path_b, step_b, 0, 100, 105, 0);
add_candidate(path_b, step_b, 0, 100, 105, 200);
add_candidate(path_b, step_b, 0, 100, 105, 400);
expected_primary = path_a;
}
SECTION("Secondary alternatives cannot decrease the strand score") {
add_candidate(path_a, step_a, 0, 100, 90, 0);
add_candidate(path_b, step_b, 0, 100, 95, 0);
add_candidate(path_b, step_b, 0, 100, 95, 200);
add_candidate(path_b, step_b, 0, 100, 95, 400);
expected_primary = path_b;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're not really testing that the strand score doesn't increase or decrease; we're testing that the path with the highest scoring thing wins, I think.

Comment thread src/unittest/surject.cpp Resolved
Comment thread src/unittest/surject.cpp Outdated
Comment on lines +257 to +259
linked_count += tags.find("SA:Z:") != string::npos;
// Each member links only to its partner, not the repeat alternative.
CHECK(std::count(tags.begin(), tags.end(), ';') == 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is counting the semicolons testing for just one SA tag?

I find it confusing to see this check right after the check to count whether the SA tag is there at all. I think it works only by relying on the SA tag being the only tag, so we have tags exactly when we have an SA tag.

This would be clearer if we were working with some kind of parsed tags.

We also aren't doing anything to check if we're linking to the correct partner and not the incorrect repeat alternative, we're just stating that we must be.

@Sagorikanag
Sagorikanag force-pushed the surject-stack-02-interval-classification branch 2 times, most recently from ffe521f to ac2e934 Compare October 7, 2026 17:09
@Sagorikanag
Sagorikanag force-pushed the surject-stack-02-interval-classification branch from ac2e934 to 0a08127 Compare October 21, 2026 12:39
Base automatically changed from surject-stack-01-tail-pruning to master October 30, 2026 17:30
@adamnovak
adamnovak force-pushed the surject-stack-02-interval-classification branch from b8020b2 to 3953ae9 Compare October 30, 2026 17:30
@adamnovak adamnovak added inference Theme/team for genome inference (calling, assembly, etc.) mapping Theme/team for mapping to pangenomes and removed inference Theme/team for genome inference (calling, assembly, etc.) labels Nov 2, 2026

@adamnovak adamnovak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good.

I still want the weird relationship between supplementary groups and strands to go away, because we're never going to call novel inversions without being able to have a supplementary group exist across strands. But this fairly clearly isn't trying to care about that, and the explanations make sense in a context where that isn't happening.

Comment thread src/surjector.hpp
Comment on lines +43 to +46
* A supplementary group consists of one non-supplementary placement
* and the supplementary pieces associated with that placement. SA tags
* link these members to each other; locally classified secondary
* alternatives are not members of the group.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are supplementary groups in principle required to live within a single strand? Are we currently restricting them to a single strand for ease of implementation? Because this isn't getting rid of choose_primary_strand(), which now looks to be assuming supplementary pieces somehow credit score to their strand.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supplementary groups do not need to stay on one strand, a read spanning an inversion can have pieces on opposite strands. This PR retains the existing path/strand-based selection. choose_primary_strand() scores each path/orientation using its local primary and supplementary pieces, while excluding secondary alternatives.

The broader handling of supplementary groups across strands will be addressed in later PRs.

@adamnovak
adamnovak merged commit 5bb0cd7 into master Nov 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mapping Theme/team for mapping to pangenomes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants