Repository navigation
surject changes: classify secondary and supplementary alignments - #5051
Conversation
2ce6896 to
e55b935
Compare
adamnovak
left a comment
There was a problem hiding this comment.
This looks pretty good, but I would adjust some comments, and in particular I would have a high-level overview in a doc comment somewhere of what the general idea is, with the idea that secondary status is something that ought to be inherited in particular ways and that something called a "supplementary group" exists.
Also, while it's pretty slick to write the test cases with SECTIONs that just set a flag, or that set up some of the input and set a flag, combining all the verification logic into one copy with a bunch of conditionals and ternaries makes it hard to see at a glance exactly what we expect to be the output for each input. This in turn makes it hard to tell if we are expecting the right output for each input.
I would favor writing things out more explicitly, even if that means we end up duplicating more of the check logic and the actual calls to the code under test. If we want to deduplicate check logic, we might consider turning it into helper functions that check that particular complex conditions are true. We want to set things up so the checks we have for an input read as much like a list of true facts about the expected output in English as we can manage.
| /// report_supplementary is enabled and are otherwise omitted. | ||
| /// | ||
| /// Classifies secondary status relative to the current input placement; | ||
| /// the caller restores inherited secondary status when emitting results. |
There was a problem hiding this comment.
We seem to be introducing a concept of "inherited secondary status" here. I'm guessing it means the case where we're surjecting a secondary graph alignment.
But we refer to the concept a couple places in this PR without really explaining it.
I would expect to see something explaining conceptual system, either here or earlier on in the header. We should set out that reads coming in can be primary or secondary, and reads going out can be primary or secondary or supplementary, and how all those are expected to relate, which would include the idea that an output derived from a secondary input ought to inherit secondary-ness from that input.
Also, we write "restore" here, but we don't have an annotate_primary, so we can't be clearing secondary status. What exactly are we requiring the caller to do? Is there a way where we could explain this as us not doing something the reader might have expected us to do (i.e. saying that we only deal with secondary status coming from within a graph placement, not with inherited secondary status), rather than us imposing a condition on what the caller must do after calling us?
| // Classify alternatives relative to this input placement. Its inherited | ||
| // secondary status is restored when surject_internal emits the results. | ||
| for (auto& surjection : surjections) { | ||
| set_is_secondary(surjection.first, false); | ||
| } |
There was a problem hiding this comment.
This isn't removing any secondary status, so I think the comment is mostly about something that happens elsewhere that isn't really connected to what happens here.
| // Classify alternatives relative to this input placement. Its inherited | |
| // secondary status is restored when surject_internal emits the results. | |
| for (auto& surjection : surjections) { | |
| set_is_secondary(surjection.first, false); | |
| } | |
| // Classify alternatives relative to this input placement. | |
| for (auto& surjection : surjections) { | |
| set_is_secondary(surjection.first, false); | |
| } |
| // Downstream path ranking and SA tags use the first non-supplementary | ||
| // candidate. Put the local primary first, preserving alternative order. | ||
| std::rotate(surjections.begin(), surjections.begin() + opt_idx, | ||
| surjections.begin() + opt_idx + 1); |
There was a problem hiding this comment.
We should explain this in terms of the function's own guarantee that the local primary alignment will be first, not in terms of what other code elsewhere might or might not rely on that guarantee for.
| // Downstream path ranking and SA tags use the first non-supplementary | |
| // candidate. Put the local primary first, preserving alternative order. | |
| std::rotate(surjections.begin(), surjections.begin() + opt_idx, | |
| surjections.begin() + opt_idx + 1); | |
| // Put the local primary first, preserving alternative order. | |
| std::rotate(surjections.begin(), surjections.begin() + opt_idx, | |
| surjections.begin() + opt_idx + 1); |
Also, if for some reason it's important that we preserve the order, so we shouldn't just swap whatever was first and whatever we want to be primary, then it needs to be in the function doc comment that we guarantee that.
| for (const auto& candidate : candidates) { | ||
| const auto& aln = candidate.first; | ||
| const bool supplementary = has_annotation(aln, "supplementary") | ||
| && get_annotation<bool>(aln, "supplementary"); | ||
| CHECK(aln.is_secondary() == (aln.name() == "A")); | ||
| CHECK(supplementary == (aln.name() == "C")); | ||
| if (!aln.is_secondary() && !supplementary) { | ||
| CHECK(aln.name() == "B"); | ||
| CHECK(aln.score() == 60); | ||
| ++primary_count; | ||
| } | ||
| } | ||
| CHECK(primary_count == 1); |
There was a problem hiding this comment.
It might make more sense to index the candidates by name in a map, so we can write "B is not secondary" and "B is not supplementary" and "B has sore 60" instead of "The alignment that is neither secondary nor supplementary is named B and has score 60".
Someone shouldn't have to be that smart to read the test code, because the test code needs to be correct by inspection, because the test code does not itself have any tests.
| surjector.add_SA_tag(output, positions, pos_graph, false); | ||
| for (const auto& aln : output) { | ||
| if (aln.name() == "A") { | ||
| CHECK_FALSE(has_annotation(aln, "tags")); | ||
| } else { | ||
| CHECK(has_annotation(aln, "tags")); | ||
| if (has_annotation(aln, "tags")) { | ||
| const string expected = aln.name() == "B" | ||
| ? "SA:Z:ref,161,+,60S40M,0,0;" | ||
| : "SA:Z:ref,101,+,60M40S,0,0;"; | ||
| CHECK(get_annotation<string>(aln, "tags") == expected); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
I feel like this is maybe a different test, or at least deserves a comment explaining that we're changing subjects. Are we now testing that the SA tag is generated by add_SA_tag() and has the correct value?
Also, do we have a way to parse the tags handy? I'm not sure it's a good idea to rely on SA being the only tag.
| SECTION("Secondary alternatives cannot increase the strand score") { | ||
| add_candidate(path_a, step_a, 0, 100, 110, 0); | ||
| add_candidate(path_b, step_b, 0, 100, 105, 0); | ||
| add_candidate(path_b, step_b, 0, 100, 105, 200); | ||
| add_candidate(path_b, step_b, 0, 100, 105, 400); | ||
| expected_primary = path_a; | ||
| } | ||
| SECTION("Secondary alternatives cannot decrease the strand score") { | ||
| add_candidate(path_a, step_a, 0, 100, 90, 0); | ||
| add_candidate(path_b, step_b, 0, 100, 95, 0); | ||
| add_candidate(path_b, step_b, 0, 100, 95, 200); | ||
| add_candidate(path_b, step_b, 0, 100, 95, 400); | ||
| expected_primary = path_b; | ||
| } |
There was a problem hiding this comment.
We're not really testing that the strand score doesn't increase or decrease; we're testing that the path with the highest scoring thing wins, I think.
| linked_count += tags.find("SA:Z:") != string::npos; | ||
| // Each member links only to its partner, not the repeat alternative. | ||
| CHECK(std::count(tags.begin(), tags.end(), ';') == 1); |
There was a problem hiding this comment.
Is counting the semicolons testing for just one SA tag?
I find it confusing to see this check right after the check to count whether the SA tag is there at all. I think it works only by relying on the SA tag being the only tag, so we have tags exactly when we have an SA tag.
This would be clearer if we were working with some kind of parsed tags.
We also aren't doing anything to check if we're linking to the correct partner and not the incorrect repeat alternative, we're just stating that we must be.
ffe521f to
ac2e934
Compare
ac2e934 to
0a08127
Compare
b8020b2 to
3953ae9
Compare
adamnovak
left a comment
There was a problem hiding this comment.
This looks good.
I still want the weird relationship between supplementary groups and strands to go away, because we're never going to call novel inversions without being able to have a supplementary group exist across strands. But this fairly clearly isn't trying to care about that, and the explanations make sense in a context where that isn't happening.
| * A supplementary group consists of one non-supplementary placement | ||
| * and the supplementary pieces associated with that placement. SA tags | ||
| * link these members to each other; locally classified secondary | ||
| * alternatives are not members of the group. |
There was a problem hiding this comment.
Are supplementary groups in principle required to live within a single strand? Are we currently restricting them to a single strand for ease of implementation? Because this isn't getting rid of choose_primary_strand(), which now looks to be assuming supplementary pieces somehow credit score to their strand.
There was a problem hiding this comment.
Supplementary groups do not need to stay on one strand, a read spanning an inversion can have pieces on opposite strands. This PR retains the existing path/strand-based selection. choose_primary_strand() scores each path/orientation using its local primary and supplementary pieces, while excluding secondary alternatives.
The broader handling of supplementary groups across strands will be addressed in later PRs.
Changelog Entry
vg surjectnow distinguishes secondary alternative placements from supplementary read segments using overlap on the read.Description
This change updates how multiple surjections on the same path and strand are classified.
The highest-scoring candidate is selected locally. Other candidates that substantially overlap it in read coordinates are retained as secondary alternatives, while candidates covering different portions of the read are treated as supplementary when supplementary output is enabled.
Secondary alternatives are excluded from
supplementary_cover()and fromchoose_primary_strand()scoring, so repeated placements of the same read bases do not artificially increase or decrease a strand’s score. Genuine supplementary pieces still contribute together.The selected local candidate is kept first so existing SA-tag behavior links it with its supplementary pieces, while locally classified secondaries are kept out of that SA group.
Tests cover secondary/supplementary classification, inherited secondary status, SA-tag membership, and repeat-count effects on strand selection.
Limitations
This layer does not fully resolve the broader problem of explicitly associating supplementary alignments with a particular primary/secondary placement and its mate. It also retains the current large-gap behavior change caused by splitting separated reference regions before supplementary reporting is considered.
Stack created with GitHub Stacks CLI • Give Feedback 💬