Repository navigation
Add vg chunk -H to include haplotype paths in chunks - #5059
Merged
Merged
Conversation
Since libbdsg 30c805e (Feb 2026), ReferencePathOverlay indexes only REFERENCE and GENERIC paths by default, and the chunker only copies paths the overlay has indexed. vg call and vg deconstruct were updated to opt back in for embedded haplotypes, but vg chunk was not, so chunks of .vg graphs silently lost their haplotype paths. Indexing haplotypes means building a full position index over every haplotype in the graph, which is too expensive to do by default on big graphs, so make it opt-in with -H/--haplotype-paths for any input, including GBZ. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changelog Entry
To be copied to the draft changelog by merger:
vg chunk -H/--haplotype-pathsincludes haplotype paths in graph chunks (works on GBZ and non-GBZ input)Description
vg chunkused to include embedded haplotype paths when chunking.vg(HashGraph/PackedGraph) graphs. That stopped in February 2026 when libbdsg30c805e(#4821/#4822, the deconstruct memory regression fix) changedReferencePathOverlayto index only REFERENCE and GENERIC paths by default. The chunker copies paths into chunks through the overlay (add_subpaths_to_subgraph→for_each_step_position_on_handle), so it only sees paths the overlay indexed.vg callandvg deconstructwere updated at the time to opt back in for embedded haplotypes, butvg chunkwas not.Turning haplotypes back on by default isn't a good idea, because the overlay builds a full position index for every path it indexes: a copy of each path's steps, a per-base offset bit vector with rank/select, and a node-to-step reverse index. On a 646-path chrY graph (2.7 Gbp of total path length), that costs about 7 s and 2 GB extra. Since gbwtgraph stopped hiding haplotypes (#4795), it would also apply to GBZ input, where a whole-genome graph would mean indexing every haplotype in it.
So this keeps the current default and adds an opt-in
-H/--haplotype-pathsflag that indexes all paths, haplotypes included, for any input.On
chrY-chm13-v2.1.vgwith-p CHM13#0#chrY:10000000-10000005 -c 100:-HTests in
30_vg_chunk.tcover default vs.-Hfor both GBZ and non-GBZ input.🤖 Generated with Claude Code