Skip to content

Add vg chunk -H to include haplotype paths in chunks - #5059

Merged
glennhickey merged 1 commit into
masterfrom
chunk-haplotype-paths
Oct 13, 2026
Merged

glennhickey merged 1 commit into
masterfrom
chunk-haplotype-paths

Conversation

@glennhickey

Copy link
Copy Markdown
Contributor

Changelog Entry

To be copied to the draft changelog by merger:

  • vg chunk -H/--haplotype-paths includes haplotype paths in graph chunks (works on GBZ and non-GBZ input)

Description

vg chunk used to include embedded haplotype paths when chunking .vg (HashGraph/PackedGraph) graphs. That stopped in February 2026 when libbdsg 30c805e (#4821/#4822, the deconstruct memory regression fix) changed ReferencePathOverlay to index only REFERENCE and GENERIC paths by default. The chunker copies paths into chunks through the overlay (add_subpaths_to_subgraph → for_each_step_position_on_handle), so it only sees paths the overlay indexed. vg call and vg deconstruct were updated at the time to opt back in for embedded haplotypes, but vg chunk was not.

Turning haplotypes back on by default isn't a good idea, because the overlay builds a full position index for every path it indexes: a copy of each path's steps, a per-base offset bit vector with rank/select, and a node-to-step reverse index. On a 646-path chrY graph (2.7 Gbp of total path length), that costs about 7 s and 2 GB extra. Since gbwtgraph stopped hiding haplotypes (#4795), it would also apply to GBZ input, where a whole-genome graph would mean indexing every haplotype in it.

So this keeps the current default and adds an opt-in -H/--haplotype-paths flag that indexes all paths, haplotypes included, for any input.

On chrY-chm13-v2.1.vg with -p CHM13#0#chrY:10000000-10000005 -c 100:

paths in chunk time max RSS
default 2 (reference only) 6.4 s 3.9 GB
-H 98 (96 haplotypes) 13.9 s 6.0 GB

Tests in 30_vg_chunk.t cover default vs. -H for both GBZ and non-GBZ input.

🤖 Generated with Claude Code

Since libbdsg 30c805e (Feb 2026), ReferencePathOverlay indexes only
REFERENCE and GENERIC paths by default, and the chunker only copies
paths the overlay has indexed. vg call and vg deconstruct were updated
to opt back in for embedded haplotypes, but vg chunk was not, so chunks
of .vg graphs silently lost their haplotype paths.

Indexing haplotypes means building a full position index over every
haplotype in the graph, which is too expensive to do by default on big
graphs, so make it opt-in with -H/--haplotype-paths for any input,
including GBZ.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@glennhickey
glennhickey merged commit 5f11892 into master Oct 13, 2026
2 checks passed
@faithokamoto
faithokamoto deleted the chunk-haplotype-paths branch October 16, 2026 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants