Skip to content

fix: recovery from archives with sparse index shards - #3

Merged
YordiLorenzo merged 2 commits into
YordiLorenzo:mainfrom
arjungopisetty:fix/issue-1-sparse-index-shards
Sep 8, 2026
Merged

fix: recovery from archives with sparse index shards#3
YordiLorenzo merged 2 commits into
YordiLorenzo:mainfrom
arjungopisetty:fix/issue-1-sparse-index-shards

Conversation

@arjungopisetty

Copy link
Copy Markdown
Contributor

Summary

  • Preserve numeric shard IDs when calculating logical index offsets.
  • Stop later shards from shifting when an earlier shard number is absent.
  • Honor -s, -n, and --sample for doctor.
  • Add regression coverage for sparse shard layouts and doctor arguments.

Root Cause

The affected archive has gaps in its shard numbering: file_chunk1.index is missing shard 27 and file_chunk4.index is missing shard 7. Cat previously compacted discovered shards into a contiguous list, so every shard after a gap was addressed 8 MiB too early and valid MP4 chunk lists were interpreted as invalid records.

The reader now derives logical offsets from each shard numeric filename.

Validation

  • Full suite: 34 passed, 9 skipped
  • Doctor: 3 sampled files rebuilt and verified
  • MVI_6400.MP4: 74,396,840 bytes rebuilt exactly
  • 9,072 chunks verified with zero skipped entries or breaks
  • MP4 restore completed successfully from Backblaze B2

Closes #1

@arjungopisetty
arjungopisetty marked this pull request as ready for review September 5, 2026 07:10
@arjungopisetty
arjungopisetty marked this pull request as draft September 5, 2026 07:12
@arjungopisetty
arjungopisetty marked this pull request as ready for review September 5, 2026 07:15
@arjungopisetty arjungopisetty changed the title Fix recovery from archives with sparse index shards fix: recovery from archives with sparse index shards Sep 5, 2026
@YordiLorenzo
YordiLorenzo merged commit ee7a121 into YordiLorenzo:main Sep 8, 2026
YordiLorenzo pushed a commit that referenced this pull request Sep 8, 2026
The docstring asserted the invariant that #3 disproved, and it was the reasoning
that led me to compute offsets from list position in the first place. Record what
a real archive actually looks like so the next person does not re-derive it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unable to restore some files from Backblaze B2

2 participants