[db] Frame snapshot storage streams into per-leaf groups

[maxkozlovsky]

Stacked on #2463 — review that first. This PR's base is max/snapshot-stream-versioning, so its diff shows only the grouping change.

Problem

Restoring into a page-encoded target buffers a whole shard before writing any of it. A page is 128 raw-adjacent slots in one trie leaf, the dump is out of order, and upsert is set-not-merge — writing a page twice drops the earlier slots instead of merging them. With no way to know a page is complete, the loader defers everything to one flush per shard, so peak memory scales with the largest shard. Sharding splits on the account-hash prefix, so a whale contract is atomic to one shard however the divider is tuned: at block 90045827 the largest shard is 740 MB against a 43 MB mean.

What this does

Page contiguity already existed in the stream — a page-encoded source expands one page leaf into slots inside a single down() on a single-threaded traverse, so each page's slots were already one contiguous run. What was missing was not bytes but a promise that a run is closed. So frame the storage stream as groups, one per source leaf, and spend the header's reserved byte on the source's page-key shift:

storage := monad_snapshot_stream_header group*
group   := [account_offset: uint64][payload_len: uint32][payload]

payload is the concatenated encode_storage_db entries of one source leaf, so the per-slot account-offset prefix collapses to one per group. The load-side predicate is one line:

group_key_shift >= storage_page_t::PAGE_KEY_SHIFT

Every target page then lies entirely inside one closed group, so the loader assembles a page when its group ends and may flush at any group boundary. Peak memory follows the flush threshold rather than the shard size, and the single-account floor disappears.

A slot-encoded source reports shift 0, which no target page fits inside, so slot-source × page-target falls back to the whole-shard accumulator with no separate branch. The threshold moves 10 GiB → 1 GiB and gains a setter; a typical shard is far below it, so the common path still does one upsert per shard.

Cost

Framing is not free on sparse storage: where a group holds one slot a 12-byte group header replaces an 8-byte per-slot prefix, and most mainnet pages hold exactly one slot. At block 90045827 the storage stream grows 2.8% (8.63 → 8.87 GiB) and the whole snapshot 2.3% (10.42 → 10.67 GiB).

Compatibility

Storage streams written before this keep loading: they report shift 0 and are read as single-slot groups, taking the previous whole-shard path.

Testing

PageGroupedStorageStream asserts the invariant the loader depends on — one page key per group, ascending slot keys, each page in exactly one group. PageToPageRestoreIndependentOfFlushThreshold covers page→page, previously untested altogether, restoring both at the default threshold and flushing after every group; agreeing on the source root is what makes the threshold a pure memory knob. HeaderlessSnapshotRestores and DumpFromSecondaryPageDb gain an unframed and a one-byte-threshold slot target, giving all six source × target × framing combinations.

End to end against an official mainnet snapshot, block 90045827, 256 shards:

• Framed page→page. Same state root as the unframed path, peak RSS 9.93 → 7.84 GB. The saving is the whole-shard accumulator going away; the threshold never engages, the largest shard being under 1 GiB.
• Pre-framing snapshot. Today's headerless layout restores into a page target at the root main produces, and into a slot target reproducing exactly the state_root in the snapshot's own block header.
• New snapshot, older code. Handed to a main binary it aborts in about a second, leaving the target database untouched.

Follow-ups, not in scope

The threshold is not wired to a CLI flag, so the bound it enables is exercised by the unit test rather than from the command line, and an unframed or slot-source stream into a page target still buffers a shard. An in-order dump would compose well, making each flush epoch contiguous in key space so a small threshold stops costing scattered re-merklization.

🤖 Generated with Claude Code