perf(l1): allocation-free leaf and extension node hashing - #7155
perf(l1): allocation-free leaf and extension node hashing#7155diegokingston wants to merge 4 commits into
Conversation
Branch nodes already build their RLP in place (#7104); leaf and extension still went through `Encoder`, which accumulates the payload in a heap `Vec` so it can prepend the list header, then copies the whole payload across. Together with the `Vec` that `encode_compact` returns, hashing one leaf cost about five allocations and two passes over its bytes. Every node's payload length is knowable before any of it is written -- for a leaf, the compact path length plus the value's RLP length; for an extension, the compact path length plus the child hash variant's width. So the header can go down first and the node can be built in place, the same shape branch already uses. * `Nibbles::encode_compact_len` derives the hex-prefix length from the nibble count alone, and `encode_compact_into` appends into a caller buffer. `encode_compact` keeps its signature for the p2p call sites and is now a thin wrapper. * `LeafNode::encode_into_vec` / `ExtensionNode::encode_into_vec` take a concrete `&mut Vec<u8>`, mirroring `BranchNode::encode_into_vec`. * `BranchNode::encode_into_vec` still routed each of its 16 child hashes through `hash.0.encode(&mut *buf)`, which takes a trait object -- so the monomorphic wrapper avoided the outer dispatch and paid 16 inner ones. Those are now direct pushes. * `NodeRef::memoize_hashes` memoized the subtrie and then called `Node::compute_hash_no_alloc`, whose first act is to memoize the subtrie again. Not exponential -- the second pass finds every `OnceLock` set -- but every branch re-walked all 16 children to learn nothing. Split out `hash_memoized` for callers that have already done the walk. `RLPEncode::encode` is deliberately left untouched, both because other callers depend on it and because it then serves as the differential oracle below. This matters most in the zkVM guest, whose bump allocator's `dealloc` is a no-op and whose `realloc` always allocates fresh and memcpys (ziskos entrypoint/src/alloc/bump.rs), so allocation churn there is permanently consumed heap rather than reused blocks. Measured on the shape the guest actually runs: a 500k-account trie arrives fully memoized (`get_embedded_root_committed` seeds every `OnceLock`), a block dirties the paths it touched, and `hash_no_commit` re-encodes only those. touched allocations heap consumed realloc copied wall 200 1,011 -> 2 52.6 KB -> 1.5K 20.7 KB -> 512B 470 -> 433 us 2,000 10,104 -> 2 512 KB -> 1.5K 203 KB -> 512B 3.53 -> 3.09 ms 20,000 101,184 -> 2 5.11 MB -> 1.5K 2.02 MB -> 512B 29.3 -> 25.3 ms Host wall-clock understates the guest case: the system allocator reuses freed blocks and can grow in place, and ZisK does neither, so the eliminated heap and memcpy columns are worth more there than they are here. Correctness. Root hash identical at every size above. The full L1 guest program run natively over a real block witness (hoodi 1265656, fixtures/cache/rpc_prover) reproduces initial_state_hash, final_state_hash and last_block_hash exactly. 49,675 differential checks against the untouched `RLPEncode` implementations -- every path length 0..=64 for leaf and extension, every single-byte value across the 0x80 boundary, every RLP header boundary (55/56/57, 255/256), and every `NodeHash` variant including the empty inline case -- 0 failures. `ethrex-trie` (61) and `ethrex-common` (171) suites pass, `no_std` builds, dependent crates build. One behavioural subtlety preserved deliberately: an extension node whose child is an empty inline hash contributes nothing to the payload, which is what the `Encoder` path produces today (`NodeHash::encode` writes `encode_raw` of an empty slice), even though `RLPEncode::length` reports 1 for that case. It only arises from a malformed trie, and `extension_child_len` and `put_extension_child` are derived from the same match so they cannot drift apart.
|
github-actions
Bot
added
L1
🤖 Codex Code ReviewNo blocking findings in the diff.
Static review-wise, the hash memoization split in crates/common/trie/node.rs looks sound and the buffer reuse changes in branch/leaf/extension hashing preserve the previous control flow. I could not run Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
Lines of code reportTotal lines added: Detailed view |
Benchmark Block Execution Results Comparison Against Main
|
1 participant
Motivation
Branch nodes already build their RLP in place (#7104); leaf and extension nodes still went through
Encoder, which accumulates a node's payload in a heapVecso it can prepend the list header, then copies the whole payload across. Together with theVecthatencode_compactreturns, hashing one leaf cost about five allocations and two passes over its bytes.This is aimed at the zkVM guest. ZisK's guest allocator is a bump pointer whose
deallocis a no-op and whosereallocalways allocates a fresh block and memcpys (ziskos/entrypoint/src/alloc/bump.rs), so allocation churn there is permanently consumed heap and unavoidable copying — a growable, reclaiming allocator hides most of this cost on the host, and the guest has neither.Description
Every node's payload length is knowable before any of it is written — for a leaf, the compact path length plus the value's RLP length; for an extension, the compact path length plus the child hash variant's width. So the header can go down first and the node can be built in place, which is the shape
BranchNode::encode_into_vecalready uses.Nibbles::encode_compact_lenderives the hex-prefix length from the nibble count alone, andencode_compact_intoappends into a caller buffer.encode_compactkeeps its signature for the p2p call sites and becomes a thin wrapper.LeafNode::encode_into_vec/ExtensionNode::encode_into_vectake a concrete&mut Vec<u8>, mirroring the branch writer.BranchNode::encode_into_vecstill routed each of its 16 child hashes throughhash.0.encode(&mut *buf), which takes a trait object — so the monomorphic wrapper added in perf(l1): give branch-node hashing a monomorphic RLP encoder #7104 avoided the outer dispatch and then paid 16 inner ones. Those are now direct pushes.NodeRef::memoize_hashesmemoized the subtrie and then calledNode::compute_hash_no_alloc, whose first act is to memoize the subtrie again. Not exponential — the second pass finds everyOnceLockalready set — but every branch re-walked all 16 children to learn nothing. Split outhash_memoizedfor callers that have already done the walk.RLPEncode::encodeis deliberately left untouched, both because other callers depend on it and because it then serves as a differential oracle for the new writers.Guest RISC-V cycles
Measured with
ziskemuon a ZisK guest ELF that builds a 2,000-account trie, memoizes every hash (asget_embedded_root_committeddoes when a witness is decoded), dirties the paths a block would touch, and re-hashes. Keccak runs through ZisK's accelerator.Both figures are for a program that also builds the trie from scratch and spends 16-17% of its cost in keccak, neither of which this touches — so the saving on the hashing itself is larger than the totals suggest.
The
keccakopcode count is identical across the two builds (3,646 and 5,986 respectively): the same hashing work is done, and what disappears is encoding overhead.dma_memcpydrops 75,895 → 64,850, which is the bump allocator'srealloccopies going away.Host
hash_no_commitover a 500k-account trie arriving fully memoized, with only the dirtied paths re-encoded:Correctness
ziskemufor both workloads above.l1::execution_programrun natively over a real block witness (hoodi 1,265,656,fixtures/cache/rpc_prover) reproducesinitial_state_hash,final_state_hashandlast_block_hashexactly.RLPEncodeimplementations: every path length 0..=64 for leaf and extension, every single-byte value across the0x80boundary, every RLP header boundary (55/56/57, 255/256), and everyNodeHashvariant including the empty inline case. 0 failures.ethrex-trie(61) andethrex-common(171) suites pass;no_stdbuilds; dependent crates build.One behaviour is preserved deliberately: an extension node whose child is an empty inline hash contributes nothing to the payload, which is what the
Encoderpath produces today (NodeHash::encodewritesencode_rawof an empty slice) even thoughRLPEncode::lengthreports 1 for that case. It only arises from a malformed trie.extension_child_lenandput_extension_childare derived from the same match so they cannot drift apart.Checklist
STORE_SCHEMA_VERSION(crates/storage/lib.rs) if the PR includes breaking changes to theStorerequiring a re-sync. — n/a, noStorechanges; trie encodings are byte-identical.