Skip to content

refactor(l1): unify the storage-trie BackendTrieDB constructors on an Option<H256> prefix - #7195

Draft
ilitteri wants to merge 1 commit into
mainfrom
consolidate-merge-new-for-storages-new-for-account
Draft

refactor(l1): unify the storage-trie BackendTrieDB constructors on an Option<H256> prefix#7195
ilitteri wants to merge 1 commit into
mainfrom
consolidate-merge-new-for-storages-new-for-account

Conversation

@ilitteri

Copy link
Copy Markdown
Collaborator

Motivation

BackendTrieDB had four constructors for the storage tables and they came in two copies of the same code. new_for_storages / new_for_storages_with_view and new_for_account_storage / new_for_account_storage_with_view set the identical STORAGE_TRIE_NODES / STORAGE_FLATKEYVALUE tables, derive last_computed_flatkeyvalue the identical way, and differ in exactly one expression: address_prefix: None versus address_prefix: Some(address_prefix).

That duplication is a liability precisely because the one differing field is the dangerous one. address_prefix feeds apply_prefix on every key path (make_key, behind get and put_batch, plus flatkeyvalue_computed directly), so picking the wrong constructor silently changes every key written to and read from the storage tables. Encoding the choice in the function name means the compiler cannot help: new_for_storages and new_for_account_storage are both in scope, both compile at every call site, and only one of them is correct at each. Any future edit to one copy also has to be mirrored into the other, and a copy that drifts is a data-corruption bug, not a style problem.

Description

The prefix is now a parameter instead of a name. new_for_storages and new_for_storages_with_view take address_prefix: Option<H256> and assign it straight to the field; new_for_account_storage and new_for_account_storage_with_view are deleted. Net 20 non-comment, non-blank lines removed from crates/storage. The surviving name is the neutral one - new_for_account_storage(db, None, ..) would have read as a contradiction.

Which value each call site passes is load-bearing, and each was derived from whether the caller already applies the prefix:

call site prefix why
Store::open_storage_trie None the BackendTrieDB is wrapped in TrieWrapper::new(.., Some(account_hash)), and TrieWrapper prepends the prefix itself (layering.rs, prefix_nibbles built in new and concatenated in get/flatkeyvalue_computed). Passing the hash here too would prefix every key twice.
Store::open_storage_trie_shared None same shape, same TrieWrapper prefix.
Store::open_direct_storage_trie Some(account_hash) no wrapper - Trie::open sits directly on the BackendTrieDB, so the prefix has to come from here.
flatkeyvalue_generator's inner storage loop Some(account_hash) no wrapper either; this is the loop that writes STORAGE_FLATKEYVALUE entries.

Both None sites are the ones that previously called a new_for_storages* constructor, and both Some sites are the ones that previously called a new_for_account_storage* constructor, so the value reaching address_prefix is unchanged at all four. new_for_accounts / new_for_accounts_with_view (the ACCOUNT_* tables) and LockedTrieDB are untouched.

The doc comment on new_for_storages now spells out the None case, because that is the invariant a future caller can get wrong and the type system will not catch.

If this change were wrong, one of these would have to be true:

  • TrieWrapper does not apply the account prefix itself, so the two None sites now under-prefix their keys. It does apply it: the prefix nibbles are built in TrieWrapper::new and concatenated onto the key on every read path in crates/storage/layering.rs.
  • The two constructor families differed in something other than the prefix - a different table, a different last_written handling. The deleted bodies are in this diff; every other field is character-for-character identical.
  • A caller outside this workspace depends on the removed names. BackendTrieDB is public, but grep -rn 'new_for_account_storage' over the repo now returns nothing, and before this change the only callers were the four in crates/storage/store.rs and two in test/tests/storage/trie_db_tests.rs.
  • The on-disk key layout moved. address_prefix is read only where keys are derived (make_key for get/put_batch, and flatkeyvalue_computed), and the value it receives at each of the four sites is the same as before, so the bytes written to STORAGE_TRIE_NODES / STORAGE_FLATKEYVALUE are identical.

No test was deleted. test_trie_db_with_address_prefix still covers the Some path; it just calls the merged constructor.

How to test

cargo fmt --all -- --check
cargo clippy --all-targets --workspace -- -D warnings
cargo clippy -p ethrex-storage --all-targets --features rocksdb -- -D warnings
cargo test -p ethrex-storage
cargo test -p ethrex-test --test ethrex_tests

Results on this branch:

  • cargo fmt --all -- --check - clean.
  • cargo clippy --all-targets --workspace -- -D warnings - clean (exit 0).
  • cargo clippy -p ethrex-storage --all-targets --features rocksdb -- -D warnings - clean; the RocksDB-gated code compiles against the new signature too, since the default gate does not build it.
  • cargo test -p ethrex-storage - 91 passed, 0 failed; doc-tests 1 passed, 1 ignored.
  • cargo test -p ethrex-test --test ethrex_tests - 964 passed, 0 failed, 1 ignored.

The targeted coverage inside that suite, for reviewers who want to run less:

  • cargo test -p ethrex-test --test ethrex_tests storage:: - 24 passed, including storage::trie_db_tests::test_trie_db_with_address_prefix (write-then-read round trip through the Some(address) path) and storage::storage_batch_tests::* (parity between the sharded and serial storage writers).
  • cargo test -p ethrex-test --test ethrex_tests blockchain::storage_sharding - 11 passed; these build storage tries through open_direct_storage_trie (the Some path) and compare sharded against serial roots, so a double-prefixed or unprefixed key shows up as a root mismatch.
  • cargo test -p ethrex-test --test ethrex_tests p2p::snap_server - 18 passed; storage_ranges_* serves storage slots back out of the same tables that were written through the changed constructors.

Checklist

  • No Store schema change, so STORE_SCHEMA_VERSION (crates/storage/lib.rs) is untouched: the only field affected is BackendTrieDB::address_prefix, every call site passes the same value it passed before, and the key bytes written to STORAGE_TRIE_NODES / STORAGE_FLATKEYVALUE are byte-identical - no re-sync is required.

…an `Option<H256>` prefix

`new_for_storages`/`new_for_storages_with_view` and
`new_for_account_storage`/`new_for_account_storage_with_view` were the same
code twice: identical `STORAGE_TRIE_NODES`/`STORAGE_FLATKEYVALUE` tables,
identical `last_computed_flatkeyvalue` derivation, differing only in
`address_prefix: None` versus `address_prefix: Some(address_prefix)`.

Encoding that one field in the function name made it the caller's job to pick
the right name out of two that both compile everywhere, and `address_prefix`
feeds `make_key`, so the wrong pick silently mis-keys every storage read and
write. Make it a parameter instead: `new_for_storages` and
`new_for_storages_with_view` now take `address_prefix: Option<H256>`, and the
`new_for_account_storage` pair is deleted.

Each call site keeps the value its old constructor hard-coded.
`open_storage_trie` and `open_storage_trie_shared` pass `None` because the
`TrieWrapper` above them already prepends `Some(account_hash)` and a second
prefix would corrupt the keys; `open_direct_storage_trie` and the storage loop
in `flatkeyvalue_generator` pass `Some(account_hash)` because no wrapper sits
between them and the trie. `new_for_accounts` and `LockedTrieDB` are untouched,
and the on-disk key layout is unchanged, so `STORE_SCHEMA_VERSION` stays put.
@github-actions github-actions Bot added the L1 Ethereum client label Aug 21, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Known Issues — intentionally skipped tests

Source: docs/known_issues.md

rpc-compat log-bearing cases excluded

Where: KNOWN_EXCLUDED_TESTS in .github/scripts/check-hive-results.sh counts out
eight hive rpc-compat cases — the four eth_getLogs cases, eth_getBlockReceipts/get-block-receipts-latest,
and three eth_getTransactionReceipt cases. They are exactly the cases whose recorded
response contains at least one log object; every case with an empty log array still runs.
Note this leaves eth_getLogs with no rpc-compat coverage at all, since all four of its
cases are in the set.

Why: ethrex populates blockTimestamp on log objects, as geth, besu, nethermind, reth
and erigon all do. hive's rpc-compat compares responses byte-exactly (jsondiff.FullMatch;
the lenient checkJSONStructure path applies only to cases upstream marks speconly), and
the corpus is pinned to execution-apis d08382ae (2025-02-10), whose recordings predate the
field — it entered the schema in execution-apis#639 and the fixtures in #846 (2026-07-22).
So the extra key cannot match, and this is a property of the pin rather than of the response.

The pin cannot move, and this is not temporary. The pin sits one commit before
execution-apis#627, which moved the test chain to a pre-merge genesis: the current corpus has
~36 proof-of-work blocks before its terminal total difficulty. ethrex does not support
pre-merge chains and will not, so importing that chain.rlp fails at block 1 —
validate_block_header has no pre-London base-fee path. Every revision carrying
blockTimestamp in its fixtures also carries that chain, so there is no revision that
satisfies both. Nor can the corpus be patched locally: rpc-compat's Dockerfile clones
ethereum/execution-apis by hard-coded URL, so the branch buildarg cannot point at a fork.

Coverage: the field itself is pinned by
block_timestamp_is_on_the_log_and_not_on_the_receipt in
crates/networking/rpc/types/receipt.rs, which asserts it is present on each log and absent
from the receipt level.

Removal: delete the entries if ethrex ever gains pre-merge chain import, or if upstream
marks these cases speconly so they are type-checked instead of compared byte-for-byte.


The stateless schema id does not identify the encoding

Where: STATELESS_INPUT_SCHEMA_ID in crates/common/types/stateless_ssz.rs.

Upstream keeps the stateless input schema id at 0x1501
(fork_index 0x15 << 8 | revision 0x01) across incompatible body changes. Three
encodings have now shipped under it: tests-zkevm@v0.6.2, then #3248 + #3278,
then #3356, which moved state, codes and public_keys from SszList to
ProgressiveList. ethrex speaks the last one.

The consequence is that the 2-byte prefix cannot be used to detect a stale or
mismatched bundle. A wrong-dialect input is accepted by the id check and then
fails later — in SSZ decode, or on a root that does not match — rather than being
rejected up front for what it is. only_amsterdam_schema_id_decodes therefore
proves less than its name suggests.

Worth raising upstream: a revision field that does not move across a body change
provides no version negotiation at all.


ZisK guest program hash changes with the unsync_cell gate

Where: crates/common/types/block.rs, transaction.rs.

The gate on the single-threaded unsync_cell::OnceCell moved from
all(feature = "eip-8025", target_arch = "riscv64") to
all(feature = "zisk", target_arch = "riscv64") when the eip-8025 feature was removed.

The guest ELFs were previously built --features "<zkvm>-build-elf,ci", which never enabled
eip-8025, so they compiled the atomic once_cell variant. bin/zisk/Cargo.toml does enable
ethrex-common/zisk, so the ZisK guest now compiles the unsafe impl Sync cell instead.
That changes the ELF bytes and therefore the program hash and verification key.

This is intended (the guest is single-threaded, so the unsync cell is sound and cheaper), but it
is a VK change rather than a no-op refactor, and the diffstat presents it as a file rename
(eip8025_cell.rsunsync_cell.rs). Anyone pinning a ZisK VK across this change must
re-register it. The stateless-validator crate now forwards ethrex-common/zisk from its own
zisk feature so the two ZisK guests do not disagree on the cell type.


Release signing key is an unprotected repository secret

Where: .github/workflows/tag_release.yaml.

MINISIGN_SECRET_KEY is a plain repository secret. There is no environment: on
finalize-release or dry-run-release-assets, and gh api repos/lambdaclass/ethrex/rulesets
shows only branch-targeted rulesets, so the github.ref_type == 'tag' condition is a workflow
check rather than an enforced boundary: anyone who can push a tag can reach the signing key.

This is a repository-settings change, not a code change, so it is recorded here rather than
fixed in the tree. Recommended:

  1. Move MINISIGN_SECRET_KEY / MINISIGN_PASSWORD into a GitHub Environment with required
    reviewers, and add environment: to the two jobs that sign.
  2. Add a ruleset targeting refs/tags/v* restricting who may create release tags.

Until then, the compromise of that key is silent and durable: signatures would still verify
against the committed .github/minisign.pub.

@github-actions

Copy link
Copy Markdown

Lines of code report

Total lines added: 2
Total lines removed: 22
Total lines changed: 24

Detailed view
+--------------------------------+-------+------+
| File                           | Lines | Diff |
+--------------------------------+-------+------+
| ethrex/crates/storage/store.rs | 5075  | +2   |
+--------------------------------+-------+------+
| ethrex/crates/storage/trie.rs  | 157   | -22  |
+--------------------------------+-------+------+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

L1 Ethereum client

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant