diff --git a/docs/developers/l2/upgrade-test.md b/docs/developers/l2/upgrade-test.md index a357b760669..a9472d0aad5 100644 --- a/docs/developers/l2/upgrade-test.md +++ b/docs/developers/l2/upgrade-test.md @@ -5,6 +5,7 @@ This is a per-release acceptance test. The goal is to verify that: 1. A node running the **previous release** (``) can be cleanly stopped. 2. The L1 contracts can be upgraded following the per-release migration guide. 3. The **new release** (``) sequencer and prover resume operation against the upgraded contracts without re-deploying from scratch. +4. Batches produced by `` and committed by `` are **proved and verified on L1** under the new release's verification key. The commands below are ready to copy-paste. The only thing you need to substitute is the version tags. Every other value (private keys, addresses, ports, fee parameters) is taken from the defaults in `crates/l2/Makefile`; feel free to override them, but the values below give you a working baseline. @@ -15,13 +16,27 @@ Replace these once at the top of your shell session: ```bash export VERSION_FROM= export VERSION_TO= -export ARCH=linux-x86_64 # or linux-aarch64, macos-aarch64 +export ARCH=linux-x86_64 export WORK=$HOME/upgrade-test # workspace mkdir -p "$WORK" ``` +> [!IMPORTANT] +> Run this on the GPU server (`l2-gpu`) with the **SP1** prover, not locally with +> `--backend exec`. The `exec` backend produces no proof, so it never consults a +> verification key — and the verification key is the thing an upgrade changes. +> Batches are committed under `keccak(VERGEN_GIT_SHA)`, baked into the binary at +> build time, and `commitBatch` refuses a commit hash it holds no key for. Until +> `$VERSION_TO`'s key is registered +> ([Step 3.6](#36-register-the-version_to-verification-key)) the upgraded +> sequencer cannot commit **anything**: every `commitBatch` reverts with +> `MissingVerificationKeyForCommit()` (selector `0xf6b9798e`) and the L2 keeps +> producing blocks locally while nothing reaches L1. Under `exec` that failure is +> invisible — the test passes and the real upgrade stops at the first batch. + ## Prerequisites +- A CUDA-capable GPU (this is why it runs on `l2-gpu`), and the `ethrex-l2-$ARCH-gpu` release asset - [`rex`](https://github.com/lambdaclass/rex) on `$PATH` (used to call `upgradeToAndCall` and read contract state) - `curl`, `jq` - `git` (to fetch genesis / fixture files for each version) @@ -71,16 +86,26 @@ cd "$WORK" git clone --branch "$VERSION_FROM" --depth 1 https://github.com/lambdaclass/ethrex.git "ethrex-$VERSION_FROM" git clone --branch "$VERSION_TO" --depth 1 https://github.com/lambdaclass/ethrex.git "ethrex-$VERSION_TO" -# Binaries -curl -L "https://github.com/lambdaclass/ethrex/releases/download/$VERSION_FROM/ethrex-l2-$ARCH" -o "ethrex-$VERSION_FROM/ethrex" -curl -L "https://github.com/lambdaclass/ethrex/releases/download/$VERSION_TO/ethrex-l2-$ARCH" -o "ethrex-$VERSION_TO/ethrex" -chmod +x "ethrex-$VERSION_FROM/ethrex" "ethrex-$VERSION_TO/ethrex" +# Binaries. The `-gpu` asset is the one that can prove with SP1 on a GPU. +# Remove before downloading: curl-ing over a binary that is currently running +# reuses the inode and gets the live process killed. +for V in "$VERSION_FROM" "$VERSION_TO"; do + rm -f "ethrex-$V/ethrex" + curl -fL "https://github.com/lambdaclass/ethrex/releases/download/$V/ethrex-l2-$ARCH-gpu" -o "ethrex-$V/ethrex" + chmod +x "ethrex-$V/ethrex" + "ethrex-$V/ethrex" --version +done -"ethrex-$VERSION_FROM/ethrex" --version -"ethrex-$VERSION_TO/ethrex" --version +# Verification keys ship in each release's contracts tarball; both versions' +# keys are needed — VERSION_FROM's to deploy, VERSION_TO's for Step 3.6. +for V in "$VERSION_FROM" "$VERSION_TO"; do + mkdir -p "contracts-$V" + curl -fL "https://github.com/lambdaclass/ethrex/releases/download/$V/ethrex-contracts.tar.gz" \ + | tar xz -C "contracts-$V" +done ``` -> Use the `ethrex-l2-*` asset (the plain `ethrex-*` asset is an L1-only build and does not have the `l2` subcommand). +> Use an `ethrex-l2-*` asset (the plain `ethrex-*` asset is an L1-only build and does not have the `l2` subcommand), and specifically the `-gpu` variant — the others carry no CUDA support, so `--backend sp1` falls back to CPU proving and a batch takes long enough to look like a hang. ### 0.1 Save the `$VERSION_FROM` L2 genesis @@ -125,6 +150,8 @@ COMPILE_CONTRACTS=true ./ethrex l2 deploy \ --on-chain-proposer-owner 0x4417092b70a3e5f10dc504d0947dd256b965fc62 \ --bridge-owner 0x4417092b70a3e5f10dc504d0947dd256b965fc62 \ --bridge-owner-pk 0x941e103320615d394a55708be13e45994c7d93b932b064dbcb2b511fe3254e2e \ + --sp1 true \ + --sp1-vk-path "$WORK/contracts-$VERSION_FROM/ethrex-riscv32im-succinct-zkvm-vk-bn254" \ --deposit-rich \ --private-keys-file-path fixtures/keys/private_keys_l1.txt \ --genesis-l1-path fixtures/genesis/l1.json \ @@ -132,6 +159,12 @@ COMPILE_CONTRACTS=true ./ethrex l2 deploy \ --env-file-path cmd/.env ``` +`--sp1 true` is what makes the OnChainProposer require a real proof, and +`--sp1-vk-path` registers `$VERSION_FROM`'s verification key against +`$VERSION_FROM`'s commit hash. Both matter: without them the deployment accepts +anything and [Step 3.6](#36-register-the-version_to-verification-key) has nothing +to prove. + After this finishes, capture the addresses we'll need later: ```bash @@ -176,7 +209,7 @@ set -a; source cmd/.env; set +a cd "$WORK/ethrex-$VERSION_FROM" ./ethrex l2 prover \ --proof-coordinators tcp://127.0.0.1:3900 \ - --backend exec + --backend sp1 ``` ### 1.5 Confirm the stack is healthy @@ -405,6 +438,60 @@ rex send "$ETHREX_WATCHER_BRIDGE_ADDRESS" 'setL2GasLimit(uint256)' --vk \ + --on-chain-proposer --timelock --private-key +``` + +The sha is the one the new binary prints after `HEAD-` in `ethrex --version`; the +key is the `ethrex-riscv32im-succinct-zkvm-vk-bn254` file from the new release's +`ethrex-contracts.tar.gz`. See +[Registering a new verification key](../fundamentals/upgrades.md#registering-a-new-verification-key) +for what the command does and why the Timelock is in the path. + +Keys are stored per commit hash, not per version, so the old key stays valid — +which is what lets a rollback keep verifying. + ## From v7 to v8 ### Database migration (local node) diff --git a/docs/l2/fundamentals/upgrades.md b/docs/l2/fundamentals/upgrades.md index 39b2ce99ec6..ad3705fd0c1 100644 --- a/docs/l2/fundamentals/upgrades.md +++ b/docs/l2/fundamentals/upgrades.md @@ -6,25 +6,47 @@ Each committed batch stores the git commit hash of the sequencer build that prod ## Registering a new verification key -To allow proofs from a new sequencer/prover build, register its verification key against the commit hash: - -1. Compute the commit hash as the Keccak-256 of the (reduced) git commit. For example, the commit `9219410` produces `b9105485bc4ba523201eaaf76478a47b259fa7399bbed795cf19294861b7fc57`. -2. From the OnChainProposer owner account, send the upgrade transaction. Example (replace addresses and keys with your values): - ``` - rex send \ - "upgradeSP1VerificationKey(bytes32,bytes32)" \ - \ - \ - --private-key - ``` -3. (Optional) Verify the mapping entry: - ``` - rex call \ - "verificationKeys(bytes32,uint8)(bytes32)" \ - \ - - ``` - `1` is the SP1 verifier ID, `2` is RISC0. +This is not optional on an upgrade. `commitBatch` rejects a commit hash it holds +no key for, so an upgraded sequencer commits **nothing** until its key is +registered: every commit reverts with `MissingVerificationKeyForCommit()` +(selector `0xf6b9798e`) while the L2 keeps producing blocks, which reads as a +stuck committer rather than a missing key. Nothing is lost — register the key and +the pending batch is committed on the next attempt. + +Use `rex l2 register-vk`, which derives the commit hash, routes the call, and +reads the mapping back: + +``` +rex l2 register-vk \ + --commit \ + --vk \ + --on-chain-proposer \ + --timelock \ + --private-key +``` + +Add `--prover risc0` for a RISC0 deployment, and `--dry-run` to print the derived +commit hash and the key currently on chain without sending. + +Two details this gets right, and which are easy to get wrong by hand: + +- **The hashed value is the full git sha the binary reports**, as ASCII — the + segment after `HEAD-` in `ethrex --version`. Hashing an abbreviated sha + produces a different key, and the only symptom is that commits keep reverting. +- **`upgradeSP1VerificationKey` is `onlyOwner`, and that owner is the Timelock** + in any deployment made since v9.0.0, so the call must be routed through + `emergencyExecute` (Security Council) or Governance `schedule` + `execute`. + Sending it straight to the OnChainProposer from an EOA reverts with + `OwnableUnauthorizedAccount`. + +To read a mapping entry directly (`1` is the SP1 verifier ID, `2` is RISC0): + +``` +rex l2 call \ + "verificationKeys(bytes32,uint8)(bytes32)" \ + \ + +``` ### Verification key artifacts