From a27e4a6d472dd8b997ab3f64773c8476cf9c1933 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 24 Aug 2026 19:06:57 +0200 Subject: [PATCH 01/61] docs: Add design documents for the PEN migration to Base Specifies a one-way migration of the native PEN token from the Pendulum parachain to a fixed-supply ERC-20 on Base (150,000,000 PEN, 18 decimals), with the full issuance pre-minted into a migration vault and released as holders migrate. - pen-base-migration-prd: requirements, decisions, component specs, threat model, acceptance criteria and rollout. - adr-001: why a purpose-built one-way migration over existing bridge infrastructure or a snapshot-and-claim, and the sub-decisions within it (pre-mint vs mint-on-demand, on-chain approvals, burn vs lock). - pen-token-contract-standards: which ERC-20 extensions the token implements and which are deliberately excluded. - pen-governance-guide: the post-migration hybrid governance model, with worked examples of both tracks and the treasury structure. - pen-migration-window-analysis: on-chain analysis of vesting, staking and governance locks sizing the migration window. - pen-base-migration-community-overview: holder-facing summary. --- docs/adr-001-pen-base-migration-approach.md | 150 +++++++++++ docs/pen-base-migration-community-overview.md | 187 ++++++++++++++ docs/pen-base-migration-prd.md | 241 ++++++++++++++++++ docs/pen-governance-guide.md | 224 ++++++++++++++++ docs/pen-migration-window-analysis.md | 117 +++++++++ docs/pen-token-contract-standards.md | 59 +++++ 6 files changed, 978 insertions(+) create mode 100644 docs/adr-001-pen-base-migration-approach.md create mode 100644 docs/pen-base-migration-community-overview.md create mode 100644 docs/pen-base-migration-prd.md create mode 100644 docs/pen-governance-guide.md create mode 100644 docs/pen-migration-window-analysis.md create mode 100644 docs/pen-token-contract-standards.md diff --git a/docs/adr-001-pen-base-migration-approach.md b/docs/adr-001-pen-base-migration-approach.md new file mode 100644 index 000000000..3a6f17843 --- /dev/null +++ b/docs/adr-001-pen-base-migration-approach.md @@ -0,0 +1,150 @@ +# ADR-001: Approach for migrating PEN from Pendulum to Base + +**Status:** Accepted +**Date:** 2026-07-07 +**Deciders:** Pendulum / SatoshiPay team +**Companion doc:** [pen-base-migration-prd.md](pen-base-migration-prd.md) (the full requirements for the chosen approach) + +## Context + +PEN is the native token of the Pendulum parachain (Substrate, Polkadot, 12 decimals, sr25519 accounts). We want to move it to Base as an ERC-20. Hard requirements that shaped the decision: + +1. `totalSupply()` on Base must equal PEN's maximum issuance **from day one**, so trackers (DefiLlama, CoinGecko) never show a confusing supply split across two chains. +2. Users migrate by giving up PEN on Pendulum and receiving it on Base (lock/burn-and-release). +3. This is a **migration**, not a bridge product: finite lifetime, one direction, and the token contract that remains afterwards should be maximally trustless and boring. + +The fundamental constraint behind everything below: **Base cannot cheaply verify Pendulum state.** Real cryptographic verification of Polkadot finality on an EVM chain requires an on-chain light client (BEEFY signature verification, validator-set tracking). That is what Snowbridge and Hyperbridge are, and each took a dedicated team years. Every approach is therefore a different answer to the question: *what do we trust instead, and how do we bound the damage if that trust fails?* + +Additional constraints: sr25519 signatures cannot be affordably verified on the EVM (rules out direct "prove you own this Substrate account" claims on Base); we control the Pendulum runtime (this repo), so adding a pallet is cheap for us; PEN teleport to AssetHub already shipped (#553), so the Snowbridge route is partially paved. + +## Decision + +**Option A — a purpose-built one-way migration:** a small `token-migration` pallet on Pendulum (burn + event with a unique nonce and target H160), a fixed-supply ERC-20 on Base with the entire max issuance pre-minted into a MigrationVault, and a 3-of-4 attestor set (initially team-operated, each on its own node — PRD D4) that watches relay-finalized Pendulum events and submits **on-chain approvals** to the vault; the third matching approval releases the tokens. + +Post-migration governance is **hybrid** (Option G4 below): OZ Governor + Timelock for Base-side contracts and treasury, Snapshot + executor Safe for off-chain/cross-chain matters, technical committee retained for Pendulum runtime actions. + +## Options considered — migration mechanism + +### Option A: Purpose-built one-way migration (attestor-based) — CHOSEN + +| Dimension | Assessment | +|---|---| +| Complexity | Medium — ~150-line pallet, ~300-line vault, 4 small daemons; 3–6 weeks + security review | +| Trust model | 3-of-4 designated attestors (initially team-operated, PRD D4); damage bounded by rate caps + pause + independent monitor + separation of duties | +| Meets supply requirement | Yes, by construction (pre-mint to vault) | +| UX | One extrinsic on Pendulum, tokens arrive on Base automatically | +| Ongoing burden | Attestor ops for the migration window only; nothing permanent | + +**Pros:** +- We control both ends; the design can be exactly as simple as the problem requires. +- The token contract itself ends up with **zero trust assumptions** (no mint function, no owner, no proxy) — the trusted component (vault + attestors) is temporary and rate-limited. +- Pre-minting satisfies the day-one supply requirement trivially. +- One-way design halves the attack surface of a bridge (no Base→Pendulum attestation). +- The on-chain-approvals variant needs no coordination infrastructure at all. + +**Cons:** +- The attestor set is a real trust assumption (mitigated by independence, caps, monitoring, pause — see PRD §8). +- We own the operational burden: key ceremonies, monitoring, runbooks, gas funding. +- Custom code needs careful adversarial review (though the surface is small and standard). + +### Option B1: Ride existing infrastructure — AssetHub → Snowbridge → Ethereum → Base standard bridge + +| Dimension | Assessment | +|---|---| +| Complexity | Low code, very high integration/UX complexity (4 hops) | +| Trust model | Strongest available (light-client bridges + canonical rollup bridge) | +| Meets supply requirement | **No** | +| UX | Multi-hop, multi-wallet, slow, fee-laden | +| Ongoing burden | Dependent on three external bridge systems | + +**Pros:** trust-minimized end to end; almost no code to write (teleport to AssetHub already shipped); no attestors to operate. + +**Cons — and why it was rejected:** +- The Base token would be a bridge-wrapped representation whose supply reflects only what has been bridged — **fails the day-one total-supply requirement outright.** +- We would not control the Base contract (created by the OP standard bridge), so no `ERC20Votes`, no governance integration, no say in metadata. +- Four-hop UX (Pendulum → AssetHub → Ethereum → Base) is unacceptable for a general holder base, and each hop has its own fees, delays, and failure modes. +- This route is designed for *bridging*, and it is a fine answer to "make PEN reachable"; it is a poor answer to "migrate PEN's home." + +### Option B2: Hyperbridge (ISMP) as message channel + +| Dimension | Assessment | +|---|---| +| Complexity | High — ISMP pallet-stack integration into the runtime, dependency on external relayer economics | +| Trust model | Consensus proofs (BEEFY) — trust-minimized, no attestor set of our own | +| Meets supply requirement | Only with the same pre-mint-to-vault construction as Option A | +| UX | Good (direct Polkadot↔Base messaging) | +| Ongoing burden | Permanent runtime dependency on ISMP pallets across all future SDK upgrades | + +**Pros:** genuinely trust-minimized without building a light client ourselves; direct route to Base; would be the right backbone if we ever wanted a permanent two-way bridge. + +**Cons — and why it was rejected:** +- Heavy runtime integration for a mechanism we intend to run for a bounded migration window, then decommission. +- Adds a permanent maintenance tax: the ISMP pallet stack must survive every Polkadot-SDK upgrade this repo goes through. +- Still needs the vault/pre-mint construction to satisfy the supply requirement, so it replaces only the attestor layer — the most easily bounded part of Option A — at the highest integration cost. +- **Revisit trigger:** if two-way bridging ever becomes a product requirement, re-evaluate Hyperbridge before extending Option A. + +### Option C: Snapshot + Merkle-claim airdrop + +| Dimension | Assessment | +|---|---| +| Complexity | Low on Base (Merkle distributor), but a hard identity problem | +| Trust model | Trustless claims on Base — but only after a trusted registration/snapshot step | +| Meets supply requirement | Yes (pre-mint to distributor) | +| UX | Hard cutover; claim flow; registration prerequisite | +| Ongoing burden | Low | + +**Pros:** the Base side is fully trustless once the Merkle root is set; minimal infrastructure; clean if the chain is being shut down on a fixed date. + +**Cons — and why it was rejected:** +- **sr25519 cannot be verified on the EVM**, so users cannot prove ownership of their Pendulum account in the claim contract. They would have to register a Base address *on Pendulum before the snapshot* — which is already half of Option A's pallet, without its flexibility. +- Forces a hard cutover: balances frozen at block X, one shot at the Merkle root, no way to accommodate late unstakers (staking/vesting locks mean many holders *cannot* be ready at an arbitrary snapshot date). +- Whoever computes the Merkle root is a single trusted party at one critical moment — concentration of the same trust Option A spreads across 5 parties and time. +- Only appropriate for a scheduled chain shutdown, which is not (yet) the plan. + +## Options considered — key sub-decisions within Option A + +### Supply model: pre-mint to vault (chosen) vs. mint-on-demand + +Mint-on-demand is the classic bridge pattern but fails the day-one supply requirement (`totalSupply` grows with migrations) and — worse — requires a live minter privilege on the token forever, making infinite mint the top attack scenario. Pre-minting the max issuance to the vault makes `totalSupply` correct from deployment, lets the token ship with **no mint function at all**, and caps the worst case at the vault's remaining balance. Circulating supply is reported to trackers as `totalSupply − vault balance`. This resolved what initially looked like a conflict between the "full supply visible" and "lock and mint" requirements: migration becomes lock/burn-and-**release**. + +### Attestation transport: on-chain approvals (chosen) vs. off-chain signature aggregation vs. light client + +- **On-chain approvals** (chosen): each attestor sends `approve(nonce, recipient, amount)` directly to the vault; the contract counts distinct-attestor approvals of the identical tuple and executes on the k-th. The chain is the coordinator — no signature-collection service, no API, no gossip; attestors share nothing but the contract address. Cost: k transactions per migration instead of one (cents on Base). Chosen for operational simplicity and attestor independence. +- **Off-chain aggregation** (Wormhole-style): attestors sign EIP-712 payloads, a service collects k signatures, anyone submits one `release(..., sigs[])` transaction. One tx per migration and user-self-serve claims, but requires building and operating a coordination service — rejected as unnecessary at migration volumes. +- **Light client / consensus proofs:** correct in the limit, disproportionate for a one-way migration (see Options B1/B2). + +### Pendulum-side effect: burn vs. lock — OPEN (PRD D1, recommendation: burn) + +Burn keeps the global invariant (`PEN on Pendulum + released on Base = max issuance`) trivially auditable and leaves no honeypot account on the Substrate side. Lock only makes sense if reverse flow is ever plausible — which the one-way decision forecloses. Kept open in the PRD only until the Pendulum chain end-state discussion concludes. + +## Options considered — post-migration governance + +The forcing fact: **migrated PEN cannot vote on Pendulum.** Whether burned or locked, it is invisible to `pallet-democracy`/referenda, so on-chain governance power on Pendulum shrinks to the unmigrated remainder — an adversely-selected and ever-cheaper-to-capture electorate. + +| Option | Pros | Cons | Verdict | +|---|---|---|---| +| **G1: Keep Pendulum as the governance chain** | No new infrastructure; familiar tooling | Governance token has left the chain — legitimacy collapses and capture gets cheaper daily; permanent coretime + collator + SDK-upgrade burden just to host votes | Rejected | +| **G2: Snapshot + executor Safe only** | Free, gasless, fastest to ship; works for any decision scope | Execution is trusted (Safe could ignore votes); weak optics for treasury-scale decisions | Rejected as sole mechanism; retained as a component | +| **G3: Full on-chain Governor + Timelock only** | Trustless execution; the setup investors recognize (Tally) | Only natively controls Base-side things; gas-cost voting UX; overkill for off-chain/cross-chain decisions | Rejected as sole mechanism; retained as a component | +| **G4: Hybrid (G2 + G3 + Pendulum technical committee) — CHOSEN** | Trustless where the assets live (Base treasury, vault parameters); pragmatic everywhere else; technical committee keeps the chain patchable without pretending it is token-governed | Two venues to operate; requires clear scoping of what is decided where | **Chosen** | + +Consequences for the token contract: `ERC20Votes` must be included at deployment (immutable token — cannot be retrofitted). Snapshot quorums must be defined against circulating supply with the vault address excluded, or they are unreachable early in the migration. + +## Trade-off analysis (summary) + +The decisive requirement was **day-one supply correctness**, which only a self-deployed, pre-minted token satisfies — eliminating B1 outright and reducing B2 to "a more expensive attestor replacement." Between A and C, the sr25519 problem means C secretly contains A's registration pallet anyway, while adding a hard-cutover constraint that conflicts with staking/vesting lock realities. Within A, every sub-choice followed one principle: **make the permanent artifact (the token) trustless and boring, and confine all trust into a temporary, rate-limited, monitored, pausable component.** + +## Consequences + +**Easier:** tracker/investor-facing supply story (correct from day one); security review (small, standard surfaces); incident response (caps + pause + single trusted component); eventual decommissioning (turn off attestors, sweep vault per governance vote). + +**Harder:** we own attestor operations (key ceremonies, monitoring, gas funding, external-operator onboarding); users must trust the attestor set during the window (mitigated, not eliminated); no reverse path if anyone regrets migrating. + +**To revisit:** Hyperbridge if two-way bridging ever becomes a requirement; the Pendulum chain end-state (interacts with burn-vs-lock, D1); governance venue consolidation once migration completes. + +## Action items + +1. [ ] Resolve PRD open decisions D1–D6 (see [PRD §4.2](pen-base-migration-prd.md)) +2. [ ] Spec the `token-migration` pallet in this repo +3. [ ] Draft `PEN.sol` + `MigrationVault.sol` and run internal adversarial reviews (PRD §9) +4. [ ] Open attestor-operator conversations (D4) and exchange coordination (PRD §11) diff --git a/docs/pen-base-migration-community-overview.md b/docs/pen-base-migration-community-overview.md new file mode 100644 index 000000000..6f78eebe0 --- /dev/null +++ b/docs/pen-base-migration-community-overview.md @@ -0,0 +1,187 @@ +# PEN → Base Migration: Community Overview + +This document accompanies the community discussion post about migrating PEN +from the Pendulum parachain to Base. It explains the working design in plain +language so holders can evaluate it. **It is not a governance proposal** — the +final parameters (migration window, attestor operators, caps, guardian, quorum +mechanics, end-of-window policy) will be fixed in a formal proposal after the +community discussion. + +For the full engineering specification, see the +[technical PRD](pen-base-migration-prd.md). Deeper companions: +[governance guide](pen-governance-guide.md), +[migration-window analysis](pen-migration-window-analysis.md), +[approach rationale (ADR-001)](adr-001-pen-base-migration-approach.md). + +## The design in one paragraph + +PEN becomes a **fixed-supply ERC-20 on Base**: exactly 150,000,000 PEN, +18 decimals, with the **entire supply minted exactly once at deployment** into +a migration vault. The token +has **no mint function, no owner authority, and no upgradeable proxy** — its +supply can never be increased by anyone. Holders migrate one-way: transferable +PEN is removed from circulation on Pendulum (the working design burns it), and +the vault on Base releases the same amount to the holder's Base address after +independent verification. There is no Base→Pendulum path. + +## What happens when you migrate + +1. You call the migration function on Pendulum and provide your Base (EVM) + address. +2. Your transferable PEN is removed from Pendulum circulation, and Pendulum + emits a migration event with a unique ID, your Base address, and the amount. +3. After Pendulum reaches relay-chain finality, each attestor service + independently observes the event from its own node. +4. When the Base vault has **three matching approvals** for the identical + (ID, address, amount), it releases your PEN on Base. + +End to end this normally completes within minutes of finality. **The action is +irreversible** — the UI enforces address checksum validation, warns when the +destination is a smart contract, requires an explicit confirmation, and +recommends a small test migration for large amounts. + +**Only freely transferable PEN can migrate.** Staked, vesting, locked, or +reserved balances must first be freed (unstake, claim vested tokens, remove +governance votes). The UI shows your locked balance and what to do about it. + +## Your amount does not change + +Moving from 12 decimals (Pendulum) to 18 decimals (Base) is an exact technical +conversion of base units by 10⁶. **1 PEN on Pendulum = 1 PEN on Base.** Your +amount and your share of supply are unchanged. + +## Supply transparency — and why exactly 150 million + +- `totalSupply()` on Base equals the full maximum supply from day one. +- The vault's balance is **excluded from circulating supply** — only migrated + tokens count as circulating. +- Every release is publicly verifiable on Base against a finalized Pendulum + burn; an independent monitor continuously checks that + `vault balance + released = total supply` and that nothing was ever released + without a matching burn. + +One detail we want to state explicitly rather than have discovered later: +Pendulum's live on-chain issuance today is slightly **below** 150 million +(~149.93M) — an untidy artifact of the chain's history (fee burns and similar), +not a meaningful tokenomics figure, and one that keeps drifting slightly as +fees continue to be burned. The Base token is deliberately set to a **clean, +canonical 150,000,000**, which is the right constant for trackers, +integrations, and an immutable token contract. + +What happens to the difference (~67,000 PEN, about 0.045% of supply): + +- It **cannot be released by the migration** — releases require a matching + burn on Pendulum, and no burns can ever exist for tokens that were never in + circulation there. It sits inert in the vault. +- It is **excluded from circulating supply** for the entire migration. +- At window close it moves — together with any unmigrated remainder — to the + **community treasury**, via the same governed, timelocked sweep. It is not + allocated to the team or any individual; only a public governance decision + can ever spend it. + +Net effect: every holder's conversion stays exactly 1:1, and the rounding +delta ends up under community control rather than as a strange decimal baked +into the token forever. + +## Security model, stated honestly + +Base cannot cryptographically verify Pendulum state (that would require an +on-chain Polkadot light client — a multi-year effort). For a finite, one-way +migration the design instead uses a **3-of-4 attestor model with strict damage +limits**: + +- Each attestor watches finalized Pendulum events **from its own node**, so no + single faulty or malicious RPC node can feed all attestors wrong data. +- A release needs **3 of 4** attestors to approve the identical migration + tuple; each key is isolated on separate infrastructure. +- The attestors may initially be **team-operated** (Pendulum currently has no + external node operators). **This is a meaningful trust trade-off, not a claim + of trustlessness.** + +Because operator independence is limited at the start, the protections that +actually carry the security are: + +- **Rate caps** — a per-release maximum and a rolling 24-hour cap on total + releases, so even a full compromise of the attestor set is limited to a + pre-agreed daily amount before it can be stopped. +- **A fast pause guardian** — a separate Safe (held by people who do not hold + attestor keys) that can freeze all releases in a single transaction. +- **A ≥48-hour timelock** on every sensitive change: unpausing, cap changes, + attestor changes, and any movement of unmigrated supply. Nothing sensitive + can happen silently or instantly. +- **Independent monitoring** — a watchdog on separate infrastructure that + verifies every release against a finalized Pendulum event and alerts (and + can auto-pause) on any inconsistency or attestor outage. + +The final proposal will publish the concrete parameters: the attestor set and +its independence arrangements, the cap values, the guardian Safe and its +threshold, and the monitoring setup. + +## The migration window + +The window's length is one of the main questions of the community discussion. +Two facts frame it: + +- **An earliest close date is not an automatic sweep.** No unmigrated PEN + moves just because the window elapsed. Moving any remainder (to a + treasury-controlled address, a burn, or another approved path) requires a + separate governance decision and a timelocked execution — and extending the + window is always an option. +- **On-chain locks do not force a long window.** Analysis of live chain state + ([details](pen-migration-window-analysis.md)) shows almost all genuinely + time-locked (vesting) PEN unlocks within a few months; a small residue can + be force-unlocked by referendum if needed. Most "locked" balances (staking, + already-vested tokens, governance votes) can be freed by their holders at + any time within hours. + +## Governance after migration + +The intended model is hybrid: + +- **On-chain (binding):** an OpenZeppelin Governor + Timelock on Base controls + the Base-side contracts and treasury; visible and votable via Tally. +- **Off-chain (signaling):** Snapshot for broader community decisions, executed + by an elected Safe. +- **Pendulum side:** the existing technical committee retains narrowly-scoped + authority (security patches, emergency actions) while the chain runs — with + no discretionary control over Base-side PEN or its treasury. + +A hard design requirement: **the migration vault must neither vote nor make +quorum unreachable.** Quorum is defined against circulating supply with the +vault explicitly excluded, in both Snapshot and on-chain governance. See the +[governance guide](pen-governance-guide.md) for worked examples of both tracks. + +## What this migration does NOT do + +- No two-way bridge between Base and Pendulum. +- No change to the 1:1 conversion, the maximum supply, or any holder's share. +- No migration of non-PEN assets (Spacewalk-wrapped and XCM assets are + unaffected). +- No automatic migration of staked/locked/vesting balances, and no automatic + migration of exchange-held PEN — unless an exchange announces a supported + process, expect to withdraw to self-custody and use the public migration + route. +- No immediate shutdown of the Pendulum chain (its longer-term operating model + is a separate discussion). +- No new centralized-exchange listing funded as part of the migration, and no + new PEN staking program on Base. + +## Rollout, in order + +1. Community discussion (the post this document accompanies). +2. Team response + formal governance proposal fixing the final parameters. +3. Testnet deployment and end-to-end testing (Foucoco + Base Sepolia). +4. Security reviews, key-management ceremonies, monitoring setup, and + operational drills. +5. Mainnet soft launch with conservative caps (team + invited large holders). +6. Public launch of the migration UI; caps raised via governance. +7. Tracker updates (total vs. circulating supply methodology). +8. Admin handover to token holders: during launch, the vault's settings + (caps, attestor set, pause/unpause) are administered by a team multisig so + the system can be wired and verified quickly. As the final step, that admin + power is transferred to the community governance structure — the + token-holder Governor behind the 48-hour timelock — after which no + sensitive parameter can change without a public on-chain vote. + +Nothing is irreversible until the formal proposal is approved and the reviewed +contracts and operational setup are live. diff --git a/docs/pen-base-migration-prd.md b/docs/pen-base-migration-prd.md new file mode 100644 index 000000000..6c3b5746d --- /dev/null +++ b/docs/pen-base-migration-prd.md @@ -0,0 +1,241 @@ +# PRD: PEN Token Migration from Pendulum to Base + +| | | +|---|---| +| **Status** | Draft v2 — all D-decisions recorded; window (D5) subject to the community discussion | +| **Date** | 2026-07-10 | +| **Owner** | Pendulum / SatoshiPay team | +| **Scope** | One-way migration of the native PEN token from the Pendulum parachain (Polkadot) to an ERC-20 on Base, plus post-migration governance | + +--- + +## 1. Summary + +We will migrate the PEN token — the native token of the Pendulum Substrate parachain — to Base as a **fixed-supply ERC-20** (150,000,000 PEN, 18 decimals). The full maximum issuance is pre-minted at deployment into a **MigrationVault** contract; the token contract has **no mint function**. Users migrate by calling a `migrate` extrinsic on Pendulum that burns their PEN and emits an event carrying their Base address. A **3-of-4 attestor set** — initially team-operated, each attestor on its own Pendulum node (D4) — observes relay-chain-finalized events and submits matching **on-chain approvals** to the vault on Base; the third matching approval releases the tokens from the vault to the user. + +Post-migration governance is **hybrid**: an OpenZeppelin Governor + Timelock on Base for on-chain control of Base-side contracts and treasury, Snapshot for off-chain/cross-chain decisions, executed by an elected Safe multisig, with a technical committee retained for Pendulum-side runtime actions for as long as the chain runs. + +The migration is **one-way**. No reverse flow (Base → Pendulum) will be built. + +## 2. Background and motivation + +- PEN currently exists only as the native token of the Pendulum parachain (12 decimals, Substrate/sr25519 accounts). +- Liquidity, investor attention, and tooling (DefiLlama, CoinGecko, Etherscan-class explorers, DeFi integrations) are concentrated in EVM ecosystems; Base is the chosen destination. +- A key requirement is that supply statistics on Base are **correct and complete from day one**: `totalSupply()` must equal PEN's maximum issuance so trackers never display a confusing split between two chains. +- Verifying Pendulum state cryptographically on an EVM chain would require an on-chain Polkadot light client (BEEFY verification) — a multi-year effort (cf. Snowbridge, Hyperbridge). For a finite-lifetime, one-way migration, a k-of-n attestation model with strict blast-radius limits is the appropriate engineering trade-off. + +## 3. Goals + +1. A live ERC-20 PEN token on Base whose `totalSupply()` equals the PEN maximum issuance from the moment of deployment. +2. A live MigrationVault on Base holding all unmigrated supply, releasing tokens only on 3-of-4 attestor agreement. +3. A `token-migration` pallet on Pendulum allowing any holder to migrate transferable PEN to a Base address of their choice. +4. Trackers (DefiLlama, CoinGecko, CoinMarketCap) display correct total and circulating supply (vault balance excluded from circulating). +5. Worst-case loss from full attestor-quorum compromise is bounded by rate caps and detected by independent monitoring within minutes. +6. A functioning hybrid governance stack on Base after migration. + +### Non-goals (explicitly out of scope) + +- **Two-way bridging** (Base → Pendulum) — the design is one-way by construction. +- **On-chain light-client verification** of Pendulum state on Base. +- **Cross-chain governance execution** (Base votes cryptographically executing Substrate calls). +- Migration of any token other than native PEN (Spacewalk-wrapped assets, XCM assets, etc. are unaffected). +- Decommissioning plan for the Pendulum chain itself (tracked separately; this PRD only requires that migration works while the chain runs). + +## 4. Decisions + +### 4.1 Locked in + +| Decision | Choice | Rationale | +|---|---|---| +| Supply model on Base | Entire max issuance pre-minted to vault at deployment; token has no mint function, no owner, no upgradeability | Correct tracker stats from day one; eliminates infinite-mint attack surface; worst case bounded by vault balance | +| Migration direction | One-way only | Halves the attack surface; no Base-side event attestation needed | +| Attestation transport | **On-chain approvals**: each attestor sends its own `approve` transaction to the vault; the k-th matching approval executes the release | No off-chain signature-coordination infrastructure; the chain is the coordinator; attestors are fully independent processes | +| Attestor threshold | 3-of-4 | Theft requires 3 simultaneously compromised keys; releases tolerate 1 offline attestor. The set is initially team-operated (D4), so the load-bearing compensations are caps, monitoring, pause, and separation of duties — not organizational independence | +| Governance | Hybrid: OZ Governor + Timelock (Base contracts/treasury) + Snapshot + executor Safe + Pendulum technical committee | On-chain teeth where the assets live; pragmatic elsewhere | +| Token extensions | `ERC20Permit` + `ERC20Votes` included at deployment | `ERC20Votes` cannot be retrofitted into an immutable token; required for future Governor voting | + +### 4.2 Decided (formerly open; recorded 2026-07-09/10) + +| # | Decision | Outcome | +|---|---|---| +| D1 | Pendulum-side effect of `migrate` | **Burn.** Keeps the invariant `Σ burned on Pendulum = Σ releasable on Base` trivially auditable and leaves no honeypot. (Implemented.) | +| D2 | Decimals on Base | **18** (scale ×10⁶ at release, in the vault only — V7). Exact conversion; no change to holder amounts, ownership share, or max supply. (Implemented.) | +| D3 | Max issuance | **Exactly 150,000,000 PEN — deliberately rounded up from the live Pendulum issuance (~149.93M, no inflation).** The live figure is an untidy artifact of the chain's history (fee burns etc.), not a meaningful tokenomics number; a clean canonical 150M is what trackers, integrators and the immutable constructor should carry, and it stays valid even as small fee burns keep nudging the live figure. Consequence: the **rounding delta (~67k PEN, ~0.045%) can never be released by migration** — no matching burns can ever exist for it — so it sits inert in the vault, is excluded from circulating supply, and at window close moves to the **community treasury via the governed, timelocked sweep (D5)**. It is not allocated to any person or team; only a governance decision can ever spend it. Per-holder conversion remains exactly 1:1. | +| D4 | Attestor set composition | **4 attestors, all initially team-operated**, one per internal RPC node, threshold **3-of-4**. Rationale: Pendulum currently has no external node operators, and onboarding them means asking outsiders to fund + run a node for the whole window. Honest consequence: organizational independence is *not* claimed — blast-radius controls (caps, monitoring, guardian pause, ≥48h timelock) and **separation of duties** (guardian + monitor operated by someone other than the attestor-key holder) carry the security. | +| D5 | Migration window end policy | Internal working target: **3-month earliest close** (`earliestSweepTimestamp ≈ deploy + 3 months`), conditional on the planned block-time improvement toward 12s; a referendum (`vesting-manager.remove_vesting_schedule`) force-unlocks any vesting residue and the permanent sentinel locks before close ([window analysis](pen-migration-window-analysis.md)). **Final window length is deliberately left to the community discussion / formal governance proposal** (the discussion post solicits 6-month / 12-month / open-ended feedback). An earliest close date is *not* an automatic sweep: moving any remainder requires a separate governance decision + timelocked execution. | +| D6 | Encumbered balances policy | **Only unstaked, freely transferable PEN migrates** (enforced by the pallet). Staked/vesting/governance-locked balances must be freed first; UI and docs surface this. No vesting locks should extend beyond the window — residue handled per D5's referendum path. | + +## 5. System overview + +``` + PENDULUM (Polkadot parachain) BASE (OP-stack L2) +┌─────────────────────────────┐ ┌──────────────────────────────┐ +│ token-migration pallet │ │ PEN ERC-20 (immutable) │ +│ migrate(amount, h160) │ │ totalSupply = max issuance │ +│ → burn PEN │ │ no mint, no owner │ +│ → event {nonce, h160, amt} │ ├──────────────────────────────┤ +└──────────┬──────────────────┘ │ MigrationVault │ + │ finalized events │ holds unmigrated supply │ + ▼ │ approve(nonce, to, amt) │ + 4 × attestor daemon ──────── Base txs ───────▶ │ 3rd matching approval │ + (own full node each, │ → transfer to user │ + relay-finality only) │ caps · pause · timelock │ + └──────────────────────────────┘ + invariant monitor (independent): Σ burned on Pendulum == Σ released on Base +``` + +**Happy path:** user calls `migrate(amount, base_address)` on Pendulum → PEN burned, event with unique `nonce` emitted → block reaches relay-chain finality → each attestor independently decodes the event and submits `approve(nonce, recipient, amount)` on Base → on the 3rd identical approval the vault transfers `amount` (decimal-adjusted) to `recipient` and marks `nonce` consumed. + +## 6. Component requirements + +### 6.1 Pendulum: `token-migration` pallet + +- **P1** — Extrinsic `migrate(amount: Balance, base_address: H160)`; atomically **burns** `amount` of transferable native PEN from the caller (D1) and emits `MigrationInitiated { nonce: u64, base_address: H160, amount: Balance }`. +- **P2** — `nonce` is a monotonically increasing storage counter; globally unique across the pallet's lifetime; never reused, including across runtime upgrades. +- **P3** — Rejects: `amount` below a configurable minimum (dust threshold ≥ existential-deposit-scale), non-transferable balance (staked, vesting-locked, reserved), and `amount` that would leave the caller between 0 and the existential deposit (must migrate to exactly 0 or stay ≥ ED). +- **P4** — Pallet is pausable via a privileged origin (technical committee / root) to halt new migrations during incidents. +- **P5** — Storage exposes cumulative migrated total for monitoring (`TotalMigrated`). +- **P6** — No knowledge of Base state; the pallet is fire-and-forget. Documentation and UI must make irreversibility explicit. +- **P7** — Deployed to Foucoco (testnet runtime) first with identical logic. + +### 6.2 Base: `PEN` ERC-20 + +- **T1** — OpenZeppelin `ERC20` + `ERC20Permit` + `ERC20Votes`. No other custom logic. Decide the EIP-6372 clock mode (block number vs. timestamp) before deployment — the Governor must use the same clock, and it cannot be changed later. +- **T2** — Constructor mints the entire max issuance (per D2/D3) to the MigrationVault address and nothing else. No `mint`/`burn` owner functions; no `Ownable`; **not upgradeable** (no proxy). +- **T3** — `name`/`symbol` consistent with existing branding (`Pendulum`, `PEN`); decimals per D2. + +### 6.3 Base: `MigrationVault` + +- **V1** — `approve(nonce, recipient, amount)` callable only by addresses in the attestor set. Approvals are counted per `keccak256(abi.encode(nonce, recipient, amount))` — attestors must agree on the *identical* tuple. A conflicting tuple for the same nonce counts separately and never merges. +- **V2** — On the k-th (k = 3) distinct-attestor approval of the same tuple: verify nonce unconsumed → verify rate caps → mark nonce consumed → `transfer(recipient, amount)`. Consumed nonces are permanent (mapping, not sequential counter — out-of-order finalization/submission must work). +- **V3** — One approval per attestor per tuple; duplicate approvals from the same attestor revert. +- **V4** — **Rate caps:** per-release maximum and rolling 24h aggregate maximum, both governance-configurable behind the timelock. Initial values sized so a full quorum compromise loses a bounded, pre-agreed amount before pause (target: < 1–2% of vault balance per day). +- **V5** — **Pause:** a guardian role (fast Safe, small threshold) can pause releases instantly. Unpause and all parameter changes (caps, attestor set, guardian) go through a TimelockController with ≥ 48h delay. +- **V6** — Attestor set changes (add/remove/replace key) via timelocked admin only; changing the set must not invalidate pending approvals in a way that permanently strands a legitimate migration (re-approval by the new set must be possible). +- **V7** — Decimal conversion 12 → 18 (if D2 = 18) happens in exactly one place (the vault, at release), as an exact ×10⁶ multiplication. +- **V8** — Not upgradeable. All flexibility comes from parameters + pause. Emits full event history (`Approved`, `Released`, `Paused`, `CapsUpdated`, `AttestorSetUpdated`) for the monitor and for public auditability. +- **V9** — End-of-window handling per D5: a timelocked function to sweep the remainder to a governance-designated destination (or burn), callable only after a hard-coded earliest timestamp. + +### 6.4 Attestor daemon (×4 instances, initially team-operated per D4) + +- **A1** — Connects **only to its own Pendulum full node** (never a shared/public RPC — one malicious or faulty node must not be able to feed all attestors wrong block data); subscribes to **relay-chain-finalized** heads; decodes `MigrationInitiated` events. Never acts on best/unfinalized blocks. +- **A2** — For each event, submits `approve(nonce, recipient, amount)` to the vault on Base, with idempotent retry (safe to resubmit; duplicates revert harmlessly) and crash-recovery from a persisted checkpoint (last processed finalized block). +- **A3** — Each instance: separate infrastructure, separate secp256k1 key (HSM or equivalent isolation), separately funded Base gas wallet with balance alerting. Operators are initially team members (D4), so **separation of duties is mandatory**: the guardian Safe and the invariant monitor must be operated by people who do not hold attestor keys — otherwise the pause button and the alarm are held by the same hands they guard against. +- **A4** — No shared code paths for event *interpretation* where avoidable is nice-to-have; at minimum, no shared runtime infrastructure or key storage. No communication between attestors — the vault contract is the only coordination point. +- **A5** — Handles runtime upgrades on Pendulum gracefully (metadata refresh) and alerts on decode failures rather than skipping events silently. + +### 6.5 Invariant monitor (independent watchdog) + +- **M1** — Runs on infrastructure separate from all attestors; reads Pendulum (`TotalMigrated`, per-nonce events) and Base (`Released` events, vault balance) independently. +- **M2** — Continuously checks: (a) every released nonce corresponds to exactly one finalized Pendulum event with matching recipient/amount; (b) Σ released ≤ Σ migrated; (c) vault balance + Σ released = max issuance. +- **M3** — On any violation: page on-call immediately and (design decision) optionally hold a guardian key to auto-pause the vault. +- **M4** — Also monitors liveness: alerts if a finalized migration event has < 3 approvals after N minutes (attestor outage detection). + +### 6.6 Migration UI + +- **U1** — Web app: connect Substrate wallet, enter/connect Base address with **EIP-55 checksum validation**, explicit irreversibility confirmation, live status tracking (finalization → approvals 0/3 → released, with Base tx link). +- **U2** — Warn when the destination is a contract address (Safe is fine; other contracts may strand funds); require an extra confirmation. +- **U3** — Surface encumbered-balance state (D6): show staked/vesting amounts and guide the user through unstaking first. +- **U4** — Encourage a small test migration for large holders as a documented pattern. + +### 6.7 Governance stack (hybrid) + +- **G1** — Snapshot space with PEN-on-Base voting strategy. **The vault address must be excluded from voting power and quorum math** — quorum thresholds must be defined against circulating supply, not `totalSupply()`, or they are unreachable early in the migration. +- **G2** — OZ Governor + TimelockController on Base using `ERC20Votes` (delegation-based). The timelock becomes the admin of the MigrationVault parameters (caps, attestor set, end-of-window sweep) after an initial bootstrap period during which a Safe holds admin (see rollout). +- **G3** — Executor Safe (elected signers) carries out Snapshot outcomes that are off-chain or on other chains; optionally hardened later with oSnap/SafeSnap. +- **G4** — Pendulum-side runtime actions remain with the existing technical committee for as long as the chain runs; its mandate post-migration is documented (security patches, pallet pause, no discretionary treasury power). +- **G5** — The pause guardian is **not** the Governor (too slow for incidents); it is a small fast Safe, itself replaceable via timelock. + +## 7. Acceptance criteria + +1. **Supply correctness:** immediately after deployment, `PEN.totalSupply()` on Base equals the confirmed max issuance (D3) and 100% sits in the vault; DefiLlama/CoinGecko display total supply = max issuance and circulating supply excluding the vault. +2. **End-to-end migration:** a user migrating X PEN on Pendulum receives exactly X (decimal-adjusted) PEN on Base after relay finality + 3 approvals, with no manual intervention, on testnet and mainnet. +3. **Conservation invariant:** at all times, Σ burned on Pendulum ≥ Σ released on Base, and vault balance + Σ released = max issuance; the monitor demonstrably alerts (staging drill) on injected violation. +4. **Replay safety:** a consumed nonce can never release twice (unit + fork-test proof); an attestor submitting the same approval twice has no effect. +5. **Quorum safety:** 2 colluding attestors cannot release anything; 1 offline attestor does not halt migrations. +6. **Caps and pause:** releases above per-tx or daily caps revert; guardian pause takes effect in one transaction and blocks all releases; every parameter change is observably delayed ≥ 48h by the timelock. +7. **No mint surface:** verified absence of any code path that increases `totalSupply()` post-constructor (review assertion). +8. **Governance live:** Snapshot space operational with vault excluded from strategy; Governor + Timelock deployed, delegation working, and admin of the vault transferred per rollout plan. +9. **Security reviews complete:** all critical/high findings from the internal adversarial review rounds (§9) resolved or formally risk-accepted before mainnet vault funding. +10. **Ops readiness:** runbooks exist and have been drill-tested for: attestor key compromise, attestor outage, invariant violation, pause/unpause, and Pendulum runtime upgrade. + +## 8. Security requirements and threat model + +**Trust assumptions:** correctness reduces to (a) ≤ 2 of the 4 attestor keys compromised at any time, (b) Polkadot relay finality is honest, (c) the vault contract is correct. There is no cryptographic verification of Pendulum state on Base. With an initially team-operated set (D4) the honest framing is: this is a **trusted, damage-bounded** design, not a trustless one — the compensations are per-key isolation, caps, independent monitoring, fast pause, and separation of duties. + +| Threat | Mitigation | +|---|---| +| Attestor key compromise (< quorum) | 3-of-4 threshold; conflicting tuples never merge; monitor flags approvals without matching Pendulum events | +| Attestor quorum compromise (≥ 3 keys) | Rate caps bound daily loss (V4); independent monitor + guardian pause (M3, V5) operated under separation of duties (A3); per-key HSM isolation. Because operators are initially one organization, caps + monitoring + pause are the primary defense, not operator independence | +| Fake/reorged Pendulum events | Attestors act only on relay-finalized blocks from their own nodes (A1); post-finality reorgs are not possible on Polkadot | +| Replay / double release | Permanent consumed-nonce mapping (V2); per-attestor per-tuple dedup (V3) | +| Malicious/typo destination address | UI checksum + contract-address warnings (U1, U2); irreversibility messaging; documented test-migration pattern | +| Infinite mint on Base | Structurally impossible — no mint function (T2) | +| Governance capture of vault params | 48h timelock on all changes (V5) gives holders and the monitor time to react; guardian can pause during the window | +| Pallet abuse (griefing with dust, nonce games) | Minimum amount (P3); nonce is pallet-internal, not user-supplied (P2) | +| Attestor gas exhaustion / outage | Funded-wallet alerting (A3); liveness monitoring (M4); 1-of-4 outage tolerance | +| RPC/supply-chain trust | Own full nodes only (A1); pinned dependencies and reproducible builds for daemon and contracts | + +**Standing security requirements:** all privileged Base keys in Safes or HSMs; no single human can both approve and change the attestor set; the guardian Safe and the invariant monitor are operated by people who hold no attestor keys (separation of duties, D4); public disclosure/bug-bounty channel before mainnet; all contracts verified on the Base explorer. + +## 9. Security assurance (decision: no external audit) + +**Decision:** no external audit is commissioned. The residual risk is +consciously accepted and carried by the threat-model mitigations of §8 — rate +caps bounding worst-case daily loss, independent monitoring with auto-pause, +the fast guardian, the ≥48h timelock on every sensitive change, separation of +duties, and the conservative-caps soft launch. This trade-off is judged +acceptable because the migration is finite, damage-bounded and pausable, and +the permanent artifact (the token) is an unmodified OpenZeppelin composition +with no mint surface. + +**What is done instead:** repeated **independent internal adversarial review +rounds** over the full stack, each documented with findings and resolutions in +the [internal security review log](pen-migration-internal-review.md), with all +critical/high findings fixed and regression-tested before mainnet vault +funding. Standing practice: every change to the fund-release path triggers a +fresh review round before deployment. + +Review focus, ranked by where the risk actually lives: + +1. **MigrationVault contract (highest priority):** approval counting and tuple hashing (V1–V3), nonce consumption, cap accounting across the 24h window, pause/timelock/role wiring, attestor-set rotation edge cases (V6), decimal conversion (V7), end-of-window sweep (V9). +2. **`token-migration` pallet:** atomicity of burn + event emission, nonce monotonicity across upgrades, balance-encumbrance checks (P3), pause origin, weight/benchmarking correctness. +3. **Attestor daemon:** event decoding against runtime metadata (including post-upgrade), finality handling (proof that it cannot act pre-finality), checkpoint/crash-recovery correctness, key handling. +4. **End-to-end trust-boundary review:** an adversarial walkthrough of the full pipeline (extrinsic → event → daemon → approval → release), explicitly attempting cross-component exploits that no single-component review would catch (e.g., decode ambiguity producing divergent tuples). +5. **Operational review (lighter):** key-management ceremony, Safe configurations, timelock parameters, monitor independence. + +**Out of review scope:** OpenZeppelin library internals, Base/OP-stack infrastructure, Polkadot finality itself, the ERC-20 beyond confirming it is an unmodified OZ composition. + +## 10. Rollout plan + +| Phase | Contents | Gate to next phase | +|---|---|---| +| **0 — Decisions & spec** | Resolve D1–D6 (done — §4.2); community discussion + formal governance proposal fixing the final window and parameters; publish tokenomics/max-issuance statement | Proposal approved | +| **1 — Build & testnet** | Pallet on Foucoco; contracts on Base Sepolia; 4 test attestors; monitor; UI; internal adversarial testing incl. chaos drills (kill attestors, inject bad approvals) | All acceptance criteria pass on testnet | +| **2 — Security reviews & drills** | Internal adversarial review rounds per §9; fix and re-verify; publish the review log; chaos + runbook drills | No open critical/high findings | +| **3 — Mainnet soft launch** | Deploy token + vault (full supply minted); production attestor ceremony; **conservative caps**; team-only + invited large-holder migrations for 1–2 weeks; vault admin held by bootstrap Safe | Soft-launch volume clean, monitor green | +| **4 — Public launch** | Runtime upgrade enabling `migrate` for all; UI public; raise caps to target; tracker submissions (DefiLlama/CoinGecko: supply endpoints, vault as non-circulating); exchange & community comms | ≥ agreed % supply migrated or T reached | +| **5 — Governance handover** | Snapshot space live from phase 4; deploy Governor + Timelock; transfer vault admin from bootstrap Safe to timelock; elect executor Safe | — | +| **6 — Window close (per D5)** | Governance vote on remainder disposition; execute sweep (V9); decommission attestors; final conservation report published | — | + +## 11. Risks and open questions + +- **Adoption risk:** slow migration leaves circulating supply small and Snapshot quorums awkward — heightened by the 3-month window (D5): mitigate with front-loaded comms, early governance cap raises (≥ ~1.7M PEN/day average throughput is required arithmetic), quorum defined on circulating supply (G1), and the option to run the infrastructure a few weeks longer if needed. +- **Unstaking delay friction (D6):** staked holders face the staking unbond period before they can migrate; comms must set expectations. +- **Attestor key concentration (D4):** with a team-operated set, a single organization holds all attestor keys — separation of duties (guardian/monitor held by non-key-holders) is the load-bearing control and must be verifiable, not aspirational. +- **Exchange coordination:** any CEX listing PEN needs a supported path (they migrate custody balances themselves via the same mechanism); start conversations in phase 1. +- **Legal/regulatory review** of the migration mechanics and any public statements about supply — not covered by this PRD, must run in parallel. +- **Pendulum chain end-state** (full sunset vs. minimal maintenance) is deliberately out of scope but interacts with D1 and G4; schedule that decision before phase 6. + +## 12. Deliverables checklist + +- [ ] `token-migration` pallet (+ benchmarks, tests) in this repo, deployed to Foucoco then Pendulum +- [ ] `PEN.sol`, `MigrationVault.sol` (+ Foundry test suite incl. fork tests and invariant tests) +- [ ] Attestor daemon (open-sourced) + operator deployment guide +- [ ] Invariant monitor + alerting integration +- [ ] Releaser service (drains cap-deferred releases; run 2 instances) +- [ ] Migration web UI +- [ ] Governor + Timelock deployment scripts; Snapshot space config (vault-excluded strategy) +- [ ] Runbooks: key compromise, attestor outage, invariant breach, pause/unpause, runtime upgrade +- [ ] Internal security-review log (published; findings and resolutions) +- [ ] Tracker submissions and public migration documentation diff --git a/docs/pen-governance-guide.md b/docs/pen-governance-guide.md new file mode 100644 index 000000000..90a02f8ae --- /dev/null +++ b/docs/pen-governance-guide.md @@ -0,0 +1,224 @@ +# PEN Governance After the Base Migration — How It Works + +| | | +|---|---| +| **Status** | Explainer (companion to the PRD/ADR) | +| **Audience** | PEN holders, contributors, prospective voters | +| **Related** | [PRD §6.7 / G1–G5](pen-base-migration-prd.md), [ADR-001 governance section](adr-001-pen-base-migration-approach.md), `contracts/src/PENGovernor.sol` | + +This is a plain-language walkthrough of the hybrid governance model, with two +real-world examples. It describes *how a decision gets made and executed* after +PEN has migrated to Base. + +## The three organs + +The whole model rests on one idea: **a decision routes to the organ that can +actually execute it.** + +- **PEN holders** are the electorate. Voting power is the delegated + `ERC20Votes` balance of PEN on Base. One practical catch: a holder has **zero + voting power until they delegate** (to themselves or a representative). + Holding tokens isn't voting; delegating is. +- **The on-chain track** — `PENGovernor` + a `TimelockController` — handles + anything that is a deterministic call to a Base contract the timelock + controls: the `MigrationVault` parameters (caps, attestor set, threshold, + guardian, remainder sweep) and any Base-side treasury the timelock owns. + Binding and trustless: nothing but a passed vote can move it. +- **The off-chain track** — Snapshot + an elected executor Safe — handles + decisions that aren't a single on-chain call: discretionary, multi-step, or + living off Base (including on the Pendulum parachain). Snapshot signals + intent gaslessly; the Safe carries it out. + +A **guardian Safe** (instant pause, no vote) and the **Pendulum technical +committee** (runtime and security actions on the parachain) sit outside the +token vote — they exist because some actions must be fast, or must run on a +chain the Base Governor can't reach. + +```mermaid +flowchart TD + H["PEN holders
delegated voting power"] + H --> A["On-chain track · binding"] + H --> B["Off-chain track · signaled"] + + A --> A1["Propose
target: vault.setCaps(…)"] + A1 --> A2["Vote · 5 days
quorum + majority"] + A2 --> A3["Queue → timelock
48-hour public delay"] + A3 --> A4["Execute
timelock calls the vault"] + A4 --> AX(["Example 1 · raise the daily cap"]) + + B --> B1["Snapshot proposal
gasless · vault excluded"] + B1 --> B2["Vote · ~7 days
PEN-on-Base holders sign"] + B2 --> B3["Executor Safe acts
elected multisig"] + B3 --> BX(["Example 2 · fund a liquidity program"]) + + G["Guardian Safe — emergency pause, no vote"] + T["Pendulum committee — runtime & security"] +``` + +The Governor's timing parameters are the deployment defaults and are themselves +governable: voting delay ~1 day, voting period 5 days, timelock delay 48 hours, +quorum a low fraction of supply at launch (raised as circulating supply grows), +plus a proposal threshold of voting power required to open a proposal. + +## Example 1 — raising the migration vault's daily cap (on-chain track) + +**The situation:** migration is live with deliberately conservative caps. +Volume picks up and legitimate migrations start getting deferred by the rolling +daily cap. The community wants to raise `dailyCap` and `perReleaseCap`. This +belongs on the on-chain track because it is exactly one deterministic action — +a call to `vault.setCaps(...)` — and the timelock is the vault's admin, so no +human needs discretion or custody. + +**How it plays out:** + +1. A delegate holding at least the proposal threshold of voting power calls + `propose(...)` with a single action: target `MigrationVault`, calldata + `setCaps(newPerRelease, newDaily)`, and a human-readable description. The + proposal appears on Tally in the pending state. +2. After the voting delay (~1 day — this is also when the voting-power snapshot + is taken, so buying tokens afterward can't influence the vote), voting opens + for five days. Delegates cast for, against, or abstain. To pass, the + proposal needs quorum and more for than against. +3. On success, anyone calls `queue(...)`, scheduling the action inside the + `TimelockController` behind a 48-hour delay. This delay is the safety valve: + for two days the exact pending change is public, and if it looks wrong the + guardian can pause the vault while the community reacts. +4. After the 48 hours, anyone calls `execute(...)`. The timelock — as the + vault's admin — makes the `setCaps` call. The cap is now raised. + +Start to finish is roughly **eight days**, and at no point does a trusted party +decide anything. The timelock is the only address that can call `setCaps`, and +it only ever acts on a vote that already passed. Attestor rotation, threshold +changes, and the end-of-window remainder sweep all run through this identical +path. + +## Example 2 — funding a liquidity and market-making program (off-chain track) + +**The situation:** the DAO wants to bootstrap PEN/USDC liquidity and retain a +market maker for six months, funded with, say, 2,000,000 PEN from the community +treasury. This does not reduce to one on-chain call — it means choosing a +market maker, negotiating terms, moving funds (possibly across venues or +chains), and exercising judgment over six months. That is what the off-chain +track is for. + +**How it plays out:** + +1. A holder posts the proposal to the forum for discussion, then creates a + Snapshot proposal. The voting strategy reads PEN balances on Base at a + snapshot block, with the `MigrationVault` address **excluded** — so the + large unmigrated balance in the vault can't vote, and quorum is measured + against circulating supply rather than total supply. +2. Voting runs for roughly a week and is **gasless**: holders sign messages, + they don't pay gas or even need to delegate. Choices can be a simple + for/against or several funding options. +3. If it passes, the elected executor Safe carries out the mandate — transfers + the PEN, contracts the market maker, and manages the engagement over its + lifetime. + +The honest trade-off: Snapshot itself is a *signal*, not an on-chain +instruction, so this track trusts the Safe signers to honor the result. That is +why the signers are elected and why, if you want to harden it, **oSnap** (UMA's +optimistic oracle) can post the Snapshot outcome on-chain so that — if +unchallenged — it becomes directly executable by the Safe, turning "the Safe +should comply" into an economic guarantee rather than a social one. + +## The treasury: where the money lives and how it's spent + +Example 2 was a treasury spend, so it's worth making the treasury structure +explicit — because the most common wrong assumption is that "the Base treasury" +is a smart contract you build and wire into the others. It isn't. + +**A treasury on Base is just an address that holds tokens.** PEN is a plain +ERC-20 — nothing gets registered or connected to it; whoever holds a balance +spends it by calling `transfer`. So you don't author a treasury contract. You +already have the right address: the `TimelockController`. In the OpenZeppelin +Governor pattern the timelock *is* the treasury and the executor at once — it +holds the reserve, and a passed proposal makes it call `transfer`. + +The recommended shape is a **split, tiered treasury** that maps straight onto +the three organs: + +| Money for… | Lives on | Held / spent by | How | +|---|---|---|---| +| Running the parachain (collators, coretime, Pendulum ops) | Pendulum | `py/trsry`, 3/5 council | existing treasury proposal, pays a Pendulum account in PEN | +| Strategic / large Base spends (reserves, partnerships, big LP) | Base | the `TimelockController` | token-holder proposal → 48h timelock → `transfer` (trustless) | +| Routine Base payouts (grants, MM retainer, small ops) | Base | an elected operating Safe with a delegated budget | Safe multisig tx, optionally Snapshot-signaled (fast) | + +The important discipline: **don't route every payout through the full +Governor.** A 48-hour timelocked proposal for a 3,000-PEN contributor grant is +governance theater. Instead, governance grants the operating Safe a periodic +budget in one action ("500k PEN + 200k USDC this quarter"); day-to-day grants +are then Safe transactions inside that mandate, and governance tops it up (or +claws it back) as needed. For recurring payments, fund a stream (Sablier or +Superfluid) so it doesn't need repeated approvals. None of these are bespoke +contracts — the Safe and the streaming tools are standard, audited, and created +through their own apps, not written by us. + +### Getting the treasury's PEN to Base + +The Pendulum treasury (`py/trsry`) is a keyless account, and the user-facing +`migrate` extrinsic needs a signed origin — so the treasury can't migrate +itself the ordinary way. The `token-migration` pallet therefore has a +governance-gated path, built as two deliberately separate steps: + +1. `set_treasury_destination(base_address)` — sets the fixed Base destination + (the timelock) **once**, as its own reviewed governance action. This is the + single security anchor: the routine migration call carries no address and so + cannot be sent to the wrong place by a typo. +2. `migrate_treasury(amount)` — burns `amount` from the treasury account and + emits the **same** `MigrationInitiated` event as a user migration (with + `who` = the treasury), so the attestors, vault and monitor handle it + identically. It always goes to the pre-set destination. + +Both are gated by the same authority that already approves treasury spends +(root or 3/5 council), and `migrate_treasury` respects the pallet pause and +keeps the treasury account alive (it can't accidentally reap itself). + +Three operational notes when you actually run it: + +- **Mind the daily cap.** A large treasury tranche competes with user + migrations for the vault's rolling daily-cap headroom and may be deferred + (delayed, never lost — it's marked pending and recoverable). Migrate in + tranches, or pass a proposal to temporarily raise the cap. +- **Circulating supply doesn't move.** Pendulum-treasury and Base-treasury are + both non-circulating, so shifting reserves between them changes nothing for + DefiLlama/CoinGecko — just add the Base treasury address to their excluded + list alongside the vault. +- **Do it inside the migration window**, coordinated, so the reserve isn't + stranded if the window later closes. + +### The quietly big win + +Once the reserve is on Base, the treasury can hold and pay **USDC and other +Base-native assets**, not just PEN — which is what contributors and market +makers usually want to be paid in, and which the Pendulum treasury structurally +cannot do today (it is native-PEN-only). You also gain the option to park +reserves in Base DeFi or provide protocol-owned liquidity. That, more than the +payout mechanics, is the real reason to move the strategic reserve to Base +rather than bridging per payout. + +## The routing rule, and three things that trip people up + +The whole model collapses to one question: **can the decision be expressed as a +deterministic call to a Base contract the timelock owns?** If yes, it goes +on-chain and executes trustlessly (Example 1). If it needs discretion, multiple +steps, or lives off Base — including anything on the Pendulum parachain, which +the Base Governor cannot reach — it goes to Snapshot and the Safe (Example 2), +or for runtime and security matters, to the Pendulum committee. + +Three practical notes that matter in real use: + +- **Delegation is a prerequisite for the on-chain track.** A holder with a + large balance but no delegation has no voting power and can't even meet the + proposal threshold. This surprises people constantly; launch communications + should make "delegate to yourself" a first-class step. +- **Emergencies deliberately skip governance.** The guardian Safe can pause the + vault in one transaction, with no vote, precisely because a 48-hour timelock + is the wrong tool for an active incident. Governance then decides the actual + fix through the slow, deliberate path. The guardian is intentionally not the + Governor. +- **Quorum is tuned for a migration in progress.** Because the vault holds most + of the supply early on, on-chain quorum is set to a low fraction at launch + and raised by governance as circulating supply grows, and Snapshot excludes + the vault outright. Otherwise quorum measured against total supply would be + unreachable in the early months. diff --git a/docs/pen-migration-window-analysis.md b/docs/pen-migration-window-analysis.md new file mode 100644 index 000000000..7b605423d --- /dev/null +++ b/docs/pen-migration-window-analysis.md @@ -0,0 +1,117 @@ +# PEN Migration: the 3-month window — analysis and conditions + +| | | +|---|---| +| **Decision** | Migration window target: **3 months**. Block time will be improved toward the 12s target; if any vesting hasn't finished by close, a **referendum force-unlocks the rest**. | +| **Verdict of this analysis** | Workable — but *conditional*. At the 12s target, all vesting finishes in ~2.3 months (fits). At today's measured ~23.6s it takes ~4.5 months (does not fit). The deciding variable is when the block-time fix lands; the referendum is the safety net that makes the plan sound either way. | +| **Data source** | Pendulum mainnet, live query at ~block 7,384,000 (2026-07-10) via `rpc-pendulum.prd.pendulumchain.tech` | + +## Why the window length is a vesting question + +Live Pendulum issuance is **fixed at ~149.93M PEN** (no inflation, confirmed; +the Base token's clean 150M includes a ~67k rounding delta that stays in the +vault and goes to the treasury at the sweep — PRD D3). Every locked +bucket except one can be freed *on demand*, with no calendar dependency: + +| Bucket | Amount | How it becomes migratable | Calendar-gated? | +|---|---|---|---| +| Freely transferable | ~110.45M | already is | no — day one | +| Vesting lock, **already vested** (stale) | ~23.9M | one `vest()` call | no — instant | +| Staked (`parachain-staking`) | ~3.75M | unstake, 2-round unbond (hours) | no | +| Governance (democracy locks) | ~1.16M | remove vote / conviction expiry | holder-managed, short | +| Reserved (identity/proxy deposits) | ~484 PEN | release the deposit | no — instant | +| **Vesting, genuinely still vesting** | **~11.87M** | **wait for the schedule (per block)** | **yes — the only real constraint** | + +So "is 3 months enough?" reduces to: does the ~11.87M of genuine vesting +finish within 3 months? + +## The block-time dependency, quantified + +Vesting releases **per block**, so wall-clock completion depends directly on +block time. The last real schedule completes **505,375 blocks** from the +snapshot. That is: + +| Average block time over the window | Vesting completes in | Fits 3 months? | +|---|---|---| +| 12s (target) | **~2.3 months** | yes, ~0.7 months margin | +| ~15.6s | ~3.0 months | exactly the break-even | +| 20s | ~3.8 months | no | +| ~23.6s (measured today, 10k-block avg) | ~4.5 months | no | + +Two concrete planning numbers fall out of this: + +- **Break-even: the window-average block time must be ≤ ~15.6s.** +- **If the chain jumps from today's ~23.6s straight to 12s, the fix must be + live within ~6 weeks of the window opening** for vesting to finish inside + 3 months (each week at the slow rate consumes roughly half a week of the + margin). + +So: improve block time *early* in the window, not toward the end. + +## The fallback that makes 3 months safe anyway + +If block production doesn't recover fast enough, some residue of the 11.87M is +still vesting at close. This is covered — with on-chain machinery that +**already exists in this runtime**: + +- The `vesting-manager` pallet exposes a **root-gated + `remove_vesting_schedule(who, index)`**. A referendum (or the same + governance track that authorizes the window close) can remove the remaining + schedules, which unlocks the tokens immediately; holders then migrate + normally before the final sweep. +- The same mechanism cleanly handles the **~30,000 PEN in 6 permanent + "never-starts" schedules** (`u32::MAX` start block) that would otherwise be + stranded under *any* finite window — 3, 6, or 12 months alike. These 6 + accounts need the referendum (or case-by-case outreach) regardless of the + window length, so they are not an argument for a longer window. + +And structurally, the close is **operational, not a hard cliff**: the sequence +is pause the pallet → settle in-flight migrations → reconcile → sweep +(runbook RB-7). If adoption or vesting lags, the infrastructure simply runs a +few weeks longer — a 3-month *target* with the option to extend, not a +contract. + +## What the shorter window changes operationally + +1. **`earliestSweepTimestamp` ≈ deploy + 3 months.** It is immutable and marks + the *earliest* allowed sweep — setting it at 3 months preserves the option + to wind down on schedule while never forcing it. +2. **Daily-cap throughput now matters.** Migrating ~150M PEN within ~90 days + needs an *average* release throughput of ~1.7M PEN/day. The PRD's + initial-cap guidance (~1–2% of vault per day = 1.5–3M/day) is compatible, + but the deliberately conservative soft-launch caps must be **raised + promptly via governance** once the launch is verified — cap raises are on + the critical path of a 3-month plan in a way they weren't at 6–12 months. +3. **Comms compress.** Holders' checklist (call `vest()`, unstake ~hours, + remove governance votes, migrate) is quick per holder, but exchanges and + passive holders need the announcement, reminders, and deadline pressure + inside a much shorter arc. The ~23.9M of *already-vested-but-stale* locks + (holders who never called `vest()`) is the strongest evidence that passive + holders exist and need active prodding. +4. **Referendum lead time counts against the window.** A Pendulum referendum + has voting + enactment periods (weeks). If block time hasn't recovered by + ~month 2, *start the unlock referendum then* — don't wait until the window + ends to begin a multi-week governance process. + +## Recommendation (under the 3-month decision) + +- Set `earliestSweepTimestamp ≈ deploy + 3 months`. +- Land the block-time improvement **within the first ~6 weeks** of the window; + track window-average block time against the ~15.6s break-even. +- Pre-draft the vesting-unlock referendum so it can be submitted at ~month 2 + if vesting is projected to overrun — it is also the vehicle for the 30k + sentinel-lock tail either way. +- Verify launch caps quickly and raise `dailyCap` early; ~1.7M PEN/day average + throughput is required arithmetic, not an optimization. +- Size the attestor + monitoring commitment to ~3 months with a soft option to + extend a few weeks. + +## Caveats + +- Live snapshot from a public RPC at one block; re-run against an internal + node at deploy time (the shape is stable, exact figures drift as schedules + progress). +- Block-time projections assume the improvement is a step change to ~12s; a + gradual ramp lands between the table rows. The break-even framing + (window-average ≤ ~15.6s) is the robust way to track it. +- Numbers are decision-grade, not accounting-grade. diff --git a/docs/pen-token-contract-standards.md b/docs/pen-token-contract-standards.md new file mode 100644 index 000000000..e45156a17 --- /dev/null +++ b/docs/pen-token-contract-standards.md @@ -0,0 +1,59 @@ +# PEN ERC-20 on Base: Token Standards and Extensions + +**Status:** Accepted +**Date:** 2026-07-07 +**Companion docs:** [PRD](pen-base-migration-prd.md) (requirement T1), [ADR-001](adr-001-pen-base-migration-approach.md) + +This document records which token standards and contract extensions the PEN ERC-20 on Base implements, which it deliberately omits, and why. The guiding principle (from ADR-001): the token is the *permanent* artifact of the migration and must be maximally trustless and boring — every line of behavior should be audited OpenZeppelin code, and every capability we exclude is an attack surface or credibility question we never have to answer. + +## Baseline + +- **OpenZeppelin Contracts 5.x** (latest audited release at implementation time), plain inheritance — no proxy, no upgradeability. +- Total composition: `ERC20 + ERC20Permit + ERC20Votes`, roughly 20 lines of custom Solidity (constructor + required overrides). +- Constructor mints the entire max issuance to the MigrationVault; there is no other supply-affecting code path. + +## Included extensions + +| Extension | Standard | Function it provides | Why included | +|---|---|---|---| +| `ERC20` | EIP-20 | Core fungible token | — | +| `ERC20Permit` | EIP-2612 | `permit(owner, spender, value, deadline, v, r, s)`: approvals via off-chain EIP-712 signatures instead of an on-chain `approve` transaction. Enables one-transaction approve-and-swap and relayer-paid (gasless) onboarding. | Table stakes for a modern token; zero added trust assumptions; expected by aggregators and wallets. | +| `ERC20Votes` | EIP-5805 | Checkpointed balance history + delegation (`delegate`, `delegateBySig`, `getVotes`, `getPastVotes`, `getPastTotalSupply`). Required by OZ Governor / Tally. | Governance is a hard requirement (hybrid model, ADR-001) and checkpointing **cannot be retrofitted** into an immutable token. Note: votes only count after delegation — holder docs must tell users to self-delegate. | +| EIP-6372 clock | (part of Votes) | `clock()` / `CLOCK_MODE()`: whether checkpoints are keyed by block number (OZ default) or timestamp. | Must be decided **before deployment** and the Governor must be deployed with the same mode. Leaning: **timestamp** on an L2 (human-legible, robust to block-cadence changes). Tracked in PRD T1. | + +### Interplay notes + +- **Smart-contract wallets and `permit`:** EIP-2612 verifies ECDSA signatures only, so Safe/ERC-4337 accounts (which sign via ERC-1271) cannot use `permit`. This is normal and acceptable: those users fall back to `approve` or to **Permit2** (Uniswap's external allowance contract), which provides signature-based, expiring approvals for *any* ERC-20 with no token-side support required. We implement nothing for Permit2; it simply works alongside. + +## Considered and excluded + +| Extension / standard | What it does | Why excluded | +|---|---|---| +| `ERC20Burnable` | Holders burn their own tokens | Breaks the two core invariants: `totalSupply()` would drift below max issuance (tracker requirement, PRD Goal 1) and the monitor's `vault + released = totalSupply` check (PRD M2) would need carve-outs. Vault-remainder disposition at window close is handled by the vault (V9), not the token. | +| `ERC20FlashMint` (EIP-3156) | Flash loans via temporary mint/burn | Directly contradicts the no-mint, fixed-`totalSupply` guarantee. | +| ERC-1363 | `transferAndCall` / `approveAndCall` — receiver contracts react to transfers atomically | Clean, finalized standard but niche adoption; integrators don't expect it; adds surface without demand. Revisit only if a concrete integration needs it (it can't be added later — acceptable, wrappers exist). | +| ERC-3009 | `transferWithAuthorization` — USDC-style gasless *transfers* with random nonces | Payments-oriented; not in OZ core; only worthwhile if PEN becomes a payments rail. Account abstraction covers the UX need without token support. | +| ERC-2771 | Meta-transactions via a trusted forwarder baked into the token | Adds a *permanent* trusted-forwarder assumption to a credibly-neutral asset. Gasless UX is solved wallet-side (ERC-4337/7702) instead. | +| `ERC20Pausable`, blacklist, fee-on-transfer, rebasing | Transfer restrictions / supply games | Break DeFi integrations and tracker math; red flags for listing teams and sophisticated holders. The *vault* is pausable; the token never is. | +| ERC-777-style transfer hooks | Sender/receiver callbacks on transfer | Reentrancy-prone design; the standard is effectively dead. | +| Upgradeability (proxy) | Post-deployment logic changes | The single biggest credibility cost for a fixed-supply token. All operational flexibility lives in the vault (parameters + pause), never in the token. | + +## The one exclusion that forecloses a future capability: ERC-7802 / SuperchainERC20 + +Because Base is an OP-stack chain, the recent **ERC-7802 (SuperchainERC20)** standard deserves an explicit decision rather than a silent default. It enables native (unwrapped) movement of a token between OP Superchain chains — but it works by granting `crosschainMint` / `crosschainBurn` rights to the SuperchainTokenBridge predeploy. That reintroduces exactly the mint authority this design eliminates, and since the token is immutable, **it cannot be added later**. + +**Decision: excluded.** The no-mint guarantee is the more valuable property for an investor-facing fixed-supply token. If Superchain presence ever matters, a wrapped representation can be built on top without touching PEN itself. + +## Resulting contract shape + +```solidity +contract PEN is ERC20, ERC20Permit, ERC20Votes { + constructor(address vault) ERC20("Pendulum", "PEN") ERC20Permit("Pendulum") { + _mint(vault, MAX_ISSUANCE); + } + // + the small required overrides (_update, nonces) and, + // if timestamp mode is chosen, clock() / CLOCK_MODE() +} +``` + +Every behavioral line is OpenZeppelin's audited code — which is precisely the property that makes the token cheap to review (PRD §9) and easy for third parties to verify. From 9cf7d89850e8ecc83454c39c8778204b4a259411 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 24 Aug 2026 19:06:58 +0200 Subject: [PATCH 02/61] pallets: Add the token-migration pallet Burns transferable native PEN and emits a MigrationInitiated event carrying a globally unique nonce and the holder's Base address, which the off-chain attestor set observes to release the equivalent amount from the vault on Base. The pallet has no knowledge of Base state. - migrate(amount, base_address): burns from the caller, rejecting amounts below a configurable minimum, the zero address, balances made non-transferable by staking/vesting/governance locks, and remainders that would strand the account below the existential deposit. - migrate_treasury(amount) and set_treasury_destination(base_address): a governance-gated path for the keyless treasury account, which cannot use the signed extrinsic. The destination is set once and reviewed separately, so the routine call carries no address. - Migrations ship paused and require an explicit governance set_paused(false), so enabling the runtime upgrade and going live are separate acts. - Nonces are globally unique and monotonic across both paths, and TotalMigrated is exposed for the invariant monitor. Includes unit tests and frame-benchmarking v2 benchmarks; weights are conservative manual estimates pending a run on reference hardware. --- Cargo.lock | 17 + Cargo.toml | 1 + pallets/token-migration/Cargo.toml | 51 +++ pallets/token-migration/src/benchmarking.rs | 80 +++++ .../token-migration/src/default_weights.rs | 41 +++ pallets/token-migration/src/lib.rs | 282 +++++++++++++++ pallets/token-migration/src/mock.rs | 139 +++++++ pallets/token-migration/src/tests.rs | 340 ++++++++++++++++++ 8 files changed, 951 insertions(+) create mode 100644 pallets/token-migration/Cargo.toml create mode 100644 pallets/token-migration/src/benchmarking.rs create mode 100644 pallets/token-migration/src/default_weights.rs create mode 100644 pallets/token-migration/src/lib.rs create mode 100644 pallets/token-migration/src/mock.rs create mode 100644 pallets/token-migration/src/tests.rs diff --git a/Cargo.lock b/Cargo.lock index 62d42714d..e531dec37 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -9029,6 +9029,7 @@ dependencies = [ "stellar-relay", "substrate-wasm-builder 5.0.0-dev (git+https://github.com/paritytech//polkadot-sdk?branch=release-polkadot-v1.6.0)", "token-chain-extension", + "token-migration", "treasury-buyout-extension", "vault-registry", "vesting-manager", @@ -15495,6 +15496,22 @@ dependencies = [ "spacewalk-primitives", ] +[[package]] +name = "token-migration" +version = "1.6.0-d" +dependencies = [ + "frame-benchmarking", + "frame-support", + "frame-system", + "pallet-balances", + "parity-scale-codec", + "scale-info", + "sp-core 21.0.0 (git+https://github.com/pendulum-chain/polkadot-sdk?rev=22dd6dee5148a0879306337bd8619c16224cc07b)", + "sp-io 23.0.0 (git+https://github.com/pendulum-chain/polkadot-sdk?rev=22dd6dee5148a0879306337bd8619c16224cc07b)", + "sp-runtime 24.0.0 (git+https://github.com/pendulum-chain/polkadot-sdk?rev=22dd6dee5148a0879306337bd8619c16224cc07b)", + "sp-std 8.0.0 (git+https://github.com/pendulum-chain/polkadot-sdk?rev=22dd6dee5148a0879306337bd8619c16224cc07b)", +] + [[package]] name = "tokio" version = "1.42.0" diff --git a/Cargo.toml b/Cargo.toml index 3557fcf41..7a9731577 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -12,6 +12,7 @@ members = [ "pallets/vesting-manager", "pallets/orml-currencies-allowance-extension", "pallets/orml-tokens-management-extension", + "pallets/token-migration", "pallets/treasury-buyout-extension", "pallets/xcm-teleport", "runtime/common", diff --git a/pallets/token-migration/Cargo.toml b/pallets/token-migration/Cargo.toml new file mode 100644 index 000000000..c4d3436ab --- /dev/null +++ b/pallets/token-migration/Cargo.toml @@ -0,0 +1,51 @@ +[package] +authors = ["Pendulum Chain"] +description = "One-way migration of the native token to Base: burns the migrated amount and emits an event for the attestor set" +edition = "2021" +name = "token-migration" +version = "1.6.0-d" + +[dependencies] +codec = { workspace = true, features = ["derive", "max-encoded-len"] } +scale-info = { workspace = true, features = ["derive"] } + +# Substrate dependencies +frame-support = { workspace = true } +frame-system = { workspace = true } +sp-core = { workspace = true } +sp-runtime = { workspace = true } +sp-std = { workspace = true } + +# benchmarking +frame-benchmarking = { workspace = true, optional = true } + +[dev-dependencies] +sp-io = { workspace = true, default-features = true } +pallet-balances = { workspace = true, default-features = true } + +[features] +default = ["std"] +std = [ + "codec/std", + "scale-info/std", + "frame-support/std", + "frame-system/std", + "sp-core/std", + "sp-runtime/std", + "sp-std/std", + "frame-benchmarking?/std" +] +runtime-benchmarks = [ + "frame-benchmarking", + "frame-benchmarking/runtime-benchmarks", + "frame-support/runtime-benchmarks", + "frame-system/runtime-benchmarks", + "pallet-balances/runtime-benchmarks", + "sp-runtime/runtime-benchmarks" +] +try-runtime = [ + "frame-support/try-runtime", + "frame-system/try-runtime", + "pallet-balances/try-runtime", + "sp-runtime/try-runtime" +] diff --git a/pallets/token-migration/src/benchmarking.rs b/pallets/token-migration/src/benchmarking.rs new file mode 100644 index 000000000..c7e6c5f21 --- /dev/null +++ b/pallets/token-migration/src/benchmarking.rs @@ -0,0 +1,80 @@ +//! Benchmarks for the token-migration pallet. +//! +//! Run against the Pendulum runtime on reference hardware to generate +//! production weights, e.g.: +//! `cargo run --release --features runtime-benchmarks -- benchmark pallet \ +//! --chain pendulum --pallet token-migration --extrinsic '*' --steps 50 --repeat 20` + +#![cfg(feature = "runtime-benchmarks")] + +use super::*; +use frame_benchmarking::v2::*; +use frame_support::traits::{EnsureOrigin, Get}; +use frame_system::RawOrigin; +use sp_runtime::traits::Bounded; + +#[benchmarks] +mod benchmarks { + use super::*; + + #[benchmark] + fn migrate() { + let caller: T::AccountId = whitelisted_caller(); + T::Currency::make_free_balance_be(&caller, BalanceOf::::max_value() / 2u32.into()); + let amount = T::MinimumMigrationAmount::get().saturating_mul(10u32.into()); + let base_address = H160::repeat_byte(0xBE); + // The pallet ships paused; benchmark the live path. + Paused::::put(false); + + #[extrinsic_call] + migrate(RawOrigin::Signed(caller.clone()), amount, base_address); + + assert_eq!(TotalMigrated::::get(), amount); + assert_eq!(NextNonce::::get(), 1); + } + + #[benchmark] + fn set_paused() -> Result<(), BenchmarkError> { + let origin = + T::PauseOrigin::try_successful_origin().map_err(|_| BenchmarkError::Weightless)?; + + #[extrinsic_call] + set_paused(origin as T::RuntimeOrigin, true); + + assert!(Paused::::get()); + Ok(()) + } + + #[benchmark] + fn set_treasury_destination() -> Result<(), BenchmarkError> { + let origin = T::TreasuryMigrateOrigin::try_successful_origin() + .map_err(|_| BenchmarkError::Weightless)?; + let base_address = H160::repeat_byte(0xBE); + + #[extrinsic_call] + set_treasury_destination(origin as T::RuntimeOrigin, base_address); + + assert_eq!(TreasuryDestination::::get(), Some(base_address)); + Ok(()) + } + + #[benchmark] + fn migrate_treasury() -> Result<(), BenchmarkError> { + let treasury = T::TreasuryAccount::get(); + T::Currency::make_free_balance_be(&treasury, BalanceOf::::max_value() / 2u32.into()); + let amount = T::MinimumMigrationAmount::get().saturating_mul(10u32.into()); + TreasuryDestination::::put(H160::repeat_byte(0xBE)); + Paused::::put(false); + let origin = T::TreasuryMigrateOrigin::try_successful_origin() + .map_err(|_| BenchmarkError::Weightless)?; + + #[extrinsic_call] + migrate_treasury(origin as T::RuntimeOrigin, amount); + + assert_eq!(TotalMigrated::::get(), amount); + assert_eq!(NextNonce::::get(), 1); + Ok(()) + } + + impl_benchmark_test_suite!(Pallet, crate::mock::ExtBuilder::build(), crate::mock::Test); +} diff --git a/pallets/token-migration/src/default_weights.rs b/pallets/token-migration/src/default_weights.rs new file mode 100644 index 000000000..eb7b7bd6a --- /dev/null +++ b/pallets/token-migration/src/default_weights.rs @@ -0,0 +1,41 @@ +//! Default weights for the token-migration pallet. +//! +//! TODO: replace with generated weights once benchmarks are added; these are +//! conservative manual estimates in the meantime (same approach as the other +//! pallets in this repo). + +use core::marker::PhantomData; +use frame_support::{traits::Get, weights::Weight}; + +pub trait WeightInfo { + fn migrate() -> Weight; + fn set_paused() -> Weight; + fn set_treasury_destination() -> Weight; + fn migrate_treasury() -> Weight; +} + +pub struct SubstrateWeight(PhantomData); + +impl WeightInfo for SubstrateWeight { + fn migrate() -> Weight { + Weight::from_parts(50_000_000, 0) + .saturating_add(T::DbWeight::get().reads(4_u64)) + .saturating_add(T::DbWeight::get().writes(4_u64)) + } + + fn set_paused() -> Weight { + Weight::from_parts(10_000_000, 0) + .saturating_add(T::DbWeight::get().writes(1_u64)) + } + + fn set_treasury_destination() -> Weight { + Weight::from_parts(12_000_000, 0) + .saturating_add(T::DbWeight::get().writes(1_u64)) + } + + fn migrate_treasury() -> Weight { + Weight::from_parts(50_000_000, 0) + .saturating_add(T::DbWeight::get().reads(5_u64)) + .saturating_add(T::DbWeight::get().writes(4_u64)) + } +} diff --git a/pallets/token-migration/src/lib.rs b/pallets/token-migration/src/lib.rs new file mode 100644 index 000000000..56c9ff354 --- /dev/null +++ b/pallets/token-migration/src/lib.rs @@ -0,0 +1,282 @@ +//! # Token Migration Pallet +//! +//! One-way migration of the native token (PEN) to Base. +//! +//! Holders call [`Pallet::migrate`] with an amount and the Base (EVM) address that +//! should receive the tokens. The amount is burned (total issuance decreases) and a +//! [`Event::MigrationInitiated`] event is emitted carrying a globally unique nonce. +//! An off-chain attestor set observes these events in finalized blocks and approves +//! the corresponding release from the MigrationVault contract on Base. +//! +//! The pallet is fire-and-forget by design: it has no knowledge of Base state and +//! migrations are irreversible. See docs/pen-base-migration-prd.md (§6.1) for the +//! full requirements this pallet implements. + +#![cfg_attr(not(feature = "std"), no_std)] + +pub use pallet::*; + +pub mod default_weights; + +#[cfg(feature = "runtime-benchmarks")] +pub mod benchmarking; + +#[cfg(test)] +mod mock; +#[cfg(test)] +mod tests; + +pub use default_weights::WeightInfo; + +use frame_support::traits::{Currency, ExistenceRequirement, WithdrawReasons}; +use sp_core::H160; +use sp_runtime::{ + traits::{CheckedSub, Saturating, Zero}, + ArithmeticError, +}; + +pub(crate) type BalanceOf = + <::Currency as Currency<::AccountId>>::Balance; + +#[frame_support::pallet] +pub mod pallet { + use super::*; + use frame_support::pallet_prelude::*; + use frame_system::pallet_prelude::*; + + #[pallet::pallet] + pub struct Pallet(_); + + #[pallet::config] + pub trait Config: frame_system::Config { + type RuntimeEvent: From> + IsType<::RuntimeEvent>; + + /// The native currency. Migrated amounts are withdrawn and burned, + /// reducing total issuance (PRD decision D1: burn). + type Currency: Currency; + + /// Smallest amount accepted by `migrate`, to keep dust-sized migrations + /// from spamming the attestor pipeline. + #[pallet::constant] + type MinimumMigrationAmount: Get>; + + /// Origin allowed to pause and unpause migrations (incident response). + type PauseOrigin: EnsureOrigin; + + /// The keyless treasury account whose funds `migrate_treasury` moves to + /// Base. Set in the runtime to the treasury pallet account. + type TreasuryAccount: Get; + + /// Origin allowed to set the treasury's Base destination and trigger a + /// treasury migration (root or a council majority in the runtime). + type TreasuryMigrateOrigin: EnsureOrigin; + + type WeightInfo: WeightInfo; + } + + #[pallet::event] + #[pallet::generate_deposit(pub(super) fn deposit_event)] + pub enum Event { + /// `amount` of the native token was burned for migration to Base. + /// The attestor set releases the equivalent amount to `base_address` + /// on Base, keyed by the globally unique `nonce`. + MigrationInitiated { + nonce: u64, + who: T::AccountId, + base_address: H160, + amount: BalanceOf, + }, + /// Migrations were paused or unpaused by the pause origin. + MigrationPauseSet { paused: bool }, + /// The fixed Base destination for treasury migrations was set. + TreasuryDestinationSet { base_address: H160 }, + } + + #[pallet::error] + pub enum Error { + /// Migrations are currently paused. + MigrationsPaused, + /// The amount is below the configured minimum migration amount. + AmountBelowMinimum, + /// The caller's free balance is lower than the requested amount. + InsufficientBalance, + /// The migration would leave a remainder below the existential deposit. + /// Migrate the entire balance or leave at least the existential deposit. + WouldLeaveDust, + /// The Base address is structurally invalid (e.g. the zero address). + /// The vault on Base would reject the release, permanently stranding + /// the burned tokens and stalling the attestor pipeline. + InvalidBaseAddress, + /// A treasury migration was attempted before the Base destination was + /// set via `set_treasury_destination`. + NoTreasuryDestination, + } + + /// Nonce of the next migration. Monotonically increasing, never reused; + /// each emitted `MigrationInitiated` event consumes one value. + #[pallet::storage] + pub type NextNonce = StorageValue<_, u64, ValueQuery>; + + /// Cumulative amount burned for migration, for the invariant monitor + /// (PRD M2: vault balance on Base + total released == max issuance). + #[pallet::storage] + pub type TotalMigrated = StorageValue<_, BalanceOf, ValueQuery>; + + /// Migrations ship **paused**. A runtime upgrade that adds this pallet + /// writes no storage, so the pallet reads as paused until governance + /// explicitly enables it with `set_paused(false)`. + /// + /// This is deliberately fail-safe rather than a one-shot storage + /// migration: the runtime upgrade that enables `migrate` and the decision + /// to go live are separate acts. If `migrate` were live on enactment, a + /// referendum enacting before the Base vault and attestor set are + /// operational would let holders burn PEN with nothing able to release it. + /// Making it a property of the storage default means it cannot be + /// defeated by forgetting to wire a migration into `Executive`, and it + /// re-arms if the value is ever cleared. + #[pallet::type_value] + pub fn DefaultPaused() -> bool { + true + } + + /// Whether migrations are paused. Defaults to `true` — see [`DefaultPaused`]. + #[pallet::storage] + pub type Paused = StorageValue<_, bool, ValueQuery, DefaultPaused>; + + /// The fixed Base destination for treasury migrations. `migrate_treasury` + /// always sends here; `None` until set by `set_treasury_destination`. + #[pallet::storage] + pub type TreasuryDestination = StorageValue<_, H160, OptionQuery>; + + #[pallet::call] + impl Pallet { + /// Burn `amount` of the caller's native tokens for migration to Base. + /// + /// The tokens are released to `base_address` on Base by the attestor set. + /// This action is IRREVERSIBLE: a wrong `base_address` means the tokens + /// are lost. The caller must either migrate their entire free balance or + /// leave at least the existential deposit behind. + #[pallet::call_index(0)] + #[pallet::weight(::WeightInfo::migrate())] + pub fn migrate( + origin: OriginFor, + #[pallet::compact] amount: BalanceOf, + base_address: H160, + ) -> DispatchResult { + let who = ensure_signed(origin)?; + ensure!(!Paused::::get(), Error::::MigrationsPaused); + ensure!( + amount >= T::MinimumMigrationAmount::get(), + Error::::AmountBelowMinimum + ); + // The vault contract rejects the zero address; burning towards it + // would emit an event no attestor can ever execute. + ensure!(base_address != H160::zero(), Error::::InvalidBaseAddress); + + let free = T::Currency::free_balance(&who); + let remainder = free.checked_sub(&amount).ok_or(Error::::InsufficientBalance)?; + ensure!( + remainder.is_zero() || remainder >= T::Currency::minimum_balance(), + Error::::WouldLeaveDust + ); + + // Fails if locks or reserves make `amount` non-transferable (staking, + // vesting and governance locks must be cleared before migrating). + let imbalance = T::Currency::withdraw( + &who, + amount, + WithdrawReasons::TRANSFER, + ExistenceRequirement::AllowDeath, + )?; + // Dropping the negative imbalance without offsetting it burns the + // withdrawn amount, i.e. total issuance decreases by `amount`. + drop(imbalance); + + // Shared accounting + event (identical to a treasury migration, so + // the attestor set decodes both the same way). + Self::emit_migration(who, base_address, amount) + } + + /// Pause or unpause migrations. Callable by the pause origin only. + #[pallet::call_index(1)] + #[pallet::weight(::WeightInfo::set_paused())] + pub fn set_paused(origin: OriginFor, paused: bool) -> DispatchResult { + T::PauseOrigin::ensure_origin(origin)?; + Paused::::put(paused); + Self::deposit_event(Event::MigrationPauseSet { paused }); + Ok(()) + } + + /// Set the fixed Base destination for treasury migrations. + /// + /// Callable by the treasury-migrate origin (root or a council majority). + /// This is the single security anchor for treasury migrations: once set, + /// `migrate_treasury` always sends here, so the routine call carries no + /// address and cannot be sent to the wrong place by a typo. + #[pallet::call_index(2)] + #[pallet::weight(::WeightInfo::set_treasury_destination())] + pub fn set_treasury_destination(origin: OriginFor, base_address: H160) -> DispatchResult { + T::TreasuryMigrateOrigin::ensure_origin(origin)?; + ensure!(base_address != H160::zero(), Error::::InvalidBaseAddress); + TreasuryDestination::::put(base_address); + Self::deposit_event(Event::TreasuryDestinationSet { base_address }); + Ok(()) + } + + /// Burn `amount` of the treasury's native tokens for migration to the + /// pre-set Base destination. + /// + /// Callable by the treasury-migrate origin. Requires a destination to + /// have been set. Emits the same `MigrationInitiated` event as a user + /// migration (with `who` = the treasury account), so the attestor set, + /// vault and monitor process it identically. + #[pallet::call_index(3)] + #[pallet::weight(::WeightInfo::migrate_treasury())] + pub fn migrate_treasury( + origin: OriginFor, + #[pallet::compact] amount: BalanceOf, + ) -> DispatchResult { + T::TreasuryMigrateOrigin::ensure_origin(origin)?; + ensure!(!Paused::::get(), Error::::MigrationsPaused); + ensure!( + amount >= T::MinimumMigrationAmount::get(), + Error::::AmountBelowMinimum + ); + let base_address = + TreasuryDestination::::get().ok_or(Error::::NoTreasuryDestination)?; + + let treasury = T::TreasuryAccount::get(); + ensure!( + T::Currency::free_balance(&treasury) >= amount, + Error::::InsufficientBalance + ); + + // KeepAlive: the treasury is a persistent system account and must + // never be reaped by a migration. + let imbalance = T::Currency::withdraw( + &treasury, + amount, + WithdrawReasons::TRANSFER, + ExistenceRequirement::KeepAlive, + )?; + drop(imbalance); + + Self::emit_migration(treasury, base_address, amount) + } + } + + impl Pallet { + /// Shared tail of every migration: consume a unique nonce, update the + /// cumulative total, and emit `MigrationInitiated`. The caller must have + /// already burned `amount` from `who`. + fn emit_migration(who: T::AccountId, base_address: H160, amount: BalanceOf) -> DispatchResult { + let nonce = NextNonce::::get(); + let next = nonce.checked_add(1).ok_or(ArithmeticError::Overflow)?; + NextNonce::::put(next); + TotalMigrated::::mutate(|total| *total = total.saturating_add(amount)); + + Self::deposit_event(Event::MigrationInitiated { nonce, who, base_address, amount }); + Ok(()) + } + } +} diff --git a/pallets/token-migration/src/mock.rs b/pallets/token-migration/src/mock.rs new file mode 100644 index 000000000..0793d0233 --- /dev/null +++ b/pallets/token-migration/src/mock.rs @@ -0,0 +1,139 @@ +use crate::{self as token_migration, default_weights::SubstrateWeight, Config}; +use frame_support::{ + parameter_types, + traits::{ConstU32, Everything}, +}; +use frame_system::EnsureRoot; +use sp_core::H256; +use sp_runtime::{ + traits::{BlakeTwo256, IdentityLookup}, + BuildStorage, +}; + +type Block = frame_system::mocking::MockBlock; + +pub const UNIT: Balance = 1_000_000_000_000; + +frame_support::construct_runtime!( + pub enum Test + { + System: frame_system, + Balances: pallet_balances, + TokenMigration: token_migration, + } +); + +pub type AccountId = u64; +pub type Balance = u128; +pub type Nonce = u64; + +parameter_types! { + pub const BlockHashCount: u64 = 250; + pub const SS58Prefix: u8 = 42; +} + +impl frame_system::Config for Test { + type Block = Block; + type BaseCallFilter = Everything; + type BlockWeights = (); + type BlockLength = (); + type DbWeight = (); + type RuntimeOrigin = RuntimeOrigin; + type RuntimeCall = RuntimeCall; + type Nonce = Nonce; + type Hash = H256; + type Hashing = BlakeTwo256; + type AccountId = AccountId; + type Lookup = IdentityLookup; + type RuntimeEvent = RuntimeEvent; + type BlockHashCount = BlockHashCount; + type Version = (); + type PalletInfo = PalletInfo; + type AccountData = pallet_balances::AccountData; + type OnNewAccount = (); + type OnKilledAccount = (); + type SystemWeightInfo = (); + type SS58Prefix = SS58Prefix; + type OnSetCode = (); + type MaxConsumers = frame_support::traits::ConstU32<16>; + type RuntimeTask = RuntimeTask; +} + +parameter_types! { + pub const MaxLocks: u32 = 50; + pub const ExistentialDeposit: Balance = 1000; + pub const MaxReserves: u32 = 50; +} + +impl pallet_balances::Config for Test { + type MaxLocks = MaxLocks; + type Balance = Balance; + type RuntimeEvent = RuntimeEvent; + type DustRemoval = (); + type ExistentialDeposit = ExistentialDeposit; + type AccountStore = System; + type WeightInfo = pallet_balances::weights::SubstrateWeight; + type MaxReserves = MaxReserves; + type ReserveIdentifier = (); + type FreezeIdentifier = (); + type MaxFreezes = (); + type MaxHolds = ConstU32<1>; + type RuntimeHoldReason = RuntimeHoldReason; + type RuntimeFreezeReason = RuntimeFreezeReason; +} + +parameter_types! { + pub const MinimumMigrationAmount: Balance = UNIT; + pub const TreasuryAccount: AccountId = 999; +} + +impl Config for Test { + type RuntimeEvent = RuntimeEvent; + type Currency = Balances; + type MinimumMigrationAmount = MinimumMigrationAmount; + type PauseOrigin = EnsureRoot; + type TreasuryAccount = TreasuryAccount; + type TreasuryMigrateOrigin = EnsureRoot; + type WeightInfo = SubstrateWeight; +} + +// ------- Constants and Genesis Config ------ // + +pub const USER: AccountId = 1; +pub const USER_INITIAL_BALANCE: Balance = 100 * UNIT; +pub const TREASURY: AccountId = 999; +pub const TREASURY_INITIAL_BALANCE: Balance = 1000 * UNIT; + +pub struct ExtBuilder; + +impl ExtBuilder { + pub fn build() -> sp_io::TestExternalities { + let mut storage = frame_system::GenesisConfig::::default().build_storage().unwrap(); + + pallet_balances::GenesisConfig:: { + balances: vec![ + (USER, USER_INITIAL_BALANCE), + (TREASURY, TREASURY_INITIAL_BALANCE), + ], + } + .assimilate_storage(&mut storage) + .unwrap(); + + sp_io::TestExternalities::from(storage) + } +} + +pub fn run_test(test: T) +where + T: FnOnce(), +{ + ExtBuilder::build().execute_with(|| { + System::set_block_number(1); + // The pallet ships paused; governance unpauses once the Base side is + // live. Mirror that here so tests exercise the normal running state. + // `migrations_ship_paused_until_governance_enables_them` covers the + // default itself, without this helper. + crate::Paused::::put(false); + test(); + }); +} diff --git a/pallets/token-migration/src/tests.rs b/pallets/token-migration/src/tests.rs new file mode 100644 index 000000000..f9a5f74ef --- /dev/null +++ b/pallets/token-migration/src/tests.rs @@ -0,0 +1,340 @@ +use crate::{mock::*, Error, Event, NextNonce, Paused, TotalMigrated, TreasuryDestination}; +use frame_support::{ + assert_noop, assert_ok, + traits::{LockableCurrency, WithdrawReasons}, +}; +use sp_core::H160; +use sp_runtime::DispatchError; + +fn base_address() -> H160 { + H160::from_low_u64_be(0xBEEF) +} + +#[test] +fn migrate_burns_amount_and_emits_event() { + run_test(|| { + let amount = 10 * UNIT; + let issuance_before = Balances::total_issuance(); + + assert_ok!(TokenMigration::migrate( + RuntimeOrigin::signed(USER), + amount, + base_address() + )); + + // The amount is burned, not moved: issuance shrinks by exactly `amount`. + assert_eq!(Balances::total_issuance(), issuance_before - amount); + assert_eq!(Balances::free_balance(USER), USER_INITIAL_BALANCE - amount); + assert_eq!(TotalMigrated::::get(), amount); + + System::assert_last_event( + Event::MigrationInitiated { + nonce: 0, + who: USER, + base_address: base_address(), + amount, + } + .into(), + ); + }); +} + +#[test] +fn nonces_are_unique_and_monotonic() { + run_test(|| { + for expected_nonce in 0u64..3 { + assert_eq!(NextNonce::::get(), expected_nonce); + assert_ok!(TokenMigration::migrate( + RuntimeOrigin::signed(USER), + UNIT, + base_address() + )); + System::assert_last_event( + Event::MigrationInitiated { + nonce: expected_nonce, + who: USER, + base_address: base_address(), + amount: UNIT, + } + .into(), + ); + } + assert_eq!(NextNonce::::get(), 3); + assert_eq!(TotalMigrated::::get(), 3 * UNIT); + }); +} + +#[test] +fn migrate_fails_below_minimum_amount() { + run_test(|| { + assert_noop!( + TokenMigration::migrate(RuntimeOrigin::signed(USER), UNIT - 1, base_address()), + Error::::AmountBelowMinimum + ); + }); +} + +#[test] +fn migrate_fails_for_zero_base_address() { + run_test(|| { + assert_noop!( + TokenMigration::migrate(RuntimeOrigin::signed(USER), UNIT, H160::zero()), + Error::::InvalidBaseAddress + ); + }); +} + +#[test] +fn migrate_fails_with_insufficient_balance() { + run_test(|| { + assert_noop!( + TokenMigration::migrate( + RuntimeOrigin::signed(USER), + USER_INITIAL_BALANCE + 1, + base_address() + ), + Error::::InsufficientBalance + ); + }); +} + +#[test] +fn migrate_fails_if_dust_would_remain() { + run_test(|| { + // Leaves a remainder of ED - 1, which the balances pallet would reap as dust. + let ed = ExistentialDeposit::get(); + assert_noop!( + TokenMigration::migrate( + RuntimeOrigin::signed(USER), + USER_INITIAL_BALANCE - (ed - 1), + base_address() + ), + Error::::WouldLeaveDust + ); + }); +} + +#[test] +fn migrate_entire_balance_works() { + run_test(|| { + assert_ok!(TokenMigration::migrate( + RuntimeOrigin::signed(USER), + USER_INITIAL_BALANCE, + base_address() + )); + assert_eq!(Balances::free_balance(USER), 0); + // Only the treasury's balance remains in issuance after the user's is burned. + assert_eq!(Balances::total_issuance(), TREASURY_INITIAL_BALANCE); + assert_eq!(TotalMigrated::::get(), USER_INITIAL_BALANCE); + }); +} + +#[test] +fn migrate_fails_for_locked_funds() { + run_test(|| { + // Lock all but 5 UNIT; migrating more than the usable balance must fail. + Balances::set_lock( + *b"lock1234", + &USER, + USER_INITIAL_BALANCE - 5 * UNIT, + WithdrawReasons::all(), + ); + assert_noop!( + TokenMigration::migrate(RuntimeOrigin::signed(USER), 10 * UNIT, base_address()), + pallet_balances::Error::::LiquidityRestrictions + ); + // Migrating within the usable balance still works. + assert_ok!(TokenMigration::migrate( + RuntimeOrigin::signed(USER), + 5 * UNIT, + base_address() + )); + }); +} + +#[test] +fn pause_blocks_migrations_and_unpause_restores_them() { + run_test(|| { + assert_ok!(TokenMigration::set_paused(RuntimeOrigin::root(), true)); + assert!(Paused::::get()); + System::assert_last_event(Event::MigrationPauseSet { paused: true }.into()); + + assert_noop!( + TokenMigration::migrate(RuntimeOrigin::signed(USER), UNIT, base_address()), + Error::::MigrationsPaused + ); + + assert_ok!(TokenMigration::set_paused(RuntimeOrigin::root(), false)); + assert_ok!(TokenMigration::migrate( + RuntimeOrigin::signed(USER), + UNIT, + base_address() + )); + }); +} + +// Fail-safe launch property: a runtime upgrade that adds this pallet must not +// enable migrations. Holders could otherwise burn PEN before the Base vault and +// attestor set are live, with nothing able to release it. +#[test] +fn migrations_ship_paused_until_governance_enables_them() { + ExtBuilder::build().execute_with(|| { + System::set_block_number(1); + + // No storage written yet — exactly the state after a runtime upgrade. + assert!(Paused::::get(), "the pallet must ship paused"); + assert_noop!( + TokenMigration::migrate(RuntimeOrigin::signed(USER), UNIT, base_address()), + Error::::MigrationsPaused + ); + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); + assert_noop!( + TokenMigration::migrate_treasury(RuntimeOrigin::root(), UNIT), + Error::::MigrationsPaused + ); + + // Going live is an explicit, separate governance act. + assert_ok!(TokenMigration::set_paused(RuntimeOrigin::root(), false)); + assert_ok!(TokenMigration::migrate(RuntimeOrigin::signed(USER), UNIT, base_address())); + }); +} + +#[test] +fn set_paused_requires_pause_origin() { + run_test(|| { + assert_noop!( + TokenMigration::set_paused(RuntimeOrigin::signed(USER), true), + DispatchError::BadOrigin + ); + }); +} + +// ---------------------------------------------------------------- treasury migration + +#[test] +fn set_treasury_destination_stores_and_emits() { + run_test(|| { + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); + assert_eq!(TreasuryDestination::::get(), Some(base_address())); + System::assert_last_event(Event::TreasuryDestinationSet { base_address: base_address() }.into()); + }); +} + +#[test] +fn set_treasury_destination_rejects_zero_and_bad_origin() { + run_test(|| { + assert_noop!( + TokenMigration::set_treasury_destination(RuntimeOrigin::root(), H160::zero()), + Error::::InvalidBaseAddress + ); + assert_noop!( + TokenMigration::set_treasury_destination(RuntimeOrigin::signed(USER), base_address()), + DispatchError::BadOrigin + ); + }); +} + +#[test] +fn migrate_treasury_burns_from_treasury_and_emits_same_event_shape() { + run_test(|| { + let amount = 10 * UNIT; + let issuance_before = Balances::total_issuance(); + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); + + assert_ok!(TokenMigration::migrate_treasury(RuntimeOrigin::root(), amount)); + + // Burned from the treasury account, issuance down by exactly `amount`. + assert_eq!(Balances::total_issuance(), issuance_before - amount); + assert_eq!(Balances::free_balance(TREASURY), TREASURY_INITIAL_BALANCE - amount); + assert_eq!(Balances::free_balance(USER), USER_INITIAL_BALANCE); + assert_eq!(TotalMigrated::::get(), amount); + assert_eq!(NextNonce::::get(), 1); + + // Same event shape as a user migration, with who = the treasury account. + System::assert_last_event( + Event::MigrationInitiated { nonce: 0, who: TREASURY, base_address: base_address(), amount }.into(), + ); + }); +} + +#[test] +fn treasury_and_user_migrations_share_the_nonce_space() { + run_test(|| { + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); + assert_ok!(TokenMigration::migrate(RuntimeOrigin::signed(USER), UNIT, base_address())); + assert_ok!(TokenMigration::migrate_treasury(RuntimeOrigin::root(), UNIT)); + assert_ok!(TokenMigration::migrate(RuntimeOrigin::signed(USER), UNIT, base_address())); + + // Nonces are globally unique across both paths. + assert_eq!(NextNonce::::get(), 3); + assert_eq!(TotalMigrated::::get(), 3 * UNIT); + }); +} + +#[test] +fn migrate_treasury_fails_without_destination() { + run_test(|| { + assert_noop!( + TokenMigration::migrate_treasury(RuntimeOrigin::root(), UNIT), + Error::::NoTreasuryDestination + ); + }); +} + +#[test] +fn migrate_treasury_requires_authorized_origin() { + run_test(|| { + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); + assert_noop!( + TokenMigration::migrate_treasury(RuntimeOrigin::signed(USER), UNIT), + DispatchError::BadOrigin + ); + }); +} + +#[test] +fn migrate_treasury_validates_amount() { + run_test(|| { + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); + // Below the minimum. + assert_noop!( + TokenMigration::migrate_treasury(RuntimeOrigin::root(), UNIT - 1), + Error::::AmountBelowMinimum + ); + // More than the treasury holds. + assert_noop!( + TokenMigration::migrate_treasury(RuntimeOrigin::root(), TREASURY_INITIAL_BALANCE + 1), + Error::::InsufficientBalance + ); + }); +} + +#[test] +fn migrate_treasury_keeps_treasury_alive() { + run_test(|| { + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); + // Draining the whole balance would reap the account; KeepAlive rejects it. + assert_noop!( + TokenMigration::migrate_treasury(RuntimeOrigin::root(), TREASURY_INITIAL_BALANCE), + pallet_balances::Error::::Expendability + ); + // Leaving at least the existential deposit works. + let keep = TREASURY_INITIAL_BALANCE - ExistentialDeposit::get(); + assert_ok!(TokenMigration::migrate_treasury(RuntimeOrigin::root(), keep)); + assert_eq!(Balances::free_balance(TREASURY), ExistentialDeposit::get()); + }); +} + +#[test] +fn migrate_treasury_respects_pause() { + run_test(|| { + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); + assert_ok!(TokenMigration::set_paused(RuntimeOrigin::root(), true)); + assert_noop!( + TokenMigration::migrate_treasury(RuntimeOrigin::root(), UNIT), + Error::::MigrationsPaused + ); + // Setting the destination is still allowed while paused (configuration). + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); + }); +} From e1f168a07ea6f205aa7829b96df66ea6c3ae8d71 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 24 Aug 2026 19:06:58 +0200 Subject: [PATCH 03/61] runtime: Wire the token-migration pallet into Pendulum Registers the pallet at index 102 with a 1 PEN minimum migration amount, the treasury account and treasury-migration origin bound to the existing treasury approval authority (root or 3/5 council), and a pause origin of root, half the council, or two thirds of the technical committee for fast incident response. Adds the pallet to the runtime's exhaustive BaseFilter call whitelist, without which every migration call would be silently rejected, and to the benchmark list. --- runtime/pendulum/Cargo.toml | 4 ++++ runtime/pendulum/src/lib.rs | 29 +++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/runtime/pendulum/Cargo.toml b/runtime/pendulum/Cargo.toml index 4657b3d8e..cb52dab7e 100644 --- a/runtime/pendulum/Cargo.toml +++ b/runtime/pendulum/Cargo.toml @@ -121,6 +121,7 @@ dia-oracle-runtime-api = { workspace = true } # Pendulum Pallets vesting-manager = { path = "../../pallets/vesting-manager", default-features = false } pallet-xcm-teleport = { path = "../../pallets/xcm-teleport", default-features = false } +token-migration = { path = "../../pallets/token-migration", default-features = false } # Polkadot pallet-xcm = { workspace = true } @@ -257,6 +258,7 @@ std = [ "parachain-staking/std", "vesting-manager/std", "pallet-xcm-teleport/std", + "token-migration/std", "price-chain-extension/std", "token-chain-extension/std", "treasury-buyout-extension/std", @@ -333,6 +335,7 @@ runtime-benchmarks = [ "staking/runtime-benchmarks", "vesting-manager/runtime-benchmarks", "pallet-xcm-teleport/runtime-benchmarks", + "token-migration/runtime-benchmarks", ] try-runtime = [ @@ -390,6 +393,7 @@ try-runtime = [ "orml-currencies-allowance-extension/try-runtime", "vesting-manager/try-runtime", "pallet-xcm-teleport/try-runtime", + "token-migration/try-runtime", "bifrost-farming/try-runtime", "zenlink-protocol/try-runtime", "treasury-buyout-extension/try-runtime", diff --git a/runtime/pendulum/src/lib.rs b/runtime/pendulum/src/lib.rs index cc11b442a..725577577 100644 --- a/runtime/pendulum/src/lib.rs +++ b/runtime/pendulum/src/lib.rs @@ -372,6 +372,7 @@ impl Contains for BaseFilter { | RuntimeCall::CumulusXcm(_) | RuntimeCall::VaultStaking(_) | RuntimeCall::XcmTeleport(_) + | RuntimeCall::TokenMigration(_) | RuntimeCall::MessageQueue(_) => true, // All pallets are allowed, but exhaustive match is defensive // in the case of adding new pallets. } @@ -1116,6 +1117,31 @@ impl pallet_xcm_teleport::Config for Runtime { type TreasuryAccount = PendulumTreasuryAccount; } +parameter_types! { + // 100 PEN (~$0.86 at $0.00858/PEN). The minimum must exceed the marginal + // Base-gas cost the five-attestor fleet spends per migration (~3 `approve` + // txs, roughly $0.01–$1 depending on Base gas), or spamming dust migrations + // becomes a cheap asymmetric gas-drain grief on every operator. 100 PEN + // dominates that cost across normal Base conditions while staying negligible + // for any real holder. Tunable via runtime upgrade. + pub const MinimumMigrationAmount: Balance = 100 * UNIT; +} + +impl token_migration::Config for Runtime { + type RuntimeEvent = RuntimeEvent; + type Currency = Balances; + type MinimumMigrationAmount = MinimumMigrationAmount; + // Root/half-council, or 2/3 of the technical committee for fast incident response. + type PauseOrigin = EitherOfDiverse< + EnsureRootOrHalfCouncil, + pallet_collective::EnsureProportionAtLeast, + >; + type TreasuryAccount = PendulumTreasuryAccount; + // Same authority that approves treasury spends: root or 3/5 council. + type TreasuryMigrateOrigin = TreasuryApproveOrigin; + type WeightInfo = token_migration::default_weights::SubstrateWeight; +} + const fn deposit(items: u32, bytes: u32) -> Balance { (items as Balance * UNIT + (bytes as Balance) * (5 * MILLIUNIT / 100)) / 10 } @@ -1692,6 +1718,8 @@ construct_runtime!( XcmTeleport: pallet_xcm_teleport = 101, + TokenMigration: token_migration = 102, + MessageQueue: pallet_message_queue = 110, } ); @@ -1725,6 +1753,7 @@ mod benches { [orml_currencies_allowance_extension, TokenAllowance] [treasury_buyout_extension, TreasuryBuyoutExtension] + [token_migration, TokenMigration] [dia_oracle, DiaOracleModule] ); From 2aec0e858f16783d9b92c0f74166f3c6f14da3f9 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 24 Aug 2026 19:07:29 +0200 Subject: [PATCH 04/61] contracts: Add the PEN token, migration vault and governance for Base Foundry project (OpenZeppelin v5.4.0) holding the Base side of the migration. PEN.sol: fixed-supply ERC20 + ERC20Permit + ERC20Votes on an EIP-6372 timestamp clock. The entire max issuance is minted to the vault in the constructor; there is no mint function, no owner and no proxy, so totalSupply is correct for trackers from day one and can never grow. MigrationVault.sol: holds the unmigrated supply and releases it on the threshold-th matching on-chain approval from the attestor set, counted per exact (nonce, recipient, amount) tuple so conflicting tuples never merge. Nonce consumption is permanent, the 12 to 18 decimal conversion happens here and nowhere else, and attestor generations ensure a release threshold can only ever be crossed inside approve(). Releases are bounded by a per-release cap and a rolling 24-hour leaky bucket; when a cap, a pause or an under-funded vault blocks a release it is recorded as pending rather than reverted, so the debt stays tracked and the fleet cannot deadlock. Pending amounts are reserved against the timelocked end-of-window sweep. A guardian can pause instantly but only the admin can unpause, so a compromised guardian can halt but never release. PENGovernor.sol plus deployment scripts for the vault, token and the Governor/Timelock handover. 37 tests including fuzz and a full propose-vote-queue-execute lifecycle. --- .gitmodules | 6 + contracts/.env.example | 30 ++ contracts/.gitignore | 3 + contracts/README.md | 49 +++ contracts/foundry.lock | 14 + contracts/foundry.toml | 12 + contracts/lib/forge-std | 1 + contracts/lib/openzeppelin-contracts | 1 + contracts/script/Deploy.s.sol | 65 +++ contracts/script/DeployGovernance.s.sol | 59 +++ contracts/src/MigrationVault.sol | 460 ++++++++++++++++++++ contracts/src/PEN.sol | 53 +++ contracts/src/PENGovernor.sol | 112 +++++ contracts/test/MigrationVault.t.sol | 533 ++++++++++++++++++++++++ contracts/test/PEN.t.sol | 76 ++++ contracts/test/PENGovernor.t.sol | 107 +++++ 16 files changed, 1581 insertions(+) create mode 100644 .gitmodules create mode 100644 contracts/.env.example create mode 100644 contracts/.gitignore create mode 100644 contracts/README.md create mode 100644 contracts/foundry.lock create mode 100644 contracts/foundry.toml create mode 160000 contracts/lib/forge-std create mode 160000 contracts/lib/openzeppelin-contracts create mode 100644 contracts/script/Deploy.s.sol create mode 100644 contracts/script/DeployGovernance.s.sol create mode 100644 contracts/src/MigrationVault.sol create mode 100644 contracts/src/PEN.sol create mode 100644 contracts/src/PENGovernor.sol create mode 100644 contracts/test/MigrationVault.t.sol create mode 100644 contracts/test/PEN.t.sol create mode 100644 contracts/test/PENGovernor.t.sol diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 000000000..460a2ec0d --- /dev/null +++ b/.gitmodules @@ -0,0 +1,6 @@ +[submodule "contracts/lib/openzeppelin-contracts"] + path = contracts/lib/openzeppelin-contracts + url = https://github.com/OpenZeppelin/openzeppelin-contracts +[submodule "contracts/lib/forge-std"] + path = contracts/lib/forge-std + url = https://github.com/foundry-rs/forge-std diff --git a/contracts/.env.example b/contracts/.env.example new file mode 100644 index 000000000..69426e3d4 --- /dev/null +++ b/contracts/.env.example @@ -0,0 +1,30 @@ +# --- Deploy.s.sol (migration stack, PRD rollout phase 3) --- +# forge script script/Deploy.s.sol --rpc-url $BASE_RPC_URL --broadcast --verify + +# Bootstrap Safe that becomes vault admin (accepts via vault.acceptAdmin()) +ADMIN_SAFE= +# Fast pause guardian (small-threshold Safe) +GUARDIAN_SAFE= +# The four attestor transaction-sender addresses (decision D4: 3-of-4, +# initially team-operated, one per internal RPC node) +ATTESTOR_1= +ATTESTOR_2= +ATTESTOR_3= +ATTESTOR_4= +# Max issuance in 18-decimal units. Decision D3: 150M PEN +# = 150000000000000000000000000 (cross-check canonical tokenomics at deploy) +MAX_ISSUANCE= +# Initial caps in 18-decimal units (PRD V4: target < 1-2% of vault per day) +PER_RELEASE_CAP= +DAILY_CAP= +# Unix timestamp before which the remainder cannot be swept. +# Decision D5: ~ deploy + 3 months (see docs/pen-migration-window-analysis.md) +EARLIEST_SWEEP_TS= + +# --- DeployGovernance.s.sol (phase 5) --- +PEN_TOKEN= +TIMELOCK_DELAY=172800 # 48h (PRD V5) +VOTING_DELAY=86400 # 1 day, seconds (timestamp clock) +VOTING_PERIOD=432000 # 5 days +PROPOSAL_THRESHOLD= # token units required to propose +QUORUM_FRACTION=4 # percent of total supply; start low (PRD G1) diff --git a/contracts/.gitignore b/contracts/.gitignore new file mode 100644 index 000000000..83f939c72 --- /dev/null +++ b/contracts/.gitignore @@ -0,0 +1,3 @@ +out/ +cache/ +broadcast/ diff --git a/contracts/README.md b/contracts/README.md new file mode 100644 index 000000000..4bf87f09f --- /dev/null +++ b/contracts/README.md @@ -0,0 +1,49 @@ +# PEN on Base — Migration Contracts + +Solidity contracts for the one-way migration of PEN from the Pendulum parachain +to Base. Design and requirements: [PRD](../docs/pen-base-migration-prd.md), +[ADR-001](../docs/adr-001-pen-base-migration-approach.md), +[token standards](../docs/pen-token-contract-standards.md). + +## Contracts + +- **`src/PEN.sol`** — fixed-supply ERC-20 (`ERC20 + ERC20Permit + ERC20Votes`, + EIP-6372 timestamp clock). The entire max issuance is minted to the + MigrationVault in the constructor; there is no mint function, no owner and no + proxy. +- **`src/MigrationVault.sol`** — holds the unmigrated supply and releases it on + the 3rd matching on-chain approval from the attestor set (per Pendulum + migration nonce). Includes per-release and daily caps, guardian pause, + timelock-friendly two-step admin, attestor rotation that retroactively + invalidates a removed attestor's approvals, and a time-locked remainder sweep + for the end of the migration window. + +## Deployment order + +The token and vault reference each other, so: + +1. Deploy `MigrationVault(admin, guardian, attestors[4], threshold=3, conversionFactor=1e6, perReleaseCap, dailyCap, earliestSweepTimestamp)` +2. Deploy `PEN(vault, maxIssuance)` — mints the full supply into the vault +3. `vault.setToken(pen)` (admin, one-time; verifies the vault holds 100% of supply) + +`conversionFactor = 1e6` converts 12-decimal pallet amounts to the 18-decimal +token; attestors always submit the raw pallet amount from the +`MigrationInitiated` event — the conversion happens in the vault and nowhere +else (PRD V7). + +## Build & test + +Requires [Foundry](https://getfoundry.sh). Dependencies are git submodules +(`lib/openzeppelin-contracts` v5.4.0, `lib/forge-std`); after a fresh clone run +`git submodule update --init --recursive` or `forge install`. + +```sh +forge build +forge test +``` + +## Open parameters (fixed at deployment, see PRD §4.2) + +- `maxIssuance` — exact figure pending decision D3 +- decimals/`conversionFactor` — 18/1e6 pending decision D2 +- attestor addresses, caps, `earliestSweepTimestamp` — decisions D4/D5 diff --git a/contracts/foundry.lock b/contracts/foundry.lock new file mode 100644 index 000000000..a4cdb73a9 --- /dev/null +++ b/contracts/foundry.lock @@ -0,0 +1,14 @@ +{ + "lib/forge-std": { + "tag": { + "name": "v1.16.2", + "rev": "bf647bd6046f2f7da30d0c2bf435e5c76a780c1b" + } + }, + "lib/openzeppelin-contracts": { + "tag": { + "name": "v5.4.0", + "rev": "c64a1edb67b6e3f4a15cca8909c9482ad33a02b0" + } + } +} \ No newline at end of file diff --git a/contracts/foundry.toml b/contracts/foundry.toml new file mode 100644 index 000000000..9c330ad81 --- /dev/null +++ b/contracts/foundry.toml @@ -0,0 +1,12 @@ +[profile.default] +src = "src" +out = "out" +libs = ["lib"] +test = "test" +solc_version = "0.8.26" +optimizer = true +optimizer_runs = 10_000 +remappings = ["@openzeppelin/=lib/openzeppelin-contracts/"] + +[fuzz] +runs = 512 diff --git a/contracts/lib/forge-std b/contracts/lib/forge-std new file mode 160000 index 000000000..bf647bd60 --- /dev/null +++ b/contracts/lib/forge-std @@ -0,0 +1 @@ +Subproject commit bf647bd6046f2f7da30d0c2bf435e5c76a780c1b diff --git a/contracts/lib/openzeppelin-contracts b/contracts/lib/openzeppelin-contracts new file mode 160000 index 000000000..c64a1edb6 --- /dev/null +++ b/contracts/lib/openzeppelin-contracts @@ -0,0 +1 @@ +Subproject commit c64a1edb67b6e3f4a15cca8909c9482ad33a02b0 diff --git a/contracts/script/Deploy.s.sol b/contracts/script/Deploy.s.sol new file mode 100644 index 000000000..27a9a2034 --- /dev/null +++ b/contracts/script/Deploy.s.sol @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: GPL-3.0-only +pragma solidity 0.8.26; + +import {Script, console} from "forge-std/Script.sol"; +import {PEN} from "../src/PEN.sol"; +import {MigrationVault} from "../src/MigrationVault.sol"; +import {IERC20} from "@openzeppelin/contracts/token/ERC20/IERC20.sol"; + +/// @notice Deploys the migration stack on Base (PRD rollout phase 3): +/// 1. MigrationVault with the deployer as interim admin +/// 2. PEN, minting the full max issuance into the vault +/// 3. vault.setToken (verifies the vault holds 100% of supply) +/// 4. hands vault admin to the bootstrap Safe (two-step; the Safe +/// must call acceptAdmin() to complete) +/// +/// Environment: +/// ADMIN_SAFE bootstrap Safe that becomes vault admin +/// GUARDIAN_SAFE fast pause guardian +/// ATTESTOR_1..4 attestor transaction-sender addresses (D4: 3-of-4) +/// MAX_ISSUANCE max issuance in 18-decimal units (decision D3) +/// PER_RELEASE_CAP initial per-release cap, 18-decimal units +/// DAILY_CAP initial daily cap, 18-decimal units +/// EARLIEST_SWEEP_TS unix timestamp before which no remainder sweep (D5) +contract Deploy is Script { + // 12-decimal pallet amounts -> 18-decimal token amounts (decision D2). + uint256 internal constant CONVERSION_FACTOR = 1e6; + uint256 internal constant THRESHOLD = 3; + + function run() external { + address adminSafe = vm.envAddress("ADMIN_SAFE"); + address guardianSafe = vm.envAddress("GUARDIAN_SAFE"); + uint256 maxIssuance = vm.envUint("MAX_ISSUANCE"); + uint256 perReleaseCap = vm.envUint("PER_RELEASE_CAP"); + uint256 dailyCap = vm.envUint("DAILY_CAP"); + uint256 earliestSweepTs = vm.envUint("EARLIEST_SWEEP_TS"); + + address[] memory attestors = new address[](4); + attestors[0] = vm.envAddress("ATTESTOR_1"); + attestors[1] = vm.envAddress("ATTESTOR_2"); + attestors[2] = vm.envAddress("ATTESTOR_3"); + attestors[3] = vm.envAddress("ATTESTOR_4"); + + vm.startBroadcast(); + + MigrationVault vault = new MigrationVault( + msg.sender, // interim admin for setToken; handed over below + guardianSafe, + attestors, + THRESHOLD, + CONVERSION_FACTOR, + perReleaseCap, + dailyCap, + earliestSweepTs + ); + PEN pen = new PEN(address(vault), maxIssuance); + vault.setToken(IERC20(address(pen))); + vault.transferAdmin(adminSafe); + + vm.stopBroadcast(); + + console.log("PEN: ", address(pen)); + console.log("MigrationVault: ", address(vault)); + console.log("NEXT STEP: the bootstrap Safe must call vault.acceptAdmin()"); + } +} diff --git a/contracts/script/DeployGovernance.s.sol b/contracts/script/DeployGovernance.s.sol new file mode 100644 index 000000000..f84fdc3c7 --- /dev/null +++ b/contracts/script/DeployGovernance.s.sol @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: GPL-3.0-only +pragma solidity 0.8.26; + +import {Script, console} from "forge-std/Script.sol"; +import {TimelockController} from "@openzeppelin/contracts/governance/TimelockController.sol"; +import {PENGovernor} from "../src/PENGovernor.sol"; +import {IVotes} from "@openzeppelin/contracts/governance/utils/IVotes.sol"; + +/// @notice Deploys the on-chain governance stack (PRD rollout phase 5): +/// TimelockController + PENGovernor, wired so that only the Governor +/// can propose to the timelock and anyone can execute after the delay. +/// The deployer's temporary timelock admin role is renounced at the +/// end, leaving the timelock self-administered. +/// +/// After this script: transfer MigrationVault admin to the timelock +/// (bootstrap Safe calls vault.transferAdmin(timelock), then a +/// governance proposal calls vault.acceptAdmin()). +/// +/// Environment: +/// PEN_TOKEN deployed PEN address +/// TIMELOCK_DELAY seconds (PRD V5: >= 48h = 172800) +/// VOTING_DELAY seconds before voting starts (timestamp clock) +/// VOTING_PERIOD seconds of voting +/// PROPOSAL_THRESHOLD token units needed to propose +/// QUORUM_FRACTION percent of total supply (start low; vault balance +/// counts toward total supply, see PRD G1) +contract DeployGovernance is Script { + function run() external { + address token = vm.envAddress("PEN_TOKEN"); + uint256 timelockDelay = vm.envUint("TIMELOCK_DELAY"); + uint48 votingDelay = uint48(vm.envUint("VOTING_DELAY")); + uint32 votingPeriod = uint32(vm.envUint("VOTING_PERIOD")); + uint256 proposalThreshold = vm.envUint("PROPOSAL_THRESHOLD"); + uint256 quorumFraction = vm.envUint("QUORUM_FRACTION"); + + vm.startBroadcast(); + + // Deployer is temporary admin so the roles below can be wired. + address[] memory empty = new address[](0); + TimelockController timelock = + new TimelockController(timelockDelay, empty, empty, msg.sender); + + PENGovernor governor = new PENGovernor( + IVotes(token), timelock, votingDelay, votingPeriod, proposalThreshold, quorumFraction + ); + + // Only the Governor proposes/cancels; anyone may execute after the delay. + timelock.grantRole(timelock.PROPOSER_ROLE(), address(governor)); + timelock.grantRole(timelock.CANCELLER_ROLE(), address(governor)); + timelock.grantRole(timelock.EXECUTOR_ROLE(), address(0)); + // Leave the timelock self-administered: changes require a proposal. + timelock.renounceRole(timelock.DEFAULT_ADMIN_ROLE(), msg.sender); + + vm.stopBroadcast(); + + console.log("TimelockController: ", address(timelock)); + console.log("PENGovernor: ", address(governor)); + } +} diff --git a/contracts/src/MigrationVault.sol b/contracts/src/MigrationVault.sol new file mode 100644 index 000000000..acabd7bee --- /dev/null +++ b/contracts/src/MigrationVault.sol @@ -0,0 +1,460 @@ +// SPDX-License-Identifier: GPL-3.0-only +pragma solidity 0.8.26; + +import {IERC20} from "@openzeppelin/contracts/token/ERC20/IERC20.sol"; +import {SafeERC20} from "@openzeppelin/contracts/token/ERC20/utils/SafeERC20.sol"; + +/// @title MigrationVault — releases pre-minted PEN as holders migrate from Pendulum +/// @notice Holds the entire unmigrated PEN supply. Each attestor independently +/// observes finalized `MigrationInitiated` events on the Pendulum +/// parachain and submits an on-chain approval for the exact +/// (nonce, recipient, amount) tuple. The `threshold`-th matching +/// approval releases the tokens (on-chain-approvals model, ADR-001). +/// +/// Trust and blast-radius model (PRD §8): +/// - fewer than `threshold` attestors can release nothing; +/// - a compromised quorum is bounded by `perReleaseCap`/`dailyCap` +/// and can be stopped by the guardian's `pause`; +/// - all parameter changes go through `admin`, expected to be a +/// TimelockController (>= 48h) after the bootstrap phase. +contract MigrationVault { + using SafeERC20 for IERC20; + + // ---------------------------------------------------------------- errors + + error NotAdmin(); + error NotGuardianOrAdmin(); + error NotAttestor(); + error NotPendingAdmin(); + error ZeroAddress(); + error RecipientIsVault(); + error TokenAlreadySet(); + error TokenNotSet(); + error VaultMustHoldFullSupply(); + error InvalidThreshold(); + error DuplicateAttestor(); + error UnknownAttestor(); + error ThresholdWouldExceedAttestors(); + error NonceAlreadyConsumed(uint64 nonce); + error AlreadyApproved(address attestor); + error NotEnoughApprovals(uint256 active, uint256 required); + error EnforcedPause(); + error NotPaused(); + error ZeroAmount(); + error ExceedsPerReleaseCap(uint256 amount, uint256 cap); + error ExceedsDailyCap(uint256 requested, uint256 available); + error SweepNotYetAllowed(uint256 earliest); + error PendingNotStale(); + error InsufficientVaultBalance(); + error ExceedsSweepable(uint256 requested, uint256 sweepable); + error SweepSettlingAfterThresholdCut(uint256 allowedFrom); + + // ---------------------------------------------------------------- events + + event TokenSet(address indexed token); + event Approved(uint64 indexed nonce, address indexed recipient, uint256 palletAmount, address indexed attestor); + event Released(uint64 indexed nonce, address indexed recipient, uint256 palletAmount, uint256 tokenAmount); + event Paused(address indexed by); + event Unpaused(address indexed by); + event AttestorAdded(address indexed attestor); + event AttestorRemoved(address indexed attestor); + event ThresholdUpdated(uint256 threshold); + event CapsUpdated(uint256 perReleaseCap, uint256 dailyCap); + event GuardianUpdated(address indexed guardian); + event AdminTransferStarted(address indexed pendingAdmin); + event AdminTransferred(address indexed newAdmin); + event RemainderSwept(address indexed to, uint256 amount); + event ReleasePending(uint64 indexed nonce, address indexed recipient, uint256 tokenAmount); + event StalePendingCleared(bytes32 indexed payload, uint256 tokenAmount); + + // ---------------------------------------------------------------- state + + /// @notice The PEN token. Set exactly once, after which the vault must + /// hold the token's entire supply (pre-mint model, ADR-001). + IERC20 public token; + + /// @notice Admin of all parameters; a TimelockController post-bootstrap. + address public admin; + address public pendingAdmin; + + /// @notice Can pause releases instantly (incident response). Unpause is + /// admin-only, so a compromised guardian can at worst halt. + address public guardian; + + mapping(address => bool) public isAttestor; + uint256 public attestorCount; + /// @notice Number of distinct active attestors that must approve the + /// identical (nonce, recipient, amount) tuple to release. + uint256 public threshold; + + /// @notice Multiplier from pallet units (12 decimals on Pendulum) to token + /// units. The decimal conversion happens here and nowhere else + /// (PRD V7); 1e6 for an 18-decimal token. + uint256 public immutable conversionFactor; + + /// @notice Maximum token units released in a single migration. + uint256 public perReleaseCap; + /// @notice Maximum token units released in any rolling 24h window (PRD V4). + /// Enforced as a leaky bucket of capacity `dailyCap` that refills + /// linearly at `dailyCap` per day: a burst is capped at `dailyCap` + /// and a second burst must wait for the bucket to refill. There is + /// no instant reset at a calendar boundary. + uint256 public dailyCap; + /// @dev Consumed allowance recorded at `windowUpdatedAt`, before decay. + uint256 public windowConsumed; + uint256 public windowUpdatedAt; + + /// @notice Earliest timestamp at which the admin may sweep the unmigrated + /// remainder (end-of-window handling, PRD V9 / decision D5). + uint256 public immutable earliestSweepTimestamp; + + bool public paused; + + /// @notice Consumed migration nonces; a nonce can never release twice. + mapping(uint64 => bool) public nonceConsumed; + + /// @dev Approvers per payload hash. Approvals are counted at release time + /// against the *current* attestor set, so removing a compromised + /// attestor retroactively invalidates its approvals (PRD V6). + mapping(bytes32 => address[]) internal _approvers; + mapping(bytes32 => mapping(address => bool)) internal _inApprovers; + + /// @dev Generation of each attestor address, bumped on every addAttestor. + /// An approval only counts while its recorded generation matches the + /// attestor's current one, so approvals from before a removal can + /// never count again after a re-add — the threshold can only ever be + /// crossed inside approve(), which maintains the pending-release + /// accounting that protects sweepRemainder. + mapping(address => uint64) public attestorGeneration; + mapping(bytes32 => mapping(address => uint64)) internal _approvalGeneration; + + /// @notice Total token units released so far (for the invariant monitor: + /// balanceOf(vault) + totalReleased == totalSupply). + uint256 public totalReleased; + + /// @notice Token units owed to threshold-approved payloads whose release + /// was deferred (pause or caps). Excluded from `sweepRemainder` + /// so a sweep can never strand an already-earned release. + uint256 public pendingApprovedAmount; + mapping(bytes32 => bool) public pendingRelease; + + /// @notice Total token units swept out via `sweepRemainder`. Tracked so + /// the invariant monitor's conservation check stays exact after a + /// window-close sweep: balanceOf(vault) + totalReleased + + /// totalSwept == totalSupply at all times. + uint256 public totalSwept; + + /// @notice Timestamp of the last threshold *decrease*. `sweepRemainder` is + /// blocked for `SWEEP_SETTLING_PERIOD` afterwards: lowering the + /// threshold can retroactively make a sub-threshold payload + /// releasable without registering it in `pendingApprovedAmount` + /// (that accounting is maintained only inside `approve()`), so the + /// delay gives the monitor and a permissionless `release()` time to + /// settle any newly-qualifying payload before a sweep could strand + /// it. See runbooks RB-6/RB-7. + uint256 public thresholdReducedAt; + uint256 public constant SWEEP_SETTLING_PERIOD = 7 days; + + // ---------------------------------------------------------------- modifiers + + modifier onlyAdmin() { + if (msg.sender != admin) revert NotAdmin(); + _; + } + + modifier onlyAttestor() { + if (!isAttestor[msg.sender]) revert NotAttestor(); + _; + } + + // ---------------------------------------------------------------- setup + + constructor( + address admin_, + address guardian_, + address[] memory attestors_, + uint256 threshold_, + uint256 conversionFactor_, + uint256 perReleaseCap_, + uint256 dailyCap_, + uint256 earliestSweepTimestamp_ + ) { + if (admin_ == address(0) || guardian_ == address(0)) revert ZeroAddress(); + if (threshold_ < 2 || threshold_ > attestors_.length) revert InvalidThreshold(); + if (conversionFactor_ == 0) revert ZeroAmount(); + + admin = admin_; + guardian = guardian_; + threshold = threshold_; + conversionFactor = conversionFactor_; + perReleaseCap = perReleaseCap_; + dailyCap = dailyCap_; + earliestSweepTimestamp = earliestSweepTimestamp_; + + for (uint256 i = 0; i < attestors_.length; i++) { + address attestor = attestors_[i]; + if (attestor == address(0)) revert ZeroAddress(); + if (isAttestor[attestor]) revert DuplicateAttestor(); + isAttestor[attestor] = true; + attestorGeneration[attestor] = 1; + emit AttestorAdded(attestor); + } + attestorCount = attestors_.length; + } + + /// @notice One-time wiring of the token, required because vault and token + /// reference each other: the vault is deployed first, then PEN + /// mints its full supply here, then the admin calls this. + function setToken(IERC20 token_) external onlyAdmin { + if (address(token) != address(0)) revert TokenAlreadySet(); + if (address(token_) == address(0)) revert ZeroAddress(); + uint256 supply = token_.totalSupply(); + if (supply == 0 || token_.balanceOf(address(this)) != supply) revert VaultMustHoldFullSupply(); + token = token_; + emit TokenSet(address(token_)); + } + + // ---------------------------------------------------------------- attestation + + /// @notice Approve the release for Pendulum migration `nonce`. `palletAmount` + /// is the burned amount in pallet units (12 decimals), exactly as + /// emitted by the `MigrationInitiated` event; the vault converts. + /// Recording approvals stays possible while paused so that releases + /// resume without re-attestation after an unpause. + function approve(uint64 nonce, address recipient, uint256 palletAmount) external onlyAttestor { + if (recipient == address(0)) revert ZeroAddress(); + // Releasing to the vault itself is a self-transfer: it leaves + // balanceOf(this) unchanged while bumping totalReleased, permanently + // breaking the monitor's conservation identity + // (balance + released + swept == totalSupply) and, with auto-pause on, + // wedging releases. It is never a legitimate migration target, so + // reject it at the single point where approvals are recorded. + if (recipient == address(this)) revert RecipientIsVault(); + if (palletAmount == 0) revert ZeroAmount(); + if (nonceConsumed[nonce]) revert NonceAlreadyConsumed(nonce); + + bytes32 payload = payloadHash(nonce, recipient, palletAmount); + uint64 generation = attestorGeneration[msg.sender]; + if (_approvalGeneration[payload][msg.sender] == generation) revert AlreadyApproved(msg.sender); + _approvalGeneration[payload][msg.sender] = generation; + if (!_inApprovers[payload][msg.sender]) { + _inApprovers[payload][msg.sender] = true; + _approvers[payload].push(msg.sender); + } + emit Approved(nonce, recipient, palletAmount, msg.sender); + + // Opportunistic release: skipped (not reverted) when paused or a cap + // is hit, so the approval is recorded either way. `release` can be + // called by anyone later to retry. + if (activeApprovals(payload) >= threshold) { + uint256 tokenAmount = palletAmount * conversionFactor; + // Insufficient balance is included here deliberately: if the vault + // was over-swept, the release is deferred (marked pending) rather + // than reverting. A revert here would roll back this approval and, + // because every attestor hits it identically, permanently + // crash-loop the fleet on this block. Deferral keeps the debt + // tracked and recoverable once the vault is refunded. + bool releasable = !paused && address(token) != address(0) && tokenAmount <= perReleaseCap + && tokenAmount <= availableDailyAllowance() + && token.balanceOf(address(this)) >= tokenAmount; + if (releasable) { + _release(nonce, recipient, palletAmount, payload); + } else if (!pendingRelease[payload]) { + // Threshold reached but deferred: account for the owed amount + // so `sweepRemainder` cannot strand it. + pendingRelease[payload] = true; + pendingApprovedAmount += tokenAmount; + emit ReleasePending(nonce, recipient, tokenAmount); + } + } + } + + /// @notice Execute a sufficiently-approved release. Callable by anyone; + /// used to retry releases deferred by pause or caps. + function release(uint64 nonce, address recipient, uint256 palletAmount) external { + if (paused) revert EnforcedPause(); + if (address(token) == address(0)) revert TokenNotSet(); + if (nonceConsumed[nonce]) revert NonceAlreadyConsumed(nonce); + + bytes32 payload = payloadHash(nonce, recipient, palletAmount); + uint256 active = activeApprovals(payload); + if (active < threshold) revert NotEnoughApprovals(active, threshold); + + uint256 tokenAmount = palletAmount * conversionFactor; + if (tokenAmount > perReleaseCap) revert ExceedsPerReleaseCap(tokenAmount, perReleaseCap); + uint256 available = availableDailyAllowance(); + if (tokenAmount > available) revert ExceedsDailyCap(tokenAmount, available); + if (token.balanceOf(address(this)) < tokenAmount) revert InsufficientVaultBalance(); + + _release(nonce, recipient, palletAmount, payload); + } + + /// @dev Caller must have verified pause state, approvals and caps. + function _release(uint64 nonce, address recipient, uint256 palletAmount, bytes32 payload) internal { + uint256 tokenAmount = palletAmount * conversionFactor; + nonceConsumed[nonce] = true; + if (pendingRelease[payload]) { + pendingRelease[payload] = false; + pendingApprovedAmount -= tokenAmount; + } + windowConsumed = _decayedConsumed() + tokenAmount; + windowUpdatedAt = block.timestamp; + totalReleased += tokenAmount; + token.safeTransfer(recipient, tokenAmount); + emit Released(nonce, recipient, palletAmount, tokenAmount); + } + + /// @notice Clear the pending-release accounting of a payload whose nonce + /// was released via a DIFFERENT (conflicting) tuple. Restricted to + /// consumed nonces: an unconsumed pending payload is still owed to + /// its migrator and must never be cleared. + function clearStalePending(uint64 nonce, address recipient, uint256 palletAmount) external onlyAdmin { + if (!nonceConsumed[nonce]) revert PendingNotStale(); + bytes32 payload = payloadHash(nonce, recipient, palletAmount); + if (!pendingRelease[payload]) revert PendingNotStale(); + pendingRelease[payload] = false; + uint256 tokenAmount = palletAmount * conversionFactor; + pendingApprovedAmount -= tokenAmount; + emit StalePendingCleared(payload, tokenAmount); + } + + // ---------------------------------------------------------------- views + + function payloadHash(uint64 nonce, address recipient, uint256 palletAmount) public pure returns (bytes32) { + return keccak256(abi.encode(nonce, recipient, palletAmount)); + } + + /// @notice Approvals for a payload counted against the current attestor + /// set and generation. Removed attestors no longer count, and a + /// re-added attestor must approve again (its pre-removal approval + /// belongs to an older generation). + function activeApprovals(bytes32 payload) public view returns (uint256 count) { + address[] storage approvers = _approvers[payload]; + for (uint256 i = 0; i < approvers.length; i++) { + address approver = approvers[i]; + if (isAttestor[approver] && _approvalGeneration[payload][approver] == attestorGeneration[approver]) { + count++; + } + } + } + + /// @notice Whether `attestor` holds a currently-valid approval for the + /// payload — i.e. it is a current attestor and its approval is + /// from its current generation. Mirrors the conditions + /// `activeApprovals` counts, so a removed attestor reports false. + function hasApproved(bytes32 payload, address attestor) public view returns (bool) { + return isAttestor[attestor] && _approvalGeneration[payload][attestor] == attestorGeneration[attestor]; + } + + function approversOf(bytes32 payload) external view returns (address[] memory) { + return _approvers[payload]; + } + + /// @dev Consumed allowance after linear refill since the last release. + function _decayedConsumed() internal view returns (uint256) { + uint256 refilled = ((block.timestamp - windowUpdatedAt) * dailyCap) / 1 days; + return windowConsumed > refilled ? windowConsumed - refilled : 0; + } + + /// @notice Token units releasable right now under the rolling daily cap. + function availableDailyAllowance() public view returns (uint256) { + uint256 consumed = _decayedConsumed(); + return dailyCap > consumed ? dailyCap - consumed : 0; + } + + // ---------------------------------------------------------------- pause + + function pause() external { + if (msg.sender != guardian && msg.sender != admin) revert NotGuardianOrAdmin(); + if (paused) revert EnforcedPause(); + paused = true; + emit Paused(msg.sender); + } + + function unpause() external onlyAdmin { + if (!paused) revert NotPaused(); + paused = false; + emit Unpaused(msg.sender); + } + + // ---------------------------------------------------------------- admin + + function addAttestor(address attestor) external onlyAdmin { + if (attestor == address(0)) revert ZeroAddress(); + if (isAttestor[attestor]) revert DuplicateAttestor(); + isAttestor[attestor] = true; + // New generation: any approvals this address recorded before a prior + // removal stop counting, so this call can never cross a threshold. + attestorGeneration[attestor] += 1; + attestorCount += 1; + emit AttestorAdded(attestor); + } + + function removeAttestor(address attestor) external onlyAdmin { + if (!isAttestor[attestor]) revert UnknownAttestor(); + if (attestorCount - 1 < threshold) revert ThresholdWouldExceedAttestors(); + isAttestor[attestor] = false; + attestorCount -= 1; + emit AttestorRemoved(attestor); + } + + function setThreshold(uint256 threshold_) external onlyAdmin { + if (threshold_ < 2 || threshold_ > attestorCount) revert InvalidThreshold(); + // A decrease can retroactively qualify a sub-threshold payload without + // routing through approve() (which maintains pendingApprovedAmount); + // gate sweeps for a settling period so it can be detected and released + // first (round-4 finding). + if (threshold_ < threshold) thresholdReducedAt = block.timestamp; + threshold = threshold_; + emit ThresholdUpdated(threshold_); + } + + function setCaps(uint256 perReleaseCap_, uint256 dailyCap_) external onlyAdmin { + perReleaseCap = perReleaseCap_; + dailyCap = dailyCap_; + emit CapsUpdated(perReleaseCap_, dailyCap_); + } + + function setGuardian(address guardian_) external onlyAdmin { + if (guardian_ == address(0)) revert ZeroAddress(); + guardian = guardian_; + emit GuardianUpdated(guardian_); + } + + function transferAdmin(address newAdmin) external onlyAdmin { + if (newAdmin == address(0)) revert ZeroAddress(); + pendingAdmin = newAdmin; + emit AdminTransferStarted(newAdmin); + } + + function acceptAdmin() external { + if (msg.sender != pendingAdmin) revert NotPendingAdmin(); + admin = msg.sender; + pendingAdmin = address(0); + emit AdminTransferred(msg.sender); + } + + /// @notice Sweep up to `amount` of the unmigrated remainder after the + /// migration window closes (destination decided by governance, + /// PRD D5). The caller must pass an explicit amount, bounded by + /// `balance − pendingApprovedAmount`, forcing a conscious + /// reconciliation against the monitor's outstanding-nonce count + /// (runbook RB-7) rather than blindly sweeping everything — + /// `pendingApprovedAmount` only reserves threshold-approved + /// releases, not migrations still gathering approvals. + function sweepRemainder(address to, uint256 amount) external onlyAdmin { + if (block.timestamp < earliestSweepTimestamp) revert SweepNotYetAllowed(earliestSweepTimestamp); + if (block.timestamp < thresholdReducedAt + SWEEP_SETTLING_PERIOD) { + revert SweepSettlingAfterThresholdCut(thresholdReducedAt + SWEEP_SETTLING_PERIOD); + } + if (to == address(0)) revert ZeroAddress(); + if (address(token) == address(0)) revert TokenNotSet(); + uint256 balanceHeld = token.balanceOf(address(this)); + // Saturating: a prior over-sweep can leave pending > balance; never revert on underflow. + uint256 sweepable = balanceHeld > pendingApprovedAmount ? balanceHeld - pendingApprovedAmount : 0; + if (amount > sweepable) revert ExceedsSweepable(amount, sweepable); + totalSwept += amount; + token.safeTransfer(to, amount); + emit RemainderSwept(to, amount); + } +} diff --git a/contracts/src/PEN.sol b/contracts/src/PEN.sol new file mode 100644 index 000000000..4a4908afd --- /dev/null +++ b/contracts/src/PEN.sol @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-3.0-only +pragma solidity 0.8.26; + +import {ERC20} from "@openzeppelin/contracts/token/ERC20/ERC20.sol"; +import {ERC20Permit} from "@openzeppelin/contracts/token/ERC20/extensions/ERC20Permit.sol"; +import {ERC20Votes} from "@openzeppelin/contracts/token/ERC20/extensions/ERC20Votes.sol"; +import {Nonces} from "@openzeppelin/contracts/utils/Nonces.sol"; + +/// @title PEN — the Pendulum token on Base +/// @notice Fixed-supply ERC-20. The entire maximum issuance is minted to the +/// MigrationVault at deployment; there is no mint function, no owner +/// and no upgradeability. Unmigrated supply sits in the vault and is +/// released as holders migrate from the Pendulum parachain. +/// +/// Extensions (see docs/pen-token-contract-standards.md): +/// - ERC20Permit (EIP-2612): signature-based approvals +/// - ERC20Votes (EIP-5805): checkpointed voting power + delegation, +/// with the EIP-6372 clock in timestamp mode +contract PEN is ERC20, ERC20Permit, ERC20Votes { + error ZeroVault(); + error ZeroIssuance(); + + /// @param vault The MigrationVault that receives the full supply. + /// @param maxIssuance The maximum issuance of PEN, in 18-decimal units + /// (PRD open decision D3 fixes the exact figure at deployment). + constructor(address vault, uint256 maxIssuance) ERC20("Pendulum", "PEN") ERC20Permit("Pendulum") { + if (vault == address(0)) revert ZeroVault(); + if (maxIssuance == 0) revert ZeroIssuance(); + _mint(vault, maxIssuance); + } + + /// @dev EIP-6372 clock in timestamp mode (PRD T1). The Governor contract + /// must be deployed with the same clock mode. + function clock() public view override returns (uint48) { + return uint48(block.timestamp); + } + + /// @dev EIP-6372 machine-readable clock description. + // solhint-disable-next-line func-name-mixedcase + function CLOCK_MODE() public pure override returns (string memory) { + return "mode=timestamp"; + } + + // ----- required overrides for the ERC20Permit/ERC20Votes composition ----- + + function _update(address from, address to, uint256 value) internal override(ERC20, ERC20Votes) { + super._update(from, to, value); + } + + function nonces(address owner) public view override(ERC20Permit, Nonces) returns (uint256) { + return super.nonces(owner); + } +} diff --git a/contracts/src/PENGovernor.sol b/contracts/src/PENGovernor.sol new file mode 100644 index 000000000..4c4c43dec --- /dev/null +++ b/contracts/src/PENGovernor.sol @@ -0,0 +1,112 @@ +// SPDX-License-Identifier: GPL-3.0-only +pragma solidity 0.8.26; + +import {Governor} from "@openzeppelin/contracts/governance/Governor.sol"; +import {GovernorSettings} from "@openzeppelin/contracts/governance/extensions/GovernorSettings.sol"; +import {GovernorCountingSimple} from "@openzeppelin/contracts/governance/extensions/GovernorCountingSimple.sol"; +import {GovernorVotes} from "@openzeppelin/contracts/governance/extensions/GovernorVotes.sol"; +import {GovernorVotesQuorumFraction} from + "@openzeppelin/contracts/governance/extensions/GovernorVotesQuorumFraction.sol"; +import {GovernorTimelockControl} from "@openzeppelin/contracts/governance/extensions/GovernorTimelockControl.sol"; +import {TimelockController} from "@openzeppelin/contracts/governance/TimelockController.sol"; +import {IVotes} from "@openzeppelin/contracts/governance/utils/IVotes.sol"; + +/// @title PENGovernor — on-chain governance for the Base-side PEN contracts +/// @notice Standard OZ Governor composition (hybrid governance model, ADR-001): +/// token-holder votes execute through a TimelockController, which is +/// the admin of the MigrationVault and the treasury after the +/// bootstrap phase. PEN uses the EIP-6372 timestamp clock, so all +/// Governor periods below are in seconds. +/// +/// Quorum caveat (PRD G1): quorum is a fraction of *total* supply, +/// which includes the unmigrated balance held by the vault. Start +/// with a low fraction while migration is in progress and raise it +/// via governance as circulating supply grows. +contract PENGovernor is + Governor, + GovernorSettings, + GovernorCountingSimple, + GovernorVotes, + GovernorVotesQuorumFraction, + GovernorTimelockControl +{ + constructor( + IVotes token, + TimelockController timelock, + uint48 votingDelay_, // seconds (timestamp clock) + uint32 votingPeriod_, // seconds + uint256 proposalThreshold_, // token units + uint256 quorumFraction // percent of total supply + ) + Governor("PENGovernor") + GovernorSettings(votingDelay_, votingPeriod_, proposalThreshold_) + GovernorVotes(token) + GovernorVotesQuorumFraction(quorumFraction) + GovernorTimelockControl(timelock) + {} + + // ----- required overrides for the Governor composition ----- + + function votingDelay() public view override(Governor, GovernorSettings) returns (uint256) { + return super.votingDelay(); + } + + function votingPeriod() public view override(Governor, GovernorSettings) returns (uint256) { + return super.votingPeriod(); + } + + function proposalThreshold() public view override(Governor, GovernorSettings) returns (uint256) { + return super.proposalThreshold(); + } + + function state(uint256 proposalId) + public + view + override(Governor, GovernorTimelockControl) + returns (ProposalState) + { + return super.state(proposalId); + } + + function proposalNeedsQueuing(uint256 proposalId) + public + view + override(Governor, GovernorTimelockControl) + returns (bool) + { + return super.proposalNeedsQueuing(proposalId); + } + + function _queueOperations( + uint256 proposalId, + address[] memory targets, + uint256[] memory values, + bytes[] memory calldatas, + bytes32 descriptionHash + ) internal override(Governor, GovernorTimelockControl) returns (uint48) { + return super._queueOperations(proposalId, targets, values, calldatas, descriptionHash); + } + + function _executeOperations( + uint256 proposalId, + address[] memory targets, + uint256[] memory values, + bytes[] memory calldatas, + bytes32 descriptionHash + ) internal override(Governor, GovernorTimelockControl) { + super._executeOperations(proposalId, targets, values, calldatas, descriptionHash); + } + + function _cancel( + address[] memory targets, + uint256[] memory values, + bytes[] memory calldatas, + bytes32 descriptionHash + ) internal override(Governor, GovernorTimelockControl) returns (uint256) { + return super._cancel(targets, values, calldatas, descriptionHash); + } + + function _executor() internal view override(Governor, GovernorTimelockControl) returns (address) { + return super._executor(); + } +} diff --git a/contracts/test/MigrationVault.t.sol b/contracts/test/MigrationVault.t.sol new file mode 100644 index 000000000..9c678d614 --- /dev/null +++ b/contracts/test/MigrationVault.t.sol @@ -0,0 +1,533 @@ +// SPDX-License-Identifier: GPL-3.0-only +pragma solidity 0.8.26; + +import {Test} from "forge-std/Test.sol"; +import {PEN} from "../src/PEN.sol"; +import {MigrationVault} from "../src/MigrationVault.sol"; +import {IERC20} from "@openzeppelin/contracts/token/ERC20/IERC20.sol"; + +contract MigrationVaultTest is Test { + uint256 internal constant MAX_ISSUANCE = 150_000_000e18; + // Pallet amounts are 12-decimal; the vault scales to the 18-decimal token. + uint256 internal constant CONVERSION_FACTOR = 1e6; + uint256 internal constant PER_RELEASE_CAP = 1_000_000e18; + uint256 internal constant DAILY_CAP = 2_000_000e18; + + MigrationVault internal vault; + PEN internal pen; + + address internal admin = makeAddr("admin"); + address internal guardian = makeAddr("guardian"); + address internal recipient = makeAddr("recipient"); + address[] internal attestors; + uint256 internal earliestSweep; + + function setUp() public { + // Decision D4: 4 attestors, threshold 3-of-4. + for (uint256 i = 0; i < 4; i++) { + attestors.push(makeAddr(string(abi.encodePacked("attestor", i)))); + } + earliestSweep = block.timestamp + 365 days; + + vault = new MigrationVault( + admin, guardian, attestors, 3, CONVERSION_FACTOR, PER_RELEASE_CAP, DAILY_CAP, earliestSweep + ); + pen = new PEN(address(vault), MAX_ISSUANCE); + vm.prank(admin); + vault.setToken(IERC20(address(pen))); + } + + function approveAs(uint256 attestorIndex, uint64 nonce, address to, uint256 palletAmount) internal { + vm.prank(attestors[attestorIndex]); + vault.approve(nonce, to, palletAmount); + } + + // ---------------------------------------------------------------- setup & wiring + + function test_SetTokenOnlyOnce() public { + vm.prank(admin); + vm.expectRevert(MigrationVault.TokenAlreadySet.selector); + vault.setToken(IERC20(address(pen))); + } + + function test_SetTokenRequiresFullSupplyInVault() public { + MigrationVault fresh = new MigrationVault( + admin, guardian, attestors, 3, CONVERSION_FACTOR, PER_RELEASE_CAP, DAILY_CAP, earliestSweep + ); + // PEN was minted to the *other* vault, so this one holds nothing. + vm.prank(admin); + vm.expectRevert(MigrationVault.VaultMustHoldFullSupply.selector); + fresh.setToken(IERC20(address(pen))); + } + + function test_ConstructorRejectsThresholdBelowTwo() public { + vm.expectRevert(MigrationVault.InvalidThreshold.selector); + new MigrationVault(admin, guardian, attestors, 1, CONVERSION_FACTOR, PER_RELEASE_CAP, DAILY_CAP, earliestSweep); + } + + // ---------------------------------------------------------------- happy path + + function test_ThresholdApprovalsRelease() public { + uint256 palletAmount = 5e12; // 5 PEN in 12-decimal pallet units + + approveAs(0, 0, recipient, palletAmount); + approveAs(1, 0, recipient, palletAmount); + assertEq(pen.balanceOf(recipient), 0, "must not release below threshold"); + + approveAs(2, 0, recipient, palletAmount); + assertEq(pen.balanceOf(recipient), 5e18, "decimal conversion 12 -> 18"); + assertTrue(vault.nonceConsumed(0)); + assertEq(vault.totalReleased(), 5e18); + // Invariant the monitor watches: vault balance + released == total supply. + assertEq(pen.balanceOf(address(vault)) + vault.totalReleased(), pen.totalSupply()); + } + + // ---------------------------------------------------------------- replay & dedup + + function test_ConsumedNonceCannotReleaseAgain() public { + uint256 palletAmount = 5e12; + approveAs(0, 0, recipient, palletAmount); + approveAs(1, 0, recipient, palletAmount); + approveAs(2, 0, recipient, palletAmount); + + vm.prank(attestors[3]); + vm.expectRevert(abi.encodeWithSelector(MigrationVault.NonceAlreadyConsumed.selector, 0)); + vault.approve(0, recipient, palletAmount); + + vm.expectRevert(abi.encodeWithSelector(MigrationVault.NonceAlreadyConsumed.selector, 0)); + vault.release(0, recipient, palletAmount); + } + + function test_SameAttestorCannotApproveTwice() public { + approveAs(0, 0, recipient, 5e12); + vm.prank(attestors[0]); + vm.expectRevert(abi.encodeWithSelector(MigrationVault.AlreadyApproved.selector, attestors[0])); + vault.approve(0, recipient, 5e12); + } + + function test_NonAttestorCannotApprove() public { + vm.prank(makeAddr("mallory")); + vm.expectRevert(MigrationVault.NotAttestor.selector); + vault.approve(0, recipient, 5e12); + } + + function test_ConflictingTuplesNeverMerge() public { + address mallory = makeAddr("mallory"); + // Two attestors approve the honest tuple, two approve a conflicting one + // for the same nonce. Neither tuple reaches the 3-of-4 threshold. + approveAs(0, 0, recipient, 5e12); + approveAs(1, 0, recipient, 5e12); + approveAs(2, 0, mallory, 5e12); + approveAs(3, 0, mallory, 5e12); + assertEq(pen.balanceOf(recipient), 0); + assertEq(pen.balanceOf(mallory), 0); + + // An attestor may approve both tuples (distinct payloads): the honest + // tuple reaches threshold and wins; the nonce is consumed. + approveAs(2, 0, recipient, 5e12); + assertEq(pen.balanceOf(recipient), 5e18); + assertEq(pen.balanceOf(mallory), 0); + } + + // ---------------------------------------------------------------- caps + + function test_PerReleaseCapDefersUntilAdminRaisesIt() public { + // 2M PEN in pallet units converts to 2Me18 > perReleaseCap. + uint256 palletAmount = 2_000_000e12; + approveAs(0, 0, recipient, palletAmount); + approveAs(1, 0, recipient, palletAmount); + approveAs(2, 0, recipient, palletAmount); + assertEq(pen.balanceOf(recipient), 0, "capped release must be deferred, not executed"); + + vm.expectRevert( + abi.encodeWithSelector(MigrationVault.ExceedsPerReleaseCap.selector, 2_000_000e18, PER_RELEASE_CAP) + ); + vault.release(0, recipient, palletAmount); + + vm.prank(admin); + vault.setCaps(3_000_000e18, 3_000_000e18); + vault.release(0, recipient, palletAmount); + assertEq(pen.balanceOf(recipient), 2_000_000e18); + } + + function test_DailyCapRefillsGraduallyOverRollingWindow() public { + uint256 palletAmount = 1_000_000e12; // converts to exactly the per-release cap + + // Consume the full daily cap (2 × 1M = DAILY_CAP). + for (uint64 nonce = 0; nonce < 2; nonce++) { + approveAs(0, nonce, recipient, palletAmount); + approveAs(1, nonce, recipient, palletAmount); + approveAs(2, nonce, recipient, palletAmount); + } + assertEq(pen.balanceOf(recipient), 2_000_000e18); + assertEq(vault.availableDailyAllowance(), 0); + + // A third release is deferred: the bucket is empty. + approveAs(0, 2, recipient, palletAmount); + approveAs(1, 2, recipient, palletAmount); + approveAs(2, 2, recipient, palletAmount); + assertEq(pen.balanceOf(recipient), 2_000_000e18); + vm.expectRevert(abi.encodeWithSelector(MigrationVault.ExceedsDailyCap.selector, 1_000_000e18, 0)); + vault.release(2, recipient, palletAmount); + + // Half a day later, exactly half the cap has refilled. + vm.warp(block.timestamp + 12 hours); + assertEq(vault.availableDailyAllowance(), 1_000_000e18); + vault.release(2, recipient, palletAmount); + assertEq(pen.balanceOf(recipient), 3_000_000e18); + } + + // The exploit the leaky bucket fixes: the old calendar-day bucket reset to + // zero at the UTC boundary, letting a compromised quorum release 2× the cap + // seconds apart. The rolling window must NOT refill instantly. + function test_DailyCapHasNoInstantResetAtBoundary() public { + // Sit one second before a UTC day boundary and consume the full cap. + vm.warp(10 days - 1); + uint256 palletAmount = 1_000_000e12; + for (uint64 nonce = 0; nonce < 2; nonce++) { + approveAs(0, nonce, recipient, palletAmount); + approveAs(1, nonce, recipient, palletAmount); + approveAs(2, nonce, recipient, palletAmount); + } + assertEq(pen.balanceOf(recipient), 2_000_000e18); + + // Cross the boundary by two seconds — negligible refill. The old + // calendar-day bucket would have fully reset to the cap here. + vm.warp(10 days + 1); + assertLt(vault.availableDailyAllowance(), 1_000e18); + approveAs(0, 2, recipient, palletAmount); + approveAs(1, 2, recipient, palletAmount); + approveAs(2, 2, recipient, palletAmount); + assertEq(pen.balanceOf(recipient), 2_000_000e18, "no instant reset at the day boundary"); + } + + // ---------------------------------------------------------------- pause + + function test_PauseBlocksReleasesButKeepsRecordingApprovals() public { + vm.prank(guardian); + vault.pause(); + + // Approvals are still recorded while paused (no re-attestation needed). + approveAs(0, 0, recipient, 5e12); + approveAs(1, 0, recipient, 5e12); + approveAs(2, 0, recipient, 5e12); + assertEq(pen.balanceOf(recipient), 0); + + vm.expectRevert(MigrationVault.EnforcedPause.selector); + vault.release(0, recipient, 5e12); + + // Guardian cannot unpause; only the admin can. + vm.prank(guardian); + vm.expectRevert(MigrationVault.NotAdmin.selector); + vault.unpause(); + + vm.prank(admin); + vault.unpause(); + vault.release(0, recipient, 5e12); + assertEq(pen.balanceOf(recipient), 5e18); + } + + function test_OnlyGuardianOrAdminCanPause() public { + vm.prank(makeAddr("mallory")); + vm.expectRevert(MigrationVault.NotGuardianOrAdmin.selector); + vault.pause(); + } + + // ---------------------------------------------------------------- attestor rotation + + function test_RemovedAttestorApprovalsStopCounting() public { + approveAs(0, 0, recipient, 5e12); + approveAs(1, 0, recipient, 5e12); + + // Attestor 0 turns out compromised and is removed: its approval must + // no longer count towards the threshold. + vm.prank(admin); + vault.removeAttestor(attestors[0]); + + approveAs(2, 0, recipient, 5e12); + assertEq(pen.balanceOf(recipient), 0, "only 2 active approvals remain"); + + // A replacement attestor is added and completes the quorum. + address replacement = makeAddr("replacement"); + vm.prank(admin); + vault.addAttestor(replacement); + vm.prank(replacement); + vault.approve(0, recipient, 5e12); + assertEq(pen.balanceOf(recipient), 5e18); + } + + function test_ReaddingAttestorNeverCrossesThresholdSilently() public { + // Two approvals, then the first approver is removed and later re-added. + approveAs(0, 0, recipient, 5e12); + approveAs(1, 0, recipient, 5e12); + + vm.startPrank(admin); + vault.removeAttestor(attestors[0]); + vault.addAttestor(attestors[0]); + vm.stopPrank(); + + // The re-add must NOT resurrect the pre-removal approval: crossing the + // threshold outside approve() would bypass pending-release accounting + // and let sweepRemainder strand the migration. + bytes32 payload = vault.payloadHash(0, recipient, 5e12); + assertEq(vault.activeApprovals(payload), 1, "old-generation approval must not count"); + assertFalse(vault.hasApproved(payload, attestors[0])); + + // The re-added attestor approves again (new generation) — allowed, and + // together with a third attestor the release executes through approve(). + approveAs(0, 0, recipient, 5e12); + assertEq(vault.activeApprovals(payload), 2); + approveAs(2, 0, recipient, 5e12); + assertEq(pen.balanceOf(recipient), 5e18); + assertEq(vault.pendingApprovedAmount(), 0); + } + + function test_CannotRemoveAttestorBelowThreshold() public { + vm.startPrank(admin); + vault.removeAttestor(attestors[0]); + vm.expectRevert(MigrationVault.ThresholdWouldExceedAttestors.selector); + vault.removeAttestor(attestors[1]); + vm.stopPrank(); + } + + // ---------------------------------------------------------------- admin & sweep + + function test_AdminFunctionsRejectNonAdmin() public { + vm.startPrank(makeAddr("mallory")); + vm.expectRevert(MigrationVault.NotAdmin.selector); + vault.setCaps(1, 1); + vm.expectRevert(MigrationVault.NotAdmin.selector); + vault.addAttestor(makeAddr("x")); + vm.expectRevert(MigrationVault.NotAdmin.selector); + vault.sweepRemainder(makeAddr("x"), 1); + vm.stopPrank(); + } + + function test_AdminTransferIsTwoStep() public { + address newAdmin = makeAddr("timelock"); + vm.prank(admin); + vault.transferAdmin(newAdmin); + assertEq(vault.admin(), admin, "no effect before acceptance"); + + vm.prank(newAdmin); + vault.acceptAdmin(); + assertEq(vault.admin(), newAdmin); + } + + function test_SweepOnlyAfterEarliestTimestamp() public { + address treasury = makeAddr("treasury"); + vm.prank(admin); + vm.expectRevert(abi.encodeWithSelector(MigrationVault.SweepNotYetAllowed.selector, earliestSweep)); + vault.sweepRemainder(treasury, MAX_ISSUANCE); + + vm.warp(earliestSweep); + vm.prank(admin); + vault.sweepRemainder(treasury, MAX_ISSUANCE); + assertEq(pen.balanceOf(treasury), MAX_ISSUANCE); + assertEq(pen.balanceOf(address(vault)), 0); + assertEq(vault.totalSwept(), MAX_ISSUANCE); + } + + // ---------------------------------------------------------------- pending-release accounting + + function test_SweepExcludesPendingApprovedReleases() public { + // A migration larger than the per-release cap reaches quorum but is + // deferred; its owed amount must survive a remainder sweep. + uint256 palletAmount = 2_000_000e12; // > perReleaseCap after conversion + approveAs(0, 0, recipient, palletAmount); + approveAs(1, 0, recipient, palletAmount); + approveAs(2, 0, recipient, palletAmount); + assertEq(vault.pendingApprovedAmount(), 2_000_000e18); + + address treasury = makeAddr("treasury"); + vm.warp(earliestSweep); + // The pending (owed) amount is not sweepable. + vm.prank(admin); + vm.expectRevert( + abi.encodeWithSelector(MigrationVault.ExceedsSweepable.selector, MAX_ISSUANCE, MAX_ISSUANCE - 2_000_000e18) + ); + vault.sweepRemainder(treasury, MAX_ISSUANCE); + + vm.prank(admin); + vault.sweepRemainder(treasury, MAX_ISSUANCE - 2_000_000e18); + assertEq(pen.balanceOf(treasury), MAX_ISSUANCE - 2_000_000e18); + assertEq(pen.balanceOf(address(vault)), 2_000_000e18, "owed amount stays in the vault"); + + // After governance raises the cap, the deferred release still succeeds. + vm.prank(admin); + vault.setCaps(3_000_000e18, 3_000_000e18); + vault.release(0, recipient, palletAmount); + assertEq(pen.balanceOf(recipient), 2_000_000e18); + assertEq(vault.pendingApprovedAmount(), 0); + } + + function test_PendingAccountingClearsOnRelease() public { + vm.prank(guardian); + vault.pause(); + + approveAs(0, 0, recipient, 5e12); + approveAs(1, 0, recipient, 5e12); + approveAs(2, 0, recipient, 5e12); + assertEq(vault.pendingApprovedAmount(), 5e18, "deferred by pause -> pending"); + + vm.prank(admin); + vault.unpause(); + vault.release(0, recipient, 5e12); + assertEq(vault.pendingApprovedAmount(), 0); + assertFalse(vault.pendingRelease(vault.payloadHash(0, recipient, 5e12))); + } + + function test_ClearStalePendingOnlyForConsumedNonce() public { + address mallory = makeAddr("mallory"); + vm.prank(guardian); + vault.pause(); + + // Both a legitimate and a conflicting tuple for nonce 0 reach quorum + // while paused (attestors may approve two different tuples). + for (uint256 i = 0; i < 3; i++) { + approveAs(i, 0, recipient, 5e12); + approveAs(i, 0, mallory, 5e12); + } + assertEq(vault.pendingApprovedAmount(), 10e18); + + // The stale (unreleased, unconsumed) pending cannot be cleared yet. + vm.prank(admin); + vm.expectRevert(MigrationVault.PendingNotStale.selector); + vault.clearStalePending(0, mallory, 5e12); + + vm.prank(admin); + vault.unpause(); + vault.release(0, recipient, 5e12); + + // Now the conflicting tuple's pending entry is stale and clearable. + vm.prank(admin); + vault.clearStalePending(0, mallory, 5e12); + assertEq(vault.pendingApprovedAmount(), 0); + } + + // A migration that was still gathering approvals when the vault was + // over-swept must NOT crash the attestor fleet, and must stay recoverable. + function test_OverSweptInFlightMigrationDefersAndRecovers() public { + // Bob's migration has 2 of 3 approvals — sub-threshold, so nothing is + // reserved in pendingApprovedAmount yet. + approveAs(0, 42, recipient, 5e12); + approveAs(1, 42, recipient, 5e12); + assertEq(vault.pendingApprovedAmount(), 0); + + // Admin sweeps the entire (unreserved) balance at window close. + address treasury = makeAddr("treasury"); + vm.warp(earliestSweep); + vm.prank(admin); + vault.sweepRemainder(treasury, MAX_ISSUANCE); + assertEq(pen.balanceOf(address(vault)), 0); + + // The 3rd approval crosses the threshold with an empty vault. This must + // NOT revert (which would crash-loop every attestor); it defers instead. + approveAs(2, 42, recipient, 5e12); + assertFalse(vault.nonceConsumed(42)); + assertEq(vault.pendingApprovedAmount(), 5e18, "owed amount now tracked as pending"); + + // A standalone release attempt reverts cleanly (distinct error). + vm.expectRevert(MigrationVault.InsufficientVaultBalance.selector); + vault.release(42, recipient, 5e12); + + // Governance refunds the vault; the release then completes — recoverable. + vm.prank(treasury); + pen.transfer(address(vault), 5e18); + vault.release(42, recipient, 5e12); + assertEq(pen.balanceOf(recipient), 5e18); + assertEq(vault.pendingApprovedAmount(), 0); + } + + // Releasing to the vault itself is a self-transfer that would break the + // monitor's conservation identity (balance unchanged, totalReleased bumped) + // and, with auto-pause on, wedge releases. It must be rejected at approve. + function test_ApproveRejectsVaultRecipient() public { + vm.prank(attestors[0]); + vm.expectRevert(MigrationVault.RecipientIsVault.selector); + vault.approve(0, address(vault), 5e12); + } + + // Guarding at approve() is sufficient: no approvals can ever accrue for a + // vault-recipient tuple, so release() can never satisfy the threshold and + // the conservation identity the monitor watches is preserved. + function test_VaultRecipientNeverReleasesAndPreservesInvariant() public { + for (uint256 i = 0; i < 3; i++) { + vm.prank(attestors[i]); + vm.expectRevert(MigrationVault.RecipientIsVault.selector); + vault.approve(7, address(vault), 5e12); + } + assertFalse(vault.nonceConsumed(7)); + assertEq( + pen.balanceOf(address(vault)) + vault.totalReleased() + vault.totalSwept(), + pen.totalSupply(), + "conservation identity intact" + ); + } + + function test_HasApprovedFalseForRemovedAttestor() public { + bytes32 payload = vault.payloadHash(0, recipient, 5e12); + approveAs(0, 0, recipient, 5e12); + assertTrue(vault.hasApproved(payload, attestors[0])); + + // Removed and never re-added (the standard RB-1 response): hasApproved + // must agree with activeApprovals and report false. + vm.prank(admin); + vault.removeAttestor(attestors[0]); + assertFalse(vault.hasApproved(payload, attestors[0])); + assertEq(vault.activeApprovals(payload), 0); + } + + // Lowering the threshold can retroactively qualify a sub-threshold payload + // outside approve(), so a sweep is blocked for a settling period afterwards + // — giving ops time to release the now-qualifying payload first. + function test_ThresholdCutBlocksSweepDuringSettling() public { + uint256 settle = vault.SWEEP_SETTLING_PERIOD(); + address treasury = makeAddr("treasury"); + + // A payload sits at 2 approvals under threshold 3 — sub-threshold, so + // nothing is reserved in pendingApprovedAmount. + approveAs(0, 5, recipient, 5e12); + approveAs(1, 5, recipient, 5e12); + assertEq(vault.pendingApprovedAmount(), 0); + + // Reach the sweep window, then lower the threshold within it. + vm.warp(earliestSweep); + vm.prank(admin); + vault.setThreshold(2); + uint256 reducedAt = block.timestamp; + + // The sweep is blocked during settling, even though earliestSweep passed. + vm.prank(admin); + vm.expectRevert( + abi.encodeWithSelector(MigrationVault.SweepSettlingAfterThresholdCut.selector, reducedAt + settle) + ); + vault.sweepRemainder(treasury, 1e18); + + // Ops release the now-qualifying payload during settling (permissionless). + vault.release(5, recipient, 5e12); + assertEq(pen.balanceOf(recipient), 5e18); + + // After settling, the sweep proceeds normally. + vm.warp(reducedAt + settle); + vm.prank(admin); + vault.sweepRemainder(treasury, 1e18); + assertEq(pen.balanceOf(treasury), 1e18); + } + + // ---------------------------------------------------------------- fuzz + + function testFuzz_ReleasePreservesSupplyInvariant(uint64 nonce, uint96 palletAmount) public { + palletAmount = uint96(bound(palletAmount, 1, PER_RELEASE_CAP / CONVERSION_FACTOR)); + approveAs(0, nonce, recipient, palletAmount); + approveAs(1, nonce, recipient, palletAmount); + approveAs(2, nonce, recipient, palletAmount); + + assertEq(pen.balanceOf(recipient), uint256(palletAmount) * CONVERSION_FACTOR); + // Conservation incl. the sweep accumulator (monitor's M2b formula). + assertEq( + pen.balanceOf(address(vault)) + vault.totalReleased() + vault.totalSwept(), + pen.totalSupply() + ); + } +} diff --git a/contracts/test/PEN.t.sol b/contracts/test/PEN.t.sol new file mode 100644 index 000000000..5c35db0e4 --- /dev/null +++ b/contracts/test/PEN.t.sol @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-3.0-only +pragma solidity 0.8.26; + +import {Test} from "forge-std/Test.sol"; +import {PEN} from "../src/PEN.sol"; + +contract PENTest is Test { + // Placeholder until PRD decision D3 confirms the exact figure. + uint256 internal constant MAX_ISSUANCE = 150_000_000e18; + + PEN internal pen; + address internal vault = makeAddr("vault"); + address internal alice; + uint256 internal alicePk; + + function setUp() public { + (alice, alicePk) = makeAddrAndKey("alice"); + pen = new PEN(vault, MAX_ISSUANCE); + } + + function test_FullSupplyMintedToVault() public view { + assertEq(pen.totalSupply(), MAX_ISSUANCE); + assertEq(pen.balanceOf(vault), MAX_ISSUANCE); + assertEq(pen.decimals(), 18); + } + + function test_RevertWhen_ZeroVaultOrZeroIssuance() public { + vm.expectRevert(PEN.ZeroVault.selector); + new PEN(address(0), MAX_ISSUANCE); + vm.expectRevert(PEN.ZeroIssuance.selector); + new PEN(vault, 0); + } + + function test_ClockIsTimestampMode() public { + vm.warp(1_900_000_000); + assertEq(pen.clock(), uint48(1_900_000_000)); + assertEq(pen.CLOCK_MODE(), "mode=timestamp"); + } + + function test_PermitSetsAllowance() public { + address spender = makeAddr("spender"); + uint256 value = 123e18; + uint256 deadline = block.timestamp + 1 hours; + + bytes32 structHash = keccak256( + abi.encode( + keccak256("Permit(address owner,address spender,uint256 value,uint256 nonce,uint256 deadline)"), + alice, + spender, + value, + pen.nonces(alice), + deadline + ) + ); + bytes32 digest = keccak256(abi.encodePacked("\x19\x01", pen.DOMAIN_SEPARATOR(), structHash)); + (uint8 v, bytes32 r, bytes32 s) = vm.sign(alicePk, digest); + + pen.permit(alice, spender, value, deadline, v, r, s); + assertEq(pen.allowance(alice, spender), value); + } + + function test_VotesRequireDelegation() public { + vm.prank(vault); + pen.transfer(alice, 1_000e18); + + assertEq(pen.getVotes(alice), 0); + vm.prank(alice); + pen.delegate(alice); + assertEq(pen.getVotes(alice), 1_000e18); + + // Checkpoints are queryable by past timestamp (EIP-6372 timestamp mode). + uint256 before = block.timestamp; + vm.warp(before + 1 days); + assertEq(pen.getPastVotes(alice, before), 1_000e18); + } +} diff --git a/contracts/test/PENGovernor.t.sol b/contracts/test/PENGovernor.t.sol new file mode 100644 index 000000000..9597ee0a0 --- /dev/null +++ b/contracts/test/PENGovernor.t.sol @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: GPL-3.0-only +pragma solidity 0.8.26; + +import {Test} from "forge-std/Test.sol"; +import {TimelockController} from "@openzeppelin/contracts/governance/TimelockController.sol"; +import {IGovernor} from "@openzeppelin/contracts/governance/IGovernor.sol"; +import {IVotes} from "@openzeppelin/contracts/governance/utils/IVotes.sol"; +import {PEN} from "../src/PEN.sol"; +import {PENGovernor} from "../src/PENGovernor.sol"; +import {MigrationVault} from "../src/MigrationVault.sol"; + +contract PENGovernorTest is Test { + uint256 internal constant MAX_ISSUANCE = 150_000_000e18; + uint256 internal constant TIMELOCK_DELAY = 2 days; + uint48 internal constant VOTING_DELAY = 1 days; + uint32 internal constant VOTING_PERIOD = 5 days; + + PEN internal pen; + PENGovernor internal governor; + TimelockController internal timelock; + MigrationVault internal vault; + + address internal alice = makeAddr("alice"); + address internal guardian = makeAddr("guardian"); + + function setUp() public { + // Token held by alice directly so she has voting power without going + // through a migration flow; the vault under governance is separate. + pen = new PEN(alice, MAX_ISSUANCE); + + address[] memory empty = new address[](0); + timelock = new TimelockController(TIMELOCK_DELAY, empty, empty, address(this)); + + governor = new PENGovernor( + IVotes(address(pen)), timelock, VOTING_DELAY, VOTING_PERIOD, 1_000e18, 4 + ); + + timelock.grantRole(timelock.PROPOSER_ROLE(), address(governor)); + timelock.grantRole(timelock.CANCELLER_ROLE(), address(governor)); + timelock.grantRole(timelock.EXECUTOR_ROLE(), address(0)); + timelock.renounceRole(timelock.DEFAULT_ADMIN_ROLE(), address(this)); + + // A vault administered by the timelock, as after governance handover. + address[] memory attestors = new address[](3); + attestors[0] = makeAddr("a0"); + attestors[1] = makeAddr("a1"); + attestors[2] = makeAddr("a2"); + vault = new MigrationVault( + address(timelock), guardian, attestors, 2, 1e6, 1e24, 2e24, block.timestamp + 365 days + ); + + vm.prank(alice); + pen.delegate(alice); + // Advance the clock so the delegation checkpoint is in the past. + vm.warp(block.timestamp + 1); + } + + function test_TimelockIsSelfAdministered() public view { + assertFalse(timelock.hasRole(timelock.DEFAULT_ADMIN_ROLE(), address(this))); + assertTrue(timelock.hasRole(timelock.PROPOSER_ROLE(), address(governor))); + assertTrue(timelock.hasRole(timelock.EXECUTOR_ROLE(), address(0))); + } + + function test_GovernorUsesTimestampClock() public view { + assertEq(governor.clock(), uint48(block.timestamp)); + assertEq(governor.CLOCK_MODE(), "mode=timestamp"); + } + + function test_FullProposalLifecycle_SetVaultCaps() public { + address[] memory targets = new address[](1); + targets[0] = address(vault); + uint256[] memory values = new uint256[](1); + bytes[] memory calldatas = new bytes[](1); + calldatas[0] = abi.encodeCall(MigrationVault.setCaps, (5e24, 9e24)); + string memory description = "Raise migration vault caps"; + + vm.prank(alice); + uint256 proposalId = governor.propose(targets, values, calldatas, description); + + vm.warp(block.timestamp + VOTING_DELAY + 1); + vm.prank(alice); + governor.castVote(proposalId, 1); // For + + vm.warp(block.timestamp + VOTING_PERIOD + 1); + assertEq(uint256(governor.state(proposalId)), uint256(IGovernor.ProposalState.Succeeded)); + + governor.queue(targets, values, calldatas, keccak256(bytes(description))); + vm.warp(block.timestamp + TIMELOCK_DELAY + 1); + governor.execute(targets, values, calldatas, keccak256(bytes(description))); + + assertEq(vault.perReleaseCap(), 5e24); + assertEq(vault.dailyCap(), 9e24); + } + + function test_ProposalBelowThresholdReverts() public { + address pleb = makeAddr("pleb"); + address[] memory targets = new address[](1); + targets[0] = address(vault); + uint256[] memory values = new uint256[](1); + bytes[] memory calldatas = new bytes[](1); + calldatas[0] = abi.encodeCall(MigrationVault.setCaps, (1, 1)); + + vm.prank(pleb); + vm.expectRevert(); + governor.propose(targets, values, calldatas, "no voting power"); + } +} From eb8ba1bd3ab7f48ccb6b28307ac1a74c37fbed4f Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 24 Aug 2026 19:07:30 +0200 Subject: [PATCH 05/61] attestor: Add the attestor daemon Run by each attestor operator. Subscribes to relay-chain-finalized heads on the operator's own Pendulum node -- never a shared or public RPC, so no single faulty node can feed the whole fleet wrong data -- decodes MigrationInitiated events and submits the matching approval to the vault on Base. Blocks are processed strictly in order and the checkpoint advances only once a block is fully handled, so a crash reprocesses at most one block and approvals are idempotent. Losing the race to peers is the normal case and is treated as a benign skip after re-checking on-chain state, never a fatal error. Tuples the vault would deterministically reject are skipped with a critical alert rather than retried, since every attestor would otherwise hit the identical revert and halt the fleet. Verifies its own membership in the attestor set at startup, and alerts on decode failures, low gas and unexpected submission errors. --- attestor/.gitignore | 3 + attestor/README.md | 77 +++ attestor/package-lock.json | 1123 +++++++++++++++++++++++++++++++++++ attestor/package.json | 21 + attestor/src/checks.test.ts | 48 ++ attestor/src/checks.ts | 35 ++ attestor/src/config.ts | 31 + attestor/src/main.ts | 243 ++++++++ attestor/src/vaultAbi.ts | 38 ++ attestor/tsconfig.json | 14 + 10 files changed, 1633 insertions(+) create mode 100644 attestor/.gitignore create mode 100644 attestor/README.md create mode 100644 attestor/package-lock.json create mode 100644 attestor/package.json create mode 100644 attestor/src/checks.test.ts create mode 100644 attestor/src/checks.ts create mode 100644 attestor/src/config.ts create mode 100644 attestor/src/main.ts create mode 100644 attestor/src/vaultAbi.ts create mode 100644 attestor/tsconfig.json diff --git a/attestor/.gitignore b/attestor/.gitignore new file mode 100644 index 000000000..9865bc15b --- /dev/null +++ b/attestor/.gitignore @@ -0,0 +1,3 @@ +node_modules/ +dist/ +checkpoint.json diff --git a/attestor/README.md b/attestor/README.md new file mode 100644 index 000000000..186c3ab7a --- /dev/null +++ b/attestor/README.md @@ -0,0 +1,77 @@ +# PEN Migration Attestor + +Daemon run by each of the four attestor operators (3-of-4, initially +team-operated — PRD §6.4 / D4). +Watches **relay-finalized** blocks on the operator's **own** Pendulum node for +`tokenMigration.MigrationInitiated` events and submits the matching +`approve(nonce, recipient, palletAmount)` transaction to the MigrationVault on +Base. The vault releases the tokens on the 3rd matching approval; attestors +never communicate with each other — the contract is the only coordination +point. + +## Non-negotiable operational rules (PRD A1–A5) + +1. **Run your own Pendulum full node** and point `PENDULUM_WS` at it. Using a + public RPC means trusting that RPC with release authority. +2. **Key isolation:** the attestor key signs only vault `approve` calls. Keep + it in an HSM/KMS signer where possible; never reuse it elsewhere. The same + address pays gas — keep it funded with Base ETH (the daemon alerts below + `MIN_GAS_BALANCE_WEI`). +3. **Separate infrastructure per operator** — different hosting, different + credentials, nothing shared with other attestors or with the monitor. +4. The daemon **exits on any decode or processing error** instead of skipping + events. Run it under a process manager (systemd example below) and page a + human when it restart-loops: a stuck attestor on a runtime upgrade usually + means the metadata changed and the daemon needs updating. + +## Configuration (environment) + +| Variable | Meaning | +|---|---| +| `PENDULUM_WS` | WebSocket of your own Pendulum node, e.g. `ws://127.0.0.1:9944` | +| `BASE_RPC_URL` | Base JSON-RPC endpoint | +| `VAULT_ADDRESS` | MigrationVault address on Base | +| `ATTESTOR_PRIVATE_KEY` | This attestor's signing key (0x-prefixed) | +| `CHECKPOINT_FILE` | Path persisting the last processed block (default `./checkpoint.json`) | +| `START_BLOCK` | First Pendulum block to scan on the very first run | +| `MIN_GAS_BALANCE_WEI` | Low-gas alert threshold (default 0.01 ETH) | +| `ALERT_WEBHOOK_URL` | Optional webhook receiving JSON alerts | +| `BASE_CHAIN_ID` | Default 8453 (Base mainnet) | + +## Run + +```sh +npm install +npm run build +npm start +``` + +### systemd example + +```ini +[Unit] +Description=PEN migration attestor +After=network-online.target + +[Service] +EnvironmentFile=/etc/pen-attestor/env +WorkingDirectory=/opt/pen-attestor +ExecStart=/usr/bin/node dist/main.js +Restart=always +RestartSec=10 + +[Install] +WantedBy=multi-user.target +``` + +## Behavior details + +- Blocks are processed strictly in order; the checkpoint advances only after + every event in a block is handled. A crash re-processes at most one block — + safe, because approvals are idempotent (`nonceConsumed`/`hasApproved` are + checked first, and duplicate submissions revert harmlessly). +- The daemon verifies at startup that its address is in the vault's attestor + set and refuses to run otherwise. +- After a Pendulum **runtime upgrade**, verify event decoding against the new + metadata on a staging node before letting the fleet advance past the + upgrade block (see docs/pen-migration-runbooks.md). diff --git a/attestor/package-lock.json b/attestor/package-lock.json new file mode 100644 index 000000000..e2acb7104 --- /dev/null +++ b/attestor/package-lock.json @@ -0,0 +1,1123 @@ +{ + "name": "pen-migration-attestor", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "pen-migration-attestor", + "version": "0.1.0", + "dependencies": { + "@polkadot/api": "^11.3.1", + "viem": "^2.21.0" + }, + "devDependencies": { + "@types/node": "^20.14.0", + "typescript": "^5.5.0" + } + }, + "node_modules/@adraffy/ens-normalize": { + "version": "1.11.1", + "resolved": "https://registry.npmjs.org/@adraffy/ens-normalize/-/ens-normalize-1.11.1.tgz", + "integrity": "sha512-nhCBV3quEgesuf7c7KYfperqSS14T8bYuvJ8PcLJp6znkZpFc0AuW4qBtr8eKVyPPe/8RSr7sglCWPU5eaxwKQ==", + "license": "MIT" + }, + "node_modules/@noble/ciphers": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@noble/ciphers/-/ciphers-1.3.0.tgz", + "integrity": "sha512-2I0gnIVPtfnMw9ee9h1dJG7tp81+8Ob3OJb3Mv37rx5L40/b0i7djjCVvGOVqc9AEIQyvyu1i6ypKdFw8R8gQw==", + "license": "MIT", + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@noble/curves": { + "version": "1.9.7", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.7.tgz", + "integrity": "sha512-gbKGcRUYIjA3/zCCNaWDciTMFI0dCkvou3TL8Zmy5Nc7sJ47a0jtOeZoTaMxkuqRo9cRhjOdZJXegxYE5FN/xw==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@noble/hashes": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/@noble/hashes/-/hashes-1.8.0.tgz", + "integrity": "sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==", + "license": "MIT", + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@polkadot-api/json-rpc-provider": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/json-rpc-provider/-/json-rpc-provider-0.0.1.tgz", + "integrity": "sha512-/SMC/l7foRjpykLTUTacIH05H3mr9ip8b5xxfwXlVezXrNVLp3Cv0GX6uItkKd+ZjzVPf3PFrDF2B2/HLSNESA==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot-api/json-rpc-provider-proxy": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/json-rpc-provider-proxy/-/json-rpc-provider-proxy-0.0.1.tgz", + "integrity": "sha512-gmVDUP8LpCH0BXewbzqXF2sdHddq1H1q+XrAW2of+KZj4woQkIGBRGTJHeBEVHe30EB+UejR1N2dT4PO/RvDdg==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot-api/metadata-builders": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/metadata-builders/-/metadata-builders-0.0.1.tgz", + "integrity": "sha512-GCI78BHDzXAF/L2pZD6Aod/yl82adqQ7ftNmKg51ixRL02JpWUA+SpUKTJE5MY1p8kiJJIo09P2um24SiJHxNA==", + "license": "MIT", + "optional": true, + "dependencies": { + "@polkadot-api/substrate-bindings": "0.0.1", + "@polkadot-api/utils": "0.0.1" + } + }, + "node_modules/@polkadot-api/observable-client": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/@polkadot-api/observable-client/-/observable-client-0.1.0.tgz", + "integrity": "sha512-GBCGDRztKorTLna/unjl/9SWZcRmvV58o9jwU2Y038VuPXZcr01jcw/1O3x+yeAuwyGzbucI/mLTDa1QoEml3A==", + "license": "MIT", + "optional": true, + "dependencies": { + "@polkadot-api/metadata-builders": "0.0.1", + "@polkadot-api/substrate-bindings": "0.0.1", + "@polkadot-api/substrate-client": "0.0.1", + "@polkadot-api/utils": "0.0.1" + }, + "peerDependencies": { + "rxjs": ">=7.8.0" + } + }, + "node_modules/@polkadot-api/substrate-bindings": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/substrate-bindings/-/substrate-bindings-0.0.1.tgz", + "integrity": "sha512-bAe7a5bOPnuFVmpv7y4BBMRpNTnMmE0jtTqRUw/+D8ZlEHNVEJQGr4wu3QQCl7k1GnSV1wfv3mzIbYjErEBocg==", + "license": "MIT", + "optional": true, + "dependencies": { + "@noble/hashes": "^1.3.1", + "@polkadot-api/utils": "0.0.1", + "@scure/base": "^1.1.1", + "scale-ts": "^1.6.0" + } + }, + "node_modules/@polkadot-api/substrate-client": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/substrate-client/-/substrate-client-0.0.1.tgz", + "integrity": "sha512-9Bg9SGc3AwE+wXONQoW8GC00N3v6lCZLW74HQzqB6ROdcm5VAHM4CB/xRzWSUF9CXL78ugiwtHx3wBcpx4H4Wg==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot-api/utils": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/utils/-/utils-0.0.1.tgz", + "integrity": "sha512-3j+pRmlF9SgiYDabSdZsBSsN5XHbpXOAce1lWj56IEEaFZVjsiCaxDOA7C9nCcgfVXuvnbxqqEGQvnY+QfBAUw==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot/api": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api/-/api-11.3.1.tgz", + "integrity": "sha512-q4kFIIHTLvKxM24b0Eo8hJevsPMme+aITJGrDML9BgdZYTRN14+cu5nXiCsQvaEamdyYj+uCXWe2OV9X7pPxsA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/api-augment": "11.3.1", + "@polkadot/api-base": "11.3.1", + "@polkadot/api-derive": "11.3.1", + "@polkadot/keyring": "^12.6.2", + "@polkadot/rpc-augment": "11.3.1", + "@polkadot/rpc-core": "11.3.1", + "@polkadot/rpc-provider": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-augment": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/types-create": "11.3.1", + "@polkadot/types-known": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "eventemitter3": "^5.0.1", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/api-augment": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api-augment/-/api-augment-11.3.1.tgz", + "integrity": "sha512-Yj+6rb6h0WwY3yJ+UGhjGW+tyMRFUMsKQuGw+eFsXdjiNU9UoXsAqA2dG7Q1F+oeX/g+y2gLGBezNoCwbl6HfA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/api-base": "11.3.1", + "@polkadot/rpc-augment": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-augment": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/api-base": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api-base/-/api-base-11.3.1.tgz", + "integrity": "sha512-b8UkNL00NN7+3QaLCwL5cKg+7YchHoKCAhwKusWHNBZkkO6Oo2BWilu0dZkPJOyqV9P389Kbd9+oH+SKs9u2VQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/rpc-core": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/util": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/api-derive": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api-derive/-/api-derive-11.3.1.tgz", + "integrity": "sha512-9dopzrh4cRuft1nANmBvMY/hEhFDu0VICMTOGxQLOl8NMfcOFPTLAN0JhSBUoicGZhV+c4vpv01NBx/7/IL1HA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/api": "11.3.1", + "@polkadot/api-augment": "11.3.1", + "@polkadot/api-base": "11.3.1", + "@polkadot/rpc-core": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/keyring": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/keyring/-/keyring-12.6.2.tgz", + "integrity": "sha512-O3Q7GVmRYm8q7HuB3S0+Yf/q/EB2egKRRU3fv9b3B7V+A52tKzA+vIwEmNVaD1g5FKW9oB97rmpggs0zaKFqHw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "12.6.2", + "@polkadot/util-crypto": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "12.6.2", + "@polkadot/util-crypto": "12.6.2" + } + }, + "node_modules/@polkadot/networks": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/networks/-/networks-12.6.2.tgz", + "integrity": "sha512-1oWtZm1IvPWqvMrldVH6NI2gBoCndl5GEwx7lAuQWGr7eNL+6Bdc5K3Z9T0MzFvDGoi2/CBqjX9dRKo39pDC/w==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "12.6.2", + "@substrate/ss58-registry": "^1.44.0", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/rpc-augment": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/rpc-augment/-/rpc-augment-11.3.1.tgz", + "integrity": "sha512-2PaDcKNju4QYQpxwVkWbRU3M0t340nMX9cMo+8awgvgL1LliV/fUDZueMKLuSS910JJMTPQ7y2pK4eQgMt08gQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/rpc-core": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/rpc-core": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/rpc-core/-/rpc-core-11.3.1.tgz", + "integrity": "sha512-KKNepsDd/mpmXcA6v/h14eFFPEzLGd7nrvx2UUXUxoZ0Fq2MH1hplP3s93k1oduNY/vOXJR2K9S4dKManA6GVQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/rpc-augment": "11.3.1", + "@polkadot/rpc-provider": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/util": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/rpc-provider": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/rpc-provider/-/rpc-provider-11.3.1.tgz", + "integrity": "sha512-pqERChoHo45hd3WAgW8UuzarRF+G/o/eXEbl0PXLubiayw4X4qCmIzmtntUcKYgxGNcYGZaG87ZU8OjN97m6UA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/keyring": "^12.6.2", + "@polkadot/types": "11.3.1", + "@polkadot/types-support": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "@polkadot/x-fetch": "^12.6.2", + "@polkadot/x-global": "^12.6.2", + "@polkadot/x-ws": "^12.6.2", + "eventemitter3": "^5.0.1", + "mock-socket": "^9.3.1", + "nock": "^13.5.0", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@substrate/connect": "0.8.10" + } + }, + "node_modules/@polkadot/types": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types/-/types-11.3.1.tgz", + "integrity": "sha512-5c7uRFXQTT11Awi6T0yFIdAfD6xGDAOz06Kp7M5S9OGNZY28wSPk5x6BYfNphWPaIBmHHewYJB5qmnrdYQAWKQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/keyring": "^12.6.2", + "@polkadot/types-augment": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/types-create": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-augment": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-augment/-/types-augment-11.3.1.tgz", + "integrity": "sha512-eR3HVpvUmB3v7q2jTWVmVfAVfb1/kuNn7ij94Zqadg/fuUq0pKqIOKwkUj3OxRM3A/5BnW3MbgparjKD3r+fyw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-codec": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-codec/-/types-codec-11.3.1.tgz", + "integrity": "sha512-i7IiiuuL+Z/jFoKTA9xeh4wGQnhnNNjMT0+1ohvlOvnFsoKZKFQQOaDPPntGJVL1JDCV+KjkN2uQKZSeW8tguQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "^12.6.2", + "@polkadot/x-bigint": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-create": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-create/-/types-create-11.3.1.tgz", + "integrity": "sha512-pBXtpz5FehcRJ6j5MzFUIUN8ZWM7z6HbqK1GxBmYbJVRElcGcOg7a/rL2pQVphU0Rx1E8bSO4thzGf4wUxSX7w==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-known": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-known/-/types-known-11.3.1.tgz", + "integrity": "sha512-3BIof7u6tn9bk3ZCIxA07iNoQ3uj4+vn3DTOjCKECozkRlt6V+kWRvqh16Hc0SHMg/QjcMb2fIu/WZhka1McUQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/networks": "^12.6.2", + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/types-create": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-support": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-support/-/types-support-11.3.1.tgz", + "integrity": "sha512-jTFz1GKyF7nI29yIOq4v0NiWTOf5yX4HahJNeFD8TcxoLhF+6tH/XXqrUXJEfbaTlSrRWiW1LZYlb+snctqKHA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/util": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/util/-/util-12.6.2.tgz", + "integrity": "sha512-l8TubR7CLEY47240uki0TQzFvtnxFIO7uI/0GoWzpYD/O62EIAMRsuY01N4DuwgKq2ZWD59WhzsLYmA5K6ksdw==", + "license": "Apache-2.0", + "peer": true, + "dependencies": { + "@polkadot/x-bigint": "12.6.2", + "@polkadot/x-global": "12.6.2", + "@polkadot/x-textdecoder": "12.6.2", + "@polkadot/x-textencoder": "12.6.2", + "@types/bn.js": "^5.1.5", + "bn.js": "^5.2.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/util-crypto": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/util-crypto/-/util-crypto-12.6.2.tgz", + "integrity": "sha512-FEWI/dJ7wDMNN1WOzZAjQoIcCP/3vz3wvAp5QQm+lOrzOLj0iDmaIGIcBkz8HVm3ErfSe/uKP0KS4jgV/ib+Mg==", + "license": "Apache-2.0", + "dependencies": { + "@noble/curves": "^1.3.0", + "@noble/hashes": "^1.3.3", + "@polkadot/networks": "12.6.2", + "@polkadot/util": "12.6.2", + "@polkadot/wasm-crypto": "^7.3.2", + "@polkadot/wasm-util": "^7.3.2", + "@polkadot/x-bigint": "12.6.2", + "@polkadot/x-randomvalues": "12.6.2", + "@scure/base": "^1.1.5", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "12.6.2" + } + }, + "node_modules/@polkadot/wasm-bridge": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-bridge/-/wasm-bridge-7.5.4.tgz", + "integrity": "sha512-6xaJVvoZbnbgpQYXNw9OHVNWjXmtcoPcWh7hlwx3NpfiLkkjljj99YS+XGZQlq7ks2fVCg7FbfknkNb8PldDaA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*", + "@polkadot/x-randomvalues": "*" + } + }, + "node_modules/@polkadot/wasm-crypto": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto/-/wasm-crypto-7.5.4.tgz", + "integrity": "sha512-1seyClxa7Jd7kQjfnCzTTTfYhTa/KUTDUaD3DMHBk5Q4ZUN1D1unJgX+v1aUeXSPxmzocdZETPJJRZjhVOqg9g==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-bridge": "7.5.4", + "@polkadot/wasm-crypto-asmjs": "7.5.4", + "@polkadot/wasm-crypto-init": "7.5.4", + "@polkadot/wasm-crypto-wasm": "7.5.4", + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*", + "@polkadot/x-randomvalues": "*" + } + }, + "node_modules/@polkadot/wasm-crypto-asmjs": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto-asmjs/-/wasm-crypto-asmjs-7.5.4.tgz", + "integrity": "sha512-ZYwxQHAJ8pPt6kYk9XFmyuFuSS+yirJLonvP+DYbxOrARRUHfN4nzp4zcZNXUuaFhpbDobDSFn6gYzye6BUotA==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*" + } + }, + "node_modules/@polkadot/wasm-crypto-init": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto-init/-/wasm-crypto-init-7.5.4.tgz", + "integrity": "sha512-U6s4Eo2rHs2n1iR01vTz/sOQ7eOnRPjaCsGWhPV+ZC/20hkVzwPAhiizu/IqMEol4tO2yiSheD4D6bn0KxUJhg==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-bridge": "7.5.4", + "@polkadot/wasm-crypto-asmjs": "7.5.4", + "@polkadot/wasm-crypto-wasm": "7.5.4", + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*", + "@polkadot/x-randomvalues": "*" + } + }, + "node_modules/@polkadot/wasm-crypto-wasm": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto-wasm/-/wasm-crypto-wasm-7.5.4.tgz", + "integrity": "sha512-PsHgLsVTu43eprwSvUGnxybtOEuHPES6AbApcs7y5ZbM2PiDMzYbAjNul098xJK/CPtrxZ0ePDFnaQBmIJyTFw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*" + } + }, + "node_modules/@polkadot/wasm-util": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-util/-/wasm-util-7.5.4.tgz", + "integrity": "sha512-hqPpfhCpRAqCIn/CYbBluhh0TXmwkJnDRjxrU9Bnqtw9nMNa97D8JuOjdd2pi0rxm+eeLQ/f1rQMp71RMM9t4w==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*" + } + }, + "node_modules/@polkadot/x-bigint": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-bigint/-/x-bigint-12.6.2.tgz", + "integrity": "sha512-HSIk60uFPX4GOFZSnIF7VYJz7WZA7tpFJsne7SzxOooRwMTWEtw3fUpFy5cYYOeLh17/kHH1Y7SVcuxzVLc74Q==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-fetch": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-fetch/-/x-fetch-12.6.2.tgz", + "integrity": "sha512-8wM/Z9JJPWN1pzSpU7XxTI1ldj/AfC8hKioBlUahZ8gUiJaOF7K9XEFCrCDLis/A1BoOu7Ne6WMx/vsJJIbDWw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "node-fetch": "^3.3.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-global": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-global/-/x-global-12.6.2.tgz", + "integrity": "sha512-a8d6m+PW98jmsYDtAWp88qS4dl8DyqUBsd0S+WgyfSMtpEXu6v9nXDgPZgwF5xdDvXhm+P0ZfVkVTnIGrScb5g==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-randomvalues": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-randomvalues/-/x-randomvalues-12.6.2.tgz", + "integrity": "sha512-Vr8uG7rH2IcNJwtyf5ebdODMcr0XjoCpUbI91Zv6AlKVYOGKZlKLYJHIwpTaKKB+7KPWyQrk4Mlym/rS7v9feg==", + "license": "Apache-2.0", + "peer": true, + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "12.6.2", + "@polkadot/wasm-util": "*" + } + }, + "node_modules/@polkadot/x-textdecoder": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-textdecoder/-/x-textdecoder-12.6.2.tgz", + "integrity": "sha512-M1Bir7tYvNappfpFWXOJcnxUhBUFWkUFIdJSyH0zs5LmFtFdbKAeiDXxSp2Swp5ddOZdZgPac294/o2TnQKN1w==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-textencoder": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-textencoder/-/x-textencoder-12.6.2.tgz", + "integrity": "sha512-4N+3UVCpI489tUJ6cv3uf0PjOHvgGp9Dl+SZRLgFGt9mvxnvpW/7+XBADRMtlG4xi5gaRK7bgl5bmY6OMDsNdw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-ws": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-ws/-/x-ws-12.6.2.tgz", + "integrity": "sha512-cGZWo7K5eRRQCRl2LrcyCYsrc3lRbTlixZh3AzgU8uX4wASVGRlNWi/Hf4TtHNe1ExCDmxabJzdIsABIfrr7xw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2", + "ws": "^8.15.1" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@scure/base": { + "version": "1.2.6", + "resolved": "https://registry.npmjs.org/@scure/base/-/base-1.2.6.tgz", + "integrity": "sha512-g/nm5FgUa//MCj1gV09zTJTaM6KBAHqLN907YVQqf7zC49+DcO4B1so4ZX07Ef10Twr6nuqYEH9GEggFXA4Fmg==", + "license": "MIT", + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/bip32": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/@scure/bip32/-/bip32-1.7.0.tgz", + "integrity": "sha512-E4FFX/N3f4B80AKWp5dP6ow+flD1LQZo/w8UnLGYZO674jS6YnYeepycOOksv+vLPSpgN35wgKgy+ybfTb2SMw==", + "license": "MIT", + "dependencies": { + "@noble/curves": "~1.9.0", + "@noble/hashes": "~1.8.0", + "@scure/base": "~1.2.5" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/bip39": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/@scure/bip39/-/bip39-1.6.0.tgz", + "integrity": "sha512-+lF0BbLiJNwVlev4eKelw1WWLaiKXw7sSl8T6FvBlWkdX+94aGJ4o8XjUdlyhTCjd8c+B3KT3JfS8P0bLRNU6A==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "~1.8.0", + "@scure/base": "~1.2.5" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@substrate/connect": { + "version": "0.8.10", + "resolved": "https://registry.npmjs.org/@substrate/connect/-/connect-0.8.10.tgz", + "integrity": "sha512-DIyQ13DDlXqVFnLV+S6/JDgiGowVRRrh18kahieJxhgvzcWicw5eLc6jpfQ0moVVLBYkO7rctB5Wreldwpva8w==", + "deprecated": "versions below 1.x are no longer maintained", + "license": "GPL-3.0-only", + "optional": true, + "dependencies": { + "@substrate/connect-extension-protocol": "^2.0.0", + "@substrate/connect-known-chains": "^1.1.4", + "@substrate/light-client-extension-helpers": "^0.0.6", + "smoldot": "2.0.22" + } + }, + "node_modules/@substrate/connect-extension-protocol": { + "version": "2.2.2", + "resolved": "https://registry.npmjs.org/@substrate/connect-extension-protocol/-/connect-extension-protocol-2.2.2.tgz", + "integrity": "sha512-t66jwrXA0s5Goq82ZtjagLNd7DPGCNjHeehRlE/gcJmJ+G56C0W+2plqOMRicJ8XGR1/YFnUSEqUFiSNbjGrAA==", + "license": "GPL-3.0-only", + "optional": true + }, + "node_modules/@substrate/connect-known-chains": { + "version": "1.10.3", + "resolved": "https://registry.npmjs.org/@substrate/connect-known-chains/-/connect-known-chains-1.10.3.tgz", + "integrity": "sha512-OJEZO1Pagtb6bNE3wCikc2wrmvEU5x7GxFFLqqbz1AJYYxSlrPCGu4N2og5YTExo4IcloNMQYFRkBGue0BKZ4w==", + "license": "GPL-3.0-only", + "optional": true + }, + "node_modules/@substrate/light-client-extension-helpers": { + "version": "0.0.6", + "resolved": "https://registry.npmjs.org/@substrate/light-client-extension-helpers/-/light-client-extension-helpers-0.0.6.tgz", + "integrity": "sha512-girltEuxQ1BvkJWmc8JJlk4ZxnlGXc/wkLcNguhY+UoDEMBK0LsdtfzQKIfrIehi4QdeSBlFEFBoI4RqPmsZzA==", + "license": "MIT", + "optional": true, + "dependencies": { + "@polkadot-api/json-rpc-provider": "0.0.1", + "@polkadot-api/json-rpc-provider-proxy": "0.0.1", + "@polkadot-api/observable-client": "0.1.0", + "@polkadot-api/substrate-client": "0.0.1", + "@substrate/connect-extension-protocol": "^2.0.0", + "@substrate/connect-known-chains": "^1.1.4", + "rxjs": "^7.8.1" + }, + "peerDependencies": { + "smoldot": "2.x" + } + }, + "node_modules/@substrate/ss58-registry": { + "version": "1.51.0", + "resolved": "https://registry.npmjs.org/@substrate/ss58-registry/-/ss58-registry-1.51.0.tgz", + "integrity": "sha512-TWDurLiPxndFgKjVavCniytBIw+t4ViOi7TYp9h/D0NMmkEc9klFTo+827eyEJ0lELpqO207Ey7uGxUa+BS1jQ==", + "license": "Apache-2.0" + }, + "node_modules/@types/bn.js": { + "version": "5.2.0", + "resolved": "https://registry.npmjs.org/@types/bn.js/-/bn.js-5.2.0.tgz", + "integrity": "sha512-DLbJ1BPqxvQhIGbeu8VbUC1DiAiahHtAYvA0ZEAa4P31F7IaArc8z3C3BRQdWX4mtLQuABG4yzp76ZrS02Ui1Q==", + "license": "MIT", + "dependencies": { + "@types/node": "*" + } + }, + "node_modules/@types/node": { + "version": "20.19.43", + "resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.43.tgz", + "integrity": "sha512-6oYBAi5ikg4Pl+kGsoYtawUMBT2zZMCvPNF7pVLnHZfd1zf38DRiWn/gT01RYCdUqkv7Fhr+C9ot4/tb+2sVvA==", + "license": "MIT", + "dependencies": { + "undici-types": "~6.21.0" + } + }, + "node_modules/abitype": { + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/abitype/-/abitype-1.2.3.tgz", + "integrity": "sha512-Ofer5QUnuUdTFsBRwARMoWKOH1ND5ehwYhJ3OJ/BQO+StkwQjHw0XyVh4vDttzHB7QOFhPHa/o413PJ82gU/Tg==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/wevm" + }, + "peerDependencies": { + "typescript": ">=5.0.4", + "zod": "^3.22.0 || ^4.0.0" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + }, + "zod": { + "optional": true + } + } + }, + "node_modules/bn.js": { + "version": "5.2.4", + "resolved": "https://registry.npmjs.org/bn.js/-/bn.js-5.2.4.tgz", + "integrity": "sha512-QL7sb18rJ1PbdsKsqPA0guxL563vIMwRHgzNrW/uzQuRGN1Cjqd/wonUBAVqHox9KwzHA6vCbM0lXx3k4iQMow==", + "license": "MIT" + }, + "node_modules/data-uri-to-buffer": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.1.tgz", + "integrity": "sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==", + "license": "MIT", + "engines": { + "node": ">= 12" + } + }, + "node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/eventemitter3": { + "version": "5.0.4", + "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz", + "integrity": "sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==", + "license": "MIT" + }, + "node_modules/fetch-blob": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.2.0.tgz", + "integrity": "sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "paypal", + "url": "https://paypal.me/jimmywarting" + } + ], + "license": "MIT", + "dependencies": { + "node-domexception": "^1.0.0", + "web-streams-polyfill": "^3.0.3" + }, + "engines": { + "node": "^12.20 || >= 14.13" + } + }, + "node_modules/formdata-polyfill": { + "version": "4.0.10", + "resolved": "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz", + "integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==", + "license": "MIT", + "dependencies": { + "fetch-blob": "^3.1.2" + }, + "engines": { + "node": ">=12.20.0" + } + }, + "node_modules/isows": { + "version": "1.0.7", + "resolved": "https://registry.npmjs.org/isows/-/isows-1.0.7.tgz", + "integrity": "sha512-I1fSfDCZL5P0v33sVqeTDSpcstAg/N+wF5HS033mogOVIp4B+oHC7oOCsA3axAbBSGTJ8QubbNmnIRN/h8U7hg==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "peerDependencies": { + "ws": "*" + } + }, + "node_modules/json-stringify-safe": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz", + "integrity": "sha512-ZClg6AaYvamvYEE82d3Iyd3vSSIjQ+odgjaTzRuO3s7toCdFKczob2i0zCh7JE8kWn17yvAWhUVxvqGwUalsRA==", + "license": "ISC" + }, + "node_modules/mock-socket": { + "version": "9.3.1", + "resolved": "https://registry.npmjs.org/mock-socket/-/mock-socket-9.3.1.tgz", + "integrity": "sha512-qxBgB7Qa2sEQgHFjj0dSigq7fX4k6Saisd5Nelwp2q8mlbAFh5dHV9JTTlF8viYJLSSWgMCZFUom8PJcMNBoJw==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "license": "MIT" + }, + "node_modules/nock": { + "version": "13.5.6", + "resolved": "https://registry.npmjs.org/nock/-/nock-13.5.6.tgz", + "integrity": "sha512-o2zOYiCpzRqSzPj0Zt/dQ/DqZeYoaQ7TUonc/xUPjCGl9WeHpNbxgVvOquXYAaJzI0M9BXV3HTzG0p8IUAbBTQ==", + "license": "MIT", + "dependencies": { + "debug": "^4.1.0", + "json-stringify-safe": "^5.0.1", + "propagate": "^2.0.0" + }, + "engines": { + "node": ">= 10.13" + } + }, + "node_modules/node-domexception": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz", + "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==", + "deprecated": "Use your platform's native DOMException instead", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "github", + "url": "https://paypal.me/jimmywarting" + } + ], + "license": "MIT", + "engines": { + "node": ">=10.5.0" + } + }, + "node_modules/node-fetch": { + "version": "3.3.2", + "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.2.tgz", + "integrity": "sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==", + "license": "MIT", + "dependencies": { + "data-uri-to-buffer": "^4.0.0", + "fetch-blob": "^3.1.4", + "formdata-polyfill": "^4.0.10" + }, + "engines": { + "node": "^12.20.0 || ^14.13.1 || >=16.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/node-fetch" + } + }, + "node_modules/ox": { + "version": "0.14.30", + "resolved": "https://registry.npmjs.org/ox/-/ox-0.14.30.tgz", + "integrity": "sha512-LI11uu+8iiM1B3CLckgd++YF1a0A2k5wDoM9ZeQMiL21BOzQs6L//BLS6hb1HSEKCyycdDIQLsVQx9MjpcC0hA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "dependencies": { + "@adraffy/ens-normalize": "^1.11.0", + "@noble/ciphers": "^1.3.0", + "@noble/curves": "1.9.1", + "@noble/hashes": "^1.8.0", + "@scure/bip32": "^1.7.0", + "@scure/bip39": "^1.6.0", + "abitype": "^1.2.3", + "eventemitter3": "5.0.1" + }, + "peerDependencies": { + "typescript": ">=5.4.0" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + } + } + }, + "node_modules/ox/node_modules/@noble/curves": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.1.tgz", + "integrity": "sha512-k11yZxZg+t+gWvBbIswW0yoJlu8cHOC7dhunwOzoWH/mXGBiYyR4YY6hAEK/3EUs4UpB8la1RfdRpeGsFHkWsA==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/ox/node_modules/eventemitter3": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.1.tgz", + "integrity": "sha512-GWkBvjiSZK87ELrYOSESUYeVIc9mvLLf/nXalMOS5dYrgZq9o5OVkbZAVM06CVxYsCwH9BDZFPlQTlPA1j4ahA==", + "license": "MIT" + }, + "node_modules/propagate": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/propagate/-/propagate-2.0.1.tgz", + "integrity": "sha512-vGrhOavPSTz4QVNuBNdcNXePNdNMaO1xj9yBeH1ScQPjk/rhg9sSlCXPhMkFuaNNW/syTvYqsnbIJxMBfRbbag==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/rxjs": { + "version": "7.8.2", + "resolved": "https://registry.npmjs.org/rxjs/-/rxjs-7.8.2.tgz", + "integrity": "sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA==", + "license": "Apache-2.0", + "peer": true, + "dependencies": { + "tslib": "^2.1.0" + } + }, + "node_modules/scale-ts": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/scale-ts/-/scale-ts-1.6.1.tgz", + "integrity": "sha512-PBMc2AWc6wSEqJYBDPcyCLUj9/tMKnLX70jLOSndMtcUoLQucP/DM0vnQo1wJAYjTrQiq8iG9rD0q6wFzgjH7g==", + "license": "MIT", + "optional": true + }, + "node_modules/smoldot": { + "version": "2.0.22", + "resolved": "https://registry.npmjs.org/smoldot/-/smoldot-2.0.22.tgz", + "integrity": "sha512-B50vRgTY6v3baYH6uCgL15tfaag5tcS2o/P5q1OiXcKGv1axZDfz2dzzMuIkVpyMR2ug11F6EAtQlmYBQd292g==", + "license": "GPL-3.0-or-later WITH Classpath-exception-2.0", + "optional": true, + "peer": true, + "dependencies": { + "ws": "^8.8.1" + } + }, + "node_modules/tslib": { + "version": "2.8.1", + "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", + "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", + "license": "0BSD" + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "devOptional": true, + "license": "Apache-2.0", + "peer": true, + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + }, + "node_modules/undici-types": { + "version": "6.21.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", + "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", + "license": "MIT" + }, + "node_modules/viem": { + "version": "2.54.6", + "resolved": "https://registry.npmjs.org/viem/-/viem-2.54.6.tgz", + "integrity": "sha512-OfybECKJYVmhiNqz+SHhed+O2h6niQ+0Wjg9J0b4bV+/QrvLgjxhfKO7hZqsuK1YtZ/0BErBKy708Zp+cU5T0Q==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "dependencies": { + "@noble/curves": "1.9.1", + "@noble/hashes": "1.8.0", + "@scure/bip32": "1.7.0", + "@scure/bip39": "1.6.0", + "abitype": "1.2.3", + "isows": "1.0.7", + "ox": "0.14.30", + "ws": "8.21.0" + }, + "peerDependencies": { + "typescript": ">=5.0.4" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + } + } + }, + "node_modules/viem/node_modules/@noble/curves": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.1.tgz", + "integrity": "sha512-k11yZxZg+t+gWvBbIswW0yoJlu8cHOC7dhunwOzoWH/mXGBiYyR4YY6hAEK/3EUs4UpB8la1RfdRpeGsFHkWsA==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/web-streams-polyfill": { + "version": "3.3.3", + "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz", + "integrity": "sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/ws": { + "version": "8.21.0", + "resolved": "https://registry.npmjs.org/ws/-/ws-8.21.0.tgz", + "integrity": "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==", + "license": "MIT", + "peer": true, + "engines": { + "node": ">=10.0.0" + }, + "peerDependencies": { + "bufferutil": "^4.0.1", + "utf-8-validate": ">=5.0.2" + }, + "peerDependenciesMeta": { + "bufferutil": { + "optional": true + }, + "utf-8-validate": { + "optional": true + } + } + } + } +} diff --git a/attestor/package.json b/attestor/package.json new file mode 100644 index 000000000..cd1f06d9a --- /dev/null +++ b/attestor/package.json @@ -0,0 +1,21 @@ +{ + "name": "pen-migration-attestor", + "version": "0.1.0", + "private": true, + "description": "Attestor daemon for the PEN migration: watches finalized MigrationInitiated events on Pendulum and submits on-chain approvals to the MigrationVault on Base", + "type": "module", + "scripts": { + "build": "tsc", + "start": "node dist/main.js", + "typecheck": "tsc --noEmit", + "test": "tsc && node --test dist/checks.test.js" + }, + "dependencies": { + "@polkadot/api": "^11.3.1", + "viem": "^2.21.0" + }, + "devDependencies": { + "@types/node": "^20.14.0", + "typescript": "^5.5.0" + } +} diff --git a/attestor/src/checks.test.ts b/attestor/src/checks.test.ts new file mode 100644 index 000000000..6068d09e9 --- /dev/null +++ b/attestor/src/checks.test.ts @@ -0,0 +1,48 @@ +/** + * Unit tests for the attestor's tuple-classification predicate. + * + * Run with `npm test` (compiles, then `node --test`). These lock in the round-6 + * fix: `isUnreleasable` must flag BOTH the zero address and the vault's own + * address (case-insensitively), so a `migrate(_, )` event is + * skipped with a critical alert instead of crash-looping the whole attestor + * fleet on the vault's `RecipientIsVault` revert. + */ + +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { isUnreleasable, ZERO_ADDRESS } from "./checks.js"; + +const VAULT = "0x1111111111111111111111111111111111111111"; +const NORMAL = "0x00000000000000000000000000000000deadbeef"; +const AMOUNT = 5_000_000_000_000n; // 5 PEN in 12-decimal pallet units + +test("a normal recipient with a non-zero amount is releasable", () => { + assert.equal(isUnreleasable(NORMAL, AMOUNT, VAULT), false); +}); + +test("the zero address is unreleasable (vault reverts ZeroAddress)", () => { + assert.equal(isUnreleasable(ZERO_ADDRESS, AMOUNT, VAULT), true); +}); + +test("the vault's own address is unreleasable (vault reverts RecipientIsVault)", () => { + // The round-6 finding: without this, a 1-PEN migration to the vault address + // crash-loops the entire fleet, since the pallet cannot reject it. + assert.equal(isUnreleasable(VAULT, AMOUNT, VAULT), true); +}); + +test("the vault address is matched case-insensitively", () => { + // The event decodes the H160 lower-cased; the configured vault address may + // be EIP-55 checksummed. The comparison must not depend on casing either way. + const vaultChecksummed = "0xAbCdEf0000000000000000000000000000000001"; + assert.equal(isUnreleasable(vaultChecksummed.toLowerCase(), AMOUNT, vaultChecksummed), true); + assert.equal(isUnreleasable(vaultChecksummed, AMOUNT, vaultChecksummed.toLowerCase()), true); +}); + +test("a zero amount is unreleasable (vault reverts ZeroAmount)", () => { + assert.equal(isUnreleasable(NORMAL, 0n, VAULT), true); +}); + +test("an unreleasable condition still wins when combined with a normal one", () => { + assert.equal(isUnreleasable(ZERO_ADDRESS, 0n, VAULT), true); + assert.equal(isUnreleasable(VAULT, 0n, VAULT), true); +}); diff --git a/attestor/src/checks.ts b/attestor/src/checks.ts new file mode 100644 index 000000000..3f5bceb13 --- /dev/null +++ b/attestor/src/checks.ts @@ -0,0 +1,35 @@ +/** + * Pure tuple-classification predicates for the attestor. + * + * Isolated from the chain-client plumbing in `main.ts` so they can be unit + * tested exhaustively (see checks.test.ts). The subtlety that has bitten a + * review round — which (nonce, recipient, amount) tuples the vault will + * *deterministically* reject — lives here, and must stay in exact lockstep with + * the input reverts at the top of `MigrationVault.approve()`. + */ + +export const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000"; + +/** + * True when the vault will deterministically reject this tuple, no matter who + * submits it or when. Such an event must be SKIPPED (with a critical alert), + * never retried: because every attestor hits the identical revert at the same + * finalized block, crash-looping on it would halt the entire fleet and block + * every migration behind it (the round-2 zero-address DoS class). + * + * `MigrationVault.approve` reverts up-front on exactly three input conditions: + * - a zero recipient (`ZeroAddress`) + * - the vault's own address (`RecipientIsVault` — a self-transfer would + * break the monitor's conservation identity) + * - a zero amount (`ZeroAmount`) + * + * The pallet rejects the zero address and sub-minimum amounts before an event + * is ever emitted, but it CANNOT know the vault's Base address (it has no + * knowledge of Base state), so a `migrate(_, )` reaches the + * attestor as a well-formed event. This gate is therefore the ONLY line of + * defence against a vault-recipient event bricking the fleet. + */ +export function isUnreleasable(recipient: string, palletAmount: bigint, vaultAddress: string): boolean { + const to = recipient.toLowerCase(); + return to === ZERO_ADDRESS || to === vaultAddress.toLowerCase() || palletAmount === 0n; +} diff --git a/attestor/src/config.ts b/attestor/src/config.ts new file mode 100644 index 000000000..deb81d6b1 --- /dev/null +++ b/attestor/src/config.ts @@ -0,0 +1,31 @@ +function required(name: string): string { + const value = process.env[name]; + if (!value) { + throw new Error(`Missing required environment variable ${name}`); + } + return value; +} + +export const config = { + /** WebSocket endpoint of THIS OPERATOR'S OWN Pendulum full node (PRD A1). + * Never point this at a public RPC: the attestor would inherit its honesty. */ + pendulumWs: required("PENDULUM_WS"), + /** Base JSON-RPC endpoint. */ + baseRpcUrl: required("BASE_RPC_URL"), + /** MigrationVault contract address on Base. */ + vaultAddress: required("VAULT_ADDRESS") as `0x${string}`, + /** This attestor's transaction-signing key (0x-prefixed, 32 bytes). + * Isolate per operator; fund with Base ETH for gas (PRD A3). */ + attestorPrivateKey: required("ATTESTOR_PRIVATE_KEY") as `0x${string}`, + /** File persisting the last fully processed finalized block (PRD A2). */ + checkpointFile: process.env.CHECKPOINT_FILE ?? "./checkpoint.json", + /** Pendulum block to start from on the very first run (the block of the + * runtime upgrade that added the token-migration pallet). */ + startBlock: Number(process.env.START_BLOCK ?? "0"), + /** Alert when the gas wallet drops below this balance (wei). */ + minGasBalanceWei: BigInt(process.env.MIN_GAS_BALANCE_WEI ?? "10000000000000000"), // 0.01 ETH + /** Optional webhook that receives JSON alerts (low gas, fatal errors). */ + alertWebhookUrl: process.env.ALERT_WEBHOOK_URL, + /** Base chain id: 8453 mainnet. */ + baseChainId: Number(process.env.BASE_CHAIN_ID ?? "8453"), +}; diff --git a/attestor/src/main.ts b/attestor/src/main.ts new file mode 100644 index 000000000..ca9c8a9bf --- /dev/null +++ b/attestor/src/main.ts @@ -0,0 +1,243 @@ +/** + * PEN migration attestor daemon (PRD §6.4). + * + * Watches RELAY-FINALIZED blocks on the operator's own Pendulum node for + * `tokenMigration.MigrationInitiated` events and submits the matching + * `approve(nonce, recipient, palletAmount)` transaction to the MigrationVault + * on Base. The vault releases the tokens on the threshold-th approval. + * + * Design invariants: + * - Only finalized blocks are read; blocks are processed strictly in order. + * - The checkpoint file is advanced only after every event in a block has + * been handled, so a crash re-processes at most one block (idempotent: + * duplicate approvals revert harmlessly and are skipped by the pre-check). + * - A decode failure is FATAL by design (PRD A5): the daemon alerts and + * exits rather than silently skipping an event; the checkpoint keeps the + * failing block next in line for after the operator intervenes. + */ + +import { readFileSync, writeFileSync } from "node:fs"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { + createPublicClient, + createWalletClient, + defineChain, + encodeAbiParameters, + http, + keccak256, +} from "viem"; +import { privateKeyToAccount } from "viem/accounts"; +import { isUnreleasable } from "./checks.js"; +import { config } from "./config.js"; +import { vaultAbi } from "./vaultAbi.js"; + +interface Checkpoint { + lastProcessedBlock: number; +} + +interface MigrationEvent { + nonce: bigint; + recipient: `0x${string}`; + palletAmount: bigint; +} + +const baseChain = defineChain({ + id: config.baseChainId, + name: "base", + nativeCurrency: { name: "Ether", symbol: "ETH", decimals: 18 }, + rpcUrls: { default: { http: [config.baseRpcUrl] } }, +}); + +const account = privateKeyToAccount(config.attestorPrivateKey); +const publicClient = createPublicClient({ chain: baseChain, transport: http(config.baseRpcUrl) }); +const walletClient = createWalletClient({ + account, + chain: baseChain, + transport: http(config.baseRpcUrl), +}); + +function log(message: string, extra?: unknown): void { + console.log(`${new Date().toISOString()} ${message}`, extra ?? ""); +} + +async function alert(subject: string, detail: unknown): Promise { + console.error(`${new Date().toISOString()} ALERT: ${subject}`, detail); + if (!config.alertWebhookUrl) return; + try { + await fetch(config.alertWebhookUrl, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ service: "pen-attestor", attestor: account.address, subject, detail: `${detail}` }), + }); + } catch (webhookError) { + console.error("alert webhook failed", webhookError); + } +} + +function loadCheckpoint(): Checkpoint { + try { + return JSON.parse(readFileSync(config.checkpointFile, "utf8")) as Checkpoint; + } catch { + return { lastProcessedBlock: config.startBlock - 1 }; + } +} + +function saveCheckpoint(checkpoint: Checkpoint): void { + writeFileSync(config.checkpointFile, JSON.stringify(checkpoint)); +} + +function payloadHash(event: MigrationEvent): `0x${string}` { + return keccak256( + encodeAbiParameters( + [{ type: "uint64" }, { type: "address" }, { type: "uint256" }], + [event.nonce, event.recipient, event.palletAmount], + ), + ); +} + +/** Extract MigrationInitiated events from one finalized Pendulum block. */ +async function migrationEventsInBlock(api: ApiPromise, blockNumber: number): Promise { + const blockHash = await api.rpc.chain.getBlockHash(blockNumber); + const apiAt = await api.at(blockHash); + const records = (await apiAt.query.system.events()) as unknown as { + event: { section: string; method: string; data: unknown[] }; + }[]; + + const events: MigrationEvent[] = []; + for (const record of records) { + const { section, method, data } = record.event; + if (section !== "tokenMigration" || method !== "MigrationInitiated") continue; + // Event shape: { nonce: u64, who: AccountId, base_address: H160, amount: u128 }. + // Guard the shape explicitly: a runtime upgrade changing the event must + // fail loudly (PRD A5), not decode garbage positionally. + if (data.length !== 4) { + throw new Error(`MigrationInitiated in block ${blockNumber} has ${data.length} fields, expected 4`); + } + const [nonce, , baseAddress, amount] = data as [ + { toBigInt(): bigint }, + unknown, + { toHex(): string }, + { toBigInt(): bigint }, + ]; + const recipient = baseAddress.toHex() as `0x${string}`; + if (!/^0x[0-9a-fA-F]{40}$/.test(recipient)) { + throw new Error(`cannot decode base_address in block ${blockNumber}: ${recipient}`); + } + events.push({ nonce: nonce.toBigInt(), recipient, palletAmount: amount.toBigInt() }); + } + return events; +} + +/** True when this migration no longer needs our approval (released, or we + * already approved). Rechecked after failures: with 4 attestors + * racing to the same event, losing the race is the NORMAL case, not an error. */ +async function alreadyHandled(event: MigrationEvent): Promise { + const consumed = await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "nonceConsumed", + args: [event.nonce], + }); + if (consumed) return true; + return publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "hasApproved", + args: [payloadHash(event), account.address], + }); +} + +/** Submit the approval for one migration event, skipping work already done. */ +async function approve(event: MigrationEvent): Promise { + const label = `nonce=${event.nonce} recipient=${event.recipient} amount=${event.palletAmount}`; + + if (isUnreleasable(event.recipient, event.palletAmount, config.vaultAddress)) { + await alert("CRITICAL: unreleasable migration event skipped permanently", label); + return; + } + + if (await alreadyHandled(event)) { + log(`skip (already released or approved): ${label}`); + return; + } + + try { + const { request } = await publicClient.simulateContract({ + account, + address: config.vaultAddress, + abi: vaultAbi, + functionName: "approve", + args: [event.nonce, event.recipient, event.palletAmount], + }); + const txHash = await walletClient.writeContract(request); + const receipt = await publicClient.waitForTransactionReceipt({ hash: txHash }); + if (receipt.status !== "success") { + throw new Error(`approve transaction reverted: ${txHash} (${label})`); + } + log(`approved: ${label} tx=${txHash}`); + } catch (error) { + // Expected race: the release landed (or our own retried tx landed) + // between our pre-check and the transaction. Benign — anything else + // is a genuine failure and propagates to the fatal handler. + if (await alreadyHandled(event)) { + log(`skip (raced, resolved on-chain): ${label}`); + return; + } + throw error; + } +} + +async function checkGasBalance(): Promise { + const balance = await publicClient.getBalance({ address: account.address }); + if (balance < config.minGasBalanceWei) { + await alert("gas balance low", `${account.address} holds ${balance} wei`); + } +} + +async function main(): Promise { + const isAttestor = await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "isAttestor", + args: [account.address], + }); + if (!isAttestor) { + throw new Error(`${account.address} is not an attestor of ${config.vaultAddress}`); + } + await checkGasBalance(); + + const api = await ApiPromise.create({ provider: new WsProvider(config.pendulumWs) }); + const checkpoint = loadCheckpoint(); + log(`attestor ${account.address} starting after block ${checkpoint.lastProcessedBlock}`); + + let processing = Promise.resolve(); + await api.rpc.chain.subscribeFinalizedHeads((head) => { + const finalized = head.number.toNumber(); + // Serialize: a slow Base transaction must not let block processing overlap. + processing = processing.then(async () => { + for (let block = checkpoint.lastProcessedBlock + 1; block <= finalized; block++) { + const events = await migrationEventsInBlock(api, block); + for (const event of events) { + await approve(event); + } + checkpoint.lastProcessedBlock = block; + saveCheckpoint(checkpoint); + } + }).catch(async (error) => { + // PRD A5: never skip an event silently. Alert and exit; the process + // manager restarts us and the checkpoint retries the failing block. + await alert("fatal error, exiting", error); + process.exit(1); + }); + }); + + setInterval( + () => void checkGasBalance().catch((error) => console.error("gas balance check failed", error)), + 10 * 60 * 1000, + ); +} + +main().catch(async (error) => { + await alert("startup failed", error); + process.exit(1); +}); diff --git a/attestor/src/vaultAbi.ts b/attestor/src/vaultAbi.ts new file mode 100644 index 000000000..4ea200986 --- /dev/null +++ b/attestor/src/vaultAbi.ts @@ -0,0 +1,38 @@ +/** Minimal MigrationVault ABI: only what the attestor needs. */ +export const vaultAbi = [ + { + type: "function", + name: "approve", + stateMutability: "nonpayable", + inputs: [ + { name: "nonce", type: "uint64" }, + { name: "recipient", type: "address" }, + { name: "palletAmount", type: "uint256" }, + ], + outputs: [], + }, + { + type: "function", + name: "nonceConsumed", + stateMutability: "view", + inputs: [{ name: "nonce", type: "uint64" }], + outputs: [{ type: "bool" }], + }, + { + type: "function", + name: "hasApproved", + stateMutability: "view", + inputs: [ + { name: "payload", type: "bytes32" }, + { name: "attestor", type: "address" }, + ], + outputs: [{ type: "bool" }], + }, + { + type: "function", + name: "isAttestor", + stateMutability: "view", + inputs: [{ name: "attestor", type: "address" }], + outputs: [{ type: "bool" }], + }, +] as const; diff --git a/attestor/tsconfig.json b/attestor/tsconfig.json new file mode 100644 index 000000000..59b02f6e5 --- /dev/null +++ b/attestor/tsconfig.json @@ -0,0 +1,14 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "outDir": "dist", + "rootDir": "src", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true + }, + "include": ["src"] +} From bb1966c7b698109c9b6e595ff2d766f9e7d3c3d3 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 24 Aug 2026 19:07:30 +0200 Subject: [PATCH 06/61] monitor: Add the independent invariant monitor Runs on infrastructure separate from every attestor and reads both chains independently. Each poll it verifies that nothing has been released without a corresponding finalized burn on Pendulum, and that the vault's balance, total released and total swept still account for the entire supply. Only a deficit alerts: a surplus is a harmless inbound transfer and must not be able to trip an auto-pause. Base reads are pinned to a single block so a release landing mid-cycle cannot produce a false alarm. Liveness tracks migrations that stay unreleased past a grace period, batching the per-nonce reads through Multicall3 and incorporating each nonce exactly once so the scan stays proportional to the pending backlog rather than to all migrations ever made. Alerts via webhook and, when configured with a guardian key, pauses the vault automatically on a conservation violation. That key must be held by someone who holds no attestor key. --- monitor/.gitignore | 2 + monitor/README.md | 39 ++ monitor/package-lock.json | 1123 ++++++++++++++++++++++++++++++++++++ monitor/package.json | 21 + monitor/src/checks.test.ts | 79 +++ monitor/src/checks.ts | 75 +++ monitor/src/main.ts | 303 ++++++++++ monitor/tsconfig.json | 14 + 8 files changed, 1656 insertions(+) create mode 100644 monitor/.gitignore create mode 100644 monitor/README.md create mode 100644 monitor/package-lock.json create mode 100644 monitor/package.json create mode 100644 monitor/src/checks.test.ts create mode 100644 monitor/src/checks.ts create mode 100644 monitor/src/main.ts create mode 100644 monitor/tsconfig.json diff --git a/monitor/.gitignore b/monitor/.gitignore new file mode 100644 index 000000000..b94707787 --- /dev/null +++ b/monitor/.gitignore @@ -0,0 +1,2 @@ +node_modules/ +dist/ diff --git a/monitor/README.md b/monitor/README.md new file mode 100644 index 000000000..2cf6c7323 --- /dev/null +++ b/monitor/README.md @@ -0,0 +1,39 @@ +# PEN Migration Invariant Monitor + +Independent watchdog for the PEN migration (PRD §6.5). **Must run on +infrastructure separate from every attestor** — its whole value is being an +independent pair of eyes on both chains. + +Checks every poll: + +| Check | Meaning | Reaction | +|---|---|---| +| M2a: `totalReleased <= TotalMigrated × conversionFactor` | Tokens may never leave the vault without a corresponding finalized burn on Pendulum. A violation is the signature of attestor-quorum compromise. | Alert + auto-pause the vault (if `GUARDIAN_PRIVATE_KEY` is set) | +| M2b: `balanceOf(vault) + totalReleased + totalSwept >= totalSupply` | Vault-internal conservation. Only a **deficit** alerts: a surplus is a harmless inbound transfer (donation, or a migration whose recipient is the vault) and is ignored, so it cannot false-trigger a pause. | Alert + auto-pause on a deficit | +| M4: every nonce older than `GRACE_SECONDS` is consumed on Base | Liveness of the attestor fleet (outage, cap deferral, pause). Per-nonce reads are batched via Multicall3 so a large backlog cannot starve the checks above. | Alert | + +## Configuration (environment) + +| Variable | Meaning | +|---|---| +| `PENDULUM_WS` | WebSocket of the monitor's own Pendulum node | +| `BASE_RPC_URL` | Base JSON-RPC endpoint (ideally a different provider than the attestors use) | +| `VAULT_ADDRESS` | MigrationVault address on Base | +| `POLL_INTERVAL_MS` | Poll cadence (default 60s) | +| `GRACE_SECONDS` | Liveness alert threshold (default 30 min) | +| `ALERT_WEBHOOK_URL` | Webhook receiving JSON alerts — wire this to paging | +| `GUARDIAN_PRIVATE_KEY` | Optional: a guardian key enabling automatic `pause()` on conservation violations | +| `BASE_CHAIN_ID` | Default 8453 | + +## Run + +```sh +npm install +npm run build +npm start +``` + +Run it under a process manager and treat "monitor down" itself as a paging +condition: an unwatched migration is the risk model failing silently. Note the +liveness state (`nonceFirstSeen`) is in-memory — after a restart, grace timers +restart from zero, which can only delay (never lose) a liveness alert. diff --git a/monitor/package-lock.json b/monitor/package-lock.json new file mode 100644 index 000000000..a48e1d8d5 --- /dev/null +++ b/monitor/package-lock.json @@ -0,0 +1,1123 @@ +{ + "name": "pen-migration-monitor", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "pen-migration-monitor", + "version": "0.1.0", + "dependencies": { + "@polkadot/api": "^11.3.1", + "viem": "^2.21.0" + }, + "devDependencies": { + "@types/node": "^20.14.0", + "typescript": "^5.5.0" + } + }, + "node_modules/@adraffy/ens-normalize": { + "version": "1.11.1", + "resolved": "https://registry.npmjs.org/@adraffy/ens-normalize/-/ens-normalize-1.11.1.tgz", + "integrity": "sha512-nhCBV3quEgesuf7c7KYfperqSS14T8bYuvJ8PcLJp6znkZpFc0AuW4qBtr8eKVyPPe/8RSr7sglCWPU5eaxwKQ==", + "license": "MIT" + }, + "node_modules/@noble/ciphers": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@noble/ciphers/-/ciphers-1.3.0.tgz", + "integrity": "sha512-2I0gnIVPtfnMw9ee9h1dJG7tp81+8Ob3OJb3Mv37rx5L40/b0i7djjCVvGOVqc9AEIQyvyu1i6ypKdFw8R8gQw==", + "license": "MIT", + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@noble/curves": { + "version": "1.9.7", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.7.tgz", + "integrity": "sha512-gbKGcRUYIjA3/zCCNaWDciTMFI0dCkvou3TL8Zmy5Nc7sJ47a0jtOeZoTaMxkuqRo9cRhjOdZJXegxYE5FN/xw==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@noble/hashes": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/@noble/hashes/-/hashes-1.8.0.tgz", + "integrity": "sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==", + "license": "MIT", + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@polkadot-api/json-rpc-provider": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/json-rpc-provider/-/json-rpc-provider-0.0.1.tgz", + "integrity": "sha512-/SMC/l7foRjpykLTUTacIH05H3mr9ip8b5xxfwXlVezXrNVLp3Cv0GX6uItkKd+ZjzVPf3PFrDF2B2/HLSNESA==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot-api/json-rpc-provider-proxy": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/json-rpc-provider-proxy/-/json-rpc-provider-proxy-0.0.1.tgz", + "integrity": "sha512-gmVDUP8LpCH0BXewbzqXF2sdHddq1H1q+XrAW2of+KZj4woQkIGBRGTJHeBEVHe30EB+UejR1N2dT4PO/RvDdg==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot-api/metadata-builders": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/metadata-builders/-/metadata-builders-0.0.1.tgz", + "integrity": "sha512-GCI78BHDzXAF/L2pZD6Aod/yl82adqQ7ftNmKg51ixRL02JpWUA+SpUKTJE5MY1p8kiJJIo09P2um24SiJHxNA==", + "license": "MIT", + "optional": true, + "dependencies": { + "@polkadot-api/substrate-bindings": "0.0.1", + "@polkadot-api/utils": "0.0.1" + } + }, + "node_modules/@polkadot-api/observable-client": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/@polkadot-api/observable-client/-/observable-client-0.1.0.tgz", + "integrity": "sha512-GBCGDRztKorTLna/unjl/9SWZcRmvV58o9jwU2Y038VuPXZcr01jcw/1O3x+yeAuwyGzbucI/mLTDa1QoEml3A==", + "license": "MIT", + "optional": true, + "dependencies": { + "@polkadot-api/metadata-builders": "0.0.1", + "@polkadot-api/substrate-bindings": "0.0.1", + "@polkadot-api/substrate-client": "0.0.1", + "@polkadot-api/utils": "0.0.1" + }, + "peerDependencies": { + "rxjs": ">=7.8.0" + } + }, + "node_modules/@polkadot-api/substrate-bindings": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/substrate-bindings/-/substrate-bindings-0.0.1.tgz", + "integrity": "sha512-bAe7a5bOPnuFVmpv7y4BBMRpNTnMmE0jtTqRUw/+D8ZlEHNVEJQGr4wu3QQCl7k1GnSV1wfv3mzIbYjErEBocg==", + "license": "MIT", + "optional": true, + "dependencies": { + "@noble/hashes": "^1.3.1", + "@polkadot-api/utils": "0.0.1", + "@scure/base": "^1.1.1", + "scale-ts": "^1.6.0" + } + }, + "node_modules/@polkadot-api/substrate-client": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/substrate-client/-/substrate-client-0.0.1.tgz", + "integrity": "sha512-9Bg9SGc3AwE+wXONQoW8GC00N3v6lCZLW74HQzqB6ROdcm5VAHM4CB/xRzWSUF9CXL78ugiwtHx3wBcpx4H4Wg==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot-api/utils": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/utils/-/utils-0.0.1.tgz", + "integrity": "sha512-3j+pRmlF9SgiYDabSdZsBSsN5XHbpXOAce1lWj56IEEaFZVjsiCaxDOA7C9nCcgfVXuvnbxqqEGQvnY+QfBAUw==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot/api": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api/-/api-11.3.1.tgz", + "integrity": "sha512-q4kFIIHTLvKxM24b0Eo8hJevsPMme+aITJGrDML9BgdZYTRN14+cu5nXiCsQvaEamdyYj+uCXWe2OV9X7pPxsA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/api-augment": "11.3.1", + "@polkadot/api-base": "11.3.1", + "@polkadot/api-derive": "11.3.1", + "@polkadot/keyring": "^12.6.2", + "@polkadot/rpc-augment": "11.3.1", + "@polkadot/rpc-core": "11.3.1", + "@polkadot/rpc-provider": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-augment": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/types-create": "11.3.1", + "@polkadot/types-known": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "eventemitter3": "^5.0.1", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/api-augment": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api-augment/-/api-augment-11.3.1.tgz", + "integrity": "sha512-Yj+6rb6h0WwY3yJ+UGhjGW+tyMRFUMsKQuGw+eFsXdjiNU9UoXsAqA2dG7Q1F+oeX/g+y2gLGBezNoCwbl6HfA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/api-base": "11.3.1", + "@polkadot/rpc-augment": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-augment": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/api-base": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api-base/-/api-base-11.3.1.tgz", + "integrity": "sha512-b8UkNL00NN7+3QaLCwL5cKg+7YchHoKCAhwKusWHNBZkkO6Oo2BWilu0dZkPJOyqV9P389Kbd9+oH+SKs9u2VQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/rpc-core": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/util": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/api-derive": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api-derive/-/api-derive-11.3.1.tgz", + "integrity": "sha512-9dopzrh4cRuft1nANmBvMY/hEhFDu0VICMTOGxQLOl8NMfcOFPTLAN0JhSBUoicGZhV+c4vpv01NBx/7/IL1HA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/api": "11.3.1", + "@polkadot/api-augment": "11.3.1", + "@polkadot/api-base": "11.3.1", + "@polkadot/rpc-core": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/keyring": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/keyring/-/keyring-12.6.2.tgz", + "integrity": "sha512-O3Q7GVmRYm8q7HuB3S0+Yf/q/EB2egKRRU3fv9b3B7V+A52tKzA+vIwEmNVaD1g5FKW9oB97rmpggs0zaKFqHw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "12.6.2", + "@polkadot/util-crypto": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "12.6.2", + "@polkadot/util-crypto": "12.6.2" + } + }, + "node_modules/@polkadot/networks": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/networks/-/networks-12.6.2.tgz", + "integrity": "sha512-1oWtZm1IvPWqvMrldVH6NI2gBoCndl5GEwx7lAuQWGr7eNL+6Bdc5K3Z9T0MzFvDGoi2/CBqjX9dRKo39pDC/w==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "12.6.2", + "@substrate/ss58-registry": "^1.44.0", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/rpc-augment": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/rpc-augment/-/rpc-augment-11.3.1.tgz", + "integrity": "sha512-2PaDcKNju4QYQpxwVkWbRU3M0t340nMX9cMo+8awgvgL1LliV/fUDZueMKLuSS910JJMTPQ7y2pK4eQgMt08gQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/rpc-core": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/rpc-core": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/rpc-core/-/rpc-core-11.3.1.tgz", + "integrity": "sha512-KKNepsDd/mpmXcA6v/h14eFFPEzLGd7nrvx2UUXUxoZ0Fq2MH1hplP3s93k1oduNY/vOXJR2K9S4dKManA6GVQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/rpc-augment": "11.3.1", + "@polkadot/rpc-provider": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/util": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/rpc-provider": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/rpc-provider/-/rpc-provider-11.3.1.tgz", + "integrity": "sha512-pqERChoHo45hd3WAgW8UuzarRF+G/o/eXEbl0PXLubiayw4X4qCmIzmtntUcKYgxGNcYGZaG87ZU8OjN97m6UA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/keyring": "^12.6.2", + "@polkadot/types": "11.3.1", + "@polkadot/types-support": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "@polkadot/x-fetch": "^12.6.2", + "@polkadot/x-global": "^12.6.2", + "@polkadot/x-ws": "^12.6.2", + "eventemitter3": "^5.0.1", + "mock-socket": "^9.3.1", + "nock": "^13.5.0", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@substrate/connect": "0.8.10" + } + }, + "node_modules/@polkadot/types": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types/-/types-11.3.1.tgz", + "integrity": "sha512-5c7uRFXQTT11Awi6T0yFIdAfD6xGDAOz06Kp7M5S9OGNZY28wSPk5x6BYfNphWPaIBmHHewYJB5qmnrdYQAWKQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/keyring": "^12.6.2", + "@polkadot/types-augment": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/types-create": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-augment": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-augment/-/types-augment-11.3.1.tgz", + "integrity": "sha512-eR3HVpvUmB3v7q2jTWVmVfAVfb1/kuNn7ij94Zqadg/fuUq0pKqIOKwkUj3OxRM3A/5BnW3MbgparjKD3r+fyw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-codec": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-codec/-/types-codec-11.3.1.tgz", + "integrity": "sha512-i7IiiuuL+Z/jFoKTA9xeh4wGQnhnNNjMT0+1ohvlOvnFsoKZKFQQOaDPPntGJVL1JDCV+KjkN2uQKZSeW8tguQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "^12.6.2", + "@polkadot/x-bigint": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-create": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-create/-/types-create-11.3.1.tgz", + "integrity": "sha512-pBXtpz5FehcRJ6j5MzFUIUN8ZWM7z6HbqK1GxBmYbJVRElcGcOg7a/rL2pQVphU0Rx1E8bSO4thzGf4wUxSX7w==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-known": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-known/-/types-known-11.3.1.tgz", + "integrity": "sha512-3BIof7u6tn9bk3ZCIxA07iNoQ3uj4+vn3DTOjCKECozkRlt6V+kWRvqh16Hc0SHMg/QjcMb2fIu/WZhka1McUQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/networks": "^12.6.2", + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/types-create": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-support": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-support/-/types-support-11.3.1.tgz", + "integrity": "sha512-jTFz1GKyF7nI29yIOq4v0NiWTOf5yX4HahJNeFD8TcxoLhF+6tH/XXqrUXJEfbaTlSrRWiW1LZYlb+snctqKHA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/util": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/util/-/util-12.6.2.tgz", + "integrity": "sha512-l8TubR7CLEY47240uki0TQzFvtnxFIO7uI/0GoWzpYD/O62EIAMRsuY01N4DuwgKq2ZWD59WhzsLYmA5K6ksdw==", + "license": "Apache-2.0", + "peer": true, + "dependencies": { + "@polkadot/x-bigint": "12.6.2", + "@polkadot/x-global": "12.6.2", + "@polkadot/x-textdecoder": "12.6.2", + "@polkadot/x-textencoder": "12.6.2", + "@types/bn.js": "^5.1.5", + "bn.js": "^5.2.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/util-crypto": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/util-crypto/-/util-crypto-12.6.2.tgz", + "integrity": "sha512-FEWI/dJ7wDMNN1WOzZAjQoIcCP/3vz3wvAp5QQm+lOrzOLj0iDmaIGIcBkz8HVm3ErfSe/uKP0KS4jgV/ib+Mg==", + "license": "Apache-2.0", + "dependencies": { + "@noble/curves": "^1.3.0", + "@noble/hashes": "^1.3.3", + "@polkadot/networks": "12.6.2", + "@polkadot/util": "12.6.2", + "@polkadot/wasm-crypto": "^7.3.2", + "@polkadot/wasm-util": "^7.3.2", + "@polkadot/x-bigint": "12.6.2", + "@polkadot/x-randomvalues": "12.6.2", + "@scure/base": "^1.1.5", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "12.6.2" + } + }, + "node_modules/@polkadot/wasm-bridge": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-bridge/-/wasm-bridge-7.5.4.tgz", + "integrity": "sha512-6xaJVvoZbnbgpQYXNw9OHVNWjXmtcoPcWh7hlwx3NpfiLkkjljj99YS+XGZQlq7ks2fVCg7FbfknkNb8PldDaA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*", + "@polkadot/x-randomvalues": "*" + } + }, + "node_modules/@polkadot/wasm-crypto": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto/-/wasm-crypto-7.5.4.tgz", + "integrity": "sha512-1seyClxa7Jd7kQjfnCzTTTfYhTa/KUTDUaD3DMHBk5Q4ZUN1D1unJgX+v1aUeXSPxmzocdZETPJJRZjhVOqg9g==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-bridge": "7.5.4", + "@polkadot/wasm-crypto-asmjs": "7.5.4", + "@polkadot/wasm-crypto-init": "7.5.4", + "@polkadot/wasm-crypto-wasm": "7.5.4", + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*", + "@polkadot/x-randomvalues": "*" + } + }, + "node_modules/@polkadot/wasm-crypto-asmjs": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto-asmjs/-/wasm-crypto-asmjs-7.5.4.tgz", + "integrity": "sha512-ZYwxQHAJ8pPt6kYk9XFmyuFuSS+yirJLonvP+DYbxOrARRUHfN4nzp4zcZNXUuaFhpbDobDSFn6gYzye6BUotA==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*" + } + }, + "node_modules/@polkadot/wasm-crypto-init": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto-init/-/wasm-crypto-init-7.5.4.tgz", + "integrity": "sha512-U6s4Eo2rHs2n1iR01vTz/sOQ7eOnRPjaCsGWhPV+ZC/20hkVzwPAhiizu/IqMEol4tO2yiSheD4D6bn0KxUJhg==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-bridge": "7.5.4", + "@polkadot/wasm-crypto-asmjs": "7.5.4", + "@polkadot/wasm-crypto-wasm": "7.5.4", + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*", + "@polkadot/x-randomvalues": "*" + } + }, + "node_modules/@polkadot/wasm-crypto-wasm": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto-wasm/-/wasm-crypto-wasm-7.5.4.tgz", + "integrity": "sha512-PsHgLsVTu43eprwSvUGnxybtOEuHPES6AbApcs7y5ZbM2PiDMzYbAjNul098xJK/CPtrxZ0ePDFnaQBmIJyTFw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*" + } + }, + "node_modules/@polkadot/wasm-util": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-util/-/wasm-util-7.5.4.tgz", + "integrity": "sha512-hqPpfhCpRAqCIn/CYbBluhh0TXmwkJnDRjxrU9Bnqtw9nMNa97D8JuOjdd2pi0rxm+eeLQ/f1rQMp71RMM9t4w==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*" + } + }, + "node_modules/@polkadot/x-bigint": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-bigint/-/x-bigint-12.6.2.tgz", + "integrity": "sha512-HSIk60uFPX4GOFZSnIF7VYJz7WZA7tpFJsne7SzxOooRwMTWEtw3fUpFy5cYYOeLh17/kHH1Y7SVcuxzVLc74Q==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-fetch": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-fetch/-/x-fetch-12.6.2.tgz", + "integrity": "sha512-8wM/Z9JJPWN1pzSpU7XxTI1ldj/AfC8hKioBlUahZ8gUiJaOF7K9XEFCrCDLis/A1BoOu7Ne6WMx/vsJJIbDWw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "node-fetch": "^3.3.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-global": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-global/-/x-global-12.6.2.tgz", + "integrity": "sha512-a8d6m+PW98jmsYDtAWp88qS4dl8DyqUBsd0S+WgyfSMtpEXu6v9nXDgPZgwF5xdDvXhm+P0ZfVkVTnIGrScb5g==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-randomvalues": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-randomvalues/-/x-randomvalues-12.6.2.tgz", + "integrity": "sha512-Vr8uG7rH2IcNJwtyf5ebdODMcr0XjoCpUbI91Zv6AlKVYOGKZlKLYJHIwpTaKKB+7KPWyQrk4Mlym/rS7v9feg==", + "license": "Apache-2.0", + "peer": true, + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "12.6.2", + "@polkadot/wasm-util": "*" + } + }, + "node_modules/@polkadot/x-textdecoder": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-textdecoder/-/x-textdecoder-12.6.2.tgz", + "integrity": "sha512-M1Bir7tYvNappfpFWXOJcnxUhBUFWkUFIdJSyH0zs5LmFtFdbKAeiDXxSp2Swp5ddOZdZgPac294/o2TnQKN1w==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-textencoder": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-textencoder/-/x-textencoder-12.6.2.tgz", + "integrity": "sha512-4N+3UVCpI489tUJ6cv3uf0PjOHvgGp9Dl+SZRLgFGt9mvxnvpW/7+XBADRMtlG4xi5gaRK7bgl5bmY6OMDsNdw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-ws": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-ws/-/x-ws-12.6.2.tgz", + "integrity": "sha512-cGZWo7K5eRRQCRl2LrcyCYsrc3lRbTlixZh3AzgU8uX4wASVGRlNWi/Hf4TtHNe1ExCDmxabJzdIsABIfrr7xw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2", + "ws": "^8.15.1" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@scure/base": { + "version": "1.2.6", + "resolved": "https://registry.npmjs.org/@scure/base/-/base-1.2.6.tgz", + "integrity": "sha512-g/nm5FgUa//MCj1gV09zTJTaM6KBAHqLN907YVQqf7zC49+DcO4B1so4ZX07Ef10Twr6nuqYEH9GEggFXA4Fmg==", + "license": "MIT", + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/bip32": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/@scure/bip32/-/bip32-1.7.0.tgz", + "integrity": "sha512-E4FFX/N3f4B80AKWp5dP6ow+flD1LQZo/w8UnLGYZO674jS6YnYeepycOOksv+vLPSpgN35wgKgy+ybfTb2SMw==", + "license": "MIT", + "dependencies": { + "@noble/curves": "~1.9.0", + "@noble/hashes": "~1.8.0", + "@scure/base": "~1.2.5" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/bip39": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/@scure/bip39/-/bip39-1.6.0.tgz", + "integrity": "sha512-+lF0BbLiJNwVlev4eKelw1WWLaiKXw7sSl8T6FvBlWkdX+94aGJ4o8XjUdlyhTCjd8c+B3KT3JfS8P0bLRNU6A==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "~1.8.0", + "@scure/base": "~1.2.5" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@substrate/connect": { + "version": "0.8.10", + "resolved": "https://registry.npmjs.org/@substrate/connect/-/connect-0.8.10.tgz", + "integrity": "sha512-DIyQ13DDlXqVFnLV+S6/JDgiGowVRRrh18kahieJxhgvzcWicw5eLc6jpfQ0moVVLBYkO7rctB5Wreldwpva8w==", + "deprecated": "versions below 1.x are no longer maintained", + "license": "GPL-3.0-only", + "optional": true, + "dependencies": { + "@substrate/connect-extension-protocol": "^2.0.0", + "@substrate/connect-known-chains": "^1.1.4", + "@substrate/light-client-extension-helpers": "^0.0.6", + "smoldot": "2.0.22" + } + }, + "node_modules/@substrate/connect-extension-protocol": { + "version": "2.2.2", + "resolved": "https://registry.npmjs.org/@substrate/connect-extension-protocol/-/connect-extension-protocol-2.2.2.tgz", + "integrity": "sha512-t66jwrXA0s5Goq82ZtjagLNd7DPGCNjHeehRlE/gcJmJ+G56C0W+2plqOMRicJ8XGR1/YFnUSEqUFiSNbjGrAA==", + "license": "GPL-3.0-only", + "optional": true + }, + "node_modules/@substrate/connect-known-chains": { + "version": "1.10.3", + "resolved": "https://registry.npmjs.org/@substrate/connect-known-chains/-/connect-known-chains-1.10.3.tgz", + "integrity": "sha512-OJEZO1Pagtb6bNE3wCikc2wrmvEU5x7GxFFLqqbz1AJYYxSlrPCGu4N2og5YTExo4IcloNMQYFRkBGue0BKZ4w==", + "license": "GPL-3.0-only", + "optional": true + }, + "node_modules/@substrate/light-client-extension-helpers": { + "version": "0.0.6", + "resolved": "https://registry.npmjs.org/@substrate/light-client-extension-helpers/-/light-client-extension-helpers-0.0.6.tgz", + "integrity": "sha512-girltEuxQ1BvkJWmc8JJlk4ZxnlGXc/wkLcNguhY+UoDEMBK0LsdtfzQKIfrIehi4QdeSBlFEFBoI4RqPmsZzA==", + "license": "MIT", + "optional": true, + "dependencies": { + "@polkadot-api/json-rpc-provider": "0.0.1", + "@polkadot-api/json-rpc-provider-proxy": "0.0.1", + "@polkadot-api/observable-client": "0.1.0", + "@polkadot-api/substrate-client": "0.0.1", + "@substrate/connect-extension-protocol": "^2.0.0", + "@substrate/connect-known-chains": "^1.1.4", + "rxjs": "^7.8.1" + }, + "peerDependencies": { + "smoldot": "2.x" + } + }, + "node_modules/@substrate/ss58-registry": { + "version": "1.51.0", + "resolved": "https://registry.npmjs.org/@substrate/ss58-registry/-/ss58-registry-1.51.0.tgz", + "integrity": "sha512-TWDurLiPxndFgKjVavCniytBIw+t4ViOi7TYp9h/D0NMmkEc9klFTo+827eyEJ0lELpqO207Ey7uGxUa+BS1jQ==", + "license": "Apache-2.0" + }, + "node_modules/@types/bn.js": { + "version": "5.2.0", + "resolved": "https://registry.npmjs.org/@types/bn.js/-/bn.js-5.2.0.tgz", + "integrity": "sha512-DLbJ1BPqxvQhIGbeu8VbUC1DiAiahHtAYvA0ZEAa4P31F7IaArc8z3C3BRQdWX4mtLQuABG4yzp76ZrS02Ui1Q==", + "license": "MIT", + "dependencies": { + "@types/node": "*" + } + }, + "node_modules/@types/node": { + "version": "20.19.43", + "resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.43.tgz", + "integrity": "sha512-6oYBAi5ikg4Pl+kGsoYtawUMBT2zZMCvPNF7pVLnHZfd1zf38DRiWn/gT01RYCdUqkv7Fhr+C9ot4/tb+2sVvA==", + "license": "MIT", + "dependencies": { + "undici-types": "~6.21.0" + } + }, + "node_modules/abitype": { + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/abitype/-/abitype-1.2.3.tgz", + "integrity": "sha512-Ofer5QUnuUdTFsBRwARMoWKOH1ND5ehwYhJ3OJ/BQO+StkwQjHw0XyVh4vDttzHB7QOFhPHa/o413PJ82gU/Tg==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/wevm" + }, + "peerDependencies": { + "typescript": ">=5.0.4", + "zod": "^3.22.0 || ^4.0.0" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + }, + "zod": { + "optional": true + } + } + }, + "node_modules/bn.js": { + "version": "5.2.4", + "resolved": "https://registry.npmjs.org/bn.js/-/bn.js-5.2.4.tgz", + "integrity": "sha512-QL7sb18rJ1PbdsKsqPA0guxL563vIMwRHgzNrW/uzQuRGN1Cjqd/wonUBAVqHox9KwzHA6vCbM0lXx3k4iQMow==", + "license": "MIT" + }, + "node_modules/data-uri-to-buffer": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.1.tgz", + "integrity": "sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==", + "license": "MIT", + "engines": { + "node": ">= 12" + } + }, + "node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/eventemitter3": { + "version": "5.0.4", + "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz", + "integrity": "sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==", + "license": "MIT" + }, + "node_modules/fetch-blob": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.2.0.tgz", + "integrity": "sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "paypal", + "url": "https://paypal.me/jimmywarting" + } + ], + "license": "MIT", + "dependencies": { + "node-domexception": "^1.0.0", + "web-streams-polyfill": "^3.0.3" + }, + "engines": { + "node": "^12.20 || >= 14.13" + } + }, + "node_modules/formdata-polyfill": { + "version": "4.0.10", + "resolved": "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz", + "integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==", + "license": "MIT", + "dependencies": { + "fetch-blob": "^3.1.2" + }, + "engines": { + "node": ">=12.20.0" + } + }, + "node_modules/isows": { + "version": "1.0.7", + "resolved": "https://registry.npmjs.org/isows/-/isows-1.0.7.tgz", + "integrity": "sha512-I1fSfDCZL5P0v33sVqeTDSpcstAg/N+wF5HS033mogOVIp4B+oHC7oOCsA3axAbBSGTJ8QubbNmnIRN/h8U7hg==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "peerDependencies": { + "ws": "*" + } + }, + "node_modules/json-stringify-safe": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz", + "integrity": "sha512-ZClg6AaYvamvYEE82d3Iyd3vSSIjQ+odgjaTzRuO3s7toCdFKczob2i0zCh7JE8kWn17yvAWhUVxvqGwUalsRA==", + "license": "ISC" + }, + "node_modules/mock-socket": { + "version": "9.3.1", + "resolved": "https://registry.npmjs.org/mock-socket/-/mock-socket-9.3.1.tgz", + "integrity": "sha512-qxBgB7Qa2sEQgHFjj0dSigq7fX4k6Saisd5Nelwp2q8mlbAFh5dHV9JTTlF8viYJLSSWgMCZFUom8PJcMNBoJw==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "license": "MIT" + }, + "node_modules/nock": { + "version": "13.5.6", + "resolved": "https://registry.npmjs.org/nock/-/nock-13.5.6.tgz", + "integrity": "sha512-o2zOYiCpzRqSzPj0Zt/dQ/DqZeYoaQ7TUonc/xUPjCGl9WeHpNbxgVvOquXYAaJzI0M9BXV3HTzG0p8IUAbBTQ==", + "license": "MIT", + "dependencies": { + "debug": "^4.1.0", + "json-stringify-safe": "^5.0.1", + "propagate": "^2.0.0" + }, + "engines": { + "node": ">= 10.13" + } + }, + "node_modules/node-domexception": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz", + "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==", + "deprecated": "Use your platform's native DOMException instead", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "github", + "url": "https://paypal.me/jimmywarting" + } + ], + "license": "MIT", + "engines": { + "node": ">=10.5.0" + } + }, + "node_modules/node-fetch": { + "version": "3.3.2", + "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.2.tgz", + "integrity": "sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==", + "license": "MIT", + "dependencies": { + "data-uri-to-buffer": "^4.0.0", + "fetch-blob": "^3.1.4", + "formdata-polyfill": "^4.0.10" + }, + "engines": { + "node": "^12.20.0 || ^14.13.1 || >=16.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/node-fetch" + } + }, + "node_modules/ox": { + "version": "0.14.30", + "resolved": "https://registry.npmjs.org/ox/-/ox-0.14.30.tgz", + "integrity": "sha512-LI11uu+8iiM1B3CLckgd++YF1a0A2k5wDoM9ZeQMiL21BOzQs6L//BLS6hb1HSEKCyycdDIQLsVQx9MjpcC0hA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "dependencies": { + "@adraffy/ens-normalize": "^1.11.0", + "@noble/ciphers": "^1.3.0", + "@noble/curves": "1.9.1", + "@noble/hashes": "^1.8.0", + "@scure/bip32": "^1.7.0", + "@scure/bip39": "^1.6.0", + "abitype": "^1.2.3", + "eventemitter3": "5.0.1" + }, + "peerDependencies": { + "typescript": ">=5.4.0" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + } + } + }, + "node_modules/ox/node_modules/@noble/curves": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.1.tgz", + "integrity": "sha512-k11yZxZg+t+gWvBbIswW0yoJlu8cHOC7dhunwOzoWH/mXGBiYyR4YY6hAEK/3EUs4UpB8la1RfdRpeGsFHkWsA==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/ox/node_modules/eventemitter3": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.1.tgz", + "integrity": "sha512-GWkBvjiSZK87ELrYOSESUYeVIc9mvLLf/nXalMOS5dYrgZq9o5OVkbZAVM06CVxYsCwH9BDZFPlQTlPA1j4ahA==", + "license": "MIT" + }, + "node_modules/propagate": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/propagate/-/propagate-2.0.1.tgz", + "integrity": "sha512-vGrhOavPSTz4QVNuBNdcNXePNdNMaO1xj9yBeH1ScQPjk/rhg9sSlCXPhMkFuaNNW/syTvYqsnbIJxMBfRbbag==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/rxjs": { + "version": "7.8.2", + "resolved": "https://registry.npmjs.org/rxjs/-/rxjs-7.8.2.tgz", + "integrity": "sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA==", + "license": "Apache-2.0", + "peer": true, + "dependencies": { + "tslib": "^2.1.0" + } + }, + "node_modules/scale-ts": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/scale-ts/-/scale-ts-1.6.1.tgz", + "integrity": "sha512-PBMc2AWc6wSEqJYBDPcyCLUj9/tMKnLX70jLOSndMtcUoLQucP/DM0vnQo1wJAYjTrQiq8iG9rD0q6wFzgjH7g==", + "license": "MIT", + "optional": true + }, + "node_modules/smoldot": { + "version": "2.0.22", + "resolved": "https://registry.npmjs.org/smoldot/-/smoldot-2.0.22.tgz", + "integrity": "sha512-B50vRgTY6v3baYH6uCgL15tfaag5tcS2o/P5q1OiXcKGv1axZDfz2dzzMuIkVpyMR2ug11F6EAtQlmYBQd292g==", + "license": "GPL-3.0-or-later WITH Classpath-exception-2.0", + "optional": true, + "peer": true, + "dependencies": { + "ws": "^8.8.1" + } + }, + "node_modules/tslib": { + "version": "2.8.1", + "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", + "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", + "license": "0BSD" + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "devOptional": true, + "license": "Apache-2.0", + "peer": true, + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + }, + "node_modules/undici-types": { + "version": "6.21.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", + "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", + "license": "MIT" + }, + "node_modules/viem": { + "version": "2.54.6", + "resolved": "https://registry.npmjs.org/viem/-/viem-2.54.6.tgz", + "integrity": "sha512-OfybECKJYVmhiNqz+SHhed+O2h6niQ+0Wjg9J0b4bV+/QrvLgjxhfKO7hZqsuK1YtZ/0BErBKy708Zp+cU5T0Q==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "dependencies": { + "@noble/curves": "1.9.1", + "@noble/hashes": "1.8.0", + "@scure/bip32": "1.7.0", + "@scure/bip39": "1.6.0", + "abitype": "1.2.3", + "isows": "1.0.7", + "ox": "0.14.30", + "ws": "8.21.0" + }, + "peerDependencies": { + "typescript": ">=5.0.4" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + } + } + }, + "node_modules/viem/node_modules/@noble/curves": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.1.tgz", + "integrity": "sha512-k11yZxZg+t+gWvBbIswW0yoJlu8cHOC7dhunwOzoWH/mXGBiYyR4YY6hAEK/3EUs4UpB8la1RfdRpeGsFHkWsA==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/web-streams-polyfill": { + "version": "3.3.3", + "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz", + "integrity": "sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/ws": { + "version": "8.21.0", + "resolved": "https://registry.npmjs.org/ws/-/ws-8.21.0.tgz", + "integrity": "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==", + "license": "MIT", + "peer": true, + "engines": { + "node": ">=10.0.0" + }, + "peerDependencies": { + "bufferutil": "^4.0.1", + "utf-8-validate": ">=5.0.2" + }, + "peerDependenciesMeta": { + "bufferutil": { + "optional": true + }, + "utf-8-validate": { + "optional": true + } + } + } + } +} diff --git a/monitor/package.json b/monitor/package.json new file mode 100644 index 000000000..f1a8b893d --- /dev/null +++ b/monitor/package.json @@ -0,0 +1,21 @@ +{ + "name": "pen-migration-monitor", + "version": "0.1.0", + "private": true, + "description": "Independent invariant monitor for the PEN migration: verifies conservation between Pendulum burns and Base releases, watches liveness, and can auto-pause the vault", + "type": "module", + "scripts": { + "build": "tsc", + "start": "node dist/main.js", + "typecheck": "tsc --noEmit", + "test": "tsc && node --test dist/checks.test.js" + }, + "dependencies": { + "@polkadot/api": "^11.3.1", + "viem": "^2.21.0" + }, + "devDependencies": { + "@types/node": "^20.14.0", + "typescript": "^5.5.0" + } +} diff --git a/monitor/src/checks.test.ts b/monitor/src/checks.test.ts new file mode 100644 index 000000000..91c0e4507 --- /dev/null +++ b/monitor/src/checks.test.ts @@ -0,0 +1,79 @@ +/** + * Unit tests for the monitor's conservation/liveness predicates. + * + * Run with `npm test` (compiles, then `node --test`). These lock in the + * round-5 fixes: the M2b check must ignore a token surplus (donation / vault + * recipient) and only fire on a real deficit. + */ + +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { isStale, newNonces, releasedExceedsMigrated, vaultConservationDeficit } from "./checks.js"; + +const CF = 1_000_000n; // 12 -> 18 decimals +const SUPPLY = 150_000_000n * 10n ** 18n; + +test("M2a: released within migrated does not fire", () => { + assert.equal(releasedExceedsMigrated(5n * CF, 5n, CF), false); + assert.equal(releasedExceedsMigrated(5n * CF, 10n, CF), false); // migrated lags releases via finality: fine +}); + +test("M2a: released exceeding migrated fires", () => { + assert.equal(releasedExceedsMigrated(6n * CF, 5n, CF), true); +}); + +test("M2b: exact conservation is not a deficit", () => { + // balance + released + swept == totalSupply + assert.equal(vaultConservationDeficit(SUPPLY - 30n, 20n, 10n, SUPPLY), false); +}); + +test("M2b: a surplus (donation / vault-recipient self-transfer) must NOT fire", () => { + // Someone transferred dust into the vault: balance is higher than the + // identity predicts, so the sum EXCEEDS totalSupply. This is harmless and + // must never trip the check (previously a strict `!=` paused the vault here). + const donation = 5n * 10n ** 18n; + assert.equal(vaultConservationDeficit(SUPPLY - 30n + donation, 20n, 10n, SUPPLY), false); + // Even a 1-wei donation must not fire. + assert.equal(vaultConservationDeficit(SUPPLY + 1n, 0n, 0n, SUPPLY), false); +}); + +test("M2b: a real deficit (tokens vanished without a counter update) fires", () => { + // balance too low for the recorded released+swept: genuine loss. + assert.equal(vaultConservationDeficit(SUPPLY - 100n, 20n, 10n, SUPPLY), true); +}); + +test("M4: staleness respects the grace period", () => { + const grace = 1800; // seconds + const now = 10_000_000; + assert.equal(isStale(now - 1000, now, grace), false); // 1s old + assert.equal(isStale(now - 1800 * 1000, now, grace), false); // exactly at grace: not yet stale + assert.equal(isStale(now - 1801 * 1000, now, grace), true); // just past grace +}); + +test("M4: newNonces incorporates each nonce exactly once (no re-add of consumed)", () => { + // Round-7 regression: the old scan re-added every nonce 0..nextNonce each + // poll, so a consumed-and-pruned nonce was re-read via Multicall forever, + // making the scan O(all migrations) instead of O(pending backlog). + const firstSeen = new Map(); + let incorporated = 0n; + + // Poll 1: five migrations exist — all freshly stamped. + for (const nonce of newNonces(incorporated, 5n)) firstSeen.set(nonce, 1_000); + incorporated = 5n; + assert.deepEqual([...firstSeen.keys()], [0n, 1n, 2n, 3n, 4n]); + + // Nonces 0,1,2 get consumed on Base and are pruned from the pending set. + firstSeen.delete(0n); + firstSeen.delete(1n); + firstSeen.delete(2n); + + // Poll 2: nextNonce unchanged — nothing to incorporate, and the consumed + // nonces must NOT reappear (the bug that this fix closes). + assert.deepEqual(newNonces(incorporated, 5n), []); + assert.deepEqual([...firstSeen.keys()], [3n, 4n]); + + // Poll 3: two new migrations arrive — only those are freshly incorporated. + for (const nonce of newNonces(incorporated, 7n)) firstSeen.set(nonce, 2_000); + incorporated = 7n; + assert.deepEqual([...firstSeen.keys()], [3n, 4n, 5n, 6n]); +}); diff --git a/monitor/src/checks.ts b/monitor/src/checks.ts new file mode 100644 index 000000000..5e7643de2 --- /dev/null +++ b/monitor/src/checks.ts @@ -0,0 +1,75 @@ +/** + * Pure conservation/liveness predicates for the invariant monitor. + * + * These are isolated from the chain-client plumbing in `main.ts` so they can be + * unit-tested exhaustively (see checks.test.ts). Every subtlety that has bitten + * a review round lives here. + */ + +/** + * (M2a) Nothing may leave the vault that was not burned on Pendulum. + * + * `totalReleased` (Base) must never exceed the burned total converted to token + * units. It lags `totalMigrated` only via finality delay, never the other way, + * because a release requires attestations of an already-finalized burn — so any + * excess is the strongest possible signal of attestor compromise. + */ +export function releasedExceedsMigrated( + totalReleased: bigint, + totalMigrated: bigint, + conversionFactor: bigint, +): boolean { + return totalReleased > totalMigrated * conversionFactor; +} + +/** + * (M2b) Vault-internal conservation. + * + * Under all legitimate contract logic `balance + released + swept` is *exactly* + * `totalSupply`: every path that removes tokens bumps a counter in lockstep + * (`_release`→`totalReleased`, `sweepRemainder`→`totalSwept`). Real loss can + * therefore only ever show up as a DEFICIT. + * + * A SURPLUS (sum > totalSupply) is harmless and must NOT trip the check: it can + * be produced permissionlessly by anyone transferring PEN into the vault (a + * plain donation, or a migration whose recipient is the vault address). Treating + * a surplus as a violation would let a single dust transfer pause the vault on + * every poll — unrecoverable until the window-close sweep, and it would train + * on-call to ignore the highest-severity alert. Hence the strict `<`. + */ +export function vaultConservationDeficit( + vaultBalance: bigint, + totalReleased: bigint, + totalSwept: bigint, + totalSupply: bigint, +): boolean { + return vaultBalance + totalReleased + totalSwept < totalSupply; +} + +/** + * (M4) Liveness: a migration the monitor has known about for longer than the + * grace period, and which is still not consumed on Base, is stalled. + */ +export function isStale(firstSeenMs: number, nowMs: number, graceSeconds: number): boolean { + return nowMs - firstSeenMs > graceSeconds * 1000; +} + +/** + * Nonces observed for the first time this poll: the half-open range + * [incorporatedUpTo, nextNonce). The caller advances its high-water mark to + * `nextNonce` after stamping these, so each nonce enters the liveness set + * EXACTLY ONCE. A nonce already seen — and possibly since consumed and pruned + * from the pending set — is never re-added. + * + * This is what keeps the per-poll `nonceConsumed` scan O(pending backlog) + * rather than O(all migrations ever created). The earlier scan re-added every + * nonce `0..nextNonce` each poll (any nonce not currently in the map), so a + * consumed-and-pruned nonce was re-inserted and re-read via Multicall every + * poll forever — silently defeating the round-5 batching optimisation and + * letting the scan grow without bound for the whole migration window (round 7). + */ +export function newNonces(incorporatedUpTo: bigint, nextNonce: bigint): bigint[] { + const fresh: bigint[] = []; + for (let nonce = incorporatedUpTo; nonce < nextNonce; nonce++) fresh.push(nonce); + return fresh; +} diff --git a/monitor/src/main.ts b/monitor/src/main.ts new file mode 100644 index 000000000..fdbcb921a --- /dev/null +++ b/monitor/src/main.ts @@ -0,0 +1,303 @@ +/** + * PEN migration invariant monitor (PRD §6.5). + * + * Runs on infrastructure SEPARATE from every attestor and reads both chains + * independently. Each poll it checks, at the finalized head of Pendulum and + * the latest Base block: + * + * (M2a) totalReleased on Base <= TotalMigrated on Pendulum * conversionFactor + * (a violation means tokens were released that were never burned — + * the strongest possible signal of attestor compromise) + * (M2b) balanceOf(vault) + totalReleased == totalSupply + * (conservation inside the vault itself) + * (M4) liveness: every migration nonce older than GRACE_SECONDS is consumed + * on Base (detects a stalled attestor fleet) + * + * On an M2a violation the monitor alerts AND — when GUARDIAN_PRIVATE_KEY is + * configured (design option in PRD M3) — pauses the vault immediately. + */ + +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { createPublicClient, createWalletClient, defineChain, http } from "viem"; +import { privateKeyToAccount } from "viem/accounts"; +import { isStale, newNonces, releasedExceedsMigrated, vaultConservationDeficit } from "./checks.js"; + +// Canonical Multicall3 deployment (same address on Base and every major chain), +// used to batch the per-nonce liveness reads into a handful of RPC round-trips. +const MULTICALL3_ADDRESS = "0xcA11bde05977b3631167028862bE2a173976CA11" as const; + +const vaultAbi = [ + { type: "function", name: "totalReleased", stateMutability: "view", inputs: [], outputs: [{ type: "uint256" }] }, + { type: "function", name: "totalSwept", stateMutability: "view", inputs: [], outputs: [{ type: "uint256" }] }, + { type: "function", name: "conversionFactor", stateMutability: "view", inputs: [], outputs: [{ type: "uint256" }] }, + { type: "function", name: "token", stateMutability: "view", inputs: [], outputs: [{ type: "address" }] }, + { type: "function", name: "paused", stateMutability: "view", inputs: [], outputs: [{ type: "bool" }] }, + { + type: "function", + name: "nonceConsumed", + stateMutability: "view", + inputs: [{ name: "nonce", type: "uint64" }], + outputs: [{ type: "bool" }], + }, + { type: "function", name: "pause", stateMutability: "nonpayable", inputs: [], outputs: [] }, +] as const; + +const erc20Abi = [ + { type: "function", name: "totalSupply", stateMutability: "view", inputs: [], outputs: [{ type: "uint256" }] }, + { + type: "function", + name: "balanceOf", + stateMutability: "view", + inputs: [{ name: "owner", type: "address" }], + outputs: [{ type: "uint256" }], + }, +] as const; + +function required(name: string): string { + const value = process.env[name]; + if (!value) throw new Error(`Missing required environment variable ${name}`); + return value; +} + +const config = { + pendulumWs: required("PENDULUM_WS"), + baseRpcUrl: required("BASE_RPC_URL"), + vaultAddress: required("VAULT_ADDRESS") as `0x${string}`, + pollIntervalMs: Number(process.env.POLL_INTERVAL_MS ?? "60000"), + /** Seconds a migration may stay unreleased before a liveness alert (M4). */ + graceSeconds: Number(process.env.GRACE_SECONDS ?? "1800"), + alertWebhookUrl: process.env.ALERT_WEBHOOK_URL, + /** Optional: enables auto-pause on a conservation violation (M3). */ + guardianPrivateKey: process.env.GUARDIAN_PRIVATE_KEY as `0x${string}` | undefined, + baseChainId: Number(process.env.BASE_CHAIN_ID ?? "8453"), +}; + +const baseChain = defineChain({ + id: config.baseChainId, + name: "base", + nativeCurrency: { name: "Ether", symbol: "ETH", decimals: 18 }, + rpcUrls: { default: { http: [config.baseRpcUrl] } }, + contracts: { multicall3: { address: MULTICALL3_ADDRESS } }, +}); +const publicClient = createPublicClient({ chain: baseChain, transport: http(config.baseRpcUrl) }); + +function log(message: string): void { + console.log(`${new Date().toISOString()} ${message}`); +} + +async function alert(subject: string, detail: string): Promise { + console.error(`${new Date().toISOString()} ALERT: ${subject} — ${detail}`); + if (!config.alertWebhookUrl) return; + try { + await fetch(config.alertWebhookUrl, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ service: "pen-monitor", subject, detail }), + }); + } catch (webhookError) { + console.error("alert webhook failed", webhookError); + } +} + +async function pauseVault(): Promise { + if (!config.guardianPrivateKey) { + await alert("AUTO-PAUSE UNAVAILABLE", "no GUARDIAN_PRIVATE_KEY configured; pause manually NOW"); + return; + } + const guardian = privateKeyToAccount(config.guardianPrivateKey); + const walletClient = createWalletClient({ account: guardian, chain: baseChain, transport: http(config.baseRpcUrl) }); + try { + const txHash = await walletClient.writeContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "pause", + }); + await alert("vault auto-paused", `tx=${txHash}`); + } catch (pauseError) { + await alert("AUTO-PAUSE FAILED", `${pauseError}; pause manually NOW`); + } +} + +/** Timestamps (ms) at which the monitor first saw each pallet nonce count. */ +const nonceFirstSeen = new Map(); + +/** Timestamp (ms) of the last liveness alert per nonce. Re-alerts are throttled + * to at most once per grace period, so a pause backlog — or a burn to a + * structurally-unreleasable address (zero / the vault, which never consumes) — + * does not re-fire the highest-severity page on every single poll and train + * on-call to ignore it. Cleared when the nonce is finally consumed. */ +const livenessAlertedAt = new Map(); + +/** High-water mark: exclusive upper bound of nonces already incorporated into + * `nonceFirstSeen`. Only nonces at or beyond this are stamped each poll, so a + * nonce that was consumed and pruned from the pending set is never re-added. + * Without it the scan re-inserted every `0..nextNonce` nonce every poll (any + * not currently in the map), re-reading consumed ones via Multicall forever + * and making the scan O(all migrations ever) instead of O(pending) — silently + * defeating the round-5 batching (round 7). */ +let nextNonceIncorporated = 0n; + +let multicallUnavailable = false; + +/** Read `nonceConsumed` for many nonces, batched through Multicall3. + * + * Falls back to plain concurrent reads if the batch call fails — e.g. on a + * chain where Multicall3 is not deployed at the canonical address, or a + * provider that rejects the batch. The fallback still works (just chattier), + * so a misconfigured multicall degrades liveness detection rather than + * crashing the whole check cycle and blinding the conservation alerts. */ +async function readNonceConsumed(nonces: bigint[]): Promise { + if (!multicallUnavailable) { + try { + return (await publicClient.multicall({ + allowFailure: false, + contracts: nonces.map((nonce) => ({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "nonceConsumed", + args: [nonce], + })), + })) as boolean[]; + } catch (multicallError) { + // Latch so we do not re-attempt (and re-log) the batch every poll. + multicallUnavailable = true; + await alert( + "multicall unavailable, using per-nonce reads", + `liveness reads fall back to individual calls; verify Multicall3 at ${MULTICALL3_ADDRESS}: ${multicallError}`, + ); + } + } + + // Fallback: read in bounded concurrent batches to avoid a request storm. + const CHUNK = 100; + const flags: boolean[] = []; + for (let start = 0; start < nonces.length; start += CHUNK) { + const chunk = nonces.slice(start, start + CHUNK); + const chunkFlags = await Promise.all( + chunk.map((nonce) => + publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "nonceConsumed", + args: [nonce], + }), + ), + ); + flags.push(...chunkFlags); + } + return flags; +} + +async function check(api: ApiPromise): Promise { + // --- Base side, pinned to one block --- + // Read Base FIRST, then Pendulum's monotonically-growing totalMigrated at a + // strictly-later snapshot: this guarantees totalMigrated >= what any Base + // release could have been attested against, so the M2a check can never + // false-positive on a burn that finalized between the two reads. Pinning + // every Base read to one block keeps totalReleased and vaultBalance from + // skewing against each other (a release landing mid-cycle). + const blockNumber = await publicClient.getBlockNumber(); + const [totalReleased, totalSwept, conversionFactor, tokenAddress] = await Promise.all([ + publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "totalReleased", blockNumber }), + publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "totalSwept", blockNumber }), + publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "conversionFactor", blockNumber }), + publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "token", blockNumber }), + ]); + const [totalSupply, vaultBalance] = await Promise.all([ + publicClient.readContract({ address: tokenAddress, abi: erc20Abi, functionName: "totalSupply", blockNumber }), + publicClient.readContract({ + address: tokenAddress, + abi: erc20Abi, + functionName: "balanceOf", + args: [config.vaultAddress], + blockNumber, + }), + ]); + + // --- Pendulum side, at the finalized head (read after Base, see above) --- + const finalizedHash = await api.rpc.chain.getFinalizedHead(); + const apiAt = await api.at(finalizedHash); + const totalMigrated = BigInt((await apiAt.query.tokenMigration.totalMigrated()).toString()); + const nextNonce = BigInt((await apiAt.query.tokenMigration.nextNonce()).toString()); + + // (M2a) Nothing may leave the vault that was not burned on Pendulum. + if (releasedExceedsMigrated(totalReleased, totalMigrated, conversionFactor)) { + await alert( + "CONSERVATION VIOLATION", + `released ${totalReleased} > migrated ${totalMigrated * conversionFactor} (token units)`, + ); + await pauseVault(); + return; + } + + // (M2b) Vault-internal conservation. Only a DEFICIT signals real loss; a + // surplus is a harmless inbound transfer (donation, or a migration whose + // recipient is the vault) and must not trip the check — otherwise a dust + // transfer would pause the vault every poll until the window-close sweep. + // totalSwept accounts for the intended end-of-window sweep. + if (vaultConservationDeficit(vaultBalance, totalReleased, totalSwept, totalSupply)) { + await alert( + "VAULT BALANCE DEFICIT", + `balance ${vaultBalance} + released ${totalReleased} + swept ${totalSwept} < supply ${totalSupply}`, + ); + await pauseVault(); + return; + } + + // (M4) Liveness: nonces the monitor has known about for longer than the + // grace period must be consumed on Base. Batch the per-nonce reads through + // Multicall3 so a large release backlog (e.g. during a pause) cannot make a + // cycle outrun the poll interval and starve the conservation checks above. + const now = Date.now(); + for (const nonce of newNonces(nextNonceIncorporated, nextNonce)) { + nonceFirstSeen.set(nonce, now); + } + nextNonceIncorporated = nextNonce; + const pendingNonces = [...nonceFirstSeen.keys()]; + if (pendingNonces.length > 0) { + const consumedFlags = await readNonceConsumed(pendingNonces); + for (let i = 0; i < pendingNonces.length; i++) { + const nonce = pendingNonces[i]; + if (consumedFlags[i]) { + nonceFirstSeen.delete(nonce); + livenessAlertedAt.delete(nonce); + } else if (isStale(nonceFirstSeen.get(nonce) ?? now, now, config.graceSeconds)) { + // Throttle: page immediately on first staleness, then at most once + // per grace period, so an ongoing outage stays visible without + // storming (a permanently-unreleasable nonce would otherwise page + // every poll forever). + const lastAlerted = livenessAlertedAt.get(nonce); + if (lastAlerted === undefined || now - lastAlerted >= config.graceSeconds * 1000) { + livenessAlertedAt.set(nonce, now); + await alert( + "LIVENESS: migration not released", + `nonce ${nonce} unreleased for over ${config.graceSeconds}s — attestor outage, cap deferral, pause, or a burn to an unreleasable address?`, + ); + } + } + } + } + + log( + `ok: migrated=${totalMigrated} released=${totalReleased} swept=${totalSwept} ` + + `pending=${nonceFirstSeen.size} vaultBalance=${vaultBalance}`, + ); +} + +async function main(): Promise { + const api = await ApiPromise.create({ provider: new WsProvider(config.pendulumWs) }); + log(`monitor started, polling every ${config.pollIntervalMs}ms`); + for (;;) { + try { + await check(api); + } catch (checkError) { + await alert("monitor check failed", `${checkError}`); + } + await new Promise((resolve) => setTimeout(resolve, config.pollIntervalMs)); + } +} + +main().catch(async (error) => { + await alert("monitor startup failed", `${error}`); + process.exit(1); +}); diff --git a/monitor/tsconfig.json b/monitor/tsconfig.json new file mode 100644 index 000000000..59b02f6e5 --- /dev/null +++ b/monitor/tsconfig.json @@ -0,0 +1,14 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "outDir": "dist", + "rootDir": "src", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true + }, + "include": ["src"] +} From 9b6a7c8fc5eb315ec4404daed230edae817e58fa Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 24 Aug 2026 19:07:30 +0200 Subject: [PATCH 07/61] releaser: Add the deferred-release service When a migration reaches the attestor threshold but a cap, a pause or an under-funded vault blocks it, the vault records it pending and emits ReleasePending instead of reverting. Those conditions heal by themselves, but the vault does not self-execute and nothing else calls the permissionless release(): attestors only submit approvals for new finalized events, and the monitor is deliberately read-only. Without this service a backlog would sit pending until an operator cleared it by hand. Kept as a separate process so the attestor's approve path and the watchdog's read-only role stay untouched. Its key holds no privilege -- release() can only pay the recipient the attestors already approved, under the same caps and pause -- so it needs gas and nothing else, and two instances can run concurrently. Failures are classified rather than treated alike: cap, pause and funding reverts retry quietly, a consumed nonce is dropped, and an amount above the per-release cap alerts because only a governance change can clear it. Scan checkpoint and pending set are persisted. --- releaser/.gitignore | 3 + releaser/README.md | 71 ++++++++++ releaser/package-lock.json | 256 ++++++++++++++++++++++++++++++++++++ releaser/package.json | 20 +++ releaser/src/checks.test.ts | 44 +++++++ releaser/src/checks.ts | 85 ++++++++++++ releaser/src/config.ts | 30 +++++ releaser/src/main.ts | 232 ++++++++++++++++++++++++++++++++ releaser/src/vaultAbi.ts | 37 ++++++ releaser/tsconfig.json | 14 ++ 10 files changed, 792 insertions(+) create mode 100644 releaser/.gitignore create mode 100644 releaser/README.md create mode 100644 releaser/package-lock.json create mode 100644 releaser/package.json create mode 100644 releaser/src/checks.test.ts create mode 100644 releaser/src/checks.ts create mode 100644 releaser/src/config.ts create mode 100644 releaser/src/main.ts create mode 100644 releaser/src/vaultAbi.ts create mode 100644 releaser/tsconfig.json diff --git a/releaser/.gitignore b/releaser/.gitignore new file mode 100644 index 000000000..03056135e --- /dev/null +++ b/releaser/.gitignore @@ -0,0 +1,3 @@ +node_modules/ +dist/ +releaser-state.json diff --git a/releaser/README.md b/releaser/README.md new file mode 100644 index 000000000..2b89312e0 --- /dev/null +++ b/releaser/README.md @@ -0,0 +1,71 @@ +# PEN Migration Releaser + +Drains **deferred** migration releases on Base. + +When a migration reaches the attestor threshold but cannot be released right +away — the rolling daily cap is exhausted, the vault is paused, or it is +under-funded — the vault records it as pending and emits `ReleasePending` +rather than reverting. Those conditions heal on their own, but the vault does +not self-execute: someone must call the permissionless `release()`. + +Nothing else does. Attestors only submit `approve` for new finalized events; +the monitor is deliberately read-only. Without this service a launch-day +backlog sits pending until an operator clears it by hand, nonce by nonce. + +## Why a separate process + +- The attestor's approve path is the most review-scarred code in the stack; + adding a second responsibility to it re-opens that risk. +- The monitor must stay a cheap, read-only, independent watchdog. +- **This key holds no privilege.** `release()` is permissionless and can only + pay the recipient the attestors already approved, subject to the same caps + and pause. It needs gas and nothing else, and must never be an attestor, + guardian or admin key. + +Run **two instances** for redundancy — the loser of a race just observes a +consumed nonce and drops it. + +## Configuration + +| Variable | Meaning | +|---|---| +| `BASE_RPC_URL` | Base JSON-RPC endpoint | +| `VAULT_ADDRESS` | MigrationVault address | +| `RELEASER_PRIVATE_KEY` | Gas-only signing key (no privileges) | +| `START_BLOCK` | Vault deployment block — where log scanning begins on a first run | +| `STATE_FILE` | Scan checkpoint + pending set (default `./releaser-state.json`) | +| `POLL_INTERVAL_MS` | Default 60s | +| `MAX_BLOCK_RANGE` | Max blocks per `eth_getLogs` (default 10,000) | +| `MIN_GAS_BALANCE_WEI` | Low-gas alert threshold | +| `ALERT_WEBHOOK_URL` | Optional JSON alert webhook | +| `BASE_CHAIN_ID` | Default 8453 | + +## Run + +```sh +npm install && npm run build && npm start +``` + +## Behaviour + +Each cycle it scans new `ReleasePending` logs, drops nonces the vault has +already consumed, then simulates and submits `release()` for the rest. +Simulating first means a still-capped release costs no gas. + +Failures are classified rather than treated alike: + +| Revert | Outcome | +|---|---| +| `ExceedsDailyCap`, `EnforcedPause`, `InsufficientVaultBalance`, `NotEnoughApprovals` | **Retry quietly** — self-heals; this is the normal backlog case | +| `NonceAlreadyConsumed` | **Done** — drop it | +| `ExceedsPerReleaseCap` | **Alert** — cannot self-heal, needs a governance `setCaps` behind the timelock | +| anything else | **Alert** | + +State (scan checkpoint + pending set) is persisted, so a restart resumes +without rescanning from the deployment block or losing pending work. + +**Known limitation:** the pending set is driven by `ReleasePending`, which the +vault emits when a threshold is crossed inside `approve()`. A payload made +releasable purely by a governance `setThreshold` *decrease* emits no such event +and is not picked up here — that path is rare, timelocked, guarded by the +7-day sweep settling period, and covered by runbook RB-6. diff --git a/releaser/package-lock.json b/releaser/package-lock.json new file mode 100644 index 000000000..6374d1ec3 --- /dev/null +++ b/releaser/package-lock.json @@ -0,0 +1,256 @@ +{ + "name": "pen-migration-releaser", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "pen-migration-releaser", + "version": "0.1.0", + "dependencies": { + "viem": "^2.21.0" + }, + "devDependencies": { + "@types/node": "^20.14.0", + "typescript": "^5.5.0" + } + }, + "node_modules/@adraffy/ens-normalize": { + "version": "1.11.1", + "resolved": "https://registry.npmjs.org/@adraffy/ens-normalize/-/ens-normalize-1.11.1.tgz", + "integrity": "sha512-nhCBV3quEgesuf7c7KYfperqSS14T8bYuvJ8PcLJp6znkZpFc0AuW4qBtr8eKVyPPe/8RSr7sglCWPU5eaxwKQ==", + "license": "MIT" + }, + "node_modules/@noble/ciphers": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@noble/ciphers/-/ciphers-1.3.0.tgz", + "integrity": "sha512-2I0gnIVPtfnMw9ee9h1dJG7tp81+8Ob3OJb3Mv37rx5L40/b0i7djjCVvGOVqc9AEIQyvyu1i6ypKdFw8R8gQw==", + "license": "MIT", + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@noble/curves": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.1.tgz", + "integrity": "sha512-k11yZxZg+t+gWvBbIswW0yoJlu8cHOC7dhunwOzoWH/mXGBiYyR4YY6hAEK/3EUs4UpB8la1RfdRpeGsFHkWsA==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@noble/hashes": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/@noble/hashes/-/hashes-1.8.0.tgz", + "integrity": "sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==", + "license": "MIT", + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/base": { + "version": "1.2.6", + "resolved": "https://registry.npmjs.org/@scure/base/-/base-1.2.6.tgz", + "integrity": "sha512-g/nm5FgUa//MCj1gV09zTJTaM6KBAHqLN907YVQqf7zC49+DcO4B1so4ZX07Ef10Twr6nuqYEH9GEggFXA4Fmg==", + "license": "MIT", + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/bip32": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/@scure/bip32/-/bip32-1.7.0.tgz", + "integrity": "sha512-E4FFX/N3f4B80AKWp5dP6ow+flD1LQZo/w8UnLGYZO674jS6YnYeepycOOksv+vLPSpgN35wgKgy+ybfTb2SMw==", + "license": "MIT", + "dependencies": { + "@noble/curves": "~1.9.0", + "@noble/hashes": "~1.8.0", + "@scure/base": "~1.2.5" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/bip39": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/@scure/bip39/-/bip39-1.6.0.tgz", + "integrity": "sha512-+lF0BbLiJNwVlev4eKelw1WWLaiKXw7sSl8T6FvBlWkdX+94aGJ4o8XjUdlyhTCjd8c+B3KT3JfS8P0bLRNU6A==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "~1.8.0", + "@scure/base": "~1.2.5" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@types/node": { + "version": "20.19.43", + "resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.43.tgz", + "integrity": "sha512-6oYBAi5ikg4Pl+kGsoYtawUMBT2zZMCvPNF7pVLnHZfd1zf38DRiWn/gT01RYCdUqkv7Fhr+C9ot4/tb+2sVvA==", + "dev": true, + "license": "MIT", + "dependencies": { + "undici-types": "~6.21.0" + } + }, + "node_modules/abitype": { + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/abitype/-/abitype-1.2.3.tgz", + "integrity": "sha512-Ofer5QUnuUdTFsBRwARMoWKOH1ND5ehwYhJ3OJ/BQO+StkwQjHw0XyVh4vDttzHB7QOFhPHa/o413PJ82gU/Tg==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/wevm" + }, + "peerDependencies": { + "typescript": ">=5.0.4", + "zod": "^3.22.0 || ^4.0.0" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + }, + "zod": { + "optional": true + } + } + }, + "node_modules/eventemitter3": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.1.tgz", + "integrity": "sha512-GWkBvjiSZK87ELrYOSESUYeVIc9mvLLf/nXalMOS5dYrgZq9o5OVkbZAVM06CVxYsCwH9BDZFPlQTlPA1j4ahA==", + "license": "MIT" + }, + "node_modules/isows": { + "version": "1.0.7", + "resolved": "https://registry.npmjs.org/isows/-/isows-1.0.7.tgz", + "integrity": "sha512-I1fSfDCZL5P0v33sVqeTDSpcstAg/N+wF5HS033mogOVIp4B+oHC7oOCsA3axAbBSGTJ8QubbNmnIRN/h8U7hg==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "peerDependencies": { + "ws": "*" + } + }, + "node_modules/ox": { + "version": "0.14.34", + "resolved": "https://registry.npmjs.org/ox/-/ox-0.14.34.tgz", + "integrity": "sha512-12seOIk7dv8eAoGQhcWaeKZxNz304IVcDvb9U5Y7JZAEVe21Nm1YMxLjhWah+su5BD4Omx4Zz0z5x3ij9M4GYQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "dependencies": { + "@adraffy/ens-normalize": "^1.11.0", + "@noble/ciphers": "^1.3.0", + "@noble/curves": "1.9.1", + "@noble/hashes": "^1.8.0", + "@scure/bip32": "^1.7.0", + "@scure/bip39": "^1.6.0", + "abitype": "^1.2.3", + "eventemitter3": "5.0.1" + }, + "peerDependencies": { + "typescript": ">=5.4.0" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + } + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "devOptional": true, + "license": "Apache-2.0", + "peer": true, + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + }, + "node_modules/undici-types": { + "version": "6.21.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", + "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/viem": { + "version": "2.55.19", + "resolved": "https://registry.npmjs.org/viem/-/viem-2.55.19.tgz", + "integrity": "sha512-4QPIX0eYPLsOBk53NKswVMkQoxuP7GlOBnB4wM6dkDokREO4QENNc3bmyPKK1PBTViXh0TPJCHLjIuU20Qi3fg==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "dependencies": { + "@noble/curves": "1.9.1", + "@noble/hashes": "1.8.0", + "@scure/bip32": "1.7.0", + "@scure/bip39": "1.6.0", + "abitype": "1.2.3", + "isows": "1.0.7", + "ox": "0.14.34", + "ws": "8.21.0" + }, + "peerDependencies": { + "typescript": ">=5.0.4" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + } + } + }, + "node_modules/ws": { + "version": "8.21.0", + "resolved": "https://registry.npmjs.org/ws/-/ws-8.21.0.tgz", + "integrity": "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==", + "license": "MIT", + "peer": true, + "engines": { + "node": ">=10.0.0" + }, + "peerDependencies": { + "bufferutil": "^4.0.1", + "utf-8-validate": ">=5.0.2" + }, + "peerDependenciesMeta": { + "bufferutil": { + "optional": true + }, + "utf-8-validate": { + "optional": true + } + } + } + } +} diff --git a/releaser/package.json b/releaser/package.json new file mode 100644 index 000000000..6f4eda818 --- /dev/null +++ b/releaser/package.json @@ -0,0 +1,20 @@ +{ + "name": "pen-migration-releaser", + "version": "0.1.0", + "private": true, + "description": "Retries deferred PEN migration releases on Base: watches ReleasePending and calls the permissionless release() once capacity allows", + "type": "module", + "scripts": { + "build": "tsc", + "start": "node dist/main.js", + "typecheck": "tsc --noEmit", + "test": "tsc && node --test dist/checks.test.js" + }, + "dependencies": { + "viem": "^2.21.0" + }, + "devDependencies": { + "@types/node": "^20.14.0", + "typescript": "^5.5.0" + } +} diff --git a/releaser/src/checks.test.ts b/releaser/src/checks.test.ts new file mode 100644 index 000000000..b7d046f6b --- /dev/null +++ b/releaser/src/checks.test.ts @@ -0,0 +1,44 @@ +import { strict as assert } from "node:assert"; +import { test } from "node:test"; +import { blockRanges, classifyReleaseFailure, toPalletAmount } from "./checks.js"; + +test("a cap-deferred release retries quietly — the case this service exists for", () => { + assert.equal(classifyReleaseFailure("ExceedsDailyCap(1000, 0)"), "retry"); + assert.equal(classifyReleaseFailure("EnforcedPause()"), "retry"); + assert.equal(classifyReleaseFailure("InsufficientVaultBalance()"), "retry"); + assert.equal(classifyReleaseFailure("NotEnoughApprovals(2, 3)"), "retry"); +}); + +test("a consumed nonce is done, not an error", () => { + assert.equal(classifyReleaseFailure("NonceAlreadyConsumed(42)"), "done"); +}); + +test("the per-release ceiling cannot self-heal and is surfaced, not swallowed", () => { + assert.equal(classifyReleaseFailure("ExceedsPerReleaseCap(5000, 3000)"), "blocked"); +}); + +test("anything unmodelled alerts a human", () => { + assert.equal(classifyReleaseFailure("TokenNotSet()"), "unexpected"); + assert.equal(classifyReleaseFailure("connection reset"), "unexpected"); +}); + +test("tokenAmount converts back to the exact palletAmount", () => { + assert.equal(toPalletAmount(5_000_000_000_000_000_000n, 1_000_000n), 5_000_000_000_000n); +}); + +test("a non-exact conversion throws rather than truncating into a wrong payload hash", () => { + assert.throws(() => toPalletAmount(1_000_001n, 1_000_000n), /not a multiple/); + assert.throws(() => toPalletAmount(10n, 0n), /invalid conversionFactor/); +}); + +test("block ranges are chunked for RPC log limits, inclusive and gapless", () => { + assert.deepEqual(blockRanges(1n, 10n, 4n), [ + [1n, 4n], + [5n, 8n], + [9n, 10n], + ]); + // A single block still produces one range. + assert.deepEqual(blockRanges(7n, 7n, 100n), [[7n, 7n]]); + // Nothing new to scan. + assert.deepEqual(blockRanges(11n, 10n, 100n), []); +}); diff --git a/releaser/src/checks.ts b/releaser/src/checks.ts new file mode 100644 index 000000000..3d9965467 --- /dev/null +++ b/releaser/src/checks.ts @@ -0,0 +1,85 @@ +/** + * Pure predicates for the releaser. + * + * Isolated from the chain-client plumbing in `main.ts` so they can be unit + * tested exhaustively (see checks.test.ts), mirroring `attestor/src/checks.ts` + * and `monitor/src/checks.ts`. + */ + +/** What to do with a pending release whose `release()` attempt failed. */ +export type ReleaseOutcome = + /** Resolved on-chain — drop it from the pending set. */ + | "done" + /** Blocked by a condition that heals on its own — keep retrying quietly. */ + | "retry" + /** Blocked by a condition that CANNOT heal without a governance action. */ + | "blocked" + /** Not a condition we model — alert a human. */ + | "unexpected"; + +/** + * Classify a failed `release()` against the vault's revert set. + * + * The distinction that matters operationally is `retry` vs `blocked`: + * + * - `ExceedsDailyCap` clears by itself as the rolling leaky bucket refills, so + * it is the normal, expected outcome of a launch-day backlog and must stay + * silent — it is precisely what this service exists to drain. + * - `ExceedsPerReleaseCap` NEVER clears on its own: the amount is above the + * per-release ceiling, so only a governance `setCaps` (behind the 48h + * timelock) can unblock it. Retrying is harmless but pointless, so it is + * surfaced instead of being swallowed. + * - `NonceAlreadyConsumed` means somebody else got there first (another + * releaser instance, or a conflicting tuple winning the nonce). Benign. + */ +export function classifyReleaseFailure(reason: string): ReleaseOutcome { + if (reason.includes("NonceAlreadyConsumed")) return "done"; + if ( + reason.includes("ExceedsDailyCap") || + reason.includes("EnforcedPause") || + reason.includes("InsufficientVaultBalance") || + // The threshold can drop below the quorum again if an attestor is + // removed after ReleasePending was emitted; a replacement approving + // restores it. + reason.includes("NotEnoughApprovals") + ) { + return "retry"; + } + if (reason.includes("ExceedsPerReleaseCap")) return "blocked"; + return "unexpected"; +} + +/** + * Convert the `tokenAmount` carried by `ReleasePending` back into the + * `palletAmount` that `release()` expects. + * + * The vault derives `tokenAmount = palletAmount * conversionFactor`, so the + * division is always exact. A non-exact result means the event and the vault's + * conversion factor disagree — a data inconsistency we must not paper over by + * silently truncating, because calling `release()` with a truncated amount + * would compute a different payload hash and never match the approvals. + */ +export function toPalletAmount(tokenAmount: bigint, conversionFactor: bigint): bigint { + if (conversionFactor <= 0n) throw new Error(`invalid conversionFactor ${conversionFactor}`); + if (tokenAmount % conversionFactor !== 0n) { + throw new Error(`tokenAmount ${tokenAmount} is not a multiple of conversionFactor ${conversionFactor}`); + } + return tokenAmount / conversionFactor; +} + +/** + * Split [fromBlock, toBlock] into chunks no larger than `maxRange` blocks. + * + * Public Base RPCs cap `eth_getLogs` ranges; a releaser restarting after a long + * outage would otherwise request a span the provider rejects and make no + * progress at all. Returns [] when there is nothing new to scan. + */ +export function blockRanges(fromBlock: bigint, toBlock: bigint, maxRange: bigint): Array<[bigint, bigint]> { + if (maxRange <= 0n) throw new Error(`invalid maxRange ${maxRange}`); + const ranges: Array<[bigint, bigint]> = []; + for (let start = fromBlock; start <= toBlock; start += maxRange) { + const end = start + maxRange - 1n; + ranges.push([start, end > toBlock ? toBlock : end]); + } + return ranges; +} diff --git a/releaser/src/config.ts b/releaser/src/config.ts new file mode 100644 index 000000000..3c6fa3d23 --- /dev/null +++ b/releaser/src/config.ts @@ -0,0 +1,30 @@ +function required(name: string): string { + const value = process.env[name]; + if (!value) { + throw new Error(`Missing required environment variable ${name}`); + } + return value; +} + +export const config = { + /** Base JSON-RPC endpoint. */ + baseRpcUrl: required("BASE_RPC_URL"), + /** MigrationVault contract address on Base. */ + vaultAddress: required("VAULT_ADDRESS") as `0x${string}`, + /** Gas-only signing key. `release()` is permissionless and can only pay the + * recipient the attestors already approved, so this key holds NO privilege + * over the vault — it must never be an attestor, guardian or admin key. */ + releaserPrivateKey: required("RELEASER_PRIVATE_KEY") as `0x${string}`, + /** File persisting the log-scan checkpoint and the pending set. */ + stateFile: process.env.STATE_FILE ?? "./releaser-state.json", + /** Base block to begin scanning `ReleasePending` from on a first run — + * set to the vault's deployment block. */ + startBlock: BigInt(process.env.START_BLOCK ?? "0"), + pollIntervalMs: Number(process.env.POLL_INTERVAL_MS ?? "60000"), + /** Max blocks per `eth_getLogs` call; public RPCs cap this. */ + maxBlockRange: BigInt(process.env.MAX_BLOCK_RANGE ?? "10000"), + /** Alert when the gas wallet drops below this balance (wei). */ + minGasBalanceWei: BigInt(process.env.MIN_GAS_BALANCE_WEI ?? "5000000000000000"), + alertWebhookUrl: process.env.ALERT_WEBHOOK_URL, + baseChainId: Number(process.env.BASE_CHAIN_ID ?? "8453"), +}; diff --git a/releaser/src/main.ts b/releaser/src/main.ts new file mode 100644 index 000000000..509dfc801 --- /dev/null +++ b/releaser/src/main.ts @@ -0,0 +1,232 @@ +/** + * PEN migration releaser. + * + * When a migration reaches the attestor threshold but cannot be released + * immediately — the rolling daily cap is exhausted, the vault is paused, or it + * is under-funded — `MigrationVault.approve` records it as pending and emits + * `ReleasePending` instead of reverting. Those deferrals heal on their own + * (the leaky bucket refills, governance unpauses), but the vault does not + * self-execute: somebody has to call the permissionless `release()`. + * + * Nothing else in the system does. Attestors only ever submit `approve` for new + * finalized events, and the monitor is a deliberately read-only watchdog. So + * without this service a launch-day backlog would sit pending until an operator + * cleared it by hand, one nonce at a time. + * + * Deliberately a separate process rather than a mode of the attestor or the + * monitor: it keeps the audited approve path untouched, keeps the watchdog + * read-only, and its key carries no privilege at all — `release()` can only pay + * the recipient the attestors already agreed on. Two instances can run + * concurrently; the loser of a race simply observes a consumed nonce. + */ + +import { readFileSync, writeFileSync } from "node:fs"; +import { createPublicClient, createWalletClient, defineChain, http } from "viem"; +import { privateKeyToAccount } from "viem/accounts"; +import { blockRanges, classifyReleaseFailure, toPalletAmount } from "./checks.js"; +import { config } from "./config.js"; +import { vaultAbi } from "./vaultAbi.js"; + +interface PendingRelease { + nonce: bigint; + recipient: `0x${string}`; + palletAmount: bigint; +} + +interface PersistedState { + /** Next Base block to scan `ReleasePending` from. */ + fromBlock: string; + /** Pending set, persisted so a restart does not lose work already scanned. */ + pending: Array<{ nonce: string; recipient: string; palletAmount: string }>; +} + +const baseChain = defineChain({ + id: config.baseChainId, + name: "base", + nativeCurrency: { name: "Ether", symbol: "ETH", decimals: 18 }, + rpcUrls: { default: { http: [config.baseRpcUrl] } }, +}); + +const account = privateKeyToAccount(config.releaserPrivateKey); +const publicClient = createPublicClient({ chain: baseChain, transport: http(config.baseRpcUrl) }); +const walletClient = createWalletClient({ account, chain: baseChain, transport: http(config.baseRpcUrl) }); + +/** nonce -> pending release. Keyed by nonce: the vault consumes a nonce once. */ +const pending = new Map(); +let fromBlock = config.startBlock; + +function log(message: string): void { + console.log(`${new Date().toISOString()} ${message}`); +} + +async function alert(subject: string, detail: string): Promise { + console.error(`${new Date().toISOString()} ALERT: ${subject} — ${detail}`); + if (!config.alertWebhookUrl) return; + try { + await fetch(config.alertWebhookUrl, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ service: "pen-releaser", releaser: account.address, subject, detail }), + }); + } catch (webhookError) { + console.error("alert webhook failed", webhookError); + } +} + +function loadState(): void { + try { + const state = JSON.parse(readFileSync(config.stateFile, "utf8")) as PersistedState; + fromBlock = BigInt(state.fromBlock); + for (const p of state.pending) { + pending.set(BigInt(p.nonce), { + nonce: BigInt(p.nonce), + recipient: p.recipient as `0x${string}`, + palletAmount: BigInt(p.palletAmount), + }); + } + } catch { + // First run: start from the configured block with an empty set. + } +} + +function saveState(): void { + const state: PersistedState = { + fromBlock: fromBlock.toString(), + pending: [...pending.values()].map((p) => ({ + nonce: p.nonce.toString(), + recipient: p.recipient, + palletAmount: p.palletAmount.toString(), + })), + }; + writeFileSync(config.stateFile, JSON.stringify(state)); +} + +/** Scan for newly deferred releases and add them to the pending set. */ +async function ingestNewPending(toBlock: bigint, conversionFactor: bigint): Promise { + for (const [start, end] of blockRanges(fromBlock, toBlock, config.maxBlockRange)) { + const logs = await publicClient.getContractEvents({ + address: config.vaultAddress, + abi: vaultAbi, + eventName: "ReleasePending", + fromBlock: start, + toBlock: end, + }); + for (const entry of logs) { + const { nonce, recipient, tokenAmount } = entry.args as { + nonce: bigint; + recipient: `0x${string}`; + tokenAmount: bigint; + }; + if (pending.has(nonce)) continue; + pending.set(nonce, { nonce, recipient, palletAmount: toPalletAmount(tokenAmount, conversionFactor) }); + log(`deferred release observed: nonce=${nonce} recipient=${recipient}`); + } + fromBlock = end + 1n; + } +} + +/** Drop entries the vault has already consumed (by us, a peer, or a rival tuple). */ +async function pruneConsumed(): Promise { + const entries = [...pending.values()]; + if (entries.length === 0) return; + const results = await publicClient.multicall({ + contracts: entries.map((p) => ({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "nonceConsumed" as const, + args: [p.nonce] as const, + })), + allowFailure: true, + }); + results.forEach((result, i) => { + if (result.status === "success" && result.result === true) { + pending.delete(entries[i].nonce); + } + }); +} + +/** Try to push each pending release through; classify what comes back. */ +async function drainPending(): Promise { + for (const p of [...pending.values()]) { + const label = `nonce=${p.nonce} recipient=${p.recipient} amount=${p.palletAmount}`; + try { + // Simulate first so a still-capped release costs no gas. + const { request } = await publicClient.simulateContract({ + account, + address: config.vaultAddress, + abi: vaultAbi, + functionName: "release", + args: [p.nonce, p.recipient, p.palletAmount], + }); + const txHash = await walletClient.writeContract(request); + const receipt = await publicClient.waitForTransactionReceipt({ hash: txHash }); + if (receipt.status !== "success") { + throw new Error(`release reverted on-chain: ${txHash}`); + } + pending.delete(p.nonce); + log(`released: ${label} tx=${txHash}`); + } catch (error) { + const reason = error instanceof Error ? error.message : String(error); + switch (classifyReleaseFailure(reason)) { + case "done": + pending.delete(p.nonce); + log(`skip (already consumed): ${label}`); + break; + case "retry": + // Expected while the daily bucket refills — stay quiet. + break; + case "blocked": + await alert( + "release blocked above the per-release cap", + `${label} — needs a governance setCaps to clear; it cannot self-heal`, + ); + break; + case "unexpected": + await alert("unexpected release failure", `${label} — ${reason}`); + break; + } + } + } +} + +async function checkGasBalance(): Promise { + const balance = await publicClient.getBalance({ address: account.address }); + if (balance < config.minGasBalanceWei) { + await alert("gas balance low", `${account.address} holds ${balance} wei`); + } +} + +async function main(): Promise { + loadState(); + const conversionFactor = await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "conversionFactor", + }); + log(`releaser ${account.address} started; scanning from block ${fromBlock}, ${pending.size} pending`); + await checkGasBalance(); + + for (;;) { + try { + const latest = await publicClient.getBlockNumber(); + await ingestNewPending(latest, conversionFactor); + await pruneConsumed(); + await drainPending(); + saveState(); + log(`ok: ${pending.size} pending, scanned through block ${fromBlock - 1n}`); + } catch (cycleError) { + await alert("releaser cycle failed", `${cycleError}`); + } + await new Promise((resolve) => setTimeout(resolve, config.pollIntervalMs)); + } +} + +setInterval( + () => void checkGasBalance().catch((error) => console.error("gas balance check failed", error)), + 10 * 60 * 1000, +); + +main().catch(async (error) => { + await alert("releaser startup failed", `${error}`); + process.exit(1); +}); diff --git a/releaser/src/vaultAbi.ts b/releaser/src/vaultAbi.ts new file mode 100644 index 000000000..60f34e2b0 --- /dev/null +++ b/releaser/src/vaultAbi.ts @@ -0,0 +1,37 @@ +/** Minimal MigrationVault ABI: only what the releaser needs. */ +export const vaultAbi = [ + { + type: "event", + name: "ReleasePending", + inputs: [ + { name: "nonce", type: "uint64", indexed: true }, + { name: "recipient", type: "address", indexed: true }, + { name: "tokenAmount", type: "uint256", indexed: false }, + ], + }, + { + type: "function", + name: "release", + stateMutability: "nonpayable", + inputs: [ + { name: "nonce", type: "uint64" }, + { name: "recipient", type: "address" }, + { name: "palletAmount", type: "uint256" }, + ], + outputs: [], + }, + { + type: "function", + name: "nonceConsumed", + stateMutability: "view", + inputs: [{ name: "nonce", type: "uint64" }], + outputs: [{ type: "bool" }], + }, + { + type: "function", + name: "conversionFactor", + stateMutability: "view", + inputs: [], + outputs: [{ type: "uint256" }], + }, +] as const; diff --git a/releaser/tsconfig.json b/releaser/tsconfig.json new file mode 100644 index 000000000..59b02f6e5 --- /dev/null +++ b/releaser/tsconfig.json @@ -0,0 +1,14 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "outDir": "dist", + "rootDir": "src", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true + }, + "include": ["src"] +} From 2fb8c1cb806fc5fa3c1949043a4f10c1852f4d44 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 24 Aug 2026 19:07:30 +0200 Subject: [PATCH 08/61] docs: Add operational runbooks, local test plan and review log - pen-migration-runbooks: procedures for attestor key compromise, attestor outage, invariant breach, pause and unpause, Pendulum runtime upgrades, attestor rotation, and the window-close sweep. - pen-migration-local-test-plan: phased runbook for validating the whole stack locally, using Anvil for Base, Chopsticks against real mainnet state for the runtime upgrade and pallet, and Zombienet for the finality-dependent end-to-end path, with pre-mainnet exit criteria. - pen-migration-implementation-overview: what was built and where. - pen-migration-internal-review: the security-assurance record. --- docs/pen-migration-implementation-overview.md | 103 +++++ docs/pen-migration-internal-review.md | 391 ++++++++++++++++++ docs/pen-migration-local-test-plan.md | 226 ++++++++++ docs/pen-migration-runbooks.md | 178 ++++++++ 4 files changed, 898 insertions(+) create mode 100644 docs/pen-migration-implementation-overview.md create mode 100644 docs/pen-migration-internal-review.md create mode 100644 docs/pen-migration-local-test-plan.md create mode 100644 docs/pen-migration-runbooks.md diff --git a/docs/pen-migration-implementation-overview.md b/docs/pen-migration-implementation-overview.md new file mode 100644 index 000000000..c5e5213be --- /dev/null +++ b/docs/pen-migration-implementation-overview.md @@ -0,0 +1,103 @@ +# PEN → Base Migration — Implementation Overview + +**Date:** 2026-07-07 +**Branches:** `feat/pen-base-migration` in this repo and in the portal repo +(`~/Documents/portal`, based on the React 19 branch `fix-issues-with-new-ss58format`; +note: `origin/staging` there is still the older Preact codebase). + +This document is the map of everything built for the migration. Design and +requirements live in the [PRD](pen-base-migration-prd.md); the approach +rationale in [ADR-001](adr-001-pen-base-migration-approach.md); the token +extension decisions in [token standards](pen-token-contract-standards.md). + +## Architecture recap (one paragraph) + +PEN holders call `tokenMigration.migrate(amount, base_address)` on Pendulum; +the amount is burned and a `MigrationInitiated` event with a unique nonce is +emitted. Four attestor daemons (initially team-operated, PRD D4) watch +relay-finalized blocks — each on its own node — and submit +`approve(nonce, recipient, amount)` to the MigrationVault on Base; the 3rd +matching approval releases pre-minted tokens. +The PEN ERC-20 has its entire max issuance minted to the vault at deployment +and no mint function — worst-case loss is bounded by the vault's rate caps, +watched by an independent monitor that can auto-pause. One-way by design. + +## Components delivered + +### Pendulum repo (`feat/pen-base-migration`) + +| Component | Location | Status | +|---|---|---| +| `token-migration` pallet | `pallets/token-migration/` | Burn-and-emit `migrate` (user) + `migrate_treasury`/`set_treasury_destination` (governance, fixed Base destination) extrinsics sharing one nonce space and event; unique nonces, dust/ED + lock handling, KeepAlive treasury withdraw, pause origin; 20 unit tests + benchmark test suite (frame-benchmarking v2) | +| Runtime wiring | `runtime/pendulum/src/lib.rs` | Pallet index 102, min amount 1 PEN, pause = root/half-council or 2/3 technical committee, added to `BaseFilter` whitelist and `define_benchmarks`; compiles with and without `runtime-benchmarks` (Foucoco intentionally skipped — production-direct decision) | +| `PEN.sol` | `contracts/src/` | Fixed-supply `ERC20 + ERC20Permit + ERC20Votes`, EIP-6372 timestamp clock, full supply minted to vault, no owner/mint/proxy | +| `MigrationVault.sol` | `contracts/src/` | 3-of-4 on-chain approvals per exact tuple, permanent nonce consumption, 12→18 decimal conversion in one place, per-release + daily caps (defer, not kill), guardian pause (approvals recorded while paused), rotation retroactively invalidates removed attestors, two-step admin, pending-release accounting protecting the timelocked remainder sweep | +| `PENGovernor.sol` | `contracts/src/` | OZ Governor composition through a TimelockController (hybrid governance, timestamp clock) | +| Deploy scripts | `contracts/script/` | `Deploy.s.sol` (vault→token→setToken dance, admin handover to bootstrap Safe), `DeployGovernance.s.sol` (timelock+governor role wiring, deployer admin renounced); parameters documented in `contracts/.env.example` | +| Contract tests | `contracts/test/` | 30 Foundry tests incl. fuzz (supply invariant), full Governor proposal lifecycle, replay/race/rotation/caps/pause/sweep-pending scenarios | +| Attestor daemon | `attestor/` | TypeScript; finalized-heads-only, strictly ordered blocks, crash-safe checkpoint, idempotent + race-tolerant approvals, fail-fast on decode errors (4-field shape asserted), startup set-membership check, low-gas/webhook alerts; ops guide in its README | +| Invariant monitor | `monitor/` | Independent watchdog: conservation checks (block-pinned reads) + per-nonce liveness; webhook alerts; optional guardian auto-pause | +| Runbooks | `docs/pen-migration-runbooks.md` | RB-1…RB-6: key compromise, outage, invariant breach, pause/unpause, runtime upgrade, attestor rotation | +| Internal security review | `docs/pen-migration-internal-review.md` | Independent adversarial pass; 2 high + 1 medium findings, all fixed (see below) | + +### Portal repo (`feat/pen-base-migration`) + +| Component | Location | Status | +|---|---|---| +| Migration page | `src/pages/migration/` | Amount validation (transferable, minimum, migrate-all-or-leave-ED), EIP-55 address validation with checksummed preview, `eth_getCode` contract-destination warning + extra confirmation, irreversibility confirmation, pause banner, locked-balance hint, post-finalization release tracking (approvals x/3 → released, BaseScan link) | +| Pallet hook | `src/hooks/migration/useMigrationPallet.tsx` | Extrinsic submission resolving at finality with the emitted nonce; pause query; on-chain constants | +| Base status hook | `src/hooks/migration/useBaseReleaseStatus.ts` | Polls the vault over plain JSON-RPC (no EVM dependency; selectors precomputed, keccak via `@polkadot/util-crypto`) | +| EVM helpers | `src/helpers/ethereum.ts` | EIP-55 checksum, payload-hash mirroring the vault's `abi.encode`, minimal `eth_call`/`eth_getCode` client | +| Config | `src/constants/migration.ts` | Vault address via `VITE_MIGRATION_VAULT_ADDRESS`, Base RPC via `VITE_BASE_RPC_URL`; page degrades gracefully when unset | +| Routing/nav | `src/app.tsx`, `src/components/Layout/links.tsx` | `/pendulum/migration`; nav item hidden on other tenants | + +## Internal security review — summary + +Adversarial review of the whole stack found and fixed: (1) attestor daemons +crash-looping on the *normal* k-of-n approval race — now a benign re-checked +skip; (2) monitor reads not pinned to one block — could false-positive a +conservation alert and auto-pause the vault; (3) `sweepRemainder` could +strand quorum-approved-but-deferred releases — now excluded via +`pendingApprovedAmount` accounting with a timelocked `clearStalePending` +restricted to consumed nonces. Details and verified-not-vulnerable list in +[pen-migration-internal-review.md](pen-migration-internal-review.md). + +## Verification status + +- Pallet: `cargo test -p token-migration` 11/11 (incl. benchmark suite). +- Runtime: `cargo check -p pendulum-runtime` clean, both feature sets. +- Contracts: `forge test` 35/35 (incl. 512-run fuzz). +- Attestor & monitor: `tsc --noEmit` clean. +- Portal: `yarn build` (tsc + vite) clean against `main`; committed through lint-staged. + +Seven internal adversarial review rounds have run; each found real issues +(sometimes in a prior round's own fix), all fixed with regression tests and +recorded in [pen-migration-internal-review.md](pen-migration-internal-review.md). +**Decision (PRD §9): no external audit is commissioned** — the residual risk is +consciously accepted and carried by the threat-model mitigations (caps, +independent monitoring + auto-pause, guardian, ≥48h timelock, separation of +duties) plus the conservative soft launch. Standing practice: any change to the +fund-release path triggers a fresh internal review round before deployment. + +## Commit map (this repo) + +`docs → pallet → contracts(core) → runtime wiring → contracts(governance) → +attestor → monitor → runbooks → benchmarks → security fixes → env template` +— see `git log` on the branch for hashes. + +## Still open (cannot be done from the repo) + +1. **Decisions D1–D6** (PRD §4.2) — all recorded as decided (burn, 18 + decimals, 150M with the rounding delta to the treasury, 3-of-4 + team-operated attestors, 3-month internal window target subject to the + community discussion, transferable-only migration). The final window and + parameters are fixed by the formal governance proposal. +2. Security sign-off before mainnet funding: no external audit will be + commissioned (PRD §9) — a final internal review pass over the shipped + revision, plus the operational drills. +3. Benchmark run on reference hardware → replace manual weights. +4. Attestor operator onboarding + key ceremonies; Safe setups (D4). +5. Exchange coordination, DefiLlama/CoinGecko supply endpoints, comms. +6. Portal deploy config: set `VITE_MIGRATION_VAULT_ADDRESS` once deployed; + decide whether the portal feature must be ported to the Preact `staging` + branch or ships with the React 19 codebase. diff --git a/docs/pen-migration-internal-review.md b/docs/pen-migration-internal-review.md new file mode 100644 index 000000000..08f61eee5 --- /dev/null +++ b/docs/pen-migration-internal-review.md @@ -0,0 +1,391 @@ +# PEN Migration — Internal Security Review (running log) + +**Date:** 2026-07-07 +**Scope:** MigrationVault.sol, PEN.sol, PENGovernor.sol, deploy scripts, +token-migration pallet, attestor daemon, invariant monitor. +**Method:** adversarial review by an independent reviewer agent against the +PRD requirements (P1–P9, V1–V9, A1–A5, M1–M4), cross-checked against the test +suite. This log is the project's security-assurance record (PRD §9): **no +external audit is commissioned** — the residual risk is consciously accepted +and carried by the threat-model mitigations (PRD §8) and the review rounds +recorded here. + +## Findings and resolutions + +### 1. HIGH — Attestor daemon treated the normal 3-of-5 race as fatal +With five independent attestors racing to approve the same event, the two +whose transactions land after the third matching approval revert with +`NonceAlreadyConsumed`. The daemon treated any revert as fatal (alert + +exit), meaning two attestors would crash-loop on nearly every migration — +alert fatigue that could mask real incidents. + +**Resolution (fixed):** on any submission failure the daemon re-checks +`nonceConsumed`/`hasApproved`; if the migration is resolved on-chain the race +is logged as benign and processing continues. Unexplained failures still +alert and exit (PRD A5 preserved). + +### 2. HIGH — Monitor reads were not pinned to one block +`totalReleased` and `balanceOf(vault)`/`totalSupply` were read in separate +batches without a block tag. A release landing between the batches would +produce a false `VAULT BALANCE MISMATCH` and — with a guardian key configured +— an unjustified auto-pause (48h+ to undo post-handover, since unpause is +timelocked). + +**Resolution (fixed):** all Base-side reads in a check cycle are pinned to a +single `blockNumber`. The Pendulum-then-Base read ordering of the M2a check +was confirmed safe by construction (burns finalize strictly before releases). + +### 3. MEDIUM — `sweepRemainder` could strand approved-but-deferred releases +A payload can reach quorum while its release is deferred (pause or caps); the +owed tokens still sit in the vault balance. Sweeping the full balance at +window close would leave such a release permanently unexecutable — the user +already burned on Pendulum. + +**Resolution (fixed):** the vault now tracks `pendingApprovedAmount` +(payloads that crossed the threshold without releasing; cleared on release). +`sweepRemainder` transfers `balance − pendingApprovedAmount`. A timelocked +`clearStalePending` exists for pending entries of *consumed* nonces only +(conflicting tuples that lost the race); unconsumed pending entries are owed +to their migrator and can never be cleared. Covered by three new tests. + +### Minor (fixed in the same pass) +- Attestor event decoding now asserts the 4-field event shape explicitly, so + a runtime upgrade that changes the event fails loudly instead of decoding + positionally into garbage. +- The attestor's periodic gas-balance check no longer swallows RPC errors. + +## Explicitly verified as not vulnerable +- **Replay/double-release:** `nonceConsumed` gates both `approve` and + `release` and is set before the transfer; conflicting tuples never merge. +- **Reentrancy:** checks-effects-interactions ordering in `_release`; the + token is hook-free OZ code. +- **Admin takeover / role wiring:** two-step admin transfer; deploy scripts + leave no dangling deployer privileges; timelock self-administered. + +## Round 2 (2026-07-07, second independent reviewer over the full diff of both repos) + +### C1. CRITICAL — Zero-address migration deadlocked the entire attestor fleet +`migrate(amount, H160::zero())` was accepted by the pallet and the portal, but +the vault deterministically rejects a zero recipient. Every attestor would hit +the same permanent revert at the same block, alert, exit, and — because the +checkpoint only advances after a block fully processes — crash-loop forever. +One 1-PEN transaction could halt every migration behind it for all five +operators simultaneously. + +**Resolution (fixed, defense in depth):** the pallet rejects +`H160::zero()` (`InvalidBaseAddress`, with test); the portal validator rejects +the zero address; the attestor statically detects vault-unreleasable tuples +(zero recipient/amount), raises a distinct CRITICAL alert, and skips past the +event instead of crash-looping — such an event can now only mean a +pallet/vault validation mismatch. + +### H1. HIGH — Re-adding a removed attestor could cross a threshold outside `approve()` +`activeApprovals` counted historical approvals against the current attestor +set, so `addAttestor` re-adding an address with stale recorded approvals could +push a payload over the threshold without running the pending-release +accounting in `approve()` — re-opening the sweep-stranding hole of round-1 +finding 3 through a rotation side door. + +**Resolution (fixed structurally):** attestor **generations**. Every +`addAttestor` bumps the address's generation and approvals only count while +their recorded generation matches — a re-added attestor must approve again, so +the threshold can only ever be crossed inside `approve()`. The public +`hasApproved` view now means "holds a currently-valid approval" (same ABI, so +the daemon keeps working and correctly re-approves after a re-add). Covered by +a regression test. + +### Round 2 explicitly verified as not vulnerable +Portal EIP-55 implementation and keccak string semantics; portal/attestor +payload-hash construction exactly mirroring the vault's `abi.encode`; +pallet↔attestor↔vault↔portal event-field alignment; replay/reentrancy/ +conflicting-tuple logic (re-confirmed); monitor block-pinning fix; +`clearStalePending` restrictions; deploy-script role wiring. + +## Round 3 (2026-07-07, third independent reviewer, focused on the round-1/2 fixes) + +All three findings trace to the round-1 `sweepRemainder`/`pendingApprovedAmount` +mechanism never being re-verified against *sub-threshold* in-flight migrations +or against the monitor's conservation formula. + +### C1(r3). CRITICAL — `sweepRemainder` could strand an in-flight migration and crash-loop the fleet +`pendingApprovedAmount` reserves only payloads that have already crossed the +threshold. A migration with 1–2 approvals at sweep time reserved nothing, so +`sweepRemainder` (which swept `balance − pendingApprovedAmount`) could remove +its tokens. When the remaining attestors then crossed the threshold, the +inline release in `approve()` reverted on insufficient balance — rolling back +the approval, and, because every attestor hit it identically, permanently +crash-looping 3-of-5 daemons and halting all future migrations. + +**Resolution (fixed):** +- `approve()` now includes vault balance in its `releasable` check, so an + under-funded release **defers** (marks pending) instead of reverting — the + fleet can never crash-loop on it, and the debt stays tracked and recoverable + after a governance refund. (`release()` gained a matching + `InsufficientVaultBalance` guard.) +- `sweepRemainder(to, amount)` now takes an explicit amount bounded by + `balance − pendingApprovedAmount` (saturating, so a prior over-sweep can't + cause an underflow revert), forcing conscious reconciliation. +- New runbook **RB-7** (window close) mandates pausing the pallet and + confirming zero outstanding nonces via the monitor before sweeping. +- Tests: `test_OverSweptInFlightMigrationDefersAndRecovers` proves the fleet + stays up and the migration recovers; sweep tests updated to the new + signature. + +### H1(r3). HIGH — Monitor's M2b check ignored `sweepRemainder` +`sweepRemainder` moved tokens out without touching `totalReleased`, so the +monitor's `balance + totalReleased == totalSupply` check would fire a +guaranteed false `VAULT BALANCE MISMATCH` — and auto-pause — on the first +legitimate window-close sweep. + +**Resolution (fixed):** the vault now tracks `totalSwept` (incremented in +`sweepRemainder`); the monitor checks `balance + totalReleased + totalSwept == +totalSupply`. RB-7 also notes a mismatch coinciding with a `RemainderSwept` +event is expected, not a compromise signal. The fuzz invariant test now +includes `totalSwept`. + +### M1(r3). MEDIUM — `hasApproved` disagreed with `activeApprovals` +`hasApproved` checked only the generation, not `isAttestor`, so it reported +`true` for an attestor removed and never re-added (the standard RB-1 outcome), +while that approval counts 0 toward releases. Low live impact (only the +attestor self-check reads it) but wrong on a public view meant for +auditability. + +**Resolution (fixed):** `hasApproved` now requires `isAttestor` too, mirroring +`activeApprovals`. Covered by `test_HasApprovedFalseForRemovedAttestor`. + +### Round 3 explicitly verified as not vulnerable +The H1 generation mechanism itself (no double-count, no stale re-match, no +double-increment of pending); C1's `isUnreleasable` completeness for the +`approve` revert set; reentrancy; governance cannot bypass the attestor quorum +or move the immutable sweep timestamp; deploy-script role wiring against the +actual vendored OZ v5.4.0 `TimelockController`; unbounded-`_approvers` +gas-griefing (not practically reachable). + +## Round 4 (2026-07-08, resumed first reviewer, full-diff pass incl. the portal UI) + +Verified all eight round-1/2/3 fixes are correctly implemented and cleared the +portal migration UI (EIP-55, payload-hash parity, finality-gated submission). +Two novel findings, both in the same class as prior rounds — a threshold +crossed outside `approve()`, and the cap backstop: + +### H1(r4). HIGH — `setThreshold` decrease could retroactively strand a payload +Lowering the threshold can make a sub-threshold payload releasable without +routing through `approve()`, so its amount is never added to +`pendingApprovedAmount`; a later `sweepRemainder` could then sweep it, and its +`release()` reverts `InsufficientVaultBalance` until governance refunds. + +**Resolution (fixed):** `setThreshold` records the time of any decrease; +`sweepRemainder` is blocked for `SWEEP_SETTLING_PERIOD` (7 days) afterwards, +giving the monitor and a permissionless `release()` time to settle any +newly-qualifying payload first. Runbook RB-6 updated. Recoverable and +detectable even absent the guard (RB-7 reconciliation shows the shortfall). +Covered by `test_ThresholdCutBlocksSweepDuringSettling`. + +### H2(r4). HIGH — Daily cap was a fixed calendar-day bucket, not a rolling window +The `currentDay` bucket reset to zero at the UTC boundary, letting a +compromised quorum release `dailyCap` at 23:59 and again at 00:00 — 2× the +intended blast-radius bound (PRD V4 specifies a *rolling* 24h maximum). + +**Resolution (fixed):** replaced with a leaky-bucket rolling limiter — +`dailyCap` capacity refilling linearly at `dailyCap`/day +(`availableDailyAllowance()`), so a burst is capped at `dailyCap` and a second +burst must wait ~24h for the bucket to refill. No instant reset at any +boundary. Covered by `test_DailyCapRefillsGraduallyOverRollingWindow` and +`test_DailyCapHasNoInstantResetAtBoundary`. Residual: over a *rolling* 24h a +full bucket plus full refill still totals up to ~2× `dailyCap`, but spread over +24h rather than instantaneous — size `dailyCap` accordingly. + +## Round 5 (2026-07-09, in-depth review focused on bricking the pipeline) + +Adversarial pass over the whole stack asking specifically where an outsider +could brick releases. The on-chain fund path (rounds 1–4) held up; the finding +was in the off-chain monitor. + +### H1(r5). HIGH — A dust PEN transfer to the vault permanently tripped the monitor's M2b check +The M2b conservation check used strict equality +(`balance + totalReleased + totalSwept != totalSupply`). Under all legitimate +contract logic that sum is *exactly* `totalSupply`, so equality could only ever +break *upward* — via tokens arriving in the vault outside the release path. +Anyone could do that permissionlessly: `PEN.transfer(vault, 1 wei)`, or a +`migrate(_, )` whose 3rd approval self-transfers into the vault. +The break is permanent (the surplus can only leave via the post-window +`sweepRemainder`), so every poll re-fired the highest-severity alert and — +with `GUARDIAN_PRIVATE_KEY` set — re-paused the vault every cycle, wedging all +releases for the rest of the window while burns kept accruing on Pendulum. + +**Resolution (fixed):** +- M2b now alerts only on a **deficit** (`balance + released + swept < + totalSupply`); a surplus is ignored. A deficit is the only direction that can + signal real loss (a genuine unauthorized release keeps the sum equal and is + caught by M2a). Alert renamed `VAULT BALANCE DEFICIT`. +- Defense in depth on-chain: `MigrationVault.approve` rejects `recipient == + address(this)` (`RecipientIsVault`), closing the self-migration variant at the + single point approvals are recorded. +- The conservation/liveness predicates were extracted to `monitor/src/checks.ts` + and unit-tested (`checks.test.ts`): surplus-does-not-fire, deficit-fires, + exact-holds. Vault side covered by `test_ApproveRejectsVaultRecipient` and + `test_VaultRecipientNeverReleasesAndPreservesInvariant`. + +### M1(r5). MEDIUM — Monitor liveness scan could starve the conservation checks +M4 read `nonceConsumed` one nonce at a time, sequentially, every poll. During a +pause or cap-deferral every migration stays unconsumed, so the scan grew with +the backlog and could push a cycle past the poll interval — starving the M2a/M2b +checks exactly when they matter most. **Fixed:** per-nonce reads are batched +through Multicall3. + +### L1(r5). LOW — M2a read ordering made safe by construction, not just by latency +The monitor read Pendulum `totalMigrated` before the Base totals. A burn +finalizing between the two reads and released before the Base read could momentarily +show `released > migrated`. It was unreachable in practice (release latency ≫ the +read gap) but is now removed outright: Base is read first, then the +monotonically-growing `totalMigrated` at a strictly-later snapshot, so M2a cannot +false-positive on an in-flight burn. + +## Round 6 (2026-07-09, review focused on outsider bricking of the off-chain fleet) + +The on-chain fund path (rounds 1–5) held up. The headline finding is the +round-5 fix re-opening the round-2 DoS class one layer out, in the attestor. + +### C1(r6). CRITICAL — A 1-PEN migration to the vault address crash-loops the whole attestor fleet +Round 5 made `MigrationVault.approve` revert `RecipientIsVault` on a +vault-recipient tuple (closing the monitor surplus-wedge). But the attestor's +`isUnreleasable` gate — which skips permanently-reverting tuples so the fleet +never crash-loops on them — was not updated in lockstep: it flagged only the +zero address and zero amount. The pallet cannot reject the vault address (it has +no knowledge of Base state), so `migrate(1 PEN, )` reaches every +attestor as a well-formed event, its `approve` reverts deterministically, the +daemon treats the revert as fatal and exits, and — the checkpoint never having +advanced past the block — reprocesses the same block on restart, forever. All +five attestors hit the same finalized block and crash-loop together, halting +every migration behind it for the price of one 1-PEN transaction. Same class as +round-2 C1 (zero-address DoS). + +**Resolution (fixed):** `isUnreleasable` now also flags `recipient == +vaultAddress` (case-insensitively), so a vault-recipient event is skipped with a +distinct CRITICAL alert instead of crash-looping — the vault-aware attestor is +the *only* component that can defend against this, since the pallet can't see +the Base address and the portal check is bypassable by calling the extrinsic +directly. The predicate was extracted to `attestor/src/checks.ts` and +unit-tested (`attestor/src/checks.test.ts`, mirroring the monitor's `checks.ts`); +the attestor previously had no unit tests, which is why the coupling gap slipped +through. **Architectural note for all future review rounds:** the set of deterministic +`approve` reverts and the attestor's `isUnreleasable` set must stay in exact +lockstep — this is the third round a vault-side "reject bad tuple" change +reopened a fleet-crash hole. A shared, tested enumeration of the reverting +preconditions would end this recurrence. + +### M1(r6). MEDIUM — Minimum migration amount was below the fleet's per-migration gas cost +`MinimumMigrationAmount` was 1 PEN (~$0.0086 at $0.00858/PEN), below the ~3 Base +`approve` txs (~$0.01–$1 depending on Base gas) the fleet spends per migration, +making dust-spam a cheap asymmetric gas-drain grief on all five operators. +**Fixed:** raised to 100 PEN (~$0.86), which dominates fleet gas cost across +normal Base conditions while staying negligible for real holders. Tunable via +runtime upgrade; revisit if the PEN price or Base gas regime shifts materially. + +### L1(r6). LOW — Monitor re-fired liveness alerts every poll for a stalled or unreleasable nonce +The M4 liveness check paged for every past-grace unconsumed nonce on every poll, +so a pause backlog — or a burn to a structurally-unreleasable address (zero or +the vault, which never consumes) — produced an unbounded alert storm, training +on-call to ignore the highest-severity page. **Fixed:** liveness re-alerts are +throttled to at most once per grace period per nonce (cleared on consumption), +preserving outage visibility without the storm. + +### Round 6 explicitly verified as not vulnerable +Monitor auto-pause cannot be weaponised by an outsider (M2b fires only on a +deficit, unreachable without stealing from the vault; M2a can't false-positive +given the Base-first read ordering); the vault's opportunistic-release path +defers rather than reverts on pause/caps/insufficient-balance, so only a +vault-side *input* revert (now fully enumerated in `isUnreleasable`) can reach +the attestor's fatal path; `pendingApprovedAmount` is not inflatable by an +outsider (pending entries require three real attestations of real burns); +`migrate` correctly refuses locked/staked/vesting balance and the dust/ED +remainder check is sound. + +## Round 7 (2026-07-09, review focused on outsider exploit/brick across the full stack) + +The on-chain fund path (rounds 1–6) held up under an independent re-derivation. +The core invariant — a release threshold can only ever be crossed inside +`approve()` (the sole exception, a `setThreshold` *decrease*, is governance-gated +and settling-period-guarded) — was re-confirmed, so `pendingApprovedAmount` is a +complete reservation against `sweepRemainder` and no outsider can strand or +double-release. The `isUnreleasable` ⇄ `approve()` revert lockstep is currently +consistent (zero recipient / vault recipient / zero amount; the nonce-consumed +and already-approved reverts are absorbed by the daemon's `alreadyHandled` +re-check). The one novel finding is off-chain, in the monitor. + +### M1(r7). MEDIUM — Monitor liveness scan re-read every nonce ever created, every poll +The round-5 fix batched the per-nonce `nonceConsumed` reads through Multicall3 to +stop the liveness scan from outrunning the poll interval. But the scan still +rebuilt its working set from scratch each poll: `for (nonce = 0; nonce < +nextNonce) if (!nonceFirstSeen.has(nonce)) set(nonce, now)` re-added *every* nonce +not currently in the map — including nonces already consumed and pruned. So each +poll re-inserted and re-read all consumed nonces via Multicall, making the scan +O(all migrations ever created) rather than O(pending backlog) and growing without +bound for the whole migration window. At high migration volume this lengthens each +check cycle (so the M2a/M2b conservation checks, which run first, fire less often) +and eventually risks the liveness Multicall failing outright. It is not a +fund-loss or correctness bug — genuinely-pending nonces keep their original +first-seen time, so no missed or false liveness alerts — but it silently negated a +fix the team believed was in place, worst exactly as the migration succeeds. + +**Resolution (fixed):** the monitor tracks a high-water mark +(`nextNonceIncorporated`) and stamps only nonces in `[nextNonceIncorporated, +nextNonce)` each poll (`newNonces` in `monitor/src/checks.ts`), so a +consumed-and-pruned nonce is never re-added and the scan is O(pending backlog) as +round 5 intended. Covered by a new `checks.test.ts` regression asserting a +consumed nonce does not reappear on the following poll. + +### C1(r7). LOW — A migration above the per-release cap looks stuck to the user +A single migration whose released amount exceeds `perReleaseCap` is burned on +Pendulum and reaches quorum, but its release defers (marked pending) until +governance raises the cap — recoverable, yet potentially a long, opaque wait. The +pallet cannot bound this (it has no knowledge of the Base-side cap). +**Resolution (fixed):** the portal migration page reads the vault's live +`perReleaseCap` and, before submission, warns that an above-cap amount will be +held until governance raises the cap, requiring an explicit confirmation and +recommending the user split into sub-cap migrations (`getPerReleaseCap` in +`src/helpers/ethereum.ts`; warning + confirmation checkbox in the migration page). + +### Round 7 — deployment / ops notes (no code change) +- **Guardian slot vs. monitor auto-pause.** The vault has a single `guardian` + address and `pause()` accepts only `guardian`/`admin`. PRD G5 wants the guardian + to be a fast human Safe; PRD M3 wants the monitor to hold a guardian key for + auto-pause. One slot cannot be both (a Safe can't be driven by the monitor's + single EOA). Decide consciously at the key ceremony (D4). Post-handover + consequence: a monitor-EOA guardian whose key leaks can pause, and `unpause` is + then a ≥48h-timelocked governance action — a bounded but real griefing halt (the + documented "guardian can at worst halt" trade-off). +- **Max issuance vs. staking inflation (PRD D3).** `MAX_ISSUANCE` is minted once + and is immutable; if Pendulum total issuance ever grew past it (e.g. via staking + inflation) during the window, late migrants could burn against a drained vault. + Confirmed **non-applicable**: staking rewards/inflation are set to zero on-chain + (`set_inflation`), so issuance is static — the genesis `InflationInfo` in + `node/src/chain_spec.rs` is historical. Because the parameter is immutable, + re-confirm at deploy time that `MAX_ISSUANCE` covers live issuance and that + rewards remain zero for the window's duration. +- **Two-step admin handover window.** `Deploy.s.sol` calls + `transferAdmin(adminSafe)`, but the deployer stays admin until the Safe calls + `acceptAdmin()`. Correct (two-step prevents a wrong-address handover), yet the + deployer key is a live admin during the gap — treat it as sensitive and complete + `acceptAdmin` promptly. + +### Round 7 explicitly verified as not vulnerable +Replay/double-release, conflicting-tuple non-merging, reentrancy (CEI + hook-free +token), attestor generations (no threshold crossing outside `approve()`), +`_approvers` bounded even under repeated rotation (`_inApprovers` dedup), +`palletAmount * conversionFactor` cannot overflow uint256, `clearStalePending` +underflow-safe and consumed-nonce-restricted; pallet burn atomicity, nonce +monotonicity/overflow guard, dust/ED check against the `withdraw(TRANSFER)` lock +enforcement, and the new `migrate_treasury`/`set_treasury_destination` pair +(origin-gated, `KeepAlive`, shared nonce space, zero-address rejection); governor +deploy-script role wiring (deployer admin renounced, executor = anyone, +self-administered timelock) and PEN↔Governor clock-mode consistency; monitor +auto-pause not weaponisable by an outsider (unchanged from round 6). + +## Residual risks and standing practices (no external audit — risk accepted) +- Every change to the fund-release path (vault release/approve/sweep logic, + pallet burn path) gets a fresh independent adversarial review round before + deployment — rounds 5–7 re-derived the round-4 fixes and the full outsider + surface, and this practice replaces the external-audit backstop. +- The attestor's positional event decode is shape-checked but still assumes + field order; re-verify against metadata after any runtime upgrade (RB-5). diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md new file mode 100644 index 000000000..85ebcd102 --- /dev/null +++ b/docs/pen-migration-local-test-plan.md @@ -0,0 +1,226 @@ +# PEN Migration — Local Test Plan + +A runbook for validating the full migration stack on a laptop, with no public +testnet involved. Work through the phases in order; each has explicit pass +criteria. Phases 1–2 are independent and can be done in either order; phase 3 +needs both. + +**Tooling, and what each one actually proves** + +| Tool | Stands in for | Proves | +|---|---|---| +| **Anvil** (Foundry) | Base | Contract behaviour, deploy script, attestor/monitor wiring | +| **Chopsticks** | Pendulum mainnet | Runtime upgrade + pallet against **real** balances, locks, vesting, treasury | +| **Zombienet** | Relay + parachain | The **relay-chain finality** path — attestors only act on finalized blocks, which Chopsticks cannot faithfully reproduce | + +Chopsticks gives realistic *state*; Zombienet gives realistic *finality*. You +need both, for different reasons. Neither requires Paseo or Foucoco. + +> Note: the public discussion post commits to testing on "Foucoco and Base +> Sepolia". Local testing does not discharge that commitment — either wire the +> pallet into the Foucoco runtime for a public run, or amend the messaging. + +--- + +## Phase 0 — Prerequisites + +```bash +# Build the runtime wasm that will be tested as the upgrade +cargo build --release -p pendulum-runtime +# -> target/release/wbuild/pendulum-runtime/pendulum_runtime.compact.compressed.wasm + +# Contracts + services +cd contracts && forge build +cd ../attestor && npm install && npm run build +cd ../monitor && npm install && npm run build +``` + +Confirm the baseline is green before starting: + +```bash +cargo test -p token-migration --features runtime-benchmarks +cd contracts && forge test +cd ../attestor && npm test +cd ../monitor && npm test +``` + +**Pass:** all suites green. + +--- + +## Phase 1 — Base side on Anvil + +Goal: the contracts behave as specified against a real EVM, and the deploy +script works with realistic parameters. + +```bash +anvil --port 8545 # terminal 1 +cd contracts +cp .env.example .env # fill in: 4 attestor addrs, Safes, caps, MAX_ISSUANCE, EARLIEST_SWEEP_TS +forge script script/Deploy.s.sol --rpc-url http://localhost:8545 --broadcast +``` + +Then verify, with `cast`: + +1. `PEN.totalSupply()` == `MAX_ISSUANCE`, and `PEN.balanceOf(vault)` == the same. +2. `vault.threshold()` == 3, `vault.attestorCount()` == 4. +3. `vault.paused()` == false; `vault.token()` == the PEN address. +4. Approve one migration from 3 attestor keys → recipient receives + `palletAmount × 1e6`; `nonceConsumed(nonce)` == true. +5. Approve from only 2 → nothing released. +6. **Cap deferral:** approve an amount above `perReleaseCap` from 3 attestors → + no release, `pendingApprovedAmount` increases, `ReleasePending` emitted. + Then `setCaps` higher and call `release(...)` → succeeds. +7. **Rolling cap:** consume the full `dailyCap`, confirm + `availableDailyAllowance()` == 0, warp 12h (`evm_increaseTime`), confirm it + has refilled by half. +8. **Guardian pause:** pause from the guardian key → `release` reverts; + unpause is rejected from the guardian and accepted from admin. +9. `sweepRemainder` reverts before `earliestSweepTimestamp`. + +**Pass:** all nine behave as described. (These mirror the Foundry suite, but +run against the deployed bytecode and the real deploy script — that is the +point.) + +--- + +## Phase 2 — Pendulum side on Chopsticks (real mainnet state) + +Goal: the runtime upgrade applies cleanly, ships **paused**, and the pallet +behaves correctly against genuine holder state — locked, vesting, staked and +whale accounts as they exist today. + +`chopsticks.yml`: + +```yaml +endpoint: wss://rpc-pendulum.prd.pendulumchain.tech +mock-signature-host: true +db: ./chopsticks-db.sqlite +port: 8000 +``` + +```bash +npx @acala-network/chopsticks@latest --config chopsticks.yml \ + --wasm-override target/release/wbuild/pendulum-runtime/pendulum_runtime.compact.compressed.wasm +``` + +Produce a block (`dev_newBlock`) so the upgrade takes effect, then check: + +1. **Ships paused (the critical one).** `tokenMigration.paused()` == `true` + immediately after the upgrade, with no storage written. Any `migrate` call + fails `MigrationsPaused`. +2. `tokenMigration.nextNonce()` == 0, `totalMigrated()` == 0, + `treasuryDestination()` == None. +3. Unpause via sudo/root (`setPaused(false)`), then run the cases below. +4. **Happy path:** fund a dev account via `dev_setStorage`, `migrate(amount, + 0x…)` → balance drops, total issuance drops by the same amount, + `MigrationInitiated` carries `{nonce, who, base_address, amount}` in that + field order (the attestor decodes positionally — this is the check that + catches event drift). +5. **Encumbered balances, against real accounts.** Pick a genuinely staked + account and a genuinely vesting account from mainnet state and confirm + `migrate` of the locked portion fails; the transferable portion succeeds. +6. **Dust/ED rule:** migrating all-but-a-sliver fails `WouldLeaveDust`; + migrating the entire free balance succeeds. +7. **Zero address:** `migrate(amount, 0x000…0)` fails `InvalidBaseAddress`. +8. **Treasury path:** `setTreasuryDestination` then `migrateTreasury` from + root — burns from the real `py/trsry` account, keeps it alive, and emits an + event identical in shape to a user migration. +9. **Nonce continuity:** several migrations across both paths share one + monotonic nonce sequence with no gaps or reuse. + +**Pass:** 1–9 all hold. Item 1 is the launch-safety property; do not proceed if +it fails. + +--- + +## Phase 3 — End-to-end with real finality (Zombienet + Anvil) + +Goal: the whole pipeline works when the Substrate side has genuine +relay-chain finality — the condition the attestors depend on. + +You already have `zombienet-macos-arm64` in the repo root. Spin up a relay +plus the Pendulum parachain with the new runtime, and run Anvil alongside with +the contracts from phase 1. + +Then start the **four attestors, the monitor and the releaser**, each with its own +`.env` — separate keys, separate checkpoint files, all pointed at the same +vault: + +```bash +PENDULUM_WS=ws://127.0.0.1:9944 BASE_RPC_URL=http://localhost:8545 \ +VAULT_ADDRESS=0x… ATTESTOR_PRIVATE_KEY=0x… CHECKPOINT_FILE=./cp1.json \ +npm start # repeat for attestors 2–4 with distinct keys/checkpoints +``` + +The releaser needs only a gas-funded key and the Base endpoint — no Pendulum +connection, and no privileges over the vault: + +```bash +BASE_RPC_URL=http://localhost:8545 VAULT_ADDRESS=0x… \ +RELEASER_PRIVATE_KEY=0x… START_BLOCK=0 npm start +``` + +Checks: + +1. **Full path:** unpause the pallet, `migrate` from a funded account → within + a block or two of finality, 3 approvals land and the recipient's PEN + balance on Anvil equals `amount × 1e6`. +2. **The race is benign.** All four attestors see the same event; the two that + lose the race log a skip and **stay running**. No crash-loop, no fatal + alert. (This is the failure mode that took three review rounds to get + right — verify it explicitly.) +3. **Restart safety:** kill an attestor mid-run, restart it → it resumes from + its checkpoint, re-derives nothing twice, no duplicate release. +4. **Outage tolerance:** stop one attestor → migrations still release (3 of 4 + remain). Stop a second → releases stop cleanly, nothing is lost, and the + monitor raises a liveness alert. Restart both → the backlog drains. +5. **Deferred releases drain by themselves.** Set a low `dailyCap`, migrate + enough to exhaust it, and confirm the excess is marked pending + (`ReleasePending`) rather than reverting — then that the releaser picks it + up and completes it as the bucket refills, with **no manual intervention**. + Restart the releaser mid-backlog and confirm it resumes from its state file. +6. **Monitor invariants:** the monitor logs `ok` with + `balance + released + swept == totalSupply` holding continuously. +7. **Conservation alarm:** manually transfer PEN out of the vault on Anvil to + create a deficit → the monitor alerts and (if `GUARDIAN_PRIVATE_KEY` is + set) auto-pauses the vault. **Then verify the reverse:** send PEN *into* + the vault → surplus is tolerated, no false alert. +8. **Portal:** run the portal against the local chain with + `VITE_MIGRATION_VAULT_ADDRESS` set to the Anvil vault; migrate through the + UI and watch the status card go 0/3 → 3/3 → released. + +**Pass:** 1–8 all hold. + +--- + +## Phase 4 — Failure drills (the runbooks) + +Rehearse each runbook once against the local stack, so the first time you run +them is not during an incident: + +| Runbook | Drill | +|---|---| +| RB-1 key compromise | Remove an attestor mid-flight; confirm its recorded approvals stop counting and a replacement can complete the quorum | +| RB-2 outage | Covered by phase 3 item 4 | +| RB-3 invariant breach | Covered by phase 3 item 6 — including the auto-pause and the recovery path | +| RB-4 pause/unpause | Pause the pallet *and* the vault; confirm the correct resume order | +| RB-5 runtime upgrade | Apply a second runtime upgrade while attestors run; confirm they keep decoding (or fail loudly rather than silently skipping) | +| RB-6 attestor rotation | Add a 5th attestor, remove an old one, confirm a re-added address must approve again | +| RB-7 window close | Warp past `earliestSweepTimestamp`, reconcile, sweep, confirm `totalSwept` and that the monitor does not false-alarm | + +--- + +## Phase 5 — Exit criteria before mainnet + +- [ ] Phases 1–4 pass end to end. +- [ ] The upgrade ships paused, verified on a Chopsticks fork of **live** + mainnet state (not a fresh chain). +- [ ] A cap-deferred release recovers correctly without manual contract + surgery. +- [ ] All four attestors survive a full run without a fatal exit. +- [ ] The monitor alerts on a real injected deficit and tolerates a surplus. +- [ ] Benchmarks re-run on reference hardware and the generated weights + replace the manual estimates. +- [ ] A dry run of the deploy script with the **final** production parameters, + reviewed by someone other than whoever wrote the `.env`. diff --git a/docs/pen-migration-runbooks.md b/docs/pen-migration-runbooks.md new file mode 100644 index 000000000..a8274fa1d --- /dev/null +++ b/docs/pen-migration-runbooks.md @@ -0,0 +1,178 @@ +# PEN Migration — Operational Runbooks + +Runbooks required by PRD acceptance criterion 10. Each must be drill-tested +before mainnet launch. Contact points, Safe addresses and paging channels are +filled in during the attestor onboarding ceremony (decision D4). + +**Shared prerequisites:** access to the alerting channel; read access to a +Pendulum node and a Base RPC; the on-call sheet mapping attestor index → +operator → contact. Escalation path everywhere: on-call engineer → migration +tech lead → guardian Safe signers → admin Safe signers. + +--- + +## RB-1: Suspected attestor key compromise + +**Trigger:** an `Approved` event from an attestor for a tuple that does not +match any finalized Pendulum `MigrationInitiated` event (monitor M2a alert, or +manual observation), or an operator reports infrastructure compromise. + +1. **Pause first, investigate second.** Any guardian signer (or the monitor's + auto-pause) calls `vault.pause()`. Releases stop; approvals keep recording. +2. Confirm the mismatch: compare the suspicious `Approved(nonce, recipient, + palletAmount, attestor)` event against the pallet's `MigrationInitiated` + events at the finalized head (`tokenMigration` section). +3. If confirmed compromised: admin Safe executes `vault.removeAttestor(x)`. + Removal retroactively invalidates all of that attestor's recorded + approvals — no released funds can result from them afterwards. +4. Operator rotates infrastructure and generates a **new** key; admin Safe + executes `vault.addAttestor(newKey)`. Never re-add a possibly-leaked key. +5. Reconcile: verify `totalReleased <= TotalMigrated × conversionFactor` and + that every consumed nonce maps 1:1 to a pallet event. If value was lost, + follow the incident-disclosure policy before unpausing. +6. Admin Safe executes `vault.unpause()`. Deferred releases can be executed by + anyone via `vault.release(nonce, recipient, palletAmount)`. + +**Rollback:** none needed; pausing is side-effect-free. + +## RB-2: Attestor outage (no key compromise) + +**Trigger:** monitor M4 liveness alert (nonce unreleased past the grace +period) or an attestor's own restart-loop/low-gas alerts. + +1. Determine how many attestors are down. With 1 of 4 down, releases + continue — treat as routine ops. With 2+ down, migrations queue up + harmlessly (approvals missing, nothing to roll back) — escalate to the + affected operators. +2. Common causes, in order of frequency: Base gas wallet empty (fund it; the + daemon logs the address), Pendulum node not synced/finalizing, checkpoint + file pointing at a pruned block (re-point `START_BLOCK` at a block the node + still has, never past unprocessed migrations), daemon restart-loop after a + runtime upgrade (see RB-5). +3. After recovery the daemon catches up from its checkpoint automatically; + duplicate approvals are impossible (pre-checks + contract dedup). +4. Verify recovery: the queued nonces release as approvals arrive; monitor + goes back to `ok`. + +## RB-3: Conservation invariant violation + +**Trigger:** monitor alert `CONSERVATION VIOLATION` or `VAULT BALANCE +DEFICIT`. This is the highest-severity alert the system can produce. + +**Note:** the monitor only alerts on a *deficit* (tokens missing), never on a +surplus. A plain inbound transfer of PEN into the vault (a donation, or a +migration whose recipient is the vault) raises the balance above the +conservation identity but is harmless and is deliberately ignored — it can no +longer false-trigger an auto-pause (round-5 fix). The vault also rejects the +vault address as a release recipient at `approve`. + +1. Auto-pause should already have fired; **verify `vault.paused() == true`** + and pause manually if not. Do not unpause until step 5. +2. Rule out monitor error: recompute both sides by hand from independent RPC + endpoints (`TotalMigrated` at the finalized head; `totalReleased`, + `balanceOf(vault)`, `totalSupply` on Base). +3. If real: identify the offending `Released` events (those whose nonce has no + matching pallet event) and the attestors who approved them → continue with + RB-1 steps 3–5 for every implicated attestor. Assume quorum compromise: + rotate **all** keys unless positively excluded. +4. Quantify the loss (sum of unmatched releases) and follow the disclosure + policy. Consider whether caps need lowering before resumption. +5. Unpause only with sign-off from the admin Safe quorum and a written + incident report. + +## RB-4: Pause / unpause (routine procedure) + +**Pause** (guardian Safe, any authorized signer, or admin): +`vault.pause()` — instant; releases stop, approvals keep recording, `migrate` +on Pendulum is unaffected (pause that separately if needed, see below). + +**Pendulum-side pause** (for pallet-level incidents or coordinated stops): +`tokenMigration.setPaused(true)` via root/half-council, or 2/3 technical +committee for fast response. This stops new burns at the source. + +**Unpause:** admin Safe executes `vault.unpause()` (or `setPaused(false)` on +the pallet via governance). After a vault unpause, deferred releases are +retried permissionlessly via `vault.release(...)` — the attestor fleet does +not need to do anything. + +**Order in a coordinated stop:** pause the pallet first (stop new burns), then +the vault. Resume in reverse order. + +## RB-5: Pendulum runtime upgrade + +**Risk:** a runtime upgrade can change event encoding or the pallet's index, +which would make attestor daemons exit on decode failure (by design, PRD A5). + +Before the upgrade is enacted: +1. Check the diff for changes to `pallets/token-migration`, its event type or + its `construct_runtime` index. No changes → notify operators, no action. +2. If the event shape changed: update and release a new daemon version; + operators deploy it **before** the upgrade block. +3. Nonce monotonicity across upgrades is a pallet invariant (P2) — any + migration touching `NextNonce` storage must preserve it; reject one that + doesn't. + +After enactment: +4. Watch the fleet: all four daemons progressing past the upgrade block, test + migration of a small amount end-to-end, monitor `ok` lines resuming. +5. If daemons exit on the upgrade block: they hold position (checkpoint stays + put) — fix decoding, redeploy, they resume without loss. + +## RB-7: Window close and remainder sweep + +**Trigger:** the migration window is closing (per decision D5) and governance +wants to sweep the unmigrated remainder to its designated destination. + +**Why this needs care:** `sweepRemainder` reserves only *threshold-approved* +pending releases (`pendingApprovedAmount`). A migration that was burned on +Pendulum but is still gathering attestor approvals is **not** reserved — sweep +it and, while the attestor fleet no longer crash-loops (the release simply +defers and stays recoverable), that user's tokens must be restored by a +governance refund before they can be released. Avoid this by reconciling +first. + +1. **Stop new burns:** pause the pallet — `tokenMigration.setPaused(true)` via + governance/technical committee (RB-4). No new migrations can start. +2. **Wait a finality + processing buffer** (at least the relay finality window + plus a generous attestor-processing margin; hours, not minutes) so every + already-finalized migration reaches the vault and is released. +3. **Reconcile via the monitor:** confirm `TotalMigrated × conversionFactor == + totalReleased + pendingApprovedAmount` and that the monitor reports **zero** + outstanding/unreleased nonces for a sustained window. Resolve any pending + or deferred releases (raise caps / unpause / `release`) before proceeding. +4. **Compute the sweep amount** off-chain: `balance − pendingApprovedAmount`, + and sanity-check it against expected unmigrated supply. Do not sweep more. +5. **Sweep:** admin (timelock) calls `sweepRemainder(destination, amount)`. + The call reverts (`ExceedsSweepable`) if the amount exceeds + `balance − pendingApprovedAmount`, as a last-line guard. +6. **Verify:** `totalSwept` increased by `amount`; the monitor's conservation + check (`balance + totalReleased + totalSwept >= totalSupply`, alerting only + on a deficit) still holds and does **not** alert (it accounts for + `totalSwept`). + +**If a still-in-flight migration was swept anyway:** its release defers with +`InsufficientVaultBalance` and is marked pending. To make the user whole, +governance transfers the owed token amount back to the vault, then anyone +calls `release(nonce, recipient, palletAmount)`. + +## RB-6: Attestor set / threshold change (planned) + +1. Admin Safe (timelocked post-handover: expect the configured delay between + scheduling and execution) calls `addAttestor` / `removeAttestor` / + `setThreshold`. Invariants enforced on-chain: threshold ≥ 2 and ≤ attestor + count. +2. Sequence for replacing an operator: `addAttestor(new)` first, wait for + their daemon to be live and approving, then `removeAttestor(old)`. +3. Removed attestors' recorded approvals stop counting immediately; pending + migrations that relied on them simply need approvals from the remaining + set (the new attestor's daemon backfills from its `START_BLOCK` — set it + to a block before the oldest unreleased migration). +4. **Lowering the threshold** (`setThreshold` to a smaller value) can make a + payload that was one approval short suddenly releasable, *without* routing + through `approve()` — so its owed amount is not registered in + `pendingApprovedAmount`. The contract guards this: `sweepRemainder` is + blocked for `SWEEP_SETTLING_PERIOD` (7 days) after any threshold decrease. + During that window, call `release(...)` on every migration that the new, + lower threshold now satisfies (the M4 liveness monitor lists them), so each + is properly released or re-registered before the next sweep. Never sweep + right after cutting the threshold. From 3c47488827bf1ddc382b448cd6f24724f72a686e Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 24 Aug 2026 19:16:27 +0200 Subject: [PATCH 09/61] contracts: Fill in the decided deployment parameters Records the agreed values in the deploy template: 150M max issuance, 1,000,000 PEN soft-launch caps to be raised to 3,000,000 by governance after the soft launch, and an earliest-sweep floor of 2027-03-01. Per-release and daily caps are set equal deliberately. A release blocked by the daily cap heals on its own as the rolling bucket refills and the releaser retries it; one above the per-release cap can only be cleared by a governance setCaps behind the timelock. Keeping them equal removes that permanently-stuck band. The earliest-sweep floor is deliberately later than the ~3-month window we intend to advertise. The two are separate numbers: closing the window, pausing the pallet and shutting down the attestors are all independent of sweeping, so a later floor costs nothing operationally -- the remainder simply waits in the vault. A shorter floor is the only irrecoverable choice, since an immutable timestamp cannot be extended afterwards, and it would downgrade the guarantee to holders from "impossible by code" to "possible via a governance vote". The docs are updated to state the target and the floor as distinct figures. Also marks which values are permanent -- max issuance, the earliest sweep timestamp and the conversion factor -- versus the addresses, caps and threshold, which governance can change after deployment. --- contracts/.env.example | 61 +++++++++++++++++++++------ docs/pen-base-migration-prd.md | 2 +- docs/pen-migration-window-analysis.md | 13 ++++-- 3 files changed, 58 insertions(+), 18 deletions(-) diff --git a/contracts/.env.example b/contracts/.env.example index 69426e3d4..70a067ec0 100644 --- a/contracts/.env.example +++ b/contracts/.env.example @@ -1,25 +1,56 @@ # --- Deploy.s.sol (migration stack, PRD rollout phase 3) --- # forge script script/Deploy.s.sol --rpc-url $BASE_RPC_URL --broadcast --verify +# +# Only three values here are PERMANENT once deployed: MAX_ISSUANCE (in the +# token) and EARLIEST_SWEEP_TS + the conversion factor (in the vault). Caps, +# attestors, threshold, guardian and admin are all changeable later by the +# admin/governance, so they do not have to be final on day one. -# Bootstrap Safe that becomes vault admin (accepts via vault.acceptAdmin()) +# Bootstrap Safe that becomes vault admin (accepts via vault.acceptAdmin()). +# Changeable later via transferAdmin; handed to the Timelock at phase 5. ADMIN_SAFE= -# Fast pause guardian (small-threshold Safe) +# Fast pause guardian (small-threshold Safe). Can pause but NOT unpause, so a +# compromised guardian can only halt, never release. Its signers must hold no +# attestor keys. Changeable later via setGuardian. GUARDIAN_SAFE= # The four attestor transaction-sender addresses (decision D4: 3-of-4, -# initially team-operated, one per internal RPC node) +# initially team-operated, one per internal RPC node). +# Changeable later via addAttestor / removeAttestor. ATTESTOR_1= ATTESTOR_2= ATTESTOR_3= ATTESTOR_4= -# Max issuance in 18-decimal units. Decision D3: 150M PEN -# = 150000000000000000000000000 (cross-check canonical tokenomics at deploy) -MAX_ISSUANCE= -# Initial caps in 18-decimal units (PRD V4: target < 1-2% of vault per day) -PER_RELEASE_CAP= -DAILY_CAP= -# Unix timestamp before which the remainder cannot be swept. -# Decision D5: ~ deploy + 3 months (see docs/pen-migration-window-analysis.md) -EARLIEST_SWEEP_TS= + +# PERMANENT. Max issuance in 18-decimal units. Decision D3: 150M PEN. +MAX_ISSUANCE=150000000000000000000000000 + +# Caps in 18-decimal units. Set equal to each other on purpose: a release +# blocked by the DAILY cap heals by itself as the rolling bucket refills and +# the releaser retries it, whereas one above the PER-RELEASE cap can only be +# cleared by a governance setCaps behind the 48h timelock. Keeping them equal +# removes that permanently-stuck band entirely. +# +# Soft launch: 1,000,000 PEN (below, active). Raise both to 3,000,000 by +# governance once the soft-launch checks pass — 3M/day is ~2% of supply per +# day, which clears the full supply well inside the migration window. +PER_RELEASE_CAP=1000000000000000000000000 +DAILY_CAP=1000000000000000000000000 + +# PERMANENT. Unix timestamp before which the remainder CANNOT be swept. This is +# a floor on the earliest a sweep could happen, not a deadline for holders: +# closing the window is a separate governance decision and can be deferred +# indefinitely. See docs/pen-migration-window-analysis.md. +# +# Deliberately set beyond the ~3-month window we intend to advertise: a later +# floor costs nothing operationally (the remainder simply waits in the vault, +# and closing the window, pausing the pallet and shutting down the attestors +# are all independent of sweeping), while a shorter one would permanently +# downgrade the guarantee to holders from "impossible by code" to "possible +# via a governance vote". It also sits inside the range the community +# discussion consulted on. +# +# 1803859200 = 2027-03-01 00:00 UTC +EARLIEST_SWEEP_TS=1803859200 # --- DeployGovernance.s.sol (phase 5) --- PEN_TOKEN= @@ -27,4 +58,8 @@ TIMELOCK_DELAY=172800 # 48h (PRD V5) VOTING_DELAY=86400 # 1 day, seconds (timestamp clock) VOTING_PERIOD=432000 # 5 days PROPOSAL_THRESHOLD= # token units required to propose -QUORUM_FRACTION=4 # percent of total supply; start low (PRD G1) +# Percent of TOTAL supply (fixed at 150M), counting For+Abstain votes actually +# cast. Set this against the delegated supply observed at handover, not a guess +# now: the vault's unmigrated balance is in the denominator but never votes, so +# too high a figure makes governance unusable early (PRD G1). +QUORUM_FRACTION=2 diff --git a/docs/pen-base-migration-prd.md b/docs/pen-base-migration-prd.md index 6c3b5746d..268e142e3 100644 --- a/docs/pen-base-migration-prd.md +++ b/docs/pen-base-migration-prd.md @@ -62,7 +62,7 @@ The migration is **one-way**. No reverse flow (Base → Pendulum) will be built. | D2 | Decimals on Base | **18** (scale ×10⁶ at release, in the vault only — V7). Exact conversion; no change to holder amounts, ownership share, or max supply. (Implemented.) | | D3 | Max issuance | **Exactly 150,000,000 PEN — deliberately rounded up from the live Pendulum issuance (~149.93M, no inflation).** The live figure is an untidy artifact of the chain's history (fee burns etc.), not a meaningful tokenomics number; a clean canonical 150M is what trackers, integrators and the immutable constructor should carry, and it stays valid even as small fee burns keep nudging the live figure. Consequence: the **rounding delta (~67k PEN, ~0.045%) can never be released by migration** — no matching burns can ever exist for it — so it sits inert in the vault, is excluded from circulating supply, and at window close moves to the **community treasury via the governed, timelocked sweep (D5)**. It is not allocated to any person or team; only a governance decision can ever spend it. Per-holder conversion remains exactly 1:1. | | D4 | Attestor set composition | **4 attestors, all initially team-operated**, one per internal RPC node, threshold **3-of-4**. Rationale: Pendulum currently has no external node operators, and onboarding them means asking outsiders to fund + run a node for the whole window. Honest consequence: organizational independence is *not* claimed — blast-radius controls (caps, monitoring, guardian pause, ≥48h timelock) and **separation of duties** (guardian + monitor operated by someone other than the attestor-key holder) carry the security. | -| D5 | Migration window end policy | Internal working target: **3-month earliest close** (`earliestSweepTimestamp ≈ deploy + 3 months`), conditional on the planned block-time improvement toward 12s; a referendum (`vesting-manager.remove_vesting_schedule`) force-unlocks any vesting residue and the permanent sentinel locks before close ([window analysis](pen-migration-window-analysis.md)). **Final window length is deliberately left to the community discussion / formal governance proposal** (the discussion post solicits 6-month / 12-month / open-ended feedback). An earliest close date is *not* an automatic sweep: moving any remainder requires a separate governance decision + timelocked execution. | +| D5 | Migration window end policy | Target migration window **~3 months**, with the immutable on-chain floor set later at **`earliestSweepTimestamp` = 2027-03-01** so the remainder provably cannot move before then. The two are deliberately different numbers: the target is what we aim to communicate and operate to, the floor is the guarantee to holders. Conditional on the planned block-time improvement toward 12s; a referendum (`vesting-manager.remove_vesting_schedule`) force-unlocks any vesting residue and the permanent sentinel locks before close ([window analysis](pen-migration-window-analysis.md)). **Final window length is deliberately left to the community discussion / formal governance proposal** (the discussion post solicits 6-month / 12-month / open-ended feedback). An earliest close date is *not* an automatic sweep: moving any remainder requires a separate governance decision + timelocked execution. | | D6 | Encumbered balances policy | **Only unstaked, freely transferable PEN migrates** (enforced by the pallet). Staked/vesting/governance-locked balances must be freed first; UI and docs surface this. No vesting locks should extend beyond the window — residue handled per D5's referendum path. | ## 5. System overview diff --git a/docs/pen-migration-window-analysis.md b/docs/pen-migration-window-analysis.md index 7b605423d..8f4727358 100644 --- a/docs/pen-migration-window-analysis.md +++ b/docs/pen-migration-window-analysis.md @@ -73,9 +73,14 @@ contract. ## What the shorter window changes operationally -1. **`earliestSweepTimestamp` ≈ deploy + 3 months.** It is immutable and marks - the *earliest* allowed sweep — setting it at 3 months preserves the option - to wind down on schedule while never forcing it. +1. **The advertised window and the on-chain floor are separate numbers.** We + aim to complete migration in ~3 months, but `earliestSweepTimestamp` — the + immutable moment before which the remainder provably cannot move — is set + to **2027-03-01**, roughly six months out. A later floor costs nothing + operationally: closing the window, pausing the pallet and shutting down the + attestors are all independent of sweeping, so the remainder simply waits in + the vault. A shorter floor would be the only irrecoverable choice, since it + cannot be extended afterwards. 2. **Daily-cap throughput now matters.** Migrating ~150M PEN within ~90 days needs an *average* release throughput of ~1.7M PEN/day. The PRD's initial-cap guidance (~1–2% of vault per day = 1.5–3M/day) is compatible, @@ -95,7 +100,7 @@ contract. ## Recommendation (under the 3-month decision) -- Set `earliestSweepTimestamp ≈ deploy + 3 months`. +- Set `earliestSweepTimestamp` = **1803859200** (2027-03-01 00:00 UTC). - Land the block-time improvement **within the first ~6 weeks** of the window; track window-average block time against the ~15.6s break-even. - Pre-draft the vesting-unlock referendum so it can be submitted at ~month 2 From a6f81006eb6f320d16a0a0de4e28cd28e7e7587b Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Thu, 27 Aug 2026 16:31:01 +0200 Subject: [PATCH 10/61] testing: Add a local validation harness for the migration Automates phase 2 of the local test plan: the Pendulum side against a Chopsticks fork of live mainnet state, with the new runtime applied as a wasm override. Exercises what the unit tests cannot -- that the upgrade applies to real storage, that it ships paused, and that migrate behaves against genuine holder state including vesting, staking locks and the real treasury account. Exits non-zero on failure so it can gate a step. Two Chopsticks behaviours are worked around and documented, because both produce silently wrong results rather than errors: storage overrides are applied after extrinsics within a block, so every write gets its own block; and the human-readable setStorage form treats a falsy value as a deletion, which for Paused means it reads back as its `true` default, so that key is written as raw 0x00. The config sets no `db:` deliberately -- a persisted database carries forward blocks the harness produced, which would make the ships-paused assertion pass or fail for the wrong reason. The script also refuses to run against a chain that is not fresh. --- testing/.gitignore | 2 + testing/README.md | 51 ++ testing/chopsticks.yml | 17 + testing/package-lock.json | 927 ++++++++++++++++++++++++++++++++ testing/package.json | 15 + testing/src/harness.mjs | 45 ++ testing/src/phase2-pendulum.mjs | 238 ++++++++ 7 files changed, 1295 insertions(+) create mode 100644 testing/.gitignore create mode 100644 testing/README.md create mode 100644 testing/chopsticks.yml create mode 100644 testing/package-lock.json create mode 100644 testing/package.json create mode 100644 testing/src/harness.mjs create mode 100644 testing/src/phase2-pendulum.mjs diff --git a/testing/.gitignore b/testing/.gitignore new file mode 100644 index 000000000..c3610e597 --- /dev/null +++ b/testing/.gitignore @@ -0,0 +1,2 @@ +node_modules/ +.chopsticks-db.sqlite* diff --git a/testing/README.md b/testing/README.md new file mode 100644 index 000000000..fec564ab3 --- /dev/null +++ b/testing/README.md @@ -0,0 +1,51 @@ +# PEN Migration — local validation harness + +Automates the checks in [`docs/pen-migration-local-test-plan.md`](../docs/pen-migration-local-test-plan.md) +so they run as a command with pass/fail output, instead of a manual checklist. +Exits non-zero on any failure, so it can gate a deployment step. + +## Phase 2 — Pendulum on Chopsticks + +Exercises what unit tests cannot: that the runtime upgrade applies to **real +mainnet storage**, that it ships **paused**, and that `migrate` behaves against +genuine holder state — whales, vesting schedules, staking locks and the real +`py/trsry` treasury account. + +```bash +# 1. build the runtime that will be proposed as the upgrade +cargo build --release -p pendulum-runtime + +# 2. fork mainnet with that runtime applied +npx @acala-network/chopsticks@latest --config testing/chopsticks.yml \ + --wasm-override target/release/wbuild/pendulum-runtime/pendulum_runtime.compact.compressed.wasm + +# 3. run the checks +cd testing && npm install +node src/phase2-pendulum.mjs +``` + +### Two things that will bite you + +**Chopsticks must be fresh for every run.** The phase asserts the state of a +*just-upgraded* chain, so a chain that has already run the harness will report +`nextNonce != 0` and the script refuses to continue. The config deliberately +sets no `db:` — a persisted database carries forward the blocks the harness +itself produced, which would make the ships-paused check pass or fail for the +wrong reason. Restart Chopsticks between runs. + +**Storage overrides land after extrinsics in the same block.** `dev_setStorage` +followed immediately by a transaction means the transaction sees the *old* +state. Every helper here produces a block after writing storage. Related: the +human-readable object form treats a falsy value as a *deletion*, so setting +`Paused: false` deletes the key — and because `Paused` defaults to `true`, it +reads back paused. The harness writes that key as a raw `0x00` instead. + +There is no sudo pallet on Pendulum, so root-only calls (`migrate_treasury`, +`set_paused`) cannot be dispatched here. The harness simulates their effect via +`dev_setStorage`; the extrinsics themselves are covered by the pallet's unit +tests. + +## Still to add + +Phase 1 (contracts on Anvil) and phase 3 (end-to-end with Zombienet, four +attestors, the monitor and the releaser) are still manual — see the test plan. diff --git a/testing/chopsticks.yml b/testing/chopsticks.yml new file mode 100644 index 000000000..a4fb332dc --- /dev/null +++ b/testing/chopsticks.yml @@ -0,0 +1,17 @@ +# Forks live Pendulum mainnet state so the runtime upgrade and the pallet are +# exercised against real balances, vesting schedules, staking locks and the +# real treasury account -- not a fresh genesis. +# +# npx @acala-network/chopsticks@latest --config testing/chopsticks.yml \ +# --wasm-override target/release/wbuild/pendulum-runtime/pendulum_runtime.compact.compressed.wasm +# +# mock-signature-host lets the harness submit extrinsics from any account +# without real signatures, so we can drive migrations from dev keys. +endpoint: wss://rpc-pendulum.prd.pendulumchain.tech +mock-signature-host: true +block: null +# No `db:` on purpose. A persisted database carries forward the blocks this +# harness itself produces, so a re-run would start already-unpaused and the +# ships-paused check -- the single most important assertion in this phase -- +# would silently pass or fail for the wrong reason. Every run forks afresh. +port: 8000 diff --git a/testing/package-lock.json b/testing/package-lock.json new file mode 100644 index 000000000..0c728562d --- /dev/null +++ b/testing/package-lock.json @@ -0,0 +1,927 @@ +{ + "name": "pen-migration-testing", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "pen-migration-testing", + "version": "0.1.0", + "dependencies": { + "@polkadot/api": "^11.3.1", + "@polkadot/keyring": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2" + } + }, + "node_modules/@noble/curves": { + "version": "1.9.7", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.7.tgz", + "integrity": "sha512-gbKGcRUYIjA3/zCCNaWDciTMFI0dCkvou3TL8Zmy5Nc7sJ47a0jtOeZoTaMxkuqRo9cRhjOdZJXegxYE5FN/xw==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@noble/hashes": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/@noble/hashes/-/hashes-1.8.0.tgz", + "integrity": "sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==", + "license": "MIT", + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@polkadot-api/json-rpc-provider": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/json-rpc-provider/-/json-rpc-provider-0.0.1.tgz", + "integrity": "sha512-/SMC/l7foRjpykLTUTacIH05H3mr9ip8b5xxfwXlVezXrNVLp3Cv0GX6uItkKd+ZjzVPf3PFrDF2B2/HLSNESA==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot-api/json-rpc-provider-proxy": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/json-rpc-provider-proxy/-/json-rpc-provider-proxy-0.0.1.tgz", + "integrity": "sha512-gmVDUP8LpCH0BXewbzqXF2sdHddq1H1q+XrAW2of+KZj4woQkIGBRGTJHeBEVHe30EB+UejR1N2dT4PO/RvDdg==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot-api/metadata-builders": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/metadata-builders/-/metadata-builders-0.0.1.tgz", + "integrity": "sha512-GCI78BHDzXAF/L2pZD6Aod/yl82adqQ7ftNmKg51ixRL02JpWUA+SpUKTJE5MY1p8kiJJIo09P2um24SiJHxNA==", + "license": "MIT", + "optional": true, + "dependencies": { + "@polkadot-api/substrate-bindings": "0.0.1", + "@polkadot-api/utils": "0.0.1" + } + }, + "node_modules/@polkadot-api/observable-client": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/@polkadot-api/observable-client/-/observable-client-0.1.0.tgz", + "integrity": "sha512-GBCGDRztKorTLna/unjl/9SWZcRmvV58o9jwU2Y038VuPXZcr01jcw/1O3x+yeAuwyGzbucI/mLTDa1QoEml3A==", + "license": "MIT", + "optional": true, + "dependencies": { + "@polkadot-api/metadata-builders": "0.0.1", + "@polkadot-api/substrate-bindings": "0.0.1", + "@polkadot-api/substrate-client": "0.0.1", + "@polkadot-api/utils": "0.0.1" + }, + "peerDependencies": { + "rxjs": ">=7.8.0" + } + }, + "node_modules/@polkadot-api/substrate-bindings": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/substrate-bindings/-/substrate-bindings-0.0.1.tgz", + "integrity": "sha512-bAe7a5bOPnuFVmpv7y4BBMRpNTnMmE0jtTqRUw/+D8ZlEHNVEJQGr4wu3QQCl7k1GnSV1wfv3mzIbYjErEBocg==", + "license": "MIT", + "optional": true, + "dependencies": { + "@noble/hashes": "^1.3.1", + "@polkadot-api/utils": "0.0.1", + "@scure/base": "^1.1.1", + "scale-ts": "^1.6.0" + } + }, + "node_modules/@polkadot-api/substrate-client": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/substrate-client/-/substrate-client-0.0.1.tgz", + "integrity": "sha512-9Bg9SGc3AwE+wXONQoW8GC00N3v6lCZLW74HQzqB6ROdcm5VAHM4CB/xRzWSUF9CXL78ugiwtHx3wBcpx4H4Wg==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot-api/utils": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/@polkadot-api/utils/-/utils-0.0.1.tgz", + "integrity": "sha512-3j+pRmlF9SgiYDabSdZsBSsN5XHbpXOAce1lWj56IEEaFZVjsiCaxDOA7C9nCcgfVXuvnbxqqEGQvnY+QfBAUw==", + "license": "MIT", + "optional": true + }, + "node_modules/@polkadot/api": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api/-/api-11.3.1.tgz", + "integrity": "sha512-q4kFIIHTLvKxM24b0Eo8hJevsPMme+aITJGrDML9BgdZYTRN14+cu5nXiCsQvaEamdyYj+uCXWe2OV9X7pPxsA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/api-augment": "11.3.1", + "@polkadot/api-base": "11.3.1", + "@polkadot/api-derive": "11.3.1", + "@polkadot/keyring": "^12.6.2", + "@polkadot/rpc-augment": "11.3.1", + "@polkadot/rpc-core": "11.3.1", + "@polkadot/rpc-provider": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-augment": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/types-create": "11.3.1", + "@polkadot/types-known": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "eventemitter3": "^5.0.1", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/api-augment": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api-augment/-/api-augment-11.3.1.tgz", + "integrity": "sha512-Yj+6rb6h0WwY3yJ+UGhjGW+tyMRFUMsKQuGw+eFsXdjiNU9UoXsAqA2dG7Q1F+oeX/g+y2gLGBezNoCwbl6HfA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/api-base": "11.3.1", + "@polkadot/rpc-augment": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-augment": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/api-base": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api-base/-/api-base-11.3.1.tgz", + "integrity": "sha512-b8UkNL00NN7+3QaLCwL5cKg+7YchHoKCAhwKusWHNBZkkO6Oo2BWilu0dZkPJOyqV9P389Kbd9+oH+SKs9u2VQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/rpc-core": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/util": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/api-derive": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/api-derive/-/api-derive-11.3.1.tgz", + "integrity": "sha512-9dopzrh4cRuft1nANmBvMY/hEhFDu0VICMTOGxQLOl8NMfcOFPTLAN0JhSBUoicGZhV+c4vpv01NBx/7/IL1HA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/api": "11.3.1", + "@polkadot/api-augment": "11.3.1", + "@polkadot/api-base": "11.3.1", + "@polkadot/rpc-core": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/keyring": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/keyring/-/keyring-12.6.2.tgz", + "integrity": "sha512-O3Q7GVmRYm8q7HuB3S0+Yf/q/EB2egKRRU3fv9b3B7V+A52tKzA+vIwEmNVaD1g5FKW9oB97rmpggs0zaKFqHw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "12.6.2", + "@polkadot/util-crypto": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "12.6.2", + "@polkadot/util-crypto": "12.6.2" + } + }, + "node_modules/@polkadot/networks": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/networks/-/networks-12.6.2.tgz", + "integrity": "sha512-1oWtZm1IvPWqvMrldVH6NI2gBoCndl5GEwx7lAuQWGr7eNL+6Bdc5K3Z9T0MzFvDGoi2/CBqjX9dRKo39pDC/w==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "12.6.2", + "@substrate/ss58-registry": "^1.44.0", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/rpc-augment": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/rpc-augment/-/rpc-augment-11.3.1.tgz", + "integrity": "sha512-2PaDcKNju4QYQpxwVkWbRU3M0t340nMX9cMo+8awgvgL1LliV/fUDZueMKLuSS910JJMTPQ7y2pK4eQgMt08gQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/rpc-core": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/rpc-core": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/rpc-core/-/rpc-core-11.3.1.tgz", + "integrity": "sha512-KKNepsDd/mpmXcA6v/h14eFFPEzLGd7nrvx2UUXUxoZ0Fq2MH1hplP3s93k1oduNY/vOXJR2K9S4dKManA6GVQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/rpc-augment": "11.3.1", + "@polkadot/rpc-provider": "11.3.1", + "@polkadot/types": "11.3.1", + "@polkadot/util": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/rpc-provider": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/rpc-provider/-/rpc-provider-11.3.1.tgz", + "integrity": "sha512-pqERChoHo45hd3WAgW8UuzarRF+G/o/eXEbl0PXLubiayw4X4qCmIzmtntUcKYgxGNcYGZaG87ZU8OjN97m6UA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/keyring": "^12.6.2", + "@polkadot/types": "11.3.1", + "@polkadot/types-support": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "@polkadot/x-fetch": "^12.6.2", + "@polkadot/x-global": "^12.6.2", + "@polkadot/x-ws": "^12.6.2", + "eventemitter3": "^5.0.1", + "mock-socket": "^9.3.1", + "nock": "^13.5.0", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@substrate/connect": "0.8.10" + } + }, + "node_modules/@polkadot/types": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types/-/types-11.3.1.tgz", + "integrity": "sha512-5c7uRFXQTT11Awi6T0yFIdAfD6xGDAOz06Kp7M5S9OGNZY28wSPk5x6BYfNphWPaIBmHHewYJB5qmnrdYQAWKQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/keyring": "^12.6.2", + "@polkadot/types-augment": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/types-create": "11.3.1", + "@polkadot/util": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2", + "rxjs": "^7.8.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-augment": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-augment/-/types-augment-11.3.1.tgz", + "integrity": "sha512-eR3HVpvUmB3v7q2jTWVmVfAVfb1/kuNn7ij94Zqadg/fuUq0pKqIOKwkUj3OxRM3A/5BnW3MbgparjKD3r+fyw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-codec": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-codec/-/types-codec-11.3.1.tgz", + "integrity": "sha512-i7IiiuuL+Z/jFoKTA9xeh4wGQnhnNNjMT0+1ohvlOvnFsoKZKFQQOaDPPntGJVL1JDCV+KjkN2uQKZSeW8tguQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "^12.6.2", + "@polkadot/x-bigint": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-create": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-create/-/types-create-11.3.1.tgz", + "integrity": "sha512-pBXtpz5FehcRJ6j5MzFUIUN8ZWM7z6HbqK1GxBmYbJVRElcGcOg7a/rL2pQVphU0Rx1E8bSO4thzGf4wUxSX7w==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/types-codec": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-known": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-known/-/types-known-11.3.1.tgz", + "integrity": "sha512-3BIof7u6tn9bk3ZCIxA07iNoQ3uj4+vn3DTOjCKECozkRlt6V+kWRvqh16Hc0SHMg/QjcMb2fIu/WZhka1McUQ==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/networks": "^12.6.2", + "@polkadot/types": "11.3.1", + "@polkadot/types-codec": "11.3.1", + "@polkadot/types-create": "11.3.1", + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/types-support": { + "version": "11.3.1", + "resolved": "https://registry.npmjs.org/@polkadot/types-support/-/types-support-11.3.1.tgz", + "integrity": "sha512-jTFz1GKyF7nI29yIOq4v0NiWTOf5yX4HahJNeFD8TcxoLhF+6tH/XXqrUXJEfbaTlSrRWiW1LZYlb+snctqKHA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/util": "^12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/util": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/util/-/util-12.6.2.tgz", + "integrity": "sha512-l8TubR7CLEY47240uki0TQzFvtnxFIO7uI/0GoWzpYD/O62EIAMRsuY01N4DuwgKq2ZWD59WhzsLYmA5K6ksdw==", + "license": "Apache-2.0", + "peer": true, + "dependencies": { + "@polkadot/x-bigint": "12.6.2", + "@polkadot/x-global": "12.6.2", + "@polkadot/x-textdecoder": "12.6.2", + "@polkadot/x-textencoder": "12.6.2", + "@types/bn.js": "^5.1.5", + "bn.js": "^5.2.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/util-crypto": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/util-crypto/-/util-crypto-12.6.2.tgz", + "integrity": "sha512-FEWI/dJ7wDMNN1WOzZAjQoIcCP/3vz3wvAp5QQm+lOrzOLj0iDmaIGIcBkz8HVm3ErfSe/uKP0KS4jgV/ib+Mg==", + "license": "Apache-2.0", + "dependencies": { + "@noble/curves": "^1.3.0", + "@noble/hashes": "^1.3.3", + "@polkadot/networks": "12.6.2", + "@polkadot/util": "12.6.2", + "@polkadot/wasm-crypto": "^7.3.2", + "@polkadot/wasm-util": "^7.3.2", + "@polkadot/x-bigint": "12.6.2", + "@polkadot/x-randomvalues": "12.6.2", + "@scure/base": "^1.1.5", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "12.6.2" + } + }, + "node_modules/@polkadot/wasm-bridge": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-bridge/-/wasm-bridge-7.5.4.tgz", + "integrity": "sha512-6xaJVvoZbnbgpQYXNw9OHVNWjXmtcoPcWh7hlwx3NpfiLkkjljj99YS+XGZQlq7ks2fVCg7FbfknkNb8PldDaA==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*", + "@polkadot/x-randomvalues": "*" + } + }, + "node_modules/@polkadot/wasm-crypto": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto/-/wasm-crypto-7.5.4.tgz", + "integrity": "sha512-1seyClxa7Jd7kQjfnCzTTTfYhTa/KUTDUaD3DMHBk5Q4ZUN1D1unJgX+v1aUeXSPxmzocdZETPJJRZjhVOqg9g==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-bridge": "7.5.4", + "@polkadot/wasm-crypto-asmjs": "7.5.4", + "@polkadot/wasm-crypto-init": "7.5.4", + "@polkadot/wasm-crypto-wasm": "7.5.4", + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*", + "@polkadot/x-randomvalues": "*" + } + }, + "node_modules/@polkadot/wasm-crypto-asmjs": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto-asmjs/-/wasm-crypto-asmjs-7.5.4.tgz", + "integrity": "sha512-ZYwxQHAJ8pPt6kYk9XFmyuFuSS+yirJLonvP+DYbxOrARRUHfN4nzp4zcZNXUuaFhpbDobDSFn6gYzye6BUotA==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*" + } + }, + "node_modules/@polkadot/wasm-crypto-init": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto-init/-/wasm-crypto-init-7.5.4.tgz", + "integrity": "sha512-U6s4Eo2rHs2n1iR01vTz/sOQ7eOnRPjaCsGWhPV+ZC/20hkVzwPAhiizu/IqMEol4tO2yiSheD4D6bn0KxUJhg==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-bridge": "7.5.4", + "@polkadot/wasm-crypto-asmjs": "7.5.4", + "@polkadot/wasm-crypto-wasm": "7.5.4", + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*", + "@polkadot/x-randomvalues": "*" + } + }, + "node_modules/@polkadot/wasm-crypto-wasm": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-crypto-wasm/-/wasm-crypto-wasm-7.5.4.tgz", + "integrity": "sha512-PsHgLsVTu43eprwSvUGnxybtOEuHPES6AbApcs7y5ZbM2PiDMzYbAjNul098xJK/CPtrxZ0ePDFnaQBmIJyTFw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/wasm-util": "7.5.4", + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*" + } + }, + "node_modules/@polkadot/wasm-util": { + "version": "7.5.4", + "resolved": "https://registry.npmjs.org/@polkadot/wasm-util/-/wasm-util-7.5.4.tgz", + "integrity": "sha512-hqPpfhCpRAqCIn/CYbBluhh0TXmwkJnDRjxrU9Bnqtw9nMNa97D8JuOjdd2pi0rxm+eeLQ/f1rQMp71RMM9t4w==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.7.0" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "*" + } + }, + "node_modules/@polkadot/x-bigint": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-bigint/-/x-bigint-12.6.2.tgz", + "integrity": "sha512-HSIk60uFPX4GOFZSnIF7VYJz7WZA7tpFJsne7SzxOooRwMTWEtw3fUpFy5cYYOeLh17/kHH1Y7SVcuxzVLc74Q==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-fetch": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-fetch/-/x-fetch-12.6.2.tgz", + "integrity": "sha512-8wM/Z9JJPWN1pzSpU7XxTI1ldj/AfC8hKioBlUahZ8gUiJaOF7K9XEFCrCDLis/A1BoOu7Ne6WMx/vsJJIbDWw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "node-fetch": "^3.3.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-global": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-global/-/x-global-12.6.2.tgz", + "integrity": "sha512-a8d6m+PW98jmsYDtAWp88qS4dl8DyqUBsd0S+WgyfSMtpEXu6v9nXDgPZgwF5xdDvXhm+P0ZfVkVTnIGrScb5g==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-randomvalues": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-randomvalues/-/x-randomvalues-12.6.2.tgz", + "integrity": "sha512-Vr8uG7rH2IcNJwtyf5ebdODMcr0XjoCpUbI91Zv6AlKVYOGKZlKLYJHIwpTaKKB+7KPWyQrk4Mlym/rS7v9feg==", + "license": "Apache-2.0", + "peer": true, + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "@polkadot/util": "12.6.2", + "@polkadot/wasm-util": "*" + } + }, + "node_modules/@polkadot/x-textdecoder": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-textdecoder/-/x-textdecoder-12.6.2.tgz", + "integrity": "sha512-M1Bir7tYvNappfpFWXOJcnxUhBUFWkUFIdJSyH0zs5LmFtFdbKAeiDXxSp2Swp5ddOZdZgPac294/o2TnQKN1w==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-textencoder": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-textencoder/-/x-textencoder-12.6.2.tgz", + "integrity": "sha512-4N+3UVCpI489tUJ6cv3uf0PjOHvgGp9Dl+SZRLgFGt9mvxnvpW/7+XBADRMtlG4xi5gaRK7bgl5bmY6OMDsNdw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@polkadot/x-ws": { + "version": "12.6.2", + "resolved": "https://registry.npmjs.org/@polkadot/x-ws/-/x-ws-12.6.2.tgz", + "integrity": "sha512-cGZWo7K5eRRQCRl2LrcyCYsrc3lRbTlixZh3AzgU8uX4wASVGRlNWi/Hf4TtHNe1ExCDmxabJzdIsABIfrr7xw==", + "license": "Apache-2.0", + "dependencies": { + "@polkadot/x-global": "12.6.2", + "tslib": "^2.6.2", + "ws": "^8.15.1" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@scure/base": { + "version": "1.2.6", + "resolved": "https://registry.npmjs.org/@scure/base/-/base-1.2.6.tgz", + "integrity": "sha512-g/nm5FgUa//MCj1gV09zTJTaM6KBAHqLN907YVQqf7zC49+DcO4B1so4ZX07Ef10Twr6nuqYEH9GEggFXA4Fmg==", + "license": "MIT", + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@substrate/connect": { + "version": "0.8.10", + "resolved": "https://registry.npmjs.org/@substrate/connect/-/connect-0.8.10.tgz", + "integrity": "sha512-DIyQ13DDlXqVFnLV+S6/JDgiGowVRRrh18kahieJxhgvzcWicw5eLc6jpfQ0moVVLBYkO7rctB5Wreldwpva8w==", + "deprecated": "versions below 1.x are no longer maintained", + "license": "GPL-3.0-only", + "optional": true, + "dependencies": { + "@substrate/connect-extension-protocol": "^2.0.0", + "@substrate/connect-known-chains": "^1.1.4", + "@substrate/light-client-extension-helpers": "^0.0.6", + "smoldot": "2.0.22" + } + }, + "node_modules/@substrate/connect-extension-protocol": { + "version": "2.2.2", + "resolved": "https://registry.npmjs.org/@substrate/connect-extension-protocol/-/connect-extension-protocol-2.2.2.tgz", + "integrity": "sha512-t66jwrXA0s5Goq82ZtjagLNd7DPGCNjHeehRlE/gcJmJ+G56C0W+2plqOMRicJ8XGR1/YFnUSEqUFiSNbjGrAA==", + "license": "GPL-3.0-only", + "optional": true + }, + "node_modules/@substrate/connect-known-chains": { + "version": "1.10.3", + "resolved": "https://registry.npmjs.org/@substrate/connect-known-chains/-/connect-known-chains-1.10.3.tgz", + "integrity": "sha512-OJEZO1Pagtb6bNE3wCikc2wrmvEU5x7GxFFLqqbz1AJYYxSlrPCGu4N2og5YTExo4IcloNMQYFRkBGue0BKZ4w==", + "license": "GPL-3.0-only", + "optional": true + }, + "node_modules/@substrate/light-client-extension-helpers": { + "version": "0.0.6", + "resolved": "https://registry.npmjs.org/@substrate/light-client-extension-helpers/-/light-client-extension-helpers-0.0.6.tgz", + "integrity": "sha512-girltEuxQ1BvkJWmc8JJlk4ZxnlGXc/wkLcNguhY+UoDEMBK0LsdtfzQKIfrIehi4QdeSBlFEFBoI4RqPmsZzA==", + "license": "MIT", + "optional": true, + "dependencies": { + "@polkadot-api/json-rpc-provider": "0.0.1", + "@polkadot-api/json-rpc-provider-proxy": "0.0.1", + "@polkadot-api/observable-client": "0.1.0", + "@polkadot-api/substrate-client": "0.0.1", + "@substrate/connect-extension-protocol": "^2.0.0", + "@substrate/connect-known-chains": "^1.1.4", + "rxjs": "^7.8.1" + }, + "peerDependencies": { + "smoldot": "2.x" + } + }, + "node_modules/@substrate/ss58-registry": { + "version": "1.51.0", + "resolved": "https://registry.npmjs.org/@substrate/ss58-registry/-/ss58-registry-1.51.0.tgz", + "integrity": "sha512-TWDurLiPxndFgKjVavCniytBIw+t4ViOi7TYp9h/D0NMmkEc9klFTo+827eyEJ0lELpqO207Ey7uGxUa+BS1jQ==", + "license": "Apache-2.0" + }, + "node_modules/@types/bn.js": { + "version": "5.2.0", + "resolved": "https://registry.npmjs.org/@types/bn.js/-/bn.js-5.2.0.tgz", + "integrity": "sha512-DLbJ1BPqxvQhIGbeu8VbUC1DiAiahHtAYvA0ZEAa4P31F7IaArc8z3C3BRQdWX4mtLQuABG4yzp76ZrS02Ui1Q==", + "license": "MIT", + "dependencies": { + "@types/node": "*" + } + }, + "node_modules/@types/node": { + "version": "26.4.0", + "resolved": "https://registry.npmjs.org/@types/node/-/node-26.4.0.tgz", + "integrity": "sha512-faiGnoIrLH/V8cibOMEAZ8pMw6oXqSukl29ra4mN8GdaB2ZewzeaLj+INpV5N+Z1eKWzY+IzaIZH2EIR6YZRNQ==", + "license": "MIT", + "dependencies": { + "undici-types": "~8.3.0" + } + }, + "node_modules/bn.js": { + "version": "5.2.5", + "resolved": "https://registry.npmjs.org/bn.js/-/bn.js-5.2.5.tgz", + "integrity": "sha512-Vq886eXykuP5E6HcKSSStP3bJgrE6In5WKxVUvJ8XGpWWYs2xZHWqUwzCtGgEtBcxyd57KBFDPFoUfNzdaHCNg==", + "license": "MIT" + }, + "node_modules/data-uri-to-buffer": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.1.tgz", + "integrity": "sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==", + "license": "MIT", + "engines": { + "node": ">= 12" + } + }, + "node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/eventemitter3": { + "version": "5.0.4", + "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz", + "integrity": "sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==", + "license": "MIT" + }, + "node_modules/fetch-blob": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.2.0.tgz", + "integrity": "sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "paypal", + "url": "https://paypal.me/jimmywarting" + } + ], + "license": "MIT", + "dependencies": { + "node-domexception": "^1.0.0", + "web-streams-polyfill": "^3.0.3" + }, + "engines": { + "node": "^12.20 || >= 14.13" + } + }, + "node_modules/formdata-polyfill": { + "version": "4.0.10", + "resolved": "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz", + "integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==", + "license": "MIT", + "dependencies": { + "fetch-blob": "^3.1.2" + }, + "engines": { + "node": ">=12.20.0" + } + }, + "node_modules/json-stringify-safe": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz", + "integrity": "sha512-ZClg6AaYvamvYEE82d3Iyd3vSSIjQ+odgjaTzRuO3s7toCdFKczob2i0zCh7JE8kWn17yvAWhUVxvqGwUalsRA==", + "license": "ISC" + }, + "node_modules/mock-socket": { + "version": "9.3.1", + "resolved": "https://registry.npmjs.org/mock-socket/-/mock-socket-9.3.1.tgz", + "integrity": "sha512-qxBgB7Qa2sEQgHFjj0dSigq7fX4k6Saisd5Nelwp2q8mlbAFh5dHV9JTTlF8viYJLSSWgMCZFUom8PJcMNBoJw==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "license": "MIT" + }, + "node_modules/nock": { + "version": "13.5.6", + "resolved": "https://registry.npmjs.org/nock/-/nock-13.5.6.tgz", + "integrity": "sha512-o2zOYiCpzRqSzPj0Zt/dQ/DqZeYoaQ7TUonc/xUPjCGl9WeHpNbxgVvOquXYAaJzI0M9BXV3HTzG0p8IUAbBTQ==", + "license": "MIT", + "dependencies": { + "debug": "^4.1.0", + "json-stringify-safe": "^5.0.1", + "propagate": "^2.0.0" + }, + "engines": { + "node": ">= 10.13" + } + }, + "node_modules/node-domexception": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz", + "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==", + "deprecated": "Use your platform's native DOMException instead", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/jimmywarting" + }, + { + "type": "github", + "url": "https://paypal.me/jimmywarting" + } + ], + "license": "MIT", + "engines": { + "node": ">=10.5.0" + } + }, + "node_modules/node-fetch": { + "version": "3.3.2", + "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.2.tgz", + "integrity": "sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==", + "license": "MIT", + "dependencies": { + "data-uri-to-buffer": "^4.0.0", + "fetch-blob": "^3.1.4", + "formdata-polyfill": "^4.0.10" + }, + "engines": { + "node": "^12.20.0 || ^14.13.1 || >=16.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/node-fetch" + } + }, + "node_modules/propagate": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/propagate/-/propagate-2.0.1.tgz", + "integrity": "sha512-vGrhOavPSTz4QVNuBNdcNXePNdNMaO1xj9yBeH1ScQPjk/rhg9sSlCXPhMkFuaNNW/syTvYqsnbIJxMBfRbbag==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/rxjs": { + "version": "7.8.2", + "resolved": "https://registry.npmjs.org/rxjs/-/rxjs-7.8.2.tgz", + "integrity": "sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA==", + "license": "Apache-2.0", + "peer": true, + "dependencies": { + "tslib": "^2.1.0" + } + }, + "node_modules/scale-ts": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/scale-ts/-/scale-ts-1.6.1.tgz", + "integrity": "sha512-PBMc2AWc6wSEqJYBDPcyCLUj9/tMKnLX70jLOSndMtcUoLQucP/DM0vnQo1wJAYjTrQiq8iG9rD0q6wFzgjH7g==", + "license": "MIT", + "optional": true + }, + "node_modules/smoldot": { + "version": "2.0.22", + "resolved": "https://registry.npmjs.org/smoldot/-/smoldot-2.0.22.tgz", + "integrity": "sha512-B50vRgTY6v3baYH6uCgL15tfaag5tcS2o/P5q1OiXcKGv1axZDfz2dzzMuIkVpyMR2ug11F6EAtQlmYBQd292g==", + "license": "GPL-3.0-or-later WITH Classpath-exception-2.0", + "optional": true, + "peer": true, + "dependencies": { + "ws": "^8.8.1" + } + }, + "node_modules/tslib": { + "version": "2.8.1", + "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", + "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", + "license": "0BSD" + }, + "node_modules/undici-types": { + "version": "8.3.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.3.0.tgz", + "integrity": "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==", + "license": "MIT" + }, + "node_modules/web-streams-polyfill": { + "version": "3.3.3", + "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz", + "integrity": "sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/ws": { + "version": "8.21.3", + "resolved": "https://registry.npmjs.org/ws/-/ws-8.21.3.tgz", + "integrity": "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw==", + "license": "MIT", + "engines": { + "node": ">=10.0.0" + }, + "peerDependencies": { + "bufferutil": "^4.0.1", + "utf-8-validate": ">=5.0.2" + }, + "peerDependenciesMeta": { + "bufferutil": { + "optional": true + }, + "utf-8-validate": { + "optional": true + } + } + } + } +} diff --git a/testing/package.json b/testing/package.json new file mode 100644 index 000000000..f48d1d519 --- /dev/null +++ b/testing/package.json @@ -0,0 +1,15 @@ +{ + "name": "pen-migration-testing", + "version": "0.1.0", + "private": true, + "description": "Local validation harness for the PEN migration: automates the checks in docs/pen-migration-local-test-plan.md", + "type": "module", + "scripts": { + "phase2": "node src/phase2-pendulum.mjs" + }, + "dependencies": { + "@polkadot/api": "^11.3.1", + "@polkadot/keyring": "^12.6.2", + "@polkadot/util-crypto": "^12.6.2" + } +} diff --git a/testing/src/harness.mjs b/testing/src/harness.mjs new file mode 100644 index 000000000..9e13d9a5f --- /dev/null +++ b/testing/src/harness.mjs @@ -0,0 +1,45 @@ +/** + * Minimal check runner shared by the phase scripts. + * + * Prints one line per check and exits non-zero if any fail, so the harness can + * gate a deployment step rather than just producing output a human has to read. + */ + +const results = []; + +export async function check(name, fn) { + try { + await fn(); + results.push({ name, ok: true }); + console.log(` PASS ${name}`); + } catch (error) { + const reason = error instanceof Error ? error.message : String(error); + results.push({ name, ok: false, reason }); + console.log(` FAIL ${name}`); + console.log(` ${reason}`); + } +} + +export function section(title) { + console.log(`\n${title}`); +} + +export function summarise() { + const failed = results.filter((r) => !r.ok); + console.log(`\n${results.length - failed.length}/${results.length} checks passed`); + if (failed.length > 0) { + console.log("\nFailed:"); + for (const f of failed) console.log(` - ${f.name}: ${f.reason}`); + } + return failed.length === 0; +} + +export function assert(condition, message) { + if (!condition) throw new Error(message); +} + +export function assertEq(actual, expected, label) { + const a = typeof actual === "bigint" ? actual.toString() : String(actual); + const e = typeof expected === "bigint" ? expected.toString() : String(expected); + if (a !== e) throw new Error(`${label}: expected ${e}, got ${a}`); +} diff --git a/testing/src/phase2-pendulum.mjs b/testing/src/phase2-pendulum.mjs new file mode 100644 index 000000000..b689fcb94 --- /dev/null +++ b/testing/src/phase2-pendulum.mjs @@ -0,0 +1,238 @@ +/** + * Phase 2 of docs/pen-migration-local-test-plan.md — the Pendulum side, run + * against a Chopsticks fork of live mainnet state with the new runtime + * applied as a wasm override. + * + * This is the phase that exercises what unit tests cannot: that the upgrade + * applies to real storage, that it ships PAUSED, and that migrate() behaves + * correctly against genuine holder state (whales, vesting, staking locks, the + * real treasury account). + * + * Start Chopsticks first: + * npx @acala-network/chopsticks@latest --config testing/chopsticks.yml \ + * --wasm-override target/release/wbuild/pendulum-runtime/pendulum_runtime.compact.compressed.wasm + * then: node testing/src/phase2-pendulum.mjs + */ + +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { Keyring } from "@polkadot/keyring"; +import { cryptoWaitReady } from "@polkadot/util-crypto"; +import { assert, assertEq, check, section, summarise } from "./harness.mjs"; + +const ENDPOINT = process.env.CHOPSTICKS_WS ?? "ws://127.0.0.1:8000"; +const UNIT = 10n ** 12n; +const BASE_ADDR = "0x00000000000000000000000000000000000000Be"; +const ZERO_ADDR = "0x0000000000000000000000000000000000000000"; +// The keyless Pendulum treasury account (py/trsry), as it exists on mainnet. +const TREASURY = "0x6d6f646c70792f74727372790000000000000000000000000000000000000000"; + +const api = await ApiPromise.create({ provider: new WsProvider(ENDPOINT), noInitWarn: true }); +// Read the pallet's real bounds rather than assuming them: the harness must +// track whatever the runtime actually ships. +const MIN = BigInt(api.consts.tokenMigration.minimumMigrationAmount.toString()); +const ED = BigInt(api.consts.balances.existentialDeposit.toString()); +const AMOUNT = MIN * 2n; // comfortably above the minimum +const FUND = MIN * 100n; // plenty for fees and several migrations +await cryptoWaitReady(); +const keyring = new Keyring({ type: "sr25519", ss58Format: api.registry.chainSS58 ?? 56 }); +const alice = keyring.addFromUri("//Alice"); + +/** Produce a block so submitted extrinsics are applied. */ +const newBlock = () => api.rpc("dev_newBlock"); + +/** Set storage through Chopsticks, standing in for a governance action. + * Raw [key, value] pairs on purpose: the human-readable object form treats a + * falsy value as a deletion, and deleting `Paused` makes it read back as its + * `true` default -- the opposite of what we want. */ +const setStorageRaw = (pairs) => api.rpc("dev_setStorage", pairs); +const setStorage = (values) => api.rpc("dev_setStorage", values); + +/** Submit, produce a block, and report whether the extrinsic succeeded. */ +async function submit(tx, signer) { + await tx.signAndSend(signer); + await newBlock(); + const events = await api.query.system.events(); + let failure; + for (const record of events) { + const { section: s, method, data } = record.event; + if (s !== "system" || method !== "ExtrinsicFailed") continue; + const [dispatchError] = data; + if (dispatchError.isModule) { + const meta = api.registry.findMetaError(dispatchError.asModule); + failure = meta.name; + } else { + failure = dispatchError.toString(); + } + } + const migrationEvent = events + .map((r) => r.event) + .find((e) => e.section === "tokenMigration" && e.method === "MigrationInitiated"); + return { ok: !failure, error: failure, events, migrationEvent }; +} + +const paused = async () => (await api.query.tokenMigration.paused()).toPrimitive() === true; +const freeOf = async (who) => BigInt((await api.query.system.account(who)).data.free.toString()); +const issuance = async () => BigInt((await api.query.balances.totalIssuance()).toString()); + +async function fund(who, amount) { + await setStorage({ System: { Account: [[[who], { providers: 1, data: { free: amount.toString() } }]] } }); + // Chopsticks applies storage overrides after extrinsics within a block, so + // the funding needs its own block before anything can spend it. + await newBlock(); +} + +console.log(`Phase 2 — Pendulum on Chopsticks (${ENDPOINT})`); + +{ + const nonce = BigInt((await api.query.tokenMigration.nextNonce()).toString()); + if (nonce !== 0n) { + console.error( + `\nThis chain is not fresh (nextNonce=${nonce}). Phase 2 asserts the state of a\n` + + "just-upgraded chain, so restart Chopsticks before re-running.\n", + ); + await api.disconnect(); + process.exit(2); + } +} + +section("Upgrade applied and fresh state"); + +await check("the token-migration pallet exists in the upgraded runtime", async () => { + assert(api.query.tokenMigration !== undefined, "tokenMigration not present in metadata"); + assert(api.tx.tokenMigration?.migrate !== undefined, "migrate extrinsic missing"); +}); + +await check("SHIPS PAUSED — no storage written, pallet reads as paused", async () => { + assert(await paused(), "expected paused() == true immediately after the upgrade"); +}); + +await check("counters start at zero and no treasury destination is set", async () => { + assertEq(BigInt((await api.query.tokenMigration.nextNonce()).toString()), 0n, "nextNonce"); + assertEq(BigInt((await api.query.tokenMigration.totalMigrated()).toString()), 0n, "totalMigrated"); + assert((await api.query.tokenMigration.treasuryDestination()).isNone, "treasuryDestination should be None"); +}); + +await check("migrate is rejected while paused", async () => { + await fund(alice.address, FUND); + const r = await submit(api.tx.tokenMigration.migrate(AMOUNT, BASE_ADDR), alice); + assertEq(r.error, "MigrationsPaused", "expected MigrationsPaused"); +}); + +section("After governance enables migrations"); + +await check("unpausing works and is the separate, explicit act", async () => { + // SCALE bool false == 0x00, written directly to the storage key. + await setStorageRaw([[api.query.tokenMigration.paused.key(), "0x00"]]); + await newBlock(); + assert(!(await paused()), "expected paused() == false after enabling"); +}); + +await check("happy path burns the amount and reduces total issuance", async () => { + await fund(alice.address, FUND); + const before = await freeOf(alice.address); + const issuedBefore = await issuance(); + const amount = AMOUNT; + + const r = await submit(api.tx.tokenMigration.migrate(amount, BASE_ADDR), alice); + assert(r.ok, `migrate failed: ${r.error}`); + + const spent = before - (await freeOf(alice.address)); + assert(spent >= amount, `balance fell by ${spent}, expected at least ${amount}`); + assertEq(issuedBefore - (await issuance()), amount, "issuance drop (burn, not transfer)"); +}); + +await check("MigrationInitiated field ORDER is {nonce, who, base_address, amount}", async () => { + // The attestor decodes this event positionally, so field order is a + // wire-compatibility contract, not a cosmetic detail. + await fund(alice.address, FUND); + const amount = AMOUNT; + const r = await submit(api.tx.tokenMigration.migrate(amount, BASE_ADDR), alice); + assert(r.migrationEvent, "no MigrationInitiated event emitted"); + const data = r.migrationEvent.data; + assertEq(data.length, 4, "event field count"); + assert(!Number.isNaN(Number(data[0].toString())), "field 0 (nonce) should be numeric"); + assertEq(data[1].toString(), alice.address, "field 1 (who)"); + assertEq(data[2].toHex().toLowerCase(), BASE_ADDR.toLowerCase(), "field 2 (base_address)"); + assertEq(BigInt(data[3].toString().replaceAll(",", "")), amount, "field 3 (amount)"); +}); + +await check("the zero Base address is rejected", async () => { + const r = await submit(api.tx.tokenMigration.migrate(AMOUNT, ZERO_ADDR), alice); + assertEq(r.error, "InvalidBaseAddress", "expected InvalidBaseAddress"); +}); + +await check("an amount below the minimum is rejected", async () => { + const r = await submit(api.tx.tokenMigration.migrate(MIN - 1n, BASE_ADDR), alice); + assertEq(r.error, "AmountBelowMinimum", "expected AmountBelowMinimum"); +}); + +await check("leaving a dust remainder below the existential deposit is rejected", async () => { + await fund(alice.address, FUND); + const free = await freeOf(alice.address); + const r = await submit(api.tx.tokenMigration.migrate(free - (ED - 1n), BASE_ADDR), alice); + assertEq(r.error, "WouldLeaveDust", "expected WouldLeaveDust"); +}); + +section("Against real mainnet holder state"); + +await check("the fork really carries mainnet state, not a blank chain", async () => { + // If this fails, every other 'real state' claim in this phase is void. + const issued = await issuance(); + assert(issued > 100_000_000n * UNIT, `total issuance looks wrong: ${issued}`); + const treasuryFree = await freeOf(TREASURY); + assert(treasuryFree > 0n, "the real treasury account has no balance in the fork"); + let lockedAccounts = 0; + for (const [, value] of await api.query.balances.locks.entries()) { + if (value.length > 0) lockedAccounts++; + if (lockedAccounts >= 50) break; + } + assert(lockedAccounts > 0, "no locked accounts in the fork — state did not load"); +}); + +await check("a locked balance cannot be migrated", async () => { + // Lock most of a funded account, then try to migrate past the free portion. + await fund(alice.address, FUND); + const lockId = "0x76657374696e6720"; // b"vesting " + const locked = FUND - MIN; // leave less than the minimum unlocked + // Locks and AccountData.frozen must both be set: the withdraw path checks + // `frozen`, and writing Balances.Locks alone does not recompute it. + await setStorage({ + System: { + Account: [[[alice.address], { providers: 1, consumers: 1, data: { free: FUND.toString(), frozen: locked.toString() } }]], + }, + Balances: { Locks: [[[alice.address], [{ id: lockId, amount: locked.toString(), reasons: "All" }]]] }, + }); + await newBlock(); + const r = await submit(api.tx.tokenMigration.migrate(FUND / 2n, BASE_ADDR), alice); + assert(!r.ok, "expected a locked balance to be refused"); + // Clear the lock again so later checks operate on a clean account. + await setStorage({ Balances: { Locks: [[[alice.address], []]] } }); + await newBlock(); +}); + +await check("the real treasury account is present and funded in the fork", async () => { + // The keyless py/trsry account is what migrate_treasury burns from. Its + // root-gated execution cannot be driven here (Pendulum has no sudo pallet, + // so root only comes from a referendum); that path is covered by the + // pallet's unit tests. What phase 2 can confirm is that the account this + // runtime will burn from really exists in mainnet state and holds a + // balance, and that the destination is settable. + const treasuryFree = await freeOf(TREASURY); + assert(treasuryFree > 0n, "real treasury account has no balance in the fork"); + await setStorage({ TokenMigration: { TreasuryDestination: BASE_ADDR } }); + await newBlock(); + const dest = await api.query.tokenMigration.treasuryDestination(); + assert(dest.isSome, "treasury destination did not persist"); + assertEq(dest.unwrap().toHex().toLowerCase(), BASE_ADDR.toLowerCase(), "treasury destination"); +}); + +await check("nonces are unique and monotonic across all migrations so far", async () => { + const next = BigInt((await api.query.tokenMigration.nextNonce()).toString()); + assert(next > 0n, "expected at least one migration to have consumed a nonce"); + const migrated = BigInt((await api.query.tokenMigration.totalMigrated()).toString()); + assert(migrated > 0n, "TotalMigrated should have advanced"); +}); + +const ok = summarise(); +await api.disconnect(); +process.exit(ok ? 0 : 1); From b3e31feed3377da4b5a93f243a354a03cf7ead09 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Thu, 27 Aug 2026 16:32:41 +0200 Subject: [PATCH 11/61] docs: Refresh the implementation overview and settle the testnet plan Brings the overview back in line with what is actually built: the releaser and the local test harness were missing, test counts and runbook range were stale, and it still referenced the superseded branches. Verification status is now a table covering every suite, including the 14/14 Chopsticks run against live mainnet state. Corrects the documented minimum migration amount to the 100 PEN the runtime actually ships, with the reason it is set there: it has to dominate the attestor fleet's per-migration Base gas, or dust spam becomes an asymmetric gas-drain grief. Records that Foucoco is not part of validation -- the chain is no longer live -- so the plan is this local stack plus Base Sepolia for the contracts, and the discussion post's reference to Foucoco needs correcting in the formal proposal. --- docs/pen-migration-implementation-overview.md | 47 +++++++++---------- docs/pen-migration-local-test-plan.md | 7 +-- 2 files changed, 26 insertions(+), 28 deletions(-) diff --git a/docs/pen-migration-implementation-overview.md b/docs/pen-migration-implementation-overview.md index c5e5213be..7aa2c3f98 100644 --- a/docs/pen-migration-implementation-overview.md +++ b/docs/pen-migration-implementation-overview.md @@ -1,9 +1,7 @@ # PEN → Base Migration — Implementation Overview -**Date:** 2026-07-07 -**Branches:** `feat/pen-base-migration` in this repo and in the portal repo -(`~/Documents/portal`, based on the React 19 branch `fix-issues-with-new-ss58format`; -note: `origin/staging` there is still the older Preact codebase). +**Branches:** `feat/pen-to-base-migration` in this repo (PR #559) and +`feat/pen-base-migration` in the portal repo (PR #655). This document is the map of everything built for the migration. Design and requirements live in the [PRD](pen-base-migration-prd.md); the approach @@ -24,27 +22,29 @@ watched by an independent monitor that can auto-pause. One-way by design. ## Components delivered -### Pendulum repo (`feat/pen-base-migration`) +### Pendulum repo (`feat/pen-to-base-migration`) | Component | Location | Status | |---|---|---| -| `token-migration` pallet | `pallets/token-migration/` | Burn-and-emit `migrate` (user) + `migrate_treasury`/`set_treasury_destination` (governance, fixed Base destination) extrinsics sharing one nonce space and event; unique nonces, dust/ED + lock handling, KeepAlive treasury withdraw, pause origin; 20 unit tests + benchmark test suite (frame-benchmarking v2) | -| Runtime wiring | `runtime/pendulum/src/lib.rs` | Pallet index 102, min amount 1 PEN, pause = root/half-council or 2/3 technical committee, added to `BaseFilter` whitelist and `define_benchmarks`; compiles with and without `runtime-benchmarks` (Foucoco intentionally skipped — production-direct decision) | +| `token-migration` pallet | `pallets/token-migration/` | Burn-and-emit `migrate` (user) + `migrate_treasury`/`set_treasury_destination` (governance, fixed Base destination) extrinsics sharing one nonce space and event; unique nonces, dust/ED + lock handling, KeepAlive treasury withdraw, ships paused, pause origin; 21 unit tests + benchmark test suite (frame-benchmarking v2) | +| Runtime wiring | `runtime/pendulum/src/lib.rs` | Pallet index 102, minimum migration amount 100 PEN (sized to dominate the attestor fleet's per-migration Base gas, so dust spam cannot grief it), pause = root/half-council or 2/3 technical committee, added to `BaseFilter` whitelist and `define_benchmarks`; compiles with and without `runtime-benchmarks` (Foucoco intentionally skipped — that chain is no longer live; validation is local plus Base Sepolia) | | `PEN.sol` | `contracts/src/` | Fixed-supply `ERC20 + ERC20Permit + ERC20Votes`, EIP-6372 timestamp clock, full supply minted to vault, no owner/mint/proxy | | `MigrationVault.sol` | `contracts/src/` | 3-of-4 on-chain approvals per exact tuple, permanent nonce consumption, 12→18 decimal conversion in one place, per-release + daily caps (defer, not kill), guardian pause (approvals recorded while paused), rotation retroactively invalidates removed attestors, two-step admin, pending-release accounting protecting the timelocked remainder sweep | | `PENGovernor.sol` | `contracts/src/` | OZ Governor composition through a TimelockController (hybrid governance, timestamp clock) | | Deploy scripts | `contracts/script/` | `Deploy.s.sol` (vault→token→setToken dance, admin handover to bootstrap Safe), `DeployGovernance.s.sol` (timelock+governor role wiring, deployer admin renounced); parameters documented in `contracts/.env.example` | -| Contract tests | `contracts/test/` | 30 Foundry tests incl. fuzz (supply invariant), full Governor proposal lifecycle, replay/race/rotation/caps/pause/sweep-pending scenarios | +| Contract tests | `contracts/test/` | 37 Foundry tests incl. fuzz (supply invariant), full Governor proposal lifecycle, replay/race/rotation/caps/pause/sweep-pending scenarios | | Attestor daemon | `attestor/` | TypeScript; finalized-heads-only, strictly ordered blocks, crash-safe checkpoint, idempotent + race-tolerant approvals, fail-fast on decode errors (4-field shape asserted), startup set-membership check, low-gas/webhook alerts; ops guide in its README | -| Invariant monitor | `monitor/` | Independent watchdog: conservation checks (block-pinned reads) + per-nonce liveness; webhook alerts; optional guardian auto-pause | -| Runbooks | `docs/pen-migration-runbooks.md` | RB-1…RB-6: key compromise, outage, invariant breach, pause/unpause, runtime upgrade, attestor rotation | -| Internal security review | `docs/pen-migration-internal-review.md` | Independent adversarial pass; 2 high + 1 medium findings, all fixed (see below) | +| Invariant monitor | `monitor/` | Independent watchdog: conservation checks (block-pinned reads) + per-nonce liveness batched via Multicall3; webhook alerts; optional guardian auto-pause | +| Releaser | `releaser/` | Drains cap-deferred releases via the permissionless `release()`; unprivileged gas-only key; classifies self-healing vs governance-blocked failures | +| Test harness | `testing/` | Automates phase 2 of the local test plan against a Chopsticks fork of live mainnet state | +| Runbooks | `docs/pen-migration-runbooks.md` | RB-1…RB-7: key compromise, outage, invariant breach, pause/unpause, runtime upgrade, attestor rotation, window close | +| Internal security review | `docs/pen-migration-internal-review.md` | The project's security-assurance record across seven adversarial rounds | -### Portal repo (`feat/pen-base-migration`) +### Portal repo (`feat/pen-base-migration`, PR #655) | Component | Location | Status | |---|---|---| -| Migration page | `src/pages/migration/` | Amount validation (transferable, minimum, migrate-all-or-leave-ED), EIP-55 address validation with checksummed preview, `eth_getCode` contract-destination warning + extra confirmation, irreversibility confirmation, pause banner, locked-balance hint, post-finalization release tracking (approvals x/3 → released, BaseScan link) | +| Migration page | `src/pages/migration/` | Amount validation (transferable, minimum, migrate-all-or-leave-ED, per-release-cap warning), EIP-55 address validation with checksummed preview, `eth_getCode` contract-destination warning + extra confirmation, irreversibility confirmation, pause banner, locked-balance hint, post-finalization release tracking (approvals x/3 → released, BaseScan link) | | Pallet hook | `src/hooks/migration/useMigrationPallet.tsx` | Extrinsic submission resolving at finality with the emitted nonce; pause query; on-chain constants | | Base status hook | `src/hooks/migration/useBaseReleaseStatus.ts` | Polls the vault over plain JSON-RPC (no EVM dependency; selectors precomputed, keccak via `@polkadot/util-crypto`) | | EVM helpers | `src/helpers/ethereum.ts` | EIP-55 checksum, payload-hash mirroring the vault's `abi.encode`, minimal `eth_call`/`eth_getCode` client | @@ -64,11 +64,14 @@ restricted to consumed nonces. Details and verified-not-vulnerable list in ## Verification status -- Pallet: `cargo test -p token-migration` 11/11 (incl. benchmark suite). -- Runtime: `cargo check -p pendulum-runtime` clean, both feature sets. -- Contracts: `forge test` 35/35 (incl. 512-run fuzz). -- Attestor & monitor: `tsc --noEmit` clean. -- Portal: `yarn build` (tsc + vite) clean against `main`; committed through lint-staged. +| Suite | Result | +|---|---| +| `cargo test -p token-migration` | 21 (22 with `runtime-benchmarks`) | +| `cargo check -p pendulum-runtime` | clean, both feature sets | +| `forge test` | 37, incl. 512-run fuzz and a full Governor lifecycle | +| `attestor` / `monitor` / `releaser` | 6 / 7 / 7 | +| Portal `yarn build` (tsc + vite) | clean against `main` | +| `testing/src/phase2-pendulum.mjs` | 14/14 against a Chopsticks fork of live mainnet state | Seven internal adversarial review rounds have run; each found real issues (sometimes in a prior round's own fix), all fixed with regression tests and @@ -79,12 +82,6 @@ independent monitoring + auto-pause, guardian, ≥48h timelock, separation of duties) plus the conservative soft launch. Standing practice: any change to the fund-release path triggers a fresh internal review round before deployment. -## Commit map (this repo) - -`docs → pallet → contracts(core) → runtime wiring → contracts(governance) → -attestor → monitor → runbooks → benchmarks → security fixes → env template` -— see `git log` on the branch for hashes. - ## Still open (cannot be done from the repo) 1. **Decisions D1–D6** (PRD §4.2) — all recorded as decided (burn, 18 @@ -95,7 +92,7 @@ attestor → monitor → runbooks → benchmarks → security fixes → env temp 2. Security sign-off before mainnet funding: no external audit will be commissioned (PRD §9) — a final internal review pass over the shipped revision, plus the operational drills. -3. Benchmark run on reference hardware → replace manual weights. +3. Benchmark weights generated locally; regenerate on production hardware if the launch timeline allows. 4. Attestor operator onboarding + key ceremonies; Safe setups (D4). 5. Exchange coordination, DefiLlama/CoinGecko supply endpoints, comms. 6. Portal deploy config: set `VITE_MIGRATION_VAULT_ADDRESS` once deployed; diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 85ebcd102..7a9e6f242 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -16,9 +16,10 @@ needs both. Chopsticks gives realistic *state*; Zombienet gives realistic *finality*. You need both, for different reasons. Neither requires Paseo or Foucoco. -> Note: the public discussion post commits to testing on "Foucoco and Base -> Sepolia". Local testing does not discharge that commitment — either wire the -> pallet into the Foucoco runtime for a public run, or amend the messaging. +> Foucoco is **not** used: that chain is no longer live. Validation is this +> local stack plus a public run on Base Sepolia for the contracts. The +> discussion post's reference to Foucoco should be corrected when the formal +> proposal is published. --- From 298a54b75c6638b04cb5b930bc993fa80b26a5a0 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Thu, 27 Aug 2026 16:40:44 +0200 Subject: [PATCH 12/61] pallets: Replace estimated weights with benchmarked ones Generated with the benchmark CLI over 50 steps / 20 repeats, replacing the hand-written estimates. The estimates were conservative rather than unsafe -- migrate was charged 50ms against a measured 19ms, and 4 reads / 4 writes against an actual 3 / 2 -- so nothing was under-charged, but the real figures also carry proof sizes, which the estimates omitted entirely. Restructured to the repo's weights convention (trait, SubstrateWeight and a () impl) since the generated template omits the trait definition, and the header records how to regenerate. Also documents a pre-existing blocker found while doing this: `--chain pendulum` fails for EVERY pallet in this repo, because CurrencyId and OracleKey serialise their variants first-letter-lowercased (`native`, `xCM`, `exchangeRate`) but deserialise expecting the original casing, so the benchmark CLI cannot read back the genesis it just built. The workaround -- build-spec, rewrite the variant names, pass the patched file -- is recorded in the file header. --- .../token-migration/src/default_weights.rs | 118 +++++++++++++++--- 1 file changed, 102 insertions(+), 16 deletions(-) diff --git a/pallets/token-migration/src/default_weights.rs b/pallets/token-migration/src/default_weights.rs index eb7b7bd6a..1f6f654f6 100644 --- a/pallets/token-migration/src/default_weights.rs +++ b/pallets/token-migration/src/default_weights.rs @@ -1,12 +1,43 @@ -//! Default weights for the token-migration pallet. +//! Autogenerated weights for `token_migration`. //! -//! TODO: replace with generated weights once benchmarks are added; these are -//! conservative manual estimates in the meantime (same approach as the other -//! pallets in this repo). +//! THIS FILE WAS AUTO-GENERATED USING THE SUBSTRATE BENCHMARK CLI VERSION 4.0.0-dev +//! DATE: 2026-08-27, STEPS: `50`, REPEAT: `20` +//! WASM-EXECUTION: `Compiled`, DB CACHE: 1024 +//! +//! Generated on a development machine, not on collator-grade reference +//! hardware, so treat these as measured-but-provisional: they are strictly +//! better than the hand-written estimates they replace, but should be +//! regenerated on production hardware before the runtime upgrade if the +//! timeline allows. +//! +//! Regenerate with: +//! +//! ```text +//! cargo build --release --features runtime-benchmarks -p pendulum-node +//! ./target/release/pendulum-node benchmark pallet \ +//! --chain --pallet token-migration --extrinsic '*' \ +//! --steps 50 --repeat 20 \ +//! --template .maintain/frame-weight-template.hbs \ +//! --output pallets/token-migration/src/default_weights.rs +//! ``` +//! +//! NOTE: `--chain pendulum` currently fails for every pallet in this repo with +//! `Invalid JSON blob: unknown variant ...`. `CurrencyId` and `OracleKey` +//! serialise their variants first-letter-lowercased (`native`, `xCM`, +//! `exchangeRate`) but deserialise expecting the original casing, so the +//! benchmark CLI cannot read back the genesis it just built. Until that is +//! fixed upstream, generate a spec with `build-spec`, rewrite those variant +//! names to their PascalCase form, and pass the patched file as `--chain`. + +#![cfg_attr(rustfmt, rustfmt_skip)] +#![allow(unused_parens)] +#![allow(unused_imports)] +#![allow(missing_docs)] +use frame_support::{traits::Get, weights::{Weight, constants::RocksDbWeight}}; use core::marker::PhantomData; -use frame_support::{traits::Get, weights::Weight}; +/// Weight functions needed for `token_migration`. pub trait WeightInfo { fn migrate() -> Weight; fn set_paused() -> Weight; @@ -14,28 +45,83 @@ pub trait WeightInfo { fn migrate_treasury() -> Weight; } +/// Weights for `token_migration` using the Substrate node and recommended hardware. pub struct SubstrateWeight(PhantomData); - impl WeightInfo for SubstrateWeight { + /// Storage: `TokenMigration::Paused` (r:1 w:0) + /// Proof: `TokenMigration::Paused` (`max_values`: Some(1), `max_size`: Some(1), added: 496, mode: `MaxEncodedLen`) + /// Storage: `TokenMigration::NextNonce` (r:1 w:1) + /// Proof: `TokenMigration::NextNonce` (`max_values`: Some(1), `max_size`: Some(8), added: 503, mode: `MaxEncodedLen`) + /// Storage: `TokenMigration::TotalMigrated` (r:1 w:1) + /// Proof: `TokenMigration::TotalMigrated` (`max_values`: Some(1), `max_size`: Some(16), added: 511, mode: `MaxEncodedLen`) fn migrate() -> Weight { - Weight::from_parts(50_000_000, 0) - .saturating_add(T::DbWeight::get().reads(4_u64)) - .saturating_add(T::DbWeight::get().writes(4_u64)) + // Proof Size summary in bytes: + // Measured: `166` + // Estimated: `1501` + // Minimum execution time: 15_000_000 picoseconds. + Weight::from_parts(19_000_000, 1501) + .saturating_add(T::DbWeight::get().reads(3_u64)) + .saturating_add(T::DbWeight::get().writes(2_u64)) } - + /// Storage: `TokenMigration::Paused` (r:0 w:1) + /// Proof: `TokenMigration::Paused` (`max_values`: Some(1), `max_size`: Some(1), added: 496, mode: `MaxEncodedLen`) fn set_paused() -> Weight { - Weight::from_parts(10_000_000, 0) + // Proof Size summary in bytes: + // Measured: `0` + // Estimated: `0` + // Minimum execution time: 3_000_000 picoseconds. + Weight::from_parts(4_000_000, 0) .saturating_add(T::DbWeight::get().writes(1_u64)) } - + /// Storage: `TokenMigration::TreasuryDestination` (r:0 w:1) + /// Proof: `TokenMigration::TreasuryDestination` (`max_values`: Some(1), `max_size`: Some(20), added: 515, mode: `MaxEncodedLen`) fn set_treasury_destination() -> Weight { - Weight::from_parts(12_000_000, 0) + // Proof Size summary in bytes: + // Measured: `0` + // Estimated: `0` + // Minimum execution time: 4_000_000 picoseconds. + Weight::from_parts(4_000_000, 0) .saturating_add(T::DbWeight::get().writes(1_u64)) } - + /// Storage: `TokenMigration::Paused` (r:1 w:0) + /// Proof: `TokenMigration::Paused` (`max_values`: Some(1), `max_size`: Some(1), added: 496, mode: `MaxEncodedLen`) + /// Storage: `TokenMigration::TreasuryDestination` (r:1 w:0) + /// Proof: `TokenMigration::TreasuryDestination` (`max_values`: Some(1), `max_size`: Some(20), added: 515, mode: `MaxEncodedLen`) + /// Storage: `System::Account` (r:1 w:1) + /// Proof: `System::Account` (`max_values`: None, `max_size`: Some(128), added: 2603, mode: `MaxEncodedLen`) + /// Storage: `TokenMigration::NextNonce` (r:1 w:1) + /// Proof: `TokenMigration::NextNonce` (`max_values`: Some(1), `max_size`: Some(8), added: 503, mode: `MaxEncodedLen`) + /// Storage: `TokenMigration::TotalMigrated` (r:1 w:1) + /// Proof: `TokenMigration::TotalMigrated` (`max_values`: Some(1), `max_size`: Some(16), added: 511, mode: `MaxEncodedLen`) fn migrate_treasury() -> Weight { - Weight::from_parts(50_000_000, 0) + // Proof Size summary in bytes: + // Measured: `550` + // Estimated: `3593` + // Minimum execution time: 18_000_000 picoseconds. + Weight::from_parts(24_000_000, 3593) .saturating_add(T::DbWeight::get().reads(5_u64)) - .saturating_add(T::DbWeight::get().writes(4_u64)) + .saturating_add(T::DbWeight::get().writes(3_u64)) + } +} + +// For backwards compatibility and tests. +impl WeightInfo for () { + fn migrate() -> Weight { + Weight::from_parts(19_000_000, 1501) + .saturating_add(RocksDbWeight::get().reads(3_u64)) + .saturating_add(RocksDbWeight::get().writes(2_u64)) + } + fn set_paused() -> Weight { + Weight::from_parts(4_000_000, 0) + .saturating_add(RocksDbWeight::get().writes(1_u64)) + } + fn set_treasury_destination() -> Weight { + Weight::from_parts(4_000_000, 0) + .saturating_add(RocksDbWeight::get().writes(1_u64)) + } + fn migrate_treasury() -> Weight { + Weight::from_parts(24_000_000, 3593) + .saturating_add(RocksDbWeight::get().reads(5_u64)) + .saturating_add(RocksDbWeight::get().writes(3_u64)) } } From 7c1b91cc109270a8e3657f80ae25b70272473969 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Thu, 27 Aug 2026 18:38:35 +0200 Subject: [PATCH 13/61] attestor: Pad the gas limit when submitting an approval The same approve() call executes one of two very different paths depending on what has landed by the time it is mined: either it merely records an approval, or it is the one that crosses the threshold and therefore performs the release, including an ERC-20 transfer. Gas estimated while the cheap path applied does not cover the expensive one, and with four attestors racing the same migration that reordering is the normal case rather than an edge case. The result was an OutOfGas revert for whichever attestor landed third. That reads as an unexplained failure, so the daemon alerted and exited -- the same fleet-crash class the earlier race handling was meant to close, reached by a different route. Under a process manager it would restart, reprocess the same block and can hit it again. Found by the end-to-end harness, which is the only place the daemons race each other; no unit test can reach it. --- attestor/src/main.ts | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/attestor/src/main.ts b/attestor/src/main.ts index ca9c8a9bf..844717d1e 100644 --- a/attestor/src/main.ts +++ b/attestor/src/main.ts @@ -35,6 +35,11 @@ interface Checkpoint { lastProcessedBlock: number; } +/** Multiplier applied to the estimated gas for `approve`. See the note at the + * call site: the same call can take the cheap record path or the expensive + * threshold-crossing release path. */ +const GAS_LIMIT_MULTIPLIER = 4n; + interface MigrationEvent { nonce: bigint; recipient: `0x${string}`; @@ -169,7 +174,23 @@ async function approve(event: MigrationEvent): Promise { functionName: "approve", args: [event.nonce, event.recipient, event.palletAmount], }); - const txHash = await walletClient.writeContract(request); + // Pad the gas limit generously. The SAME approve call executes one of two + // very different paths depending on what has landed by the time it is + // mined: either it merely records an approval, or it is the one that + // crosses the threshold and therefore performs the release, including an + // ERC-20 transfer. Gas estimated while the cheap path applied is not + // enough for the expensive one, and with several attestors racing the + // same migration that reordering is the normal case, not an edge case. + // An under-estimate reverts OutOfGas, which reads as an unexplained + // failure and takes the daemon down. + const gasLimit = (request.gas ?? (await publicClient.estimateContractGas({ + account, + address: config.vaultAddress, + abi: vaultAbi, + functionName: "approve", + args: [event.nonce, event.recipient, event.palletAmount], + }))) * GAS_LIMIT_MULTIPLIER; + const txHash = await walletClient.writeContract({ ...request, gas: gasLimit }); const receipt = await publicClient.waitForTransactionReceipt({ hash: txHash }); if (receipt.status !== "success") { throw new Error(`approve transaction reverted: ${txHash} (${label})`); From dcf5401f408c26ddf7ee0fe6ccff259935a48d3b Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Thu, 27 Aug 2026 18:38:35 +0200 Subject: [PATCH 14/61] releaser: Declare Multicall3 and fall back to per-nonce reads viem refuses to batch unless the chain definition names a multicall3 address, even when the contract is present on-chain. The releaser's custom chain definition did not, so every cycle that had anything pending threw ChainDoesNotSupportContract -- which is precisely the cycle in which the releaser matters. It would have silently drained nothing in production while logging a cycle failure each poll. Declares the canonical Multicall3 address and, as the monitor already does, degrades to individual reads when the predeploy is absent (a local devnet). Batching is an optimisation, never a requirement. --- releaser/src/main.ts | 65 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 52 insertions(+), 13 deletions(-) diff --git a/releaser/src/main.ts b/releaser/src/main.ts index 509dfc801..e17e95516 100644 --- a/releaser/src/main.ts +++ b/releaser/src/main.ts @@ -40,13 +40,23 @@ interface PersistedState { pending: Array<{ nonce: string; recipient: string; palletAmount: string }>; } +/** Canonical Multicall3, deployed at the same address on Base and every major + * chain. viem refuses to batch unless the chain definition declares it, even + * when the contract is present on-chain, so it has to be named here. */ +const MULTICALL3 = "0xcA11bde05977b3631167028862bE2a173976CA11" as const; + const baseChain = defineChain({ id: config.baseChainId, name: "base", nativeCurrency: { name: "Ether", symbol: "ETH", decimals: 18 }, rpcUrls: { default: { http: [config.baseRpcUrl] } }, + contracts: { multicall3: { address: MULTICALL3 } }, }); +/** Set once Multicall3 turns out to be unavailable (a local devnet without the + * predeploy), after which reads fall back to one call per nonce. */ +let multicallUnavailable = false; + const account = privateKeyToAccount(config.releaserPrivateKey); const publicClient = createPublicClient({ chain: baseChain, transport: http(config.baseRpcUrl) }); const walletClient = createWalletClient({ account, chain: baseChain, transport: http(config.baseRpcUrl) }); @@ -125,23 +135,52 @@ async function ingestNewPending(toBlock: bigint, conversionFactor: bigint): Prom } } +/** Read `nonceConsumed` for many nonces, batched where possible. + * + * Batching is an optimisation, never a requirement: a chain without the + * Multicall3 predeploy must degrade to individual reads rather than failing + * the cycle. Before this fallback existed a missing predeploy threw on every + * cycle that had anything pending -- which is precisely when the releaser + * matters -- so it silently never drained a single deferred release. */ +async function readConsumed(nonces: bigint[]): Promise { + const single = (nonce: bigint) => + publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "nonceConsumed", + args: [nonce], + }); + + if (!multicallUnavailable) { + try { + const results = await publicClient.multicall({ + contracts: nonces.map((nonce) => ({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "nonceConsumed" as const, + args: [nonce] as const, + })), + allowFailure: true, + }); + return results.map((r) => r.status === "success" && r.result === true); + } catch (error) { + multicallUnavailable = true; + await alert( + "multicall unavailable, using per-nonce reads", + `verify Multicall3 at ${MULTICALL3}: ${error}`, + ); + } + } + return Promise.all(nonces.map(single)); +} + /** Drop entries the vault has already consumed (by us, a peer, or a rival tuple). */ async function pruneConsumed(): Promise { const entries = [...pending.values()]; if (entries.length === 0) return; - const results = await publicClient.multicall({ - contracts: entries.map((p) => ({ - address: config.vaultAddress, - abi: vaultAbi, - functionName: "nonceConsumed" as const, - args: [p.nonce] as const, - })), - allowFailure: true, - }); - results.forEach((result, i) => { - if (result.status === "success" && result.result === true) { - pending.delete(entries[i].nonce); - } + const consumed = await readConsumed(entries.map((p) => p.nonce)); + entries.forEach((entry, i) => { + if (consumed[i]) pending.delete(entry.nonce); }); } From 58e48b2580fd2ba1c2127dce04a6198e87d11660 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Thu, 27 Aug 2026 18:38:35 +0200 Subject: [PATCH 15/61] testing: Add phase 1 and phase 3 to the validation harness Phase 1 deploys with the real Deploy.s.sol -- including its two-step admin handover -- and asserts against the deployed bytecode: supply, attestor set, the release path, both caps, guardian asymmetry, the sweep floor and the conservation identity. 11 checks. Phase 3 runs the whole system together: four attestors, the monitor and the releaser against Chopsticks and Anvil. It asserts that a burn on the Substrate side arrives on Base unattended, that losing the approval race does not kill a daemon, that the fleet tolerates one attestor down and stops cleanly at two, that a recovered attestor drains the backlog, and that a cap-deferred release is drained by the releaser with no manual step. 7 checks. ABIs are loaded from the Foundry artifacts rather than hand-maintained, so viem can decode the vault's custom errors by name -- without that a revert is a bare selector and every negative assertion is unreadable. The README records the traps that cost real debugging time: a wasm built with --features runtime-benchmarks cannot be used as a Chopsticks override, attestor START_BLOCK must be the chain head rather than 0, and strays from an aborted run must be killed or they rewrite the checkpoint files a fresh run just cleared. --- testing/.gitignore | 1 + testing/README.md | 58 +++++++++- testing/chopsticks-e2e.yml | 16 +++ testing/package-lock.json | 213 ++++++++++++++++++++++++++++++++++-- testing/package.json | 9 +- testing/src/abi.mjs | 27 +++++ testing/src/anvil.mjs | 61 +++++++++++ testing/src/daemons.mjs | 96 ++++++++++++++++ testing/src/deploy.mjs | 57 ++++++++++ testing/src/phase1-base.mjs | 186 +++++++++++++++++++++++++++++++ testing/src/phase3-e2e.mjs | 196 +++++++++++++++++++++++++++++++++ 11 files changed, 902 insertions(+), 18 deletions(-) create mode 100644 testing/chopsticks-e2e.yml create mode 100644 testing/src/abi.mjs create mode 100644 testing/src/anvil.mjs create mode 100644 testing/src/daemons.mjs create mode 100644 testing/src/deploy.mjs create mode 100644 testing/src/phase1-base.mjs create mode 100644 testing/src/phase3-e2e.mjs diff --git a/testing/.gitignore b/testing/.gitignore index c3610e597..50f41b552 100644 --- a/testing/.gitignore +++ b/testing/.gitignore @@ -1,2 +1,3 @@ node_modules/ .chopsticks-db.sqlite* +.chopsticks-e2e.sqlite* diff --git a/testing/README.md b/testing/README.md index fec564ab3..067d29390 100644 --- a/testing/README.md +++ b/testing/README.md @@ -45,7 +45,59 @@ There is no sudo pallet on Pendulum, so root-only calls (`migrate_treasury`, `dev_setStorage`; the extrinsics themselves are covered by the pallet's unit tests. -## Still to add +## Phase 1 — contracts on Anvil -Phase 1 (contracts on Anvil) and phase 3 (end-to-end with Zombienet, four -attestors, the monitor and the releaser) are still manual — see the test plan. +Deploys with the **real** `script/Deploy.s.sol`, including its two-step admin +handover, then asserts against the deployed bytecode. Unit tests exercise the +contract; this exercises the thing we ship and the script that ships it. + +```bash +anvil --port 8545 +node testing/src/phase1-base.mjs +``` + +The daily cap is a **rolling leaky bucket and is shared state across checks**. +It refills proportionally to the *current* `dailyCap`, so lowering the cap +slows the decay of consumption already recorded — always `setCaps` first and +warp afterwards, never the reverse. + +## Phase 3 — the whole pipeline together + +Runs four attestors, the monitor and the releaser against Chopsticks and Anvil, +and asserts that a burn on the Substrate side arrives on Base with no manual +step, that the fleet tolerates outages, and that a cap-deferred release drains +itself. + +```bash +anvil --port 8545 +npx @acala-network/chopsticks@latest --config testing/chopsticks-e2e.yml \ + --wasm-override target/release/wbuild/pendulum-runtime/pendulum_runtime.compact.compressed.wasm +# build the services once: (cd attestor && npm run build), same for monitor/ and releaser/ +node testing/src/phase3-e2e.mjs +``` + +Chopsticks stands in for Pendulum rather than Zombienet: it reports finalized +heads, which is what the attestors subscribe to, and it carries real state. +What it does **not** reproduce is genuine relay-chain finality timing — lag, +and the possibility of a fork before finality. That remains a manual Zombienet +exercise. + +Phase 3 uses `chopsticks-e2e.yml`, which *does* cache to a `db`: unlike phase 2 +it makes no claims about a just-upgraded chain, and the cache keeps a long run +alive when the upstream public RPC drops the connection, which it does. + +### Things that cost real debugging time here + +- **Never reuse a wasm built with `--features runtime-benchmarks`.** It + references host functions Chopsticks does not provide and fails with + `Unresolved function ext_benchmarking_add_to_whitelist_version_1`. Building + the node with that feature silently overwrites the runtime wasm, so rebuild + with `cargo build --release -p pendulum-runtime` afterwards. +- **`START_BLOCK` for the attestors must be the chain head**, not 0. A fork + sits at ~7.6M blocks and a daemon starting from 0 walks every one of them. + The same applies in production: it is the block the pallet went live at. +- **Kill stray daemons between runs.** An aborted run leaves processes that + keep writing the same checkpoint files, so a fresh attestor loads a stale + checkpoint moments after the harness cleared it. `killStrays()` handles this, + and `stopAndWait` is used wherever a test depends on a daemon really being + down — signalling alone lets it land one more transaction. diff --git a/testing/chopsticks-e2e.yml b/testing/chopsticks-e2e.yml new file mode 100644 index 000000000..fdb38af4f --- /dev/null +++ b/testing/chopsticks-e2e.yml @@ -0,0 +1,16 @@ +# Fork config for phase 3 (end-to-end), distinct from chopsticks.yml. +# +# Two differences from the phase 2 config, both deliberate: +# +# * `db:` is enabled. Phase 3 does not assert the state of a just-upgraded +# chain -- that is phase 2's job -- so carrying state forward is harmless, +# and the cache makes a long run resilient to the upstream public RPC +# dropping the connection mid-test, which it does. +# * `block:` pins a height, so the fork is reproducible and the cache +# actually hits instead of chasing the chain head. +# +# Delete testing/.chopsticks-e2e.sqlite* to refetch from scratch. +endpoint: wss://rpc-pendulum.prd.pendulumchain.tech +mock-signature-host: true +db: ./testing/.chopsticks-e2e.sqlite +port: 8000 diff --git a/testing/package-lock.json b/testing/package-lock.json index 0c728562d..a2f0dd793 100644 --- a/testing/package-lock.json +++ b/testing/package-lock.json @@ -10,7 +10,26 @@ "dependencies": { "@polkadot/api": "^11.3.1", "@polkadot/keyring": "^12.6.2", - "@polkadot/util-crypto": "^12.6.2" + "@polkadot/util-crypto": "^12.6.2", + "viem": "^2.21.0" + } + }, + "node_modules/@adraffy/ens-normalize": { + "version": "1.11.1", + "resolved": "https://registry.npmjs.org/@adraffy/ens-normalize/-/ens-normalize-1.11.1.tgz", + "integrity": "sha512-nhCBV3quEgesuf7c7KYfperqSS14T8bYuvJ8PcLJp6znkZpFc0AuW4qBtr8eKVyPPe/8RSr7sglCWPU5eaxwKQ==", + "license": "MIT" + }, + "node_modules/@noble/ciphers": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@noble/ciphers/-/ciphers-1.3.0.tgz", + "integrity": "sha512-2I0gnIVPtfnMw9ee9h1dJG7tp81+8Ob3OJb3Mv37rx5L40/b0i7djjCVvGOVqc9AEIQyvyu1i6ypKdFw8R8gQw==", + "license": "MIT", + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" } }, "node_modules/@noble/curves": { @@ -627,6 +646,33 @@ "url": "https://paulmillr.com/funding/" } }, + "node_modules/@scure/bip32": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/@scure/bip32/-/bip32-1.7.0.tgz", + "integrity": "sha512-E4FFX/N3f4B80AKWp5dP6ow+flD1LQZo/w8UnLGYZO674jS6YnYeepycOOksv+vLPSpgN35wgKgy+ybfTb2SMw==", + "license": "MIT", + "dependencies": { + "@noble/curves": "~1.9.0", + "@noble/hashes": "~1.8.0", + "@scure/base": "~1.2.5" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/bip39": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/@scure/bip39/-/bip39-1.6.0.tgz", + "integrity": "sha512-+lF0BbLiJNwVlev4eKelw1WWLaiKXw7sSl8T6FvBlWkdX+94aGJ4o8XjUdlyhTCjd8c+B3KT3JfS8P0bLRNU6A==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "~1.8.0", + "@scure/base": "~1.2.5" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, "node_modules/@substrate/connect": { "version": "0.8.10", "resolved": "https://registry.npmjs.org/@substrate/connect/-/connect-0.8.10.tgz", @@ -698,6 +744,27 @@ "undici-types": "~8.3.0" } }, + "node_modules/abitype": { + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/abitype/-/abitype-1.2.3.tgz", + "integrity": "sha512-Ofer5QUnuUdTFsBRwARMoWKOH1ND5ehwYhJ3OJ/BQO+StkwQjHw0XyVh4vDttzHB7QOFhPHa/o413PJ82gU/Tg==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/wevm" + }, + "peerDependencies": { + "typescript": ">=5.0.4", + "zod": "^3.22.0 || ^4.0.0" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + }, + "zod": { + "optional": true + } + } + }, "node_modules/bn.js": { "version": "5.2.5", "resolved": "https://registry.npmjs.org/bn.js/-/bn.js-5.2.5.tgz", @@ -771,6 +838,21 @@ "node": ">=12.20.0" } }, + "node_modules/isows": { + "version": "1.0.7", + "resolved": "https://registry.npmjs.org/isows/-/isows-1.0.7.tgz", + "integrity": "sha512-I1fSfDCZL5P0v33sVqeTDSpcstAg/N+wF5HS033mogOVIp4B+oHC7oOCsA3axAbBSGTJ8QubbNmnIRN/h8U7hg==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "peerDependencies": { + "ws": "*" + } + }, "node_modules/json-stringify-safe": { "version": "5.0.1", "resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz", @@ -844,6 +926,57 @@ "url": "https://opencollective.com/node-fetch" } }, + "node_modules/ox": { + "version": "0.14.34", + "resolved": "https://registry.npmjs.org/ox/-/ox-0.14.34.tgz", + "integrity": "sha512-12seOIk7dv8eAoGQhcWaeKZxNz304IVcDvb9U5Y7JZAEVe21Nm1YMxLjhWah+su5BD4Omx4Zz0z5x3ij9M4GYQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "dependencies": { + "@adraffy/ens-normalize": "^1.11.0", + "@noble/ciphers": "^1.3.0", + "@noble/curves": "1.9.1", + "@noble/hashes": "^1.8.0", + "@scure/bip32": "^1.7.0", + "@scure/bip39": "^1.6.0", + "abitype": "^1.2.3", + "eventemitter3": "5.0.1" + }, + "peerDependencies": { + "typescript": ">=5.4.0" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + } + } + }, + "node_modules/ox/node_modules/@noble/curves": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.1.tgz", + "integrity": "sha512-k11yZxZg+t+gWvBbIswW0yoJlu8cHOC7dhunwOzoWH/mXGBiYyR4YY6hAEK/3EUs4UpB8la1RfdRpeGsFHkWsA==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/ox/node_modules/eventemitter3": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.1.tgz", + "integrity": "sha512-GWkBvjiSZK87ELrYOSESUYeVIc9mvLLf/nXalMOS5dYrgZq9o5OVkbZAVM06CVxYsCwH9BDZFPlQTlPA1j4ahA==", + "license": "MIT" + }, "node_modules/propagate": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/propagate/-/propagate-2.0.1.tgz", @@ -870,17 +1003,6 @@ "license": "MIT", "optional": true }, - "node_modules/smoldot": { - "version": "2.0.22", - "resolved": "https://registry.npmjs.org/smoldot/-/smoldot-2.0.22.tgz", - "integrity": "sha512-B50vRgTY6v3baYH6uCgL15tfaag5tcS2o/P5q1OiXcKGv1axZDfz2dzzMuIkVpyMR2ug11F6EAtQlmYBQd292g==", - "license": "GPL-3.0-or-later WITH Classpath-exception-2.0", - "optional": true, - "peer": true, - "dependencies": { - "ws": "^8.8.1" - } - }, "node_modules/tslib": { "version": "2.8.1", "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", @@ -893,6 +1015,72 @@ "integrity": "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==", "license": "MIT" }, + "node_modules/viem": { + "version": "2.56.0", + "resolved": "https://registry.npmjs.org/viem/-/viem-2.56.0.tgz", + "integrity": "sha512-JmkgIk4jN+im4oguLxwPv19pwSaGH9kcGSq25Ilm55106ULBBMNR3eWKKA0wZoeyLPSHIiOwZ4sGceEYEIq7LA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/wevm" + } + ], + "license": "MIT", + "dependencies": { + "@noble/curves": "1.9.1", + "@noble/hashes": "1.8.0", + "@scure/bip32": "1.7.0", + "@scure/bip39": "1.6.0", + "abitype": "1.2.3", + "isows": "1.0.7", + "ox": "0.14.34", + "ws": "8.21.0" + }, + "peerDependencies": { + "typescript": ">=5.0.4" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + } + } + }, + "node_modules/viem/node_modules/@noble/curves": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.1.tgz", + "integrity": "sha512-k11yZxZg+t+gWvBbIswW0yoJlu8cHOC7dhunwOzoWH/mXGBiYyR4YY6hAEK/3EUs4UpB8la1RfdRpeGsFHkWsA==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/viem/node_modules/ws": { + "version": "8.21.0", + "resolved": "https://registry.npmjs.org/ws/-/ws-8.21.0.tgz", + "integrity": "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==", + "license": "MIT", + "engines": { + "node": ">=10.0.0" + }, + "peerDependencies": { + "bufferutil": "^4.0.1", + "utf-8-validate": ">=5.0.2" + }, + "peerDependenciesMeta": { + "bufferutil": { + "optional": true + }, + "utf-8-validate": { + "optional": true + } + } + }, "node_modules/web-streams-polyfill": { "version": "3.3.3", "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz", @@ -907,6 +1095,7 @@ "resolved": "https://registry.npmjs.org/ws/-/ws-8.21.3.tgz", "integrity": "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw==", "license": "MIT", + "peer": true, "engines": { "node": ">=10.0.0" }, diff --git a/testing/package.json b/testing/package.json index f48d1d519..1ea48d8f9 100644 --- a/testing/package.json +++ b/testing/package.json @@ -5,11 +5,14 @@ "description": "Local validation harness for the PEN migration: automates the checks in docs/pen-migration-local-test-plan.md", "type": "module", "scripts": { - "phase2": "node src/phase2-pendulum.mjs" + "phase1": "node src/phase1-base.mjs", + "phase2": "node src/phase2-pendulum.mjs", + "phase3": "node src/phase3-e2e.mjs" }, "dependencies": { "@polkadot/api": "^11.3.1", "@polkadot/keyring": "^12.6.2", - "@polkadot/util-crypto": "^12.6.2" + "@polkadot/util-crypto": "^12.6.2", + "viem": "^2.21.0" } -} +} \ No newline at end of file diff --git a/testing/src/abi.mjs b/testing/src/abi.mjs new file mode 100644 index 000000000..bba5f6152 --- /dev/null +++ b/testing/src/abi.mjs @@ -0,0 +1,27 @@ +/** + * Contract ABIs, loaded from the Foundry build artifacts. + * + * Read from `contracts/out/` rather than hand-maintained here so they cannot + * drift from the compiled contracts -- and, importantly, so viem can decode + * the vault's custom errors by name. Without the error entries a revert shows + * up only as a bare selector, which makes every negative assertion in this + * harness unreadable. + */ + +import { readFileSync } from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const OUT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../../contracts/out"); + +function abiOf(file, name) { + const p = path.join(OUT, file, `${name}.json`); + try { + return JSON.parse(readFileSync(p, "utf8")).abi; + } catch { + throw new Error(`missing artifact ${p} — run \`forge build\` in contracts/ first`); + } +} + +export const vaultAbi = abiOf("MigrationVault.sol", "MigrationVault"); +export const erc20Abi = abiOf("PEN.sol", "PEN"); diff --git a/testing/src/anvil.mjs b/testing/src/anvil.mjs new file mode 100644 index 000000000..ee5975160 --- /dev/null +++ b/testing/src/anvil.mjs @@ -0,0 +1,61 @@ +/** + * Shared Anvil/Base helpers: well-known dev accounts and a viem client pair. + * + * These are Anvil's deterministic development keys. They are public knowledge + * and exist only to drive a throwaway local chain — never reuse them anywhere + * that holds value. + */ + +import { createPublicClient, createWalletClient, defineChain, http } from "viem"; +import { privateKeyToAccount } from "viem/accounts"; + +export const RPC = process.env.BASE_RPC_URL ?? "http://127.0.0.1:8545"; + +const KEYS = [ + "0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80", // 0 deployer + "0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d", // 1 attestor A + "0x5de4111afa1a4b94908f83103eb1f1706367c2e68ca870fc3fb9a804cdab365a", // 2 attestor B + "0x7c852118294e51e653712a81e05800f419141751be58f605c371e15141b007a6", // 3 attestor C + "0x47e179ec197488593b187f80a00eb0da91f1b9d0b13f8733639f19c30a34926a", // 4 attestor D + "0x8b3a350cf5c34c9194ca85829a2df0ec3153be0318b5e2d3348e872092edffba", // 5 guardian + "0x92db14e403b83dfe3df233f83dfa3a0d7096f21ca9b0d6d6b8d88b2b4ec1564e", // 6 admin + "0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356", // 7 releaser (gas only) +]; + +export const accounts = KEYS.map((k) => privateKeyToAccount(k)); +export const keys = KEYS; +export const [deployer, attA, attB, attC, attD, guardian, admin, releaser] = accounts; +export const attestors = [attA, attB, attC, attD]; + +export const anvilChain = defineChain({ + id: Number(process.env.BASE_CHAIN_ID ?? 31337), + name: "anvil", + nativeCurrency: { name: "Ether", symbol: "ETH", decimals: 18 }, + rpcUrls: { default: { http: [RPC] } }, +}); + +export const pub = createPublicClient({ chain: anvilChain, transport: http(RPC) }); +export const wallet = (account) => createWalletClient({ account, chain: anvilChain, transport: http(RPC) }); + +/** Send a contract call and wait for it to land, returning the receipt. */ +export async function send(account, params) { + const { request } = await pub.simulateContract({ account, ...params }); + const hash = await wallet(account).writeContract(request); + return pub.waitForTransactionReceipt({ hash }); +} + +/** Attempt a call and return the revert reason instead of throwing. */ +export async function expectRevert(account, params) { + try { + await pub.simulateContract({ account, ...params }); + return null; // no revert + } catch (error) { + return error instanceof Error ? error.message : String(error); + } +} + +/** Advance the chain clock (Anvil only). */ +export async function warp(seconds) { + await pub.request({ method: "evm_increaseTime", params: [seconds] }); + await pub.request({ method: "evm_mine", params: [] }); +} diff --git a/testing/src/daemons.mjs b/testing/src/daemons.mjs new file mode 100644 index 000000000..33465afb3 --- /dev/null +++ b/testing/src/daemons.mjs @@ -0,0 +1,96 @@ +/** + * Spawns and supervises the attestor / monitor / releaser processes for the + * end-to-end phase, so a failed run cannot leave orphans behind. + */ + +import { execSync, spawn } from "node:child_process"; +import { rmSync } from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../.."); +const running = new Map(); + +/** Start a built service. Output is captured so a crash is diagnosable, and + * `exited` records whether the process died -- the attestor race regression + * is precisely "a daemon that should have kept running did not". */ +export function start(name, dir, env) { + const proc = spawn("node", ["dist/main.js"], { + cwd: path.join(ROOT, dir), + env: { ...process.env, ...env }, + stdio: ["ignore", "pipe", "pipe"], + }); + const rec = { proc, name, out: [], exited: null }; + const capture = (chunk) => { + for (const line of String(chunk).split("\n")) if (line.trim()) rec.out.push(line); + if (rec.out.length > 500) rec.out.splice(0, rec.out.length - 500); + }; + proc.stdout.on("data", capture); + proc.stderr.on("data", capture); + proc.on("exit", (code) => { rec.exited = code ?? -1; }); + running.set(name, rec); + return rec; +} + +export const get = (name) => running.get(name); +export const alive = (name) => running.get(name)?.exited === null; +export const logs = (name) => (running.get(name)?.out ?? []).join("\n"); + +export function stop(name) { + const rec = running.get(name); + if (rec && rec.exited === null) rec.proc.kill("SIGTERM"); +} + +/** Stop a daemon and WAIT until it has actually exited. + * + * `stop` alone only signals. A daemon that is mid-cycle can still land a + * transaction after the signal, so an outage test that does not wait may + * observe a quorum it thought it had removed. Escalates to SIGKILL. */ +export async function stopAndWait(name, { timeoutMs = 15_000 } = {}) { + const rec = running.get(name); + if (!rec || rec.exited !== null) return; + rec.proc.kill("SIGTERM"); + const deadline = Date.now() + timeoutMs; + while (rec.exited === null) { + if (Date.now() > deadline) { rec.proc.kill("SIGKILL"); break; } + await sleep(250); + } + while (rec.exited === null) await sleep(250); +} + +export function stopAll() { + for (const name of running.keys()) stop(name); +} + +/** Remove a service's persisted state so a run starts clean. */ +export function clearState(files) { + for (const f of files) { try { rmSync(path.join(ROOT, f)); } catch {} } +} + +export const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + +/** Poll until `fn()` is truthy or the timeout expires. */ +export async function waitFor(fn, { timeoutMs = 60_000, intervalMs = 1000, label = "condition" } = {}) { + const deadline = Date.now() + timeoutMs; + for (;;) { + const value = await fn(); + if (value) return value; + if (Date.now() > deadline) throw new Error(`timed out after ${timeoutMs}ms waiting for ${label}`); + await sleep(intervalMs); + } +} + +/** Kill any daemons left behind by an earlier aborted run. + * + * Without this a previous run's processes keep writing the same checkpoint + * files, so a fresh attestor loads a stale checkpoint moments after the + * harness cleared it and silently scans the wrong block range. */ +export function killStrays() { + try { + execSync("pkill -f 'dist/main.js' || true", { stdio: "ignore" }); + } catch { /* nothing to kill */ } +} + +process.on("exit", stopAll); +process.on("SIGINT", () => { stopAll(); process.exit(130); }); +process.on("SIGTERM", () => { stopAll(); process.exit(143); }); diff --git a/testing/src/deploy.mjs b/testing/src/deploy.mjs new file mode 100644 index 000000000..848f00b02 --- /dev/null +++ b/testing/src/deploy.mjs @@ -0,0 +1,57 @@ +/** + * Deploys the migration stack to a local Anvil using the REAL Deploy.s.sol, + * not a hand-rolled deployment. That is deliberate: the deploy script and its + * two-step admin handover are themselves part of what phase 1 validates. + */ + +import { execFileSync } from "node:child_process"; +import { readFileSync } from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { accounts, keys, RPC } from "./anvil.mjs"; + +const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../.."); +const CONTRACTS = path.join(ROOT, "contracts"); + +export const PARAMS = { + MAX_ISSUANCE: 150_000_000n * 10n ** 18n, + // Small caps so the deferral paths are reachable in a test run. + PER_RELEASE_CAP: 1_000_000n * 10n ** 18n, + DAILY_CAP: 1_000_000n * 10n ** 18n, + CONVERSION_FACTOR: 1_000_000n, +}; + +export function deployStack({ earliestSweepOffsetSeconds = 365 * 24 * 3600 } = {}) { + const [deployer, attA, attB, attC, attD, guardian, admin] = accounts; + const env = { + ...process.env, + PRIVATE_KEY: keys[0], + ADMIN_SAFE: admin.address, + GUARDIAN_SAFE: guardian.address, + ATTESTOR_1: attA.address, + ATTESTOR_2: attB.address, + ATTESTOR_3: attC.address, + ATTESTOR_4: attD.address, + MAX_ISSUANCE: PARAMS.MAX_ISSUANCE.toString(), + PER_RELEASE_CAP: PARAMS.PER_RELEASE_CAP.toString(), + DAILY_CAP: PARAMS.DAILY_CAP.toString(), + EARLIEST_SWEEP_TS: String(Math.floor(Date.now() / 1000) + earliestSweepOffsetSeconds), + }; + + execFileSync( + "forge", + ["script", "script/Deploy.s.sol", "--rpc-url", RPC, "--broadcast", "--private-key", keys[0], "-vvv"], + { cwd: CONTRACTS, env, stdio: ["ignore", "pipe", "pipe"] }, + ); + + // Read the addresses back out of the broadcast record rather than parsing + // log output, so this stays stable if the script's console output changes. + const chainId = 31337; + const file = path.join(CONTRACTS, "broadcast", "Deploy.s.sol", String(chainId), "run-latest.json"); + const run = JSON.parse(readFileSync(file, "utf8")); + const creations = run.transactions.filter((t) => t.transactionType === "CREATE"); + const vault = creations.find((t) => t.contractName === "MigrationVault")?.contractAddress; + const pen = creations.find((t) => t.contractName === "PEN")?.contractAddress; + if (!vault || !pen) throw new Error("could not locate deployed addresses in the broadcast record"); + return { vault, pen, env }; +} diff --git a/testing/src/phase1-base.mjs b/testing/src/phase1-base.mjs new file mode 100644 index 000000000..25f42221f --- /dev/null +++ b/testing/src/phase1-base.mjs @@ -0,0 +1,186 @@ +/** + * Phase 1 of docs/pen-migration-local-test-plan.md — the Base side on Anvil. + * + * These assertions mirror the Foundry suite, but run against the *deployed + * bytecode* produced by the real Deploy.s.sol, including its two-step admin + * handover. That is the point: unit tests exercise the contract, this + * exercises the thing we will actually ship and the script that ships it. + * + * anvil --port 8545 + * node testing/src/phase1-base.mjs + */ + +import { assert, assertEq, check, section, summarise } from "./harness.mjs"; +import { accounts, admin, attestors, expectRevert, guardian, pub, send, warp } from "./anvil.mjs"; +import { erc20Abi, vaultAbi } from "./abi.mjs"; +import { deployStack, PARAMS } from "./deploy.mjs"; + +const recipient = accounts[7].address; +const CF = PARAMS.CONVERSION_FACTOR; +let nonce = 0n; +const nextNonce = () => nonce++; + +console.log("Phase 1 — Base contracts on Anvil"); +console.log(" deploying via script/Deploy.s.sol …"); +const { vault, pen } = deployStack(); +console.log(` vault ${vault}\n PEN ${pen}`); + +const V = { address: vault, abi: vaultAbi }; +const P = { address: pen, abi: erc20Abi }; +const read = (c, functionName, args) => pub.readContract({ ...c, functionName, args }); + +/** Approve with attestors [from, to). Ranged rather than counted, because + * re-approving with an attestor that already signed a payload reverts + * AlreadyApproved -- which is the contract behaving correctly. */ +async function approveRange(from, to, n, recipientAddr, palletAmount) { + for (let i = from; i < to; i++) { + await send(attestors[i], { ...V, functionName: "approve", args: [n, recipientAddr, palletAmount] }); + } +} +const approveWith = (count, n, to, amt) => approveRange(0, count, n, to, amt); + +/** Fully refill the rolling daily bucket. It is shared state across checks, so + * any check that needs budget must claim it explicitly. + * + * Call this AFTER any setCaps: the bucket refills at a rate proportional to + * the CURRENT dailyCap, so lowering the cap slows the decay of consumption + * already recorded. Warping first and lowering the cap afterwards leaves the + * old consumption largely undecayed. */ +const refillBucket = () => warp(30 * 24 * 3600); + +section("Deployment"); + +await check("the full supply is minted and sits in the vault", async () => { + assertEq(await read(P, "totalSupply"), PARAMS.MAX_ISSUANCE, "totalSupply"); + assertEq(await read(P, "balanceOf", [vault]), PARAMS.MAX_ISSUANCE, "vault balance"); +}); + +await check("attestor set and threshold match the deployment parameters", async () => { + assertEq(await read(V, "threshold"), 3n, "threshold"); + assertEq(await read(V, "attestorCount"), 4n, "attestorCount"); +}); + +await check("the vault is wired to the token, unpaused, with the guardian set", async () => { + assertEq((await read(V, "token")).toLowerCase(), pen.toLowerCase(), "token"); + assertEq(await read(V, "paused"), false, "paused"); + assertEq((await read(V, "guardian")).toLowerCase(), guardian.address.toLowerCase(), "guardian"); +}); + +await check("admin handover is two-step and completes only on acceptance", async () => { + // Until the admin Safe accepts, the deployer still holds admin. This gap is + // real on mainnet too, which is why it belongs in the deployment runbook. + const before = await read(V, "admin"); + assert(before.toLowerCase() !== admin.address.toLowerCase(), "admin transferred without acceptance"); + await send(admin, { ...V, functionName: "acceptAdmin", args: [] }); + assertEq((await read(V, "admin")).toLowerCase(), admin.address.toLowerCase(), "admin after acceptance"); +}); + +section("Release path"); + +await check("three matching approvals release; two do not", async () => { + const amount = 5n * 10n ** 12n; // 5 PEN in 12-decimal pallet units + const n = nextNonce(); + await approveRange(0, 2, n, recipient, amount); + assertEq(await read(P, "balanceOf", [recipient]), 0n, "released below threshold"); + + await approveRange(2, 3, n, recipient, amount); + assertEq(await read(P, "balanceOf", [recipient]), amount * CF, "12->18 decimal conversion"); + assertEq(await read(V, "nonceConsumed", [n]), true, "nonceConsumed"); +}); + +await check("a consumed nonce cannot be released twice", async () => { + const amount = 5n * 10n ** 12n; + const n = 0n; // already consumed above + const reason = await expectRevert(attestors[3], { ...V, functionName: "approve", args: [n, recipient, amount] }); + assert(reason?.includes("NonceAlreadyConsumed"), `expected NonceAlreadyConsumed, got: ${reason}`); +}); + +section("Caps"); + +await check("an amount above the per-release cap defers instead of reverting", async () => { + const palletAmount = PARAMS.PER_RELEASE_CAP / CF + 1n; // just over the cap + const n = nextNonce(); + const pendingBefore = await read(V, "pendingApprovedAmount"); + await approveWith(3, n, recipient, palletAmount); + + assertEq(await read(V, "nonceConsumed", [n]), false, "must not release above the cap"); + const pendingAfter = await read(V, "pendingApprovedAmount"); + assert(pendingAfter > pendingBefore, "pendingApprovedAmount did not increase"); + + // It cannot self-heal — only a governance cap raise clears it. + const reason = await expectRevert(accounts[0], { ...V, functionName: "release", args: [n, recipient, palletAmount] }); + assert(reason?.includes("ExceedsPerReleaseCap"), `expected ExceedsPerReleaseCap, got: ${reason}`); + + await send(admin, { ...V, functionName: "setCaps", args: [PARAMS.PER_RELEASE_CAP * 4n, PARAMS.DAILY_CAP * 4n] }); + await send(accounts[0], { ...V, functionName: "release", args: [n, recipient, palletAmount] }); + assertEq(await read(V, "nonceConsumed", [n]), true, "release after the cap raise"); + assertEq(await read(V, "pendingApprovedAmount"), pendingBefore, "pending accounting not cleared"); +}); + +await check("the daily cap is a rolling bucket that refills over time", async () => { + // Reset to a small, known daily budget FIRST, then refill at that new rate. + const daily = 100_000n * 10n ** 18n; + await send(admin, { ...V, functionName: "setCaps", args: [daily, daily] }); + await refillBucket(); + const palletAmount = daily / CF; + + const n = nextNonce(); + await approveWith(3, n, recipient, palletAmount); + assertEq(await read(V, "nonceConsumed", [n]), true, "first release should fit the budget"); + assertEq(await read(V, "availableDailyAllowance"), 0n, "budget should be exhausted"); + + // A second release now defers, and self-heals as the bucket refills. + const n2 = nextNonce(); + await approveWith(3, n2, recipient, palletAmount); + assertEq(await read(V, "nonceConsumed", [n2]), false, "should defer on an empty bucket"); + + // Half a day back gives roughly half the budget -- gradual, not a reset. + await warp(12 * 3600); + const half = await read(V, "availableDailyAllowance"); + assert(half > daily / 3n && half < daily, `half-day refill looks wrong: ${half}`); + + // Still not enough for a full-cap release: the bucket refills, it does not + // jump. Only once it is full does the deferred release go through, and the + // releaser service is what retries it in production. + const early = await expectRevert(accounts[0], { ...V, functionName: "release", args: [n2, recipient, palletAmount] }); + assert(early?.includes("ExceedsDailyCap"), `expected ExceedsDailyCap at half refill, got: ${early}`); + + await warp(13 * 3600); + await send(accounts[0], { ...V, functionName: "release", args: [n2, recipient, palletAmount] }); + assertEq(await read(V, "nonceConsumed", [n2]), true, "deferred release after full refill"); +}); + +section("Guardian and sweep"); + +await check("the guardian can pause but cannot unpause; the admin can", async () => { + await refillBucket(); + await send(guardian, { ...V, functionName: "pause", args: [] }); + assertEq(await read(V, "paused"), true, "paused"); + + const n = nextNonce(); + const amount = 1n * 10n ** 12n; + await approveWith(3, n, recipient, amount); + assertEq(await read(V, "nonceConsumed", [n]), false, "must not release while paused"); + + const reason = await expectRevert(guardian, { ...V, functionName: "unpause", args: [] }); + assert(reason?.includes("NotAdmin"), `guardian should not unpause, got: ${reason}`); + + await send(admin, { ...V, functionName: "unpause", args: [] }); + assertEq(await read(V, "paused"), false, "unpaused by admin"); + await send(accounts[0], { ...V, functionName: "release", args: [n, recipient, amount] }); + assertEq(await read(V, "nonceConsumed", [n]), true, "deferred release after unpause"); +}); + +await check("the remainder cannot be swept before the earliest sweep timestamp", async () => { + const reason = await expectRevert(admin, { ...V, functionName: "sweepRemainder", args: [recipient, 1n] }); + assert(reason?.includes("SweepNotYetAllowed"), `expected SweepNotYetAllowed, got: ${reason}`); +}); + +await check("conservation holds: vault balance + released + swept == total supply", async () => { + const [bal, released, swept, supply] = await Promise.all([ + read(P, "balanceOf", [vault]), read(V, "totalReleased"), read(V, "totalSwept"), read(P, "totalSupply"), + ]); + assertEq(bal + released + swept, supply, "conservation identity"); +}); + +process.exit(summarise() ? 0 : 1); diff --git a/testing/src/phase3-e2e.mjs b/testing/src/phase3-e2e.mjs new file mode 100644 index 000000000..70b397ce2 --- /dev/null +++ b/testing/src/phase3-e2e.mjs @@ -0,0 +1,196 @@ +/** + * Phase 3 — the whole pipeline running together. + * + * A migration burned on the Substrate side must appear on Base without any + * manual step, driven by four independent attestor processes, an invariant + * monitor and the releaser. Every component is unit-tested in isolation; this + * is the only place they run as a system, which is where the remaining risk + * lives. + * + * Chopsticks stands in for Pendulum here rather than Zombienet: it reports + * finalized heads, which is what the attestors subscribe to, and it carries + * real mainnet state. What it does NOT reproduce is genuine relay-chain + * finality timing -- lag, and the possibility of a fork before finality. That + * remains a manual Zombienet exercise; see the README. + * + * Prerequisites: Chopsticks on :8000 (fresh), Anvil on :8545, and the services + * built (`npm run build` in attestor/, monitor/ and releaser/). + */ + +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { Keyring } from "@polkadot/keyring"; +import { cryptoWaitReady } from "@polkadot/util-crypto"; +import { assert, assertEq, check, section, summarise } from "./harness.mjs"; +import { accounts, attestors, keys, pub, releaser as releaserAcct, RPC, send } from "./anvil.mjs"; +import { erc20Abi, vaultAbi } from "./abi.mjs"; +import { deployStack, PARAMS } from "./deploy.mjs"; +import { alive, clearState, killStrays, logs, start, stopAll, stopAndWait, waitFor } from "./daemons.mjs"; + +const CHOPSTICKS = process.env.CHOPSTICKS_WS ?? "ws://127.0.0.1:8000"; +const CF = PARAMS.CONVERSION_FACTOR; + +console.log("Phase 3 — end-to-end pipeline"); +const { vault, pen } = deployStack(); +console.log(` vault ${vault}\n PEN ${pen}`); +const V = { address: vault, abi: vaultAbi }; +const P = { address: pen, abi: erc20Abi }; +const read = (c, fn, args) => pub.readContract({ ...c, functionName: fn, args }); + +// Accept admin so caps can be tuned during the run. +const admin = accounts[6]; +await send(admin, { ...V, functionName: "acceptAdmin", args: [] }); + +const api = await ApiPromise.create({ provider: new WsProvider(CHOPSTICKS), noInitWarn: true }); +await cryptoWaitReady(); +const keyring = new Keyring({ type: "sr25519", ss58Format: api.registry.chainSS58 ?? 56 }); +const alice = keyring.addFromUri("//Alice"); +const MIN = BigInt(api.consts.tokenMigration.minimumMigrationAmount.toString()); + +const newBlock = () => api.rpc("dev_newBlock"); +async function setStorage(v) { await api.rpc("dev_setStorage", v); await newBlock(); } +async function fund(who, amount) { + await setStorage({ System: { Account: [[[who], { providers: 1, data: { free: amount.toString() } }]] } }); +} + +// Enable migrations (stands in for the governance unpause). +await api.rpc("dev_setStorage", [[api.query.tokenMigration.paused.key(), "0x00"]]); +await newBlock(); + +/** Burn on Pendulum and return the emitted migration details. */ +async function migrate(amount, baseAddress) { + await fund(alice.address, MIN * 200n); + await api.tx.tokenMigration.migrate(amount, baseAddress).signAndSend(alice); + await newBlock(); + const events = await api.query.system.events(); + const ev = events.map((r) => r.event).find((e) => e.section === "tokenMigration" && e.method === "MigrationInitiated"); + assert(ev, "no MigrationInitiated event — the burn did not happen"); + return { nonce: BigInt(ev.data[0].toString()), amount: BigInt(ev.data[3].toString().replaceAll(",", "")) }; +} + +// --- start the fleet ------------------------------------------------------- +killStrays(); +clearState(["attestor/cp1.json", "attestor/cp2.json", "attestor/cp3.json", "attestor/cp4.json", + "releaser/releaser-state.json"]); +const baseEnv = { BASE_RPC_URL: RPC, VAULT_ADDRESS: vault, BASE_CHAIN_ID: "31337", POLL_INTERVAL_MS: "2000" }; +const startBlock = String(await pub.getBlockNumber()); +// Where the attestors begin scanning Pendulum. Must be the current head: a +// fork sits at ~7.4M blocks, and starting from 0 would have each daemon walk +// every historical block before reaching anything under test. +const pendulumHead = (await api.query.system.number()).toString(); +console.log(` attestors scan Pendulum from block ${pendulumHead}`); + +for (let i = 0; i < 4; i++) { + start(`attestor${i + 1}`, "attestor", { + ...baseEnv, PENDULUM_WS: CHOPSTICKS, + ATTESTOR_PRIVATE_KEY: keys[i + 1], CHECKPOINT_FILE: `./cp${i + 1}.json`, START_BLOCK: pendulumHead, + }); +} +start("monitor", "monitor", { ...baseEnv, PENDULUM_WS: CHOPSTICKS, GRACE_SECONDS: "30" }); +start("releaser", "releaser", { ...baseEnv, RELEASER_PRIVATE_KEY: keys[7], START_BLOCK: startBlock }); + +const recipient = "0x000000000000000000000000000000000000beef"; +const balanceOf = (who) => read(P, "balanceOf", [who]); + +section("The pipeline end to end"); + +await check("a burn on Pendulum arrives on Base with no manual step", async () => { + const before = await balanceOf(recipient); + const { nonce, amount } = await migrate(MIN * 2n, recipient); + try { + await waitFor(async () => (await read(V, "nonceConsumed", [nonce])) === true, + { timeoutMs: 90_000, label: "the release to land on Base" }); + } catch (e) { + // A silent pipeline is the hardest thing to debug from the outside, so + // surface what each daemon actually saw. + throw new Error(`${e.message}\n--- attestor1 ---\n${logs("attestor1")}\n--- monitor ---\n${logs("monitor")}`); + } + assertEq(await balanceOf(recipient), before + amount * CF, "released amount"); +}); + +await check("losing the approval race is benign — no attestor exits", async () => { + // Three approvals release; the fourth attestor's submission necessarily + // reverts. That is the normal case, and it must not kill the process. + for (let i = 1; i <= 4; i++) { + assert(alive(`attestor${i}`), `attestor${i} exited:\n${logs(`attestor${i}`)}`); + } +}); + +section("Resilience"); + +await check("one attestor down still releases (3 of 4)", async () => { + await stopAndWait("attestor4"); + const { nonce } = await migrate(MIN * 2n, recipient); + await waitFor(async () => (await read(V, "nonceConsumed", [nonce])) === true, + { timeoutMs: 90_000, label: "release with 3 attestors" }); +}); + +await check("two attestors down stops releases cleanly, losing nothing", async () => { + await stopAndWait("attestor3"); + const { nonce } = await migrate(MIN * 2n, recipient); + await new Promise((r) => setTimeout(r, 20_000)); + assertEq(await read(V, "nonceConsumed", [nonce]), false, "should not release below quorum"); + + // Restarting a third attestor drains the backlog without re-burning. + start("attestor3b", "attestor", { + ...baseEnv, PENDULUM_WS: CHOPSTICKS, + ATTESTOR_PRIVATE_KEY: keys[3], CHECKPOINT_FILE: "./cp3.json", START_BLOCK: pendulumHead, + }); + await waitFor(async () => (await read(V, "nonceConsumed", [nonce])) === true, + { timeoutMs: 120_000, label: "the backlog to drain after recovery" }); +}); + +section("Cap deferral and the releaser"); + +await check("a cap-deferred release is drained by the releaser, unattended", async () => { + // Squeeze the daily budget so the next migration cannot release immediately. + const tiny = MIN * CF; // one minimum-sized migration's worth + await send(admin, { ...V, functionName: "setCaps", args: [tiny * 10n, tiny] }); + // Earlier checks in this run consumed the (much larger) original budget. + // The bucket refills proportionally to the CURRENT dailyCap, so after + // lowering it the old consumption decays slowly -- warp past it, or the + // first release below has no budget at all. + await pub.request({ method: "evm_increaseTime", params: [30 * 24 * 3600] }); + await pub.request({ method: "evm_mine", params: [] }); + + const first = await migrate(MIN, recipient); + await waitFor(async () => (await read(V, "nonceConsumed", [first.nonce])) === true, + { timeoutMs: 90_000, label: "the first release to consume the budget" }); + + const second = await migrate(MIN, recipient); + await waitFor(async () => (await read(V, "pendingApprovedAmount")) > 0n, + { timeoutMs: 90_000, label: "the second release to be deferred" }); + assertEq(await read(V, "nonceConsumed", [second.nonce]), false, "should be deferred, not released"); + + // No manual release(): the releaser must pick it up as the bucket refills. + await pub.request({ method: "evm_increaseTime", params: [25 * 3600] }); + await pub.request({ method: "evm_mine", params: [] }); + try { + await waitFor(async () => (await read(V, "nonceConsumed", [second.nonce])) === true, + { timeoutMs: 120_000, label: "the releaser to drain the deferred release" }); + } catch (e) { + throw new Error(`${e.message}\n--- releaser ---\n${logs("releaser")}`); + } + assertEq(await read(V, "pendingApprovedAmount"), 0n, "pending accounting cleared"); + assert(alive("releaser"), `releaser exited:\n${logs("releaser")}`); +}); + +section("Invariants"); + +await check("the monitor reports healthy and stays running", async () => { + assert(alive("monitor"), `monitor exited:\n${logs("monitor")}`); + await waitFor(() => logs("monitor").includes("ok:"), { timeoutMs: 60_000, label: "a monitor ok line" }); + const bad = logs("monitor").match(/ALERT: (CONSERVATION VIOLATION|VAULT BALANCE MISMATCH)/); + assert(!bad, `monitor raised a conservation alert:\n${logs("monitor")}`); +}); + +await check("conservation holds across the whole run", async () => { + const [bal, released, swept, supply] = await Promise.all([ + balanceOf(vault), read(V, "totalReleased"), read(V, "totalSwept"), read(P, "totalSupply"), + ]); + assertEq(bal + released + swept, supply, "conservation identity"); +}); + +const ok = summarise(); +stopAll(); +await api.disconnect(); +process.exit(ok ? 0 : 1); From 1bb94dfb06ba649ccf8c07d552be10ce803c4ed0 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Thu, 27 Aug 2026 18:38:45 +0200 Subject: [PATCH 16/61] attestor: Ignore local checkpoint files from test runs --- attestor/.gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/attestor/.gitignore b/attestor/.gitignore index 9865bc15b..63e22d9f6 100644 --- a/attestor/.gitignore +++ b/attestor/.gitignore @@ -1,3 +1,4 @@ node_modules/ dist/ checkpoint.json +cp*.json From 7027c81afeb06544bf8a141369b8ed591e893f13 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 11:28:50 +0200 Subject: [PATCH 17/61] docs: Point the test plan at the automated harness Phases 1 and 3 were written as manual procedures and are now scripted, so the plan and the harness had drifted. Records what each script asserts, that phase 3 uses Chopsticks rather than Zombienet and why, that a Zombienet run for genuine finality timing is still outstanding, and the traps that cost debugging time when running either phase. --- docs/pen-migration-local-test-plan.md | 50 +++++++++++++++++++-------- 1 file changed, 36 insertions(+), 14 deletions(-) diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 7a9e6f242..1fc779dad 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -61,7 +61,8 @@ cp .env.example .env # fill in: 4 attestor addrs, Safes, caps, MAX_ISSUANC forge script script/Deploy.s.sol --rpc-url http://localhost:8545 --broadcast ``` -Then verify, with `cast`: +Automated by `testing/src/phase1-base.mjs` (`node testing/src/phase1-base.mjs`), +which deploys with the real script and then asserts: 1. `PEN.totalSupply()` == `MAX_ISSUANCE`, and `PEN.balanceOf(vault)` == the same. 2. `vault.threshold()` == 3, `vault.attestorCount()` == 4. @@ -79,9 +80,13 @@ Then verify, with `cast`: unpause is rejected from the guardian and accepted from admin. 9. `sweepRemainder` reverts before `earliestSweepTimestamp`. -**Pass:** all nine behave as described. (These mirror the Foundry suite, but -run against the deployed bytecode and the real deploy script — that is the -point.) +**Pass:** 11/11 from the script. These mirror the Foundry suite, but run +against the deployed bytecode and the real deploy script — that is the point. + +Note the daily cap is a rolling bucket and is **shared state across checks**, +refilling proportionally to the *current* `dailyCap`. Always `setCaps` first +and warp afterwards; warping first and lowering the cap leaves the earlier +consumption largely undecayed. --- @@ -135,18 +140,28 @@ it fails. --- -## Phase 3 — End-to-end with real finality (Zombienet + Anvil) +## Phase 3 — the whole pipeline together (Chopsticks + Anvil) + +Goal: a burn on the Substrate side reaches Base with no manual step, driven by +four attestor processes, the monitor and the releaser running as a system. +This is the only place the components race each other, and it is where the +remaining risk lives — both production bugs found during this work (attestor +gas under-estimation, releaser Multicall) were invisible to unit tests and +appeared here. -Goal: the whole pipeline works when the Substrate side has genuine -relay-chain finality — the condition the attestors depend on. +Automated by `testing/src/phase3-e2e.mjs`. Chopsticks stands in for Pendulum: +it reports finalized heads, which is what the attestors subscribe to, and it +carries real state. Use `testing/chopsticks-e2e.yml`, which caches to a `db` — +unlike phase 2 it makes no claim about a just-upgraded chain, and the cache +keeps a long run alive when the upstream public RPC drops the connection. -You already have `zombienet-macos-arm64` in the repo root. Spin up a relay -plus the Pendulum parachain with the new runtime, and run Anvil alongside with -the contracts from phase 1. +**Still manual: Zombienet.** What Chopsticks does not reproduce is genuine +relay-chain finality timing — lag, and the possibility of a fork before +finality. Running the same fleet against `zombienet-macos-arm64` with a relay +plus the Pendulum parachain remains an exercise to do before mainnet. -Then start the **four attestors, the monitor and the releaser**, each with its own -`.env` — separate keys, separate checkpoint files, all pointed at the same -vault: +The script starts the four attestors, the monitor and the releaser itself, +each with its own key and checkpoint file. To run them by hand instead: ```bash PENDULUM_WS=ws://127.0.0.1:9944 BASE_RPC_URL=http://localhost:8545 \ @@ -191,7 +206,14 @@ Checks: `VITE_MIGRATION_VAULT_ADDRESS` set to the Anvil vault; migrate through the UI and watch the status card go 0/3 → 3/3 → released. -**Pass:** 1–8 all hold. +**Pass:** 7/7 from the script. + +Three traps cost real debugging time and are worth knowing before you run it: +a wasm built with `--features runtime-benchmarks` cannot be used as a +Chopsticks override (building the node for benchmarks silently overwrites the +runtime wasm); the attestors' `START_BLOCK` must be the chain head, not 0, or +each daemon walks ~7.6M historical blocks; and strays from an aborted run keep +rewriting the checkpoint files a fresh run just cleared. --- From 9aeb8722537579e43a90e66c6cce1dfa3661d8ee Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 11:54:17 +0200 Subject: [PATCH 18/61] testing: Repair chain specs so the node can read back its own output `build-spec` serialises CurrencyId and OracleKey variants first-letter lowercased (`native`, `xCM`, `exchangeRate`) but deserialises expecting the original casing, so converting a plain spec to raw fails on a file the very same binary just wrote. This blocks both `benchmark pallet --chain pendulum` and Zombienet, which performs that conversion internally. Drive the repair from the node's own error rather than a hardcoded variant list that would drift: convert, read the rejected variant and its expected spelling off stderr, rename only exact case-insensitive matches, repeat. Genuine camelCase fields (chainType, bootNodes, tokenSymbol) are never touched because the node never complains about them. --- testing/src/fix-chainspec.mjs | 102 ++++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 testing/src/fix-chainspec.mjs diff --git a/testing/src/fix-chainspec.mjs b/testing/src/fix-chainspec.mjs new file mode 100644 index 000000000..1383317e2 --- /dev/null +++ b/testing/src/fix-chainspec.mjs @@ -0,0 +1,102 @@ +/** + * Repair a Pendulum chain spec so the node can read back its own output. + * + * `build-spec` serialises `CurrencyId`/`OracleKey` variants first-letter + * lowercased (`native`, `xCM`, `exchangeRate`) but deserialises expecting the + * original casing, so `build-spec --chain --raw` fails on a file + * the very same binary just wrote. This blocks anything that needs a raw spec: + * `benchmark pallet --chain pendulum`, and Zombienet, which performs the + * plain -> raw conversion internally. + * + * Rather than hardcode a variant list that will drift, this drives itself from + * the node's own error: it converts, reads `unknown variant `x`, expected one + * of `A`, `B`` off stderr, rewrites only the keys and string values that equal + * `x` case-insensitively, and repeats until the conversion succeeds. Field + * names that merely look similar (`chainType`, `bootNodes`, `tokenSymbol`) are + * never touched, because the node never complains about them. + * + * Usage: node testing/src/fix-chainspec.mjs + */ +import { execFileSync } from "node:child_process"; +import { readFileSync, writeFileSync } from "node:fs"; + +const [nodeBin, plainPath, rawOut] = process.argv.slice(2); +if (!nodeBin || !plainPath || !rawOut) { + console.error("usage: fix-chainspec.mjs "); + process.exit(2); +} + +const patchedPath = `${rawOut}.plain-patched.json`; +let spec = JSON.parse(readFileSync(plainPath, "utf8")); +const applied = []; + +/** Rename every object key and string value equal to `from` (case-insensitively). */ +function rename(node, from, to) { + let count = 0; + const walk = (value) => { + if (Array.isArray(value)) { + for (let i = 0; i < value.length; i++) { + if (typeof value[i] === "string" && value[i].toLowerCase() === from.toLowerCase()) { + value[i] = to; + count++; + } else walk(value[i]); + } + return; + } + if (value === null || typeof value !== "object") return; + for (const key of Object.keys(value)) { + const child = value[key]; + if (typeof child === "string" && child.toLowerCase() === from.toLowerCase()) { + value[key] = to; + count++; + } else walk(child); + if (key.toLowerCase() === from.toLowerCase() && key !== to) { + value[to] = value[key]; + delete value[key]; + count++; + } + } + }; + walk(node); + return count; +} + +for (let attempt = 1; attempt <= 40; attempt++) { + writeFileSync(patchedPath, JSON.stringify(spec)); + try { + const raw = execFileSync( + nodeBin, + ["build-spec", "--chain", patchedPath, "--raw", "--disable-default-bootnode"], + { maxBuffer: 512 * 1024 * 1024, stdio: ["ignore", "pipe", "pipe"] }, + ); + writeFileSync(rawOut, raw); + console.log(`raw spec written to ${rawOut} after ${applied.length} rename(s)`); + for (const entry of applied) console.log(` ${entry}`); + process.exit(0); + } catch (error) { + const stderr = `${error.stderr ?? ""}`; + // Two wordings, depending on how many variants the enum has: + // unknown variant `x`, expected one of `A`, `B` + // unknown variant `x`, expected `A` + const match = stderr.match(/unknown variant `([^`]+)`, expected (?:one of )?((?:`[^`]+`(?:, )?)+)/); + if (!match) { + console.error("build-spec failed for a reason this cannot repair:\n", stderr.slice(-2000)); + process.exit(1); + } + const bad = match[1]; + const expected = [...match[2].matchAll(/`([^`]+)`/g)].map((m) => m[1]); + const correct = expected.find((name) => name.toLowerCase() === bad.toLowerCase()); + if (!correct) { + console.error(`no case-insensitive match for \`${bad}\` among ${expected.join(", ")}`); + process.exit(1); + } + const renamed = rename(spec, bad, correct); + if (renamed === 0) { + console.error(`could not locate \`${bad}\` in the spec; aborting to avoid a loop`); + process.exit(1); + } + applied.push(`${bad} -> ${correct} (${renamed} site${renamed === 1 ? "" : "s"})`); + } +} +console.error("gave up after 40 attempts"); +process.exit(1); From e7ca41bbfcf1a030b72c31f2f9265cfc02fd9d48 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 11:56:05 +0200 Subject: [PATCH 19/61] testing: Add a Zombienet network for relay-finality validation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Chopsticks finalises every block it authors, so an attestor reading finalized heads there is indistinguishable from one reading best heads. This brings up a real relay plus the Pendulum collator, where finalized genuinely lags best. Two things had to be taken over from Zombienet to make this work. It cannot build this chain spec itself (build-spec cannot read back its own variant casings), so the spec is generated here and handed over as chain_spec_path — which in turn means Zombienet cannot inject the collator's authoring key, so genesis is repointed at well-known dev keys. Governance membership goes with them: this chain has no sudo pallet, and the pause origin must be drivable locally. The relay validators are named validator01/02 so the name 'alice' is free for the collator; Zombienet only derives //Alice for a node actually called alice, and a renamed collator silently gets a key that does not match genesis. Also swap the runtime the node binary embeds for the artifact we ship. A build with --features runtime-benchmarks rewrites that embedded wasm, and the result decompresses past the relay's VALIDATION_CODE_BOMB_LIMIT — the relay then rejects every candidate as PossibleBomb and the parachain never gets past its own block #1. --- .gitignore | 4 +- testing/src/make-zombienet-spec.mjs | 111 ++++++++++++++++++++++++++++ testing/zombienet.toml | 29 ++++++++ 3 files changed, 143 insertions(+), 1 deletion(-) create mode 100644 testing/src/make-zombienet-spec.mjs create mode 100644 testing/zombienet.toml diff --git a/.gitignore b/.gitignore index 5029ec793..a06687829 100644 --- a/.gitignore +++ b/.gitignore @@ -15,4 +15,6 @@ .DS_Store .idea -.vscode \ No newline at end of file +.vscode +# Locally generated Zombienet chain spec (13MB, rebuilt by make-zombienet-spec.mjs) +testing/.zombienet-pendulum-raw.json* diff --git a/testing/src/make-zombienet-spec.mjs b/testing/src/make-zombienet-spec.mjs new file mode 100644 index 000000000..4c4834e5b --- /dev/null +++ b/testing/src/make-zombienet-spec.mjs @@ -0,0 +1,111 @@ +/** + * Build a Zombienet-ready raw chain spec for the Pendulum runtime. + * + * Zombienet normally injects the collator's authoring key into genesis, but it + * can only do that for a spec it builds itself — and it cannot build this one, + * because `build-spec --chain pendulum` emits variant casings the node refuses + * to read back (see fix-chainspec.mjs). So we take over the whole pipeline: + * generate the plain spec, repoint authority/governance genesis at well-known + * dev keys, then convert to raw through the repair loop. + * + * Only the parts that must differ on a local network are touched — authorities, + * governance membership, funding and the relay id. The migration-relevant + * genesis (and therefore what the pallet sees) stays as production emits it. + * + * Usage: node testing/src/make-zombienet-spec.mjs + */ +import { execFileSync } from "node:child_process"; +import { cryptoWaitReady, encodeAddress } from "@polkadot/util-crypto"; +import { readFileSync, writeFileSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const here = dirname(fileURLToPath(import.meta.url)); +const repoRoot = resolve(here, "../.."); +const NODE_BIN = process.env.PENDULUM_NODE ?? resolve(repoRoot, "target/release/pendulum-node"); + +// Well-known dev public keys. This spec serialises AccountId as SS58, so they +// are encoded with the chain's own prefix once it is read off the spec. +// //Alice — zombienet derives the same key for a node named `alice`. +const ALICE_PUB = "0xd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d"; +// //Bob, //Charlie, //Dave — funded so the harness has accounts to migrate from. +const FUNDED_PUB = [ + ALICE_PUB, + "0x8eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48", + "0x90b5ab205c6974c9ea841be688864633dc9ca8a357843eeacf2314649965fe22", + "0x306721211d5404bd9da88e0204360a1a9ab8b87c66c1bc2fcdd37f3c2222cc20", +]; + +await cryptoWaitReady(); + +const rawOut = resolve(process.argv[2] ?? resolve(repoRoot, "testing/.zombienet-pendulum-raw.json")); +const plainPath = `${rawOut}.plain.json`; + +console.log("building plain spec from --chain pendulum ..."); +const plain = execFileSync(NODE_BIN, ["build-spec", "--chain", "pendulum", "--disable-default-bootnode"], { + maxBuffer: 512 * 1024 * 1024, +}); +const spec = JSON.parse(plain.toString()); + +// Network identity: a local rococo relay, not Polkadot. +spec.relay_chain = "rococo-local"; +spec.bootNodes = []; +spec.name = "Pendulum Local"; +spec.id = "pendulum-local"; + +const ss58Prefix = spec.properties?.ss58Format ?? 42; +const ALICE = encodeAddress(ALICE_PUB, ss58Prefix); +const FUNDED = FUNDED_PUB.map((pub) => encodeAddress(pub, ss58Prefix)); +console.log(`encoding accounts with ss58 prefix ${ss58Prefix}; alice = ${ALICE}`); + +const genesis = spec.genesis.runtimeGenesis.patch; + +// One collator (alice) authors every block. +const stake = 5_000_000_000_000_000n; +genesis.session = { keys: [[ALICE, ALICE, { aura: ALICE }]] }; +if (genesis.parachainStaking) { + genesis.parachainStaking.stakers = [[ALICE, null, Number(stake)]]; +} + +// Governance origins we must be able to drive locally: the pallet's pause +// origin is root / half-council / 2-3 technical committee, and this chain has +// no sudo pallet. +if (genesis.council) genesis.council.members = [ALICE]; +if (genesis.technicalCommittee) genesis.technicalCommittee.members = [ALICE]; + +// Fund the dev accounts on top of whatever production genesis already holds. +const endowment = 1_000_000_000_000_000_000n; +genesis.balances = genesis.balances ?? { balances: [] }; +const existing = new Map(genesis.balances.balances.map(([who, amount]) => [who, amount])); +for (const who of FUNDED) existing.set(who, Number(endowment)); +genesis.balances.balances = [...existing.entries()]; + +writeFileSync(plainPath, JSON.stringify(spec)); +console.log(`patched plain spec -> ${plainPath}`); + +console.log("converting to raw (repairing variant casings as needed) ..."); +execFileSync("node", [resolve(here, "fix-chainspec.mjs"), NODE_BIN, plainPath, rawOut], { stdio: "inherit" }); + +// Replace the runtime the node binary happens to embed with the artifact we +// actually ship. `cargo build --features runtime-benchmarks` silently rewrites +// the embedded wasm, and a benchmarking runtime decompresses past the relay's +// VALIDATION_CODE_BOMB_LIMIT (MAX_CODE_SIZE * 4 = 12 MiB) — the relay then +// rejects every candidate with `PossibleBomb` and the parachain never gets +// past its own block #1. Using the shipped artifact is both smaller and the +// thing under test. +const runtimeWasm = process.env.PENDULUM_RUNTIME_WASM ?? + resolve(repoRoot, "target/release/wbuild/pendulum-runtime/pendulum_runtime.compact.compressed.wasm"); +const rawSpec = JSON.parse(readFileSync(rawOut, "utf8")); +const CODE_KEY = "0x3a636f6465"; // :code +const embedded = rawSpec.genesis.raw.top[CODE_KEY]; +const shipped = `0x${readFileSync(runtimeWasm).toString("hex")}`; +if (embedded !== shipped) { + const mib = (hex) => ((hex.length - 2) / 2 / 1048576).toFixed(2); + console.log(`replacing embedded :code (${mib(embedded)} MiB) with ${runtimeWasm} (${mib(shipped)} MiB)`); + rawSpec.genesis.raw.top[CODE_KEY] = shipped; + writeFileSync(rawOut, JSON.stringify(rawSpec)); +} + +const written = JSON.parse(readFileSync(rawOut, "utf8")); +console.log(`\nraw spec: ${rawOut}`); +console.log(` name=${written.name} id=${written.id} para_id=${written.para_id} relay=${written.relay_chain}`); diff --git a/testing/zombienet.toml b/testing/zombienet.toml new file mode 100644 index 000000000..430da6c44 --- /dev/null +++ b/testing/zombienet.toml @@ -0,0 +1,29 @@ +# Zombienet: real relay-chain finality for the PEN migration attestors. +# Derived from the repo-root config.toml, with absolute binary paths (zombienet +# does not expand `~`) and RPC ports pinned so the harness can attach. +[settings] +timeout = 1000 + +[relaychain] +default_command = "/Users/marcel/Documents/polkadot/target/testnet/polkadot" +default_args = ["-lparachain=debug"] +chain = "rococo-local" + + [[relaychain.nodes]] + name = "validator01" + validator = true + + [[relaychain.nodes]] + name = "validator02" + validator = true + +[[parachains]] +id = 2094 +cumulus_based = true +chain_spec_path = "/Users/marcel/Documents/pendulum/testing/.zombienet-pendulum-raw.json" + + [parachains.collator] + name = "alice" + command = "/Users/marcel/Documents/pendulum/target/release/pendulum-node" + rpc_port = 9944 + args = ["-lparachain=info", "-lruntime=info"] From df0b9a8340accf4fef60ef8b375cfa6eb533e310 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 11:58:27 +0200 Subject: [PATCH 20/61] testing: Add phase 4, relay-chain finality under Zombienet MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Chopsticks finalises every block it authors, so an attestor reading finalized heads is indistinguishable there from one reading best heads — the safety property the whole design rests on was untested by construction. Against a real relay it is observable: the parachain held a steady ~2-block finality lag, and the checks assert both that finality advances (the relay is finalising parachain blocks at all) and that it lags (finality is not instant). Also re-checks the ships-paused default here, on a chain built from genesis rather than forked from mainnet state, and asserts the exact subscription the attestor uses delivers monotonically increasing heads. The collator RPC is discovered rather than fixed: Zombienet reassigns ports on every spawn, and the collator also exposes an embedded relay client, so a fixed port is as likely to report Rococo as Pendulum. --- docs/pen-migration-local-test-plan.md | 72 ++++++++++-- testing/src/phase4-zombienet.mjs | 155 ++++++++++++++++++++++++++ 2 files changed, 219 insertions(+), 8 deletions(-) create mode 100644 testing/src/phase4-zombienet.mjs diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 1fc779dad..733782d7f 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -11,7 +11,7 @@ needs both. |---|---|---| | **Anvil** (Foundry) | Base | Contract behaviour, deploy script, attestor/monitor wiring | | **Chopsticks** | Pendulum mainnet | Runtime upgrade + pallet against **real** balances, locks, vesting, treasury | -| **Zombienet** | Relay + parachain | The **relay-chain finality** path — attestors only act on finalized blocks, which Chopsticks cannot faithfully reproduce | +| **Zombienet** | Relay + parachain | The **relay-chain finality** path — attestors only act on finalized blocks, which Chopsticks cannot faithfully reproduce (phase 4) | Chopsticks gives realistic *state*; Zombienet gives realistic *finality*. You need both, for different reasons. Neither requires Paseo or Foucoco. @@ -155,10 +155,7 @@ carries real state. Use `testing/chopsticks-e2e.yml`, which caches to a `db` — unlike phase 2 it makes no claim about a just-upgraded chain, and the cache keeps a long run alive when the upstream public RPC drops the connection. -**Still manual: Zombienet.** What Chopsticks does not reproduce is genuine -relay-chain finality timing — lag, and the possibility of a fork before -finality. Running the same fleet against `zombienet-macos-arm64` with a relay -plus the Pendulum parachain remains an exercise to do before mainnet. +Genuine relay-chain finality is covered separately in phase 4. The script starts the four attestors, the monitor and the releaser itself, each with its own key and checkpoint file. To run them by hand instead: @@ -217,7 +214,66 @@ rewriting the checkpoint files a fresh run just cleared. --- -## Phase 4 — Failure drills (the runbooks) +## Phase 4 — real relay-chain finality (Zombienet) + +Goal: prove the finality gate is real. Chopsticks finalises every block it +authors, so an attestor reading finalized heads there is indistinguishable from +one reading best heads — the safety property is untested by construction. This +phase runs a genuine relay chain, where finalized lags best. + +```bash +node testing/src/make-zombienet-spec.mjs # generates testing/.zombienet-pendulum-raw.json +./zombienet-macos-arm64 spawn testing/zombienet.toml --provider native +node testing/src/phase4-zombienet.mjs # auto-discovers the collator RPC +``` + +Checks: + +1. The collator serves the **Pendulum** runtime (not the relay — the collator + exposes an embedded relay client too, and a fixed port is as likely to hit + Rococo). +2. `tokenMigration` is in the runtime metadata with all four extrinsics. +3. **Ships paused** on a chain that never wrote the storage — the same + fail-safe default phase 2 checks against forked mainnet state, here on a + chain built from genesis. +4. The parachain is authoring blocks. +5. **Finalized advances and lags best.** This is the phase: it proves both that + the relay is finalising parachain blocks at all, and that finality is not + instant. +6. The lag is strictly positive at least once — i.e. genuinely relay-driven. +7. `subscribeFinalizedHeads` delivers monotonically increasing heads, which is + the exact subscription the attestor uses. + +**Pass:** 7/7. Observed: a steady ~2-block parachain finality lag (best #9 / +finalized #7) behind a relay running its own ~3-block lag — against mainnet the +measured figure is ~2 blocks / ~47s, comfortably inside the monitor's +`GRACE_SECONDS` default of 1800. + +Three traps here cost real debugging time: + +- **A benchmarking build breaks the relay, not the node.** `cargo build + --features runtime-benchmarks` rewrites the runtime wasm embedded in the node + binary, `build-spec` propagates it into the chain spec, and the result + decompresses past the relay's `VALIDATION_CODE_BOMB_LIMIT` + (`MAX_CODE_SIZE * 4` = 12 MiB). The relay then rejects every candidate with + `PossibleBomb` and the parachain stalls at its own block #1 while the relay + looks perfectly healthy. `make-zombienet-spec.mjs` swaps in the shipped + artifact to avoid this. +- **The collator must be named `alice`.** Zombienet only derives `//Alice` for a + node with that name, and genesis pins the authority to that key. A collator + renamed by a collision (`alice-1`) silently gets a key that is not in genesis + and never authors. Hence the relay validators being `validator01`/`02`. +- **`build-spec` cannot read back its own output**, so Zombienet cannot build + this spec itself; see `fix-chainspec.mjs`. + +An older relay is fine. This ran against polkadot **0.9.40** driving a +**1.6.0** collator; the version gap does not affect finality behaviour, and the +only incompatibility encountered was the code-size limit above, which is our +artifact's problem rather than the relay's. + +--- + +## Phase 5 — Failure drills (the runbooks) Rehearse each runbook once against the local stack, so the first time you run them is not during an incident: @@ -234,9 +290,9 @@ them is not during an incident: --- -## Phase 5 — Exit criteria before mainnet +## Phase 6 — Exit criteria before mainnet -- [ ] Phases 1–4 pass end to end. +- [ ] Phases 1–5 pass end to end. - [ ] The upgrade ships paused, verified on a Chopsticks fork of **live** mainnet state (not a fresh chain). - [ ] A cap-deferred release recovers correctly without manual contract diff --git a/testing/src/phase4-zombienet.mjs b/testing/src/phase4-zombienet.mjs new file mode 100644 index 000000000..2d842f517 --- /dev/null +++ b/testing/src/phase4-zombienet.mjs @@ -0,0 +1,155 @@ +/** + * Phase 4 — real relay-chain finality (Zombienet). + * + * The one property Chopsticks cannot reproduce: Chopsticks finalises every + * block it authors, so an attestor reading finalized heads there is + * indistinguishable from one reading best heads. Against a genuine relay, + * finalized lags best, and this is what proves the attestors are gated on + * finality rather than merely appearing to be. + * + * Assumes a network spawned from testing/zombienet.toml with the collator RPC + * on 9944. Run: node testing/src/phase4-zombienet.mjs + */ +import { execSync } from "node:child_process"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { check, summarise } from "./harness.mjs"; + +const SAMPLE_SECONDS = Number(process.env.SAMPLE_SECONDS ?? 90); + +const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + +/** + * Find the collator's RPC endpoint. Zombienet reassigns ports on every spawn, + * and the collator also runs an embedded relay client, so a fixed port is as + * likely to land on Rococo as on Pendulum. Probe the listening ports and keep + * the one that reports the Pendulum runtime. + */ +async function connectToCollator() { + if (process.env.PENDULUM_WS) { + return ApiPromise.create({ provider: new WsProvider(process.env.PENDULUM_WS), noInitWarn: true }); + } + const listening = execSync( + "lsof -nP -iTCP -sTCP:LISTEN 2>/dev/null | grep pendulum | awk '{print $9}' | sed 's/.*://' | sort -un", + ) + .toString() + .trim() + .split("\n") + .filter(Boolean); + for (const port of listening) { + const url = `ws://127.0.0.1:${port}`; + // Hold the provider separately: several of these ports are not RPC at + // all (prometheus, p2p), and a provider left behind by a failed probe + // keeps retrying for the rest of the run. + const provider = new WsProvider(url, 1000); + let api; + try { + api = await Promise.race([ + ApiPromise.create({ provider, noInitWarn: true, throwOnConnect: true }), + new Promise((_, reject) => setTimeout(() => reject(new Error("timeout")), 8000)), + ]); + const chain = (await api.rpc.system.chain()).toString(); + if (/pendulum/i.test(chain)) { + console.log(`collator RPC discovered on ${url} (${chain})`); + return api; + } + await api.disconnect(); + } catch { + if (api) await api.disconnect().catch(() => {}); + else await provider.disconnect().catch(() => {}); + } + } + throw new Error(`no Pendulum collator RPC among listening ports: ${listening.join(", ")}`); +} + +const api = await connectToCollator(); + +await check("collator is a parachain running the Pendulum runtime", async () => { + const chain = (await api.rpc.system.chain()).toString(); + const name = (await api.rpc.system.name()).toString(); + if (!/pendulum/i.test(chain)) throw new Error(`unexpected chain: ${chain}`); + return `${chain} / ${name}`; +}); + +await check("token-migration pallet is present in the runtime metadata", async () => { + if (!api.query.tokenMigration) throw new Error("tokenMigration missing from metadata"); + const calls = Object.keys(api.tx.tokenMigration); + for (const required of ["migrate", "setPaused", "migrateTreasury", "setTreasuryDestination"]) { + if (!calls.includes(required)) throw new Error(`missing extrinsic ${required}`); + } + return `4 extrinsics: ${calls.join(", ")}`; +}); + +await check("pallet ships PAUSED on a chain that never wrote its storage", async () => { + const paused = await api.query.tokenMigration.paused(); + if (!paused.isTrue) throw new Error("pallet is unpaused on a fresh chain — fail-safe default broken"); + return "paused = true"; +}); + +await check("parachain blocks are being authored", async () => { + const start = (await api.rpc.chain.getHeader()).number.toNumber(); + await sleep(30_000); + const end = (await api.rpc.chain.getHeader()).number.toNumber(); + if (end <= start) throw new Error(`best head stuck at ${start}`); + return `best head ${start} -> ${end}`; +}); + +// The core of the phase: finality must ADVANCE (proving relay/collator +// interop) and must LAG best (proving it is genuine relay finality and not +// the instant self-finalisation Chopsticks does). +let sawLag = false; +let finalityAdvanced = false; +await check(`finalized head advances and lags best over ${SAMPLE_SECONDS}s`, async () => { + const firstFinal = (await api.rpc.chain.getHeader(await api.rpc.chain.getFinalizedHead())).number.toNumber(); + const samples = []; + const deadline = Date.now() + SAMPLE_SECONDS * 1000; + while (Date.now() < deadline) { + const best = (await api.rpc.chain.getHeader()).number.toNumber(); + const fin = (await api.rpc.chain.getHeader(await api.rpc.chain.getFinalizedHead())).number.toNumber(); + samples.push(best - fin); + if (best - fin > 0) sawLag = true; + await sleep(6000); + } + const lastFinal = (await api.rpc.chain.getHeader(await api.rpc.chain.getFinalizedHead())).number.toNumber(); + finalityAdvanced = lastFinal > firstFinal; + if (!finalityAdvanced) { + throw new Error(`finalized head stuck at ${firstFinal} — relay is not finalising parachain blocks`); + } + const mean = samples.reduce((a, b) => a + b, 0) / samples.length; + return `finalized ${firstFinal} -> ${lastFinal}; lag samples [${samples.join(", ")}], mean ${mean.toFixed(1)} blocks`; +}); + +await check("finality lag is strictly positive at least once (not instant finalisation)", async () => { + if (!sawLag) throw new Error("finalized never lagged best — this is not genuine relay finality"); + return "observed best > finalized, as expected against a real relay"; +}); + +await check("subscribeFinalizedHeads delivers monotonically increasing heads", async () => { + const seen = []; + await new Promise((resolve, reject) => { + let unsub; + const timer = setTimeout(() => { + if (unsub) unsub(); + seen.length >= 2 ? resolve() : reject(new Error(`only ${seen.length} finalized head(s) in 60s`)); + }, 60_000); + api.rpc.chain + .subscribeFinalizedHeads((head) => { + seen.push(head.number.toNumber()); + if (seen.length >= 3) { + clearTimeout(timer); + if (unsub) unsub(); + resolve(); + } + }) + .then((u) => { + unsub = u; + }) + .catch(reject); + }); + for (let i = 1; i < seen.length; i++) { + if (seen[i] <= seen[i - 1]) throw new Error(`non-monotonic finalized heads: ${seen.join(", ")}`); + } + return `heads ${seen.join(" -> ")}`; +}); + +await api.disconnect(); +summarise(); From 9c2add8f99663f1a0a0d4c28b2b906edd81ed78b Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 12:01:14 +0200 Subject: [PATCH 21/61] docs: Record relay-finality validation in the test plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 4 was previously described as an exercise left for before mainnet. It now exists and passes, so document it as a phase with its own pass criteria, the measured lag, and the three traps that cost debugging time — chief among them that a runtime-benchmarks build makes the relay reject every candidate while looking entirely healthy from the relay's side. Renumbers the failure drills and exit criteria to 5 and 6, and adds finality gating to the exit checklist. --- docs/pen-migration-implementation-overview.md | 5 ++++- docs/pen-migration-local-test-plan.md | 2 ++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/pen-migration-implementation-overview.md b/docs/pen-migration-implementation-overview.md index 7aa2c3f98..91c89a6cd 100644 --- a/docs/pen-migration-implementation-overview.md +++ b/docs/pen-migration-implementation-overview.md @@ -36,7 +36,7 @@ watched by an independent monitor that can auto-pause. One-way by design. | Attestor daemon | `attestor/` | TypeScript; finalized-heads-only, strictly ordered blocks, crash-safe checkpoint, idempotent + race-tolerant approvals, fail-fast on decode errors (4-field shape asserted), startup set-membership check, low-gas/webhook alerts; ops guide in its README | | Invariant monitor | `monitor/` | Independent watchdog: conservation checks (block-pinned reads) + per-nonce liveness batched via Multicall3; webhook alerts; optional guardian auto-pause | | Releaser | `releaser/` | Drains cap-deferred releases via the permissionless `release()`; unprivileged gas-only key; classifies self-healing vs governance-blocked failures | -| Test harness | `testing/` | Automates phase 2 of the local test plan against a Chopsticks fork of live mainnet state | +| Test harness | `testing/` | Automates all four phases of the local test plan: contracts on Anvil, the pallet against a Chopsticks fork of live mainnet state, the full attestor/monitor/releaser pipeline end to end, and relay-chain finality under Zombienet | | Runbooks | `docs/pen-migration-runbooks.md` | RB-1…RB-7: key compromise, outage, invariant breach, pause/unpause, runtime upgrade, attestor rotation, window close | | Internal security review | `docs/pen-migration-internal-review.md` | The project's security-assurance record across seven adversarial rounds | @@ -71,7 +71,10 @@ restricted to consumed nonces. Details and verified-not-vulnerable list in | `forge test` | 37, incl. 512-run fuzz and a full Governor lifecycle | | `attestor` / `monitor` / `releaser` | 6 / 7 / 7 | | Portal `yarn build` (tsc + vite) | clean against `main` | +| `testing/src/phase1-base.mjs` | 11/11 against the real deploy script on Anvil | | `testing/src/phase2-pendulum.mjs` | 14/14 against a Chopsticks fork of live mainnet state | +| `testing/src/phase3-e2e.mjs` | 7/7 end to end, four attestors + monitor + releaser | +| `testing/src/phase4-zombienet.mjs` | 7/7 against a real relay (~2-block parachain finality lag) | Seven internal adversarial review rounds have run; each found real issues (sometimes in a prior round's own fix), all fixed with regression tests and diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 733782d7f..192c74fcf 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -298,6 +298,8 @@ them is not during an incident: - [ ] A cap-deferred release recovers correctly without manual contract surgery. - [ ] All four attestors survive a full run without a fatal exit. +- [ ] Finality gating confirmed against a real relay: finalized lags best, and + the attestors act only on the finalized stream. - [ ] The monitor alerts on a real injected deficit and tolerates a surplus. - [ ] Benchmarks re-run on reference hardware and the generated weights replace the manual estimates. From ec4100db55db405818f64ad12e1300f82d94b462 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 16:15:36 +0200 Subject: [PATCH 22/61] testing: Add rehearsal environment with testnet guardrails MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixed throwaway keys in a gitignored .env.rehearsal, so a rehearsal can be re-run repeatedly without re-funding from a faucet each time. The important part is assertTestnet. This script is built to be run casually and often, with real keys in a real env file, and it deploys contracts and moves tokens — so the cost of it ever pointing at Base mainnet or at real Pendulum is unbounded. Both are refused explicitly, before anything is deployed or signed: chain 8453 outright, anything other than Sepolia without a deliberate override, and a Substrate endpoint that does not self-report as the local chain. --- .gitignore | 4 + testing/.env.rehearsal.example | 41 ++++++++++ testing/src/rehearsal-env.mjs | 142 +++++++++++++++++++++++++++++++++ 3 files changed, 187 insertions(+) create mode 100644 testing/.env.rehearsal.example create mode 100644 testing/src/rehearsal-env.mjs diff --git a/.gitignore b/.gitignore index a06687829..a5e09e6dd 100644 --- a/.gitignore +++ b/.gitignore @@ -18,3 +18,7 @@ .vscode # Locally generated Zombienet chain spec (13MB, rebuilt by make-zombienet-spec.mjs) testing/.zombienet-pendulum-raw.json* + +# Rehearsal secrets and run artifacts (throwaway keys, but never commit them) +testing/.env.rehearsal +testing/.rehearsal/ diff --git a/testing/.env.rehearsal.example b/testing/.env.rehearsal.example new file mode 100644 index 000000000..85814ef51 --- /dev/null +++ b/testing/.env.rehearsal.example @@ -0,0 +1,41 @@ +# Rehearsal environment — Base Sepolia + a local Zombienet Pendulum. +# +# Copy to `.env.rehearsal` (gitignored) and fill in. These keys are THROWAWAY: +# they exist only to rehearse the migration end to end and must never be reused +# for mainnet. Fund each address once from a Base Sepolia faucet; the rehearsal +# redeploys contracts on every run but reuses these EOAs. +# +# Run `node src/rehearsal.mjs --preflight` to see which addresses need funding. + +BASE_SEPOLIA_RPC_URL=https://sepolia.base.org + +# Deploys the contracts. Needs the most gas. +DEPLOYER_PRIVATE_KEY= + +# The 3-of-4 attestor set. Each submits its own approve() transactions. +ATTESTOR_1_PRIVATE_KEY= +ATTESTOR_2_PRIVATE_KEY= +ATTESTOR_3_PRIVATE_KEY= +ATTESTOR_4_PRIVATE_KEY= + +# Separation of duties: the guardian can pause but not unpause, the admin can +# do both and tune caps. Distinct keys on purpose — that asymmetry is part of +# what the rehearsal exercises. +GUARDIAN_PRIVATE_KEY= +ADMIN_PRIVATE_KEY= + +# Unprivileged, gas only. Drains cap-deferred releases. +RELEASER_PRIVATE_KEY= + +# --- cap sizing ------------------------------------------------------------ +# Sized for WALL CLOCK, not for production. The rolling bucket refills at +# DAILY_CAP per 86400s, so a daily cap of 28,800 PEN refills 100 PEN (the +# on-chain minimum migration) every ~5 minutes — enough to watch a deferred +# release drain by itself inside one session. Production values are validated +# separately on Anvil in phase 1, where time can be warped. +REHEARSAL_DAILY_CAP_PEN=28800 +REHEARSAL_PER_RELEASE_CAP_PEN=50000 + +# How far ahead the immutable sweep floor sits. Short so the sweep path is +# reachable in a long session; production is 2027-03-01. +REHEARSAL_SWEEP_OFFSET_SECONDS=3600 diff --git a/testing/src/rehearsal-env.mjs b/testing/src/rehearsal-env.mjs new file mode 100644 index 000000000..73fab4869 --- /dev/null +++ b/testing/src/rehearsal-env.mjs @@ -0,0 +1,142 @@ +/** + * Environment, clients and guardrails for the rehearsal. + * + * The single most important thing in this file is `assertTestnet`. This script + * exists to be run casually and repeatedly, with real keys in a real .env, and + * it deploys contracts and moves tokens. The cost of it ever pointing at Base + * mainnet or at real Pendulum is unbounded, so both are checked explicitly + * before anything is deployed or signed. + */ + +import { readFileSync } from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { createPublicClient, createWalletClient, defineChain, http } from "viem"; +import { privateKeyToAccount } from "viem/accounts"; + +export const TESTING = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); +export const ROOT = path.resolve(TESTING, ".."); + +export const BASE_SEPOLIA_CHAIN_ID = 84532; +export const BASE_MAINNET_CHAIN_ID = 8453; +/** Canonical Multicall3, deployed at the same address on Base Sepolia. The + * monitor and releaser batch their reads through it and viem refuses to use + * it unless the chain definition names it. */ +export const MULTICALL3 = "0xcA11bde05977b3631167028862bE2a173976CA11"; + +export const PEN_18 = 10n ** 18n; // Base side +export const PEN_12 = 10n ** 12n; // pallet side +export const CONVERSION_FACTOR = 1_000_000n; + +const REQUIRED = [ + "BASE_SEPOLIA_RPC_URL", + "DEPLOYER_PRIVATE_KEY", + "ATTESTOR_1_PRIVATE_KEY", + "ATTESTOR_2_PRIVATE_KEY", + "ATTESTOR_3_PRIVATE_KEY", + "ATTESTOR_4_PRIVATE_KEY", + "GUARDIAN_PRIVATE_KEY", + "ADMIN_PRIVATE_KEY", + "RELEASER_PRIVATE_KEY", +]; + +/** Parse a dotenv-style file without taking on a dependency. */ +function parseEnvFile(file) { + const out = {}; + let text; + try { + text = readFileSync(file, "utf8"); + } catch { + throw new Error( + `missing ${file}\n\n cp testing/.env.rehearsal.example testing/.env.rehearsal\n\nthen fill in the throwaway keys.`, + ); + } + for (const raw of text.split("\n")) { + const line = raw.trim(); + if (!line || line.startsWith("#")) continue; + const eq = line.indexOf("="); + if (eq === -1) continue; + out[line.slice(0, eq).trim()] = line.slice(eq + 1).trim(); + } + return out; +} + +export function loadEnv() { + const file = path.join(TESTING, ".env.rehearsal"); + const env = parseEnvFile(file); + const missing = REQUIRED.filter((k) => !env[k]); + if (missing.length) { + throw new Error(`${file} is missing values for:\n ${missing.join("\n ")}`); + } + for (const key of REQUIRED) { + if (key.endsWith("PRIVATE_KEY") && !/^0x[0-9a-fA-F]{64}$/.test(env[key])) { + throw new Error(`${key} is not a 32-byte hex private key`); + } + } + return env; +} + +export function buildContext(env) { + const chain = defineChain({ + id: BASE_SEPOLIA_CHAIN_ID, + name: "base-sepolia", + nativeCurrency: { name: "Ether", symbol: "ETH", decimals: 18 }, + rpcUrls: { default: { http: [env.BASE_SEPOLIA_RPC_URL] } }, + contracts: { multicall3: { address: MULTICALL3 } }, + }); + const key = (name) => env[name]; + const account = (name) => privateKeyToAccount(key(name)); + + const roles = { + deployer: account("DEPLOYER_PRIVATE_KEY"), + attestors: [1, 2, 3, 4].map((i) => account(`ATTESTOR_${i}_PRIVATE_KEY`)), + guardian: account("GUARDIAN_PRIVATE_KEY"), + admin: account("ADMIN_PRIVATE_KEY"), + releaser: account("RELEASER_PRIVATE_KEY"), + }; + const pub = createPublicClient({ chain, transport: http(env.BASE_SEPOLIA_RPC_URL) }); + const wallet = (acct) => createWalletClient({ account: acct, chain, transport: http(env.BASE_SEPOLIA_RPC_URL) }); + + return { chain, roles, pub, wallet, keys: { ...env } }; +} + +/** Send a transaction and wait for it, failing loudly on revert. */ +export async function send(ctx, acct, params) { + const { request } = await ctx.pub.simulateContract({ account: acct, ...params }); + const hash = await ctx.wallet(acct).writeContract(request); + const receipt = await ctx.pub.waitForTransactionReceipt({ hash }); + if (receipt.status !== "success") throw new Error(`transaction reverted: ${hash}`); + return receipt; +} + +/** + * Refuse to run anywhere that could cost real money. + * + * Checked before any deployment: the EVM side must be Base Sepolia, and the + * Substrate side must be the local Zombienet chain rather than Pendulum + * mainnet. `REHEARSAL_ALLOW_CHAIN_ID` exists for a deliberate move to another + * testnet and still refuses Base mainnet outright. + */ +export async function assertTestnet(ctx, substrateApi) { + const chainId = await ctx.pub.getChainId(); + const allowed = Number(process.env.REHEARSAL_ALLOW_CHAIN_ID ?? BASE_SEPOLIA_CHAIN_ID); + if (chainId === BASE_MAINNET_CHAIN_ID) { + throw new Error("REFUSING TO RUN: the RPC points at Base MAINNET (chain 8453)."); + } + if (chainId !== allowed) { + throw new Error( + `REFUSING TO RUN: expected chain ${allowed} (Base Sepolia), got ${chainId}. ` + + "Set REHEARSAL_ALLOW_CHAIN_ID only if you mean it.", + ); + } + if (substrateApi) { + const name = (await substrateApi.rpc.system.chain()).toString(); + if (!/local/i.test(name)) { + throw new Error( + `REFUSING TO RUN: the Substrate endpoint reports "${name}", which is not the local ` + + "Zombienet chain. This script burns real balances; point it at the local network.", + ); + } + } + return chainId; +} From e0eda12a1dd91b3459725844972546767ba565b0 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 16:15:37 +0200 Subject: [PATCH 23/61] testing: Extract Zombienet control into a shared module MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spawn, spec generation, collator discovery, finality readiness and teardown, so the rehearsal does not restate what phase 4 already worked out. killMatching reads the process table and filters in-process rather than shelling out to `pkill -f`. A shell running `pkill -f ` matches its own command line, because the pattern is part of it — which is how an earlier session produced waiter shells that spun forever on a condition that could never become false. It also skips its own ancestors, so a teardown can never kill its caller. --- testing/src/zombienet.mjs | 147 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 147 insertions(+) create mode 100644 testing/src/zombienet.mjs diff --git a/testing/src/zombienet.mjs b/testing/src/zombienet.mjs new file mode 100644 index 000000000..7838e48fa --- /dev/null +++ b/testing/src/zombienet.mjs @@ -0,0 +1,147 @@ +/** + * Spawning, probing and tearing down the local Zombienet network. + * + * Note `killMatching`: it deliberately does not shell out to `pkill -f`. + * `pkill -f ` run through a shell matches the shell's own command + * line, because the pattern is part of it — which is how three waiter shells + * in an earlier session ended up killing themselves or spinning forever on a + * condition that could never become false. Reading the process table and + * filtering in-process avoids the whole class of problem. + */ + +import { execFileSync, execSync, spawn } from "node:child_process"; +import path from "node:path"; +import { ROOT, TESTING } from "./rehearsal-env.mjs"; + +export const CHAIN_BINARIES = [ + "zombienet-macos", + "target/testnet/polkadot", + "target/release/pendulum-node", +]; + +export const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + +/** Kill processes whose command line contains any of `patterns`, skipping this + * process and its ancestors so a teardown can never kill its own caller. */ +export function killMatching(patterns, { signal = "SIGKILL" } = {}) { + let table; + try { + table = execSync("ps -eo pid,ppid,command", { encoding: "utf8", maxBuffer: 16 * 1024 * 1024 }); + } catch { + return 0; + } + const own = new Set(); + for (let pid = process.pid; pid && pid > 1; ) { + own.add(pid); + const row = table.split("\n").find((l) => Number(l.trim().split(/\s+/)[0]) === pid); + pid = row ? Number(row.trim().split(/\s+/)[1]) : 0; + } + let killed = 0; + for (const line of table.split("\n").slice(1)) { + const trimmed = line.trim(); + if (!trimmed) continue; + const [pidStr, , ...rest] = trimmed.split(/\s+/); + const pid = Number(pidStr); + const command = rest.join(" "); + if (!pid || own.has(pid)) continue; + if (!patterns.some((p) => command.includes(p))) continue; + try { + process.kill(pid, signal); + killed++; + } catch { /* already gone */ } + } + return killed; +} + +/** Regenerate the raw chain spec Zombienet consumes. */ +export function generateSpec(log) { + log("generating chain spec (repairing variant casings, swapping in the shipped runtime) ..."); + execFileSync("node", [path.join(TESTING, "src/make-zombienet-spec.mjs")], { + cwd: ROOT, + stdio: ["ignore", "pipe", "pipe"], + maxBuffer: 64 * 1024 * 1024, + }); +} + +export function spawnNetwork(log) { + const binary = path.join(ROOT, "zombienet-macos-arm64"); + log("spawning Zombienet (relay + Pendulum collator) ..."); + const proc = spawn(binary, ["spawn", "testing/zombienet.toml", "--provider", "native"], { + cwd: ROOT, + stdio: ["ignore", "pipe", "pipe"], + }); + const out = []; + const capture = (c) => { + for (const l of String(c).split("\n")) if (l.trim()) out.push(l.replace(/\x1b\[[0-9;]*m/g, "")); + if (out.length > 2000) out.splice(0, out.length - 2000); + }; + proc.stdout.on("data", capture); + proc.stderr.on("data", capture); + return { proc, out }; +} + +export function teardown(log) { + const killed = killMatching(CHAIN_BINARIES); + if (log) log(`torn down ${killed} chain process(es)`); + return killed; +} + +/** + * Find the collator's RPC. Zombienet reassigns ports on every spawn and the + * collator also exposes an embedded relay client, so a fixed port is as likely + * to answer as Rococo as it is as Pendulum. + */ +export async function discoverCollator(ApiPromise, WsProvider, { timeoutMs = 240_000, log } = {}) { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + let ports = []; + try { + ports = execSync( + "lsof -nP -iTCP -sTCP:LISTEN 2>/dev/null | grep pendulum | awk '{print $9}' | sed 's/.*://' | sort -un", + { encoding: "utf8" }, + ).trim().split("\n").filter(Boolean); + } catch { /* nothing listening yet */ } + + for (const port of ports) { + const url = `ws://127.0.0.1:${port}`; + // Hold the provider so a failed probe can be disconnected: several of + // these ports are not RPC at all, and an abandoned provider retries + // for the rest of the run. + const provider = new WsProvider(url, 1000); + let api; + try { + api = await Promise.race([ + ApiPromise.create({ provider, noInitWarn: true, throwOnConnect: true }), + new Promise((_, rej) => setTimeout(() => rej(new Error("timeout")), 8000)), + ]); + const chain = (await api.rpc.system.chain()).toString(); + if (/pendulum/i.test(chain)) { + if (log) log(`collator RPC on ${url} (${chain})`); + return api; + } + await api.disconnect(); + } catch { + if (api) await api.disconnect().catch(() => {}); + else await provider.disconnect().catch(() => {}); + } + } + await sleep(5000); + } + throw new Error("no Pendulum collator RPC appeared — check the Zombienet log"); +} + +/** Wait until the parachain is finalising, which is the real readiness signal: + * authoring alone does not mean the relay is backing and including candidates. */ +export async function waitForFinality(api, { timeoutMs = 300_000, log } = {}) { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + const best = (await api.rpc.chain.getHeader()).number.toNumber(); + const finalized = (await api.rpc.chain.getHeader(await api.rpc.chain.getFinalizedHead())).number.toNumber(); + if (finalized >= 1) { + if (log) log(`parachain finalising (best #${best}, finalized #${finalized})`); + return { best, finalized }; + } + await sleep(5000); + } + throw new Error("parachain never finalised a block — the relay is not including candidates"); +} From c1b265443596d72c36c300afd26572c4eea807bf Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 16:15:37 +0200 Subject: [PATCH 24/61] testing: Add phase 5, full-stack rehearsal on Base Sepolia MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Runs the whole system against real infrastructure on both sides at once: genuine relay finality from Zombienet, and a public EVM with real gas estimation, block times and RPC behaviour. Both production bugs found during this work lived exactly there and were unreachable from unit tests. Contracts are redeployed every run. The local chain is ephemeral and restarts its nonce sequence at zero on each spawn while the vault's nonceConsumed mapping is permanent, so a reused vault makes the second run re-emit nonce 0, every attestor's pre-check answer 'already handled', and the pipeline log skips while testing nothing. A guard asserts that rather than trusting the convention. Migrations are enabled through the technical-committee origin instead of a storage poke — this chain has no sudo pallet, so the rehearsal drives the same origin that will unpause mainnet. Caps are sized for wall clock: there is no evm_increaseTime on a public chain, so the rolling bucket is tuned to return the minimum migration every ~5 minutes and the deferred-drain path can be observed unaided. --- testing/src/rehearsal-deploy.mjs | 73 ++++++ testing/src/rehearsal.mjs | 389 +++++++++++++++++++++++++++++++ 2 files changed, 462 insertions(+) create mode 100644 testing/src/rehearsal-deploy.mjs create mode 100644 testing/src/rehearsal.mjs diff --git a/testing/src/rehearsal-deploy.mjs b/testing/src/rehearsal-deploy.mjs new file mode 100644 index 000000000..02326df3b --- /dev/null +++ b/testing/src/rehearsal-deploy.mjs @@ -0,0 +1,73 @@ +/** + * Deploys the migration stack to Base Sepolia using the real Deploy.s.sol. + * + * Fresh contracts on every run, deliberately. The Zombienet chain is ephemeral + * and restarts its nonce sequence at zero on each spawn, while the vault's + * `nonceConsumed` mapping is permanent — so reusing a vault across runs means + * the second run re-emits nonce 0, every attestor's `alreadyHandled` pre-check + * returns true, and the pipeline logs "skip (already released)" while testing + * nothing at all. Redeploying also means the deploy script itself is exercised + * on every cycle rather than once. + */ + +import { execFileSync } from "node:child_process"; +import { readFileSync } from "node:fs"; +import path from "node:path"; +import { BASE_SEPOLIA_CHAIN_ID, PEN_18, ROOT } from "./rehearsal-env.mjs"; + +const CONTRACTS = path.join(ROOT, "contracts"); + +export function rehearsalParams(env) { + const dailyCapPen = BigInt(env.REHEARSAL_DAILY_CAP_PEN ?? "28800"); + const perReleaseCapPen = BigInt(env.REHEARSAL_PER_RELEASE_CAP_PEN ?? "50000"); + return { + maxIssuance: 150_000_000n * PEN_18, + dailyCap: dailyCapPen * PEN_18, + perReleaseCap: perReleaseCapPen * PEN_18, + sweepOffsetSeconds: Number(env.REHEARSAL_SWEEP_OFFSET_SECONDS ?? "3600"), + // How long 100 PEN takes to refill, for the deferred-drain scenario. + refillSecondsPer100Pen: Number((100n * 86400n) / dailyCapPen), + }; +} + +export function deployToSepolia({ env, roles, log }) { + const params = rehearsalParams(env); + const sweepTs = Math.floor(Date.now() / 1000) + params.sweepOffsetSeconds; + const scriptEnv = { + ...process.env, + PRIVATE_KEY: env.DEPLOYER_PRIVATE_KEY, + ADMIN_SAFE: roles.admin.address, + GUARDIAN_SAFE: roles.guardian.address, + ATTESTOR_1: roles.attestors[0].address, + ATTESTOR_2: roles.attestors[1].address, + ATTESTOR_3: roles.attestors[2].address, + ATTESTOR_4: roles.attestors[3].address, + MAX_ISSUANCE: params.maxIssuance.toString(), + PER_RELEASE_CAP: params.perReleaseCap.toString(), + DAILY_CAP: params.dailyCap.toString(), + EARLIEST_SWEEP_TS: String(sweepTs), + }; + + log(`deploying to Base Sepolia (sweep floor in ${params.sweepOffsetSeconds}s) ...`); + execFileSync( + "forge", + [ + "script", "script/Deploy.s.sol", + "--rpc-url", env.BASE_SEPOLIA_RPC_URL, + "--broadcast", + "--private-key", env.DEPLOYER_PRIVATE_KEY, + "--slow", + ], + { cwd: CONTRACTS, env: scriptEnv, stdio: ["ignore", "pipe", "pipe"], maxBuffer: 64 * 1024 * 1024 }, + ); + + const file = path.join( + CONTRACTS, "broadcast", "Deploy.s.sol", String(BASE_SEPOLIA_CHAIN_ID), "run-latest.json", + ); + const run = JSON.parse(readFileSync(file, "utf8")); + const creations = run.transactions.filter((t) => t.transactionType === "CREATE"); + const vault = creations.find((t) => t.contractName === "MigrationVault")?.contractAddress; + const pen = creations.find((t) => t.contractName === "PEN")?.contractAddress; + if (!vault || !pen) throw new Error("could not locate deployed addresses in the broadcast record"); + return { vault, pen, params, sweepTs }; +} diff --git a/testing/src/rehearsal.mjs b/testing/src/rehearsal.mjs new file mode 100644 index 000000000..a04113955 --- /dev/null +++ b/testing/src/rehearsal.mjs @@ -0,0 +1,389 @@ +/** + * Phase 5 — full-stack rehearsal: local Zombienet Pendulum + Base Sepolia. + * + * Phases 1-4 each hold one half of the system still. This runs the whole thing + * against real infrastructure on both sides at once: genuine relay-chain + * finality on the Substrate side, and a public EVM on the Base side with real + * gas estimation, real block times and real RPC behaviour. Both production + * bugs found during this work (attestor gas under-estimation, releaser + * Multicall) lived exactly there, and neither was reachable from unit tests. + * + * It is built to be re-run casually — every failure investigation should start + * by spinning this up again — so it redeploys contracts each time, clears + * daemon state, and refuses to run anywhere that could cost real money. + * + * Usage: + * node src/rehearsal.mjs full run, then tear down + * node src/rehearsal.mjs --preflight check prerequisites and funding only + * node src/rehearsal.mjs --keep leave everything running afterwards + * node src/rehearsal.mjs --attach use an already-running Zombienet + * node src/rehearsal.mjs --skip-slow skip the wall-clock cap-refill test + */ + +import { execSync } from "node:child_process"; +import { existsSync, mkdirSync, writeFileSync } from "node:fs"; +import path from "node:path"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { encodeAbiParameters, keccak256 } from "viem"; +import { Keyring } from "@polkadot/keyring"; +import { cryptoWaitReady } from "@polkadot/util-crypto"; +import { erc20Abi, vaultAbi } from "./abi.mjs"; +import { alive, clearState, logs, start, stopAll, stopAndWait, waitFor } from "./daemons.mjs"; +import { assert, assertEq, check, section, summarise } from "./harness.mjs"; +import { + assertTestnet, buildContext, CONVERSION_FACTOR, loadEnv, PEN_12, PEN_18, ROOT, send, TESTING, +} from "./rehearsal-env.mjs"; +import { deployToSepolia, rehearsalParams } from "./rehearsal-deploy.mjs"; +import { discoverCollator, killMatching, generateSpec, sleep, spawnNetwork, teardown, waitForFinality } from "./zombienet.mjs"; + +const flags = new Set(process.argv.slice(2)); +const KEEP = flags.has("--keep"); +const ATTACH = flags.has("--attach"); +const SKIP_SLOW = flags.has("--skip-slow"); +const PREFLIGHT_ONLY = flags.has("--preflight"); + +const started = new Date(); +const stamp = started.toISOString().replace(/[:.]/g, "-"); +const runDir = path.join(TESTING, ".rehearsal", stamp); +const log = (m) => console.log(` ${m}`); + +// Minimum gas each role needs to complete a run. The attestors submit several +// approvals each; the deployer pays for two contract creations. +const FUNDING_MINIMUMS = { deployer: 30_000_000_000_000_000n, attestor: 5_000_000_000_000_000n, other: 2_000_000_000_000_000n }; + +let network = null; +let api = null; + +async function preflight(ctx) { + section("Preflight"); + + await check("services are built", () => { + for (const svc of ["attestor", "monitor", "releaser"]) { + assert(existsSync(path.join(ROOT, svc, "dist/main.js")), `${svc}/dist/main.js missing — run \`npm run build\` in ${svc}/`); + } + }); + + await check("contract artifacts exist", () => { + assert(existsSync(path.join(ROOT, "contracts/out/MigrationVault.sol/MigrationVault.json")), + "contracts/out missing — run `forge build` in contracts/"); + }); + + await check("Zombienet prerequisites present", () => { + assert(existsSync(path.join(ROOT, "zombienet-macos-arm64")), "zombienet-macos-arm64 missing from the repo root"); + assert(existsSync(path.join(ROOT, "target/release/pendulum-node")), "target/release/pendulum-node missing"); + assert(existsSync(path.join(ROOT, "target/release/wbuild/pendulum-runtime/pendulum_runtime.compact.compressed.wasm")), + "runtime wasm missing — run `cargo build --release -p pendulum-runtime`"); + }); + + await check("Base Sepolia RPC reachable and is NOT mainnet", async () => { + const id = await assertTestnet(ctx, null); + return `chain ${id}`; + }); + + await check("every role is funded", async () => { + const rows = [ + ["deployer", ctx.roles.deployer, FUNDING_MINIMUMS.deployer], + ...ctx.roles.attestors.map((a, i) => [`attestor${i + 1}`, a, FUNDING_MINIMUMS.attestor]), + ["guardian", ctx.roles.guardian, FUNDING_MINIMUMS.other], + ["admin", ctx.roles.admin, FUNDING_MINIMUMS.other], + ["releaser", ctx.roles.releaser, FUNDING_MINIMUMS.other], + ]; + const underfunded = []; + for (const [name, acct, minimum] of rows) { + const balance = await ctx.pub.getBalance({ address: acct.address }); + const eth = (Number(balance) / 1e18).toFixed(5); + console.log(` ${name.padEnd(10)} ${acct.address} ${eth} ETH`); + if (balance < minimum) underfunded.push(`${name} (${acct.address}) has ${eth} ETH`); + } + assert(underfunded.length === 0, + `fund these from a Base Sepolia faucet:\n ${underfunded.join("\n ")}`); + }); +} + +/** Enable migrations through the real governance path. + * + * This chain has no sudo pallet, so unlike the Chopsticks phase there is no + * storage poke available — which is a feature: it means the rehearsal + * exercises the same technical-committee origin that will unpause mainnet. + * The generated spec seats a single member, so a threshold of 1 executes + * immediately rather than opening a vote. */ +async function unpausePallet(alice) { + const call = api.tx.tokenMigration.setPaused(false); + const collective = api.tx.technicalCommittee ?? api.tx.council; + assert(collective, "neither technicalCommittee nor council is available to drive the pause origin"); + await new Promise((resolve, reject) => { + collective.propose(1, call, call.method.encodedLength) + .signAndSend(alice, ({ status, dispatchError }) => { + if (dispatchError) return reject(new Error(dispatchError.toString())); + if (status.isInBlock) resolve(); + }) + .catch(reject); + }); + await waitFor(async () => (await api.query.tokenMigration.paused()).isFalse, + { timeoutMs: 60_000, label: "the pallet to report unpaused" }); +} + +async function main() { + console.log(`Phase 5 — full-stack rehearsal (Zombienet + Base Sepolia)\n run ${stamp}`); + const env = loadEnv(); + const ctx = buildContext(env); + + await preflight(ctx); + if (PREFLIGHT_ONLY) { + process.exit(summarise() ? 0 : 1); + } + if (!summarise()) { + console.log("\npreflight failed — not deploying anything"); + process.exit(1); + } + + // --- bring up the Substrate side --------------------------------------- + section("Local Pendulum"); + if (!ATTACH) { + teardown(log); + generateSpec(log); + network = spawnNetwork(log); + } + api = await discoverCollator(ApiPromise, WsProvider, { log }); + await waitForFinality(api, { log }); + await assertTestnet(ctx, api); + + await cryptoWaitReady(); + const keyring = new Keyring({ type: "sr25519", ss58Format: api.registry.chainSS58 ?? 42 }); + const alice = keyring.addFromUri("//Alice"); + const MIN = BigInt(api.consts.tokenMigration.minimumMigrationAmount.toString()); + log(`minimum migration ${MIN / PEN_12} PEN`); + + await check("pallet ships paused, then unpauses through the committee origin", async () => { + assert((await api.query.tokenMigration.paused()).isTrue, "pallet was not paused on a fresh chain"); + await unpausePallet(alice); + }); + + // --- bring up the Base side -------------------------------------------- + section("Base Sepolia"); + const { vault, pen, params, sweepTs } = deployToSepolia({ env, roles: ctx.roles, log }); + log(`vault ${vault}`); + log(`PEN ${pen}`); + const V = { address: vault, abi: vaultAbi }; + const P = { address: pen, abi: erc20Abi }; + const read = (c, fn, args) => ctx.pub.readContract({ ...c, functionName: fn, args }); + + await check("fresh vault has consumed no nonce the local chain will emit", async () => { + const next = BigInt((await api.query.tokenMigration.nextNonce()).toString()); + const consumed = await read(V, "nonceConsumed", [next]); + assert(!consumed, + `vault has already consumed nonce ${next}. The local chain restarts its nonce sequence at zero on ` + + "every spawn, so a reused vault silently skips every release. Deploy fresh contracts."); + }); + + await check("admin accepts the two-step handover", async () => { + await send(ctx, ctx.roles.admin, { ...V, functionName: "acceptAdmin", args: [] }); + assertEq((await read(V, "admin", [])).toLowerCase(), ctx.roles.admin.address.toLowerCase(), "admin"); + }); + + // --- start the fleet ---------------------------------------------------- + section("Attestor fleet"); + killMatching(["dist/main.js"]); + clearState(["attestor/cp1.json", "attestor/cp2.json", "attestor/cp3.json", "attestor/cp4.json", + "releaser/releaser-state.json"]); + const pendulumWs = api._options?.provider?.endpoint ?? process.env.PENDULUM_WS; + const baseEnv = { + BASE_RPC_URL: env.BASE_SEPOLIA_RPC_URL, + VAULT_ADDRESS: vault, + BASE_CHAIN_ID: "84532", + POLL_INTERVAL_MS: "5000", + }; + const pendulumHead = (await api.query.system.number()).toString(); + const baseHead = String(await ctx.pub.getBlockNumber()); + for (let i = 0; i < 4; i++) { + start(`attestor${i + 1}`, "attestor", { + ...baseEnv, PENDULUM_WS: pendulumWs, + ATTESTOR_PRIVATE_KEY: env[`ATTESTOR_${i + 1}_PRIVATE_KEY`], + CHECKPOINT_FILE: `./cp${i + 1}.json`, START_BLOCK: pendulumHead, + }); + } + start("monitor", "monitor", { ...baseEnv, PENDULUM_WS: pendulumWs, GRACE_SECONDS: "300" }); + start("releaser", "releaser", { ...baseEnv, RELEASER_PRIVATE_KEY: env.RELEASER_PRIVATE_KEY, START_BLOCK: baseHead }); + log(`fleet started (Pendulum from #${pendulumHead}, Base from #${baseHead})`); + + const recipient = "0x000000000000000000000000000000000000beef"; + const balanceOf = (who) => read(P, "balanceOf", [who]); + + // `pendingRelease` is keyed by the payload hash, not the nonce: the vault + // commits to the exact (nonce, recipient, amount) tuple so a deferred + // release cannot be completed with different arguments later. + const payloadHash = (nonce, amount) => + keccak256(encodeAbiParameters( + [{ type: "uint64" }, { type: "address" }, { type: "uint256" }], + [nonce, recipient, amount], + )); + + /** Burn on the local chain and return the emitted migration. */ + async function migrate(amountPen) { + const amount = amountPen * PEN_12; + return new Promise((resolve, reject) => { + api.tx.tokenMigration.migrate(amount, recipient) + .signAndSend(alice, ({ status, events, dispatchError }) => { + if (dispatchError) { + const decoded = dispatchError.isModule + ? api.registry.findMetaError(dispatchError.asModule).name + : dispatchError.toString(); + return reject(new Error(`migrate failed: ${decoded}`)); + } + if (!status.isInBlock) return; + const ev = events.map((r) => r.event) + .find((e) => e.section === "tokenMigration" && e.method === "MigrationInitiated"); + if (!ev) return reject(new Error("no MigrationInitiated event")); + resolve({ + nonce: BigInt(ev.data[0].toString()), + amount: BigInt(ev.data[3].toString().replaceAll(",", "")), + }); + }) + .catch(reject); + }); + } + + const diagnose = (message) => + `${message}\n--- attestor1 ---\n${logs("attestor1")}\n--- releaser ---\n${logs("releaser")}`; + + // --- scenarios ---------------------------------------------------------- + section("End to end over real infrastructure"); + + await check("a burn on Pendulum releases on Base with no manual step", async () => { + const before = await balanceOf(recipient); + const { nonce, amount } = await migrate(200n); + try { + await waitFor(async () => (await read(V, "nonceConsumed", [nonce])) === true, + { timeoutMs: 240_000, intervalMs: 5000, label: "the release to land on Base Sepolia" }); + } catch (e) { + throw new Error(diagnose(e.message)); + } + assertEq(await balanceOf(recipient), before + amount * CONVERSION_FACTOR, "released amount"); + }); + + await check("losing the approval race is benign — every attestor survives", () => { + for (let i = 1; i <= 4; i++) { + assert(alive(`attestor${i}`), `attestor${i} exited:\n${logs(`attestor${i}`)}`); + } + }); + + await check("three of four attestors still release", async () => { + await stopAndWait("attestor4"); + const { nonce } = await migrate(150n); + await waitFor(async () => (await read(V, "nonceConsumed", [nonce])) === true, + { timeoutMs: 240_000, intervalMs: 5000, label: "release with 3 attestors" }); + }); + + await check("a restarted attestor resumes from its checkpoint without duplicating", async () => { + const consumedBefore = await read(V, "totalReleased", []); + start("attestor4", "attestor", { + ...baseEnv, PENDULUM_WS: pendulumWs, + ATTESTOR_PRIVATE_KEY: env.ATTESTOR_4_PRIVATE_KEY, + CHECKPOINT_FILE: "./cp4.json", START_BLOCK: pendulumHead, + }); + await sleep(30_000); + assert(alive("attestor4"), `attestor4 died on restart:\n${logs("attestor4")}`); + assertEq(await read(V, "totalReleased", []), consumedBefore, "totalReleased after a restart"); + }); + + await check("the monitor holds its conservation invariant", async () => { + const [balance, released, swept, supply] = await Promise.all([ + read(P, "balanceOf", [vault]), read(V, "totalReleased", []), + read(V, "totalSwept", []), read(P, "totalSupply", []), + ]); + assertEq(balance + released + swept, supply, "balance + released + swept == totalSupply"); + assert(alive("monitor"), `monitor exited:\n${logs("monitor")}`); + }); + + await check("the guardian can pause but cannot unpause; the admin can", async () => { + await send(ctx, ctx.roles.guardian, { ...V, functionName: "pause", args: [] }); + assertEq(await read(V, "paused", []), true, "paused by guardian"); + let rejected = false; + try { + await send(ctx, ctx.roles.guardian, { ...V, functionName: "unpause", args: [] }); + } catch { rejected = true; } + assert(rejected, "the guardian was able to unpause — the asymmetry is broken"); + await send(ctx, ctx.roles.admin, { ...V, functionName: "unpause", args: [] }); + assertEq(await read(V, "paused", []), false, "unpaused by admin"); + }); + + if (!SKIP_SLOW) { + await check(`a cap-deferred release drains by itself (~${params.refillSecondsPer100Pen}s of refill)`, async () => { + // Exhaust the rolling bucket, then migrate again: the excess must be + // recorded pending rather than reverting, and the releaser must pick it + // up unaided as the bucket refills. This is the one behaviour that + // cannot be faked with a time warp here, which is why it is worth the + // wall-clock wait. + const available = await read(V, "availableDailyAllowance", []); + const drainPen = available / PEN_18; + assert(drainPen > 100n, `daily cap too small to exercise: ${drainPen} PEN available`); + await migrate(drainPen); + await waitFor(async () => (await read(V, "availableDailyAllowance", [])) < PEN_18 * 100n, + { timeoutMs: 300_000, intervalMs: 5000, label: "the daily bucket to be exhausted" }); + + const { nonce, amount } = await migrate(100n); + const payload = payloadHash(nonce, amount); + await waitFor( + async () => (await read(V, "pendingRelease", [payload])) === true + || (await read(V, "nonceConsumed", [nonce])) === true, + { timeoutMs: 180_000, intervalMs: 5000, label: "the release to be deferred" }); + await waitFor(async () => (await read(V, "nonceConsumed", [nonce])) === true, + { timeoutMs: (params.refillSecondsPer100Pen + 300) * 1000, intervalMs: 10_000, + label: "the releaser to drain the deferred release as the bucket refills" }); + }); + } + + // --- record ------------------------------------------------------------- + mkdirSync(runDir, { recursive: true }); + const manifest = { + startedAt: started.toISOString(), + commit: execSync("git rev-parse HEAD", { cwd: ROOT, encoding: "utf8" }).trim(), + base: { + chainId: 84532, rpc: env.BASE_SEPOLIA_RPC_URL, vault, pen, + startBlock: baseHead, sweepTimestamp: sweepTs, + caps: { dailyCap: params.dailyCap.toString(), perReleaseCap: params.perReleaseCap.toString() }, + }, + pendulum: { ws: pendulumWs, startBlock: pendulumHead, minimumMigration: MIN.toString() }, + // Addresses only — private keys stay in .env.rehearsal and out of run artifacts. + roles: { + deployer: ctx.roles.deployer.address, + attestors: ctx.roles.attestors.map((a) => a.address), + guardian: ctx.roles.guardian.address, + admin: ctx.roles.admin.address, + releaser: ctx.roles.releaser.address, + }, + }; + writeFileSync(path.join(runDir, "manifest.json"), JSON.stringify(manifest, null, 2)); + for (const name of ["attestor1", "attestor2", "attestor3", "attestor4", "monitor", "releaser"]) { + writeFileSync(path.join(runDir, `${name}.log`), logs(name)); + } + if (network) writeFileSync(path.join(runDir, "zombienet.log"), network.out.join("\n")); + console.log(`\n run artifacts: ${path.relative(ROOT, runDir)}`); + + const ok = summarise(); + if (KEEP) { + console.log("\n --keep: leaving the network and fleet running."); + console.log(` vault ${vault} on Base Sepolia; Pendulum at ${pendulumWs}`); + console.log(" tear down with: node -e \"import('./testing/src/zombienet.mjs').then(z=>z.teardown(console.log))\""); + process.exit(ok ? 0 : 1); + } + stopAll(); + await sleep(2000); + if (!ATTACH) teardown(log); + if (api) await api.disconnect().catch(() => {}); + process.exit(ok ? 0 : 1); +} + +process.on("SIGINT", () => { + console.log("\ninterrupted — tearing down"); + stopAll(); + if (!KEEP) teardown(console.log); + process.exit(130); +}); + +main().catch(async (error) => { + console.error(`\nrehearsal aborted: ${error.message}`); + stopAll(); + if (!KEEP && !ATTACH) teardown(console.log); + process.exit(1); +}); From 53acc1e8692b29774db6a423e55ffa0651a27930 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 16:15:37 +0200 Subject: [PATCH 25/61] docs: Document the full-stack rehearsal phase --- docs/pen-migration-local-test-plan.md | 69 +++++++++++++++++++++++++-- 1 file changed, 66 insertions(+), 3 deletions(-) diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 192c74fcf..4f3b24bb8 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -12,6 +12,7 @@ needs both. | **Anvil** (Foundry) | Base | Contract behaviour, deploy script, attestor/monitor wiring | | **Chopsticks** | Pendulum mainnet | Runtime upgrade + pallet against **real** balances, locks, vesting, treasury | | **Zombienet** | Relay + parachain | The **relay-chain finality** path — attestors only act on finalized blocks, which Chopsticks cannot faithfully reproduce (phase 4) | +| **Base Sepolia** | Base | Real gas estimation, block times and RPC behaviour, against a public chain (phase 5) | Chopsticks gives realistic *state*; Zombienet gives realistic *finality*. You need both, for different reasons. Neither requires Paseo or Foucoco. @@ -273,7 +274,68 @@ artifact's problem rather than the relay's. --- -## Phase 5 — Failure drills (the runbooks) +## Phase 5 — full-stack rehearsal (Zombienet + Base Sepolia) + +Goal: run the whole system against real infrastructure on both sides at once, +with no real value at stake. Phases 1–4 each hold one half still — Anvil is +instant and single-node, Chopsticks fakes finality. This is the only phase +where genuine relay finality and a public EVM meet, which is where both +production bugs found during this work actually lived. + +```bash +cp testing/.env.rehearsal.example testing/.env.rehearsal # fill in throwaway keys +node testing/src/rehearsal.mjs --preflight # lists what needs funding +node testing/src/rehearsal.mjs +``` + +| Flag | Effect | +|---|---| +| `--preflight` | Check prerequisites and role funding, deploy nothing | +| `--keep` | Leave the network and fleet running for manual poking | +| `--attach` | Use an already-running Zombienet instead of spawning one | +| `--skip-slow` | Skip the wall-clock cap-refill scenario | + +It brings up Zombienet, waits for genuine parachain finality, unpauses the +pallet **through the technical-committee origin** (this chain has no sudo, so +unlike phase 2 there is no storage poke — the rehearsal drives the same origin +that will unpause mainnet), deploys the contracts to Base Sepolia with the real +`Deploy.s.sol`, starts four attestors plus the monitor and releaser, and runs +the scenarios. Every run writes `testing/.rehearsal//` containing a +manifest (addresses, ports, block heights, commit) and each daemon's log, so a +failed run stays diagnosable after teardown. + +Four design decisions worth knowing before you change it: + +- **Contracts are redeployed every run, deliberately.** The Zombienet chain is + ephemeral and restarts its nonce sequence at zero on each spawn, while the + vault's `nonceConsumed` mapping is permanent. Reusing a vault means the second + run re-emits nonce 0, every attestor's pre-check returns "already handled", + and the pipeline logs skips while testing nothing. A guard asserts this + explicitly rather than trusting the convention. Redeploying also exercises the + deploy script on every cycle. +- **Caps are sized for wall clock, not for production.** There is no + `evm_increaseTime` on a public chain, so the rolling bucket has to refill in + real minutes: at `DAILY_CAP` = 28,800 PEN it returns 100 PEN (the on-chain + minimum migration) every ~5 minutes. The production cap values remain + validated only in phase 1, where time can be warped — the two phases are + complementary and neither is sufficient alone. +- **It refuses to run anywhere that could cost money.** Base mainnet (chain + 8453) is rejected outright, any chain other than Sepolia needs an explicit + override, and the Substrate endpoint must self-report as the local chain. + Checked before anything is deployed or signed. +- **Teardown is part of the contract.** Stray daemons from an aborted run + rewrite the checkpoint files a fresh run just cleared, so processes are killed + by reading the process table rather than `pkill -f` — a shell running + `pkill -f ` matches its own command line, which is exactly how an + earlier session produced three waiter shells that could never terminate. + +**Pass:** all scenarios green. Keys are throwaway and testnet-only; +`testing/.env.rehearsal` is gitignored and must never hold a key that will see +mainnet. + +--- + +## Phase 6 — Failure drills (the runbooks) Rehearse each runbook once against the local stack, so the first time you run them is not during an incident: @@ -290,9 +352,9 @@ them is not during an incident: --- -## Phase 6 — Exit criteria before mainnet +## Phase 7 — Exit criteria before mainnet -- [ ] Phases 1–5 pass end to end. +- [ ] Phases 1–6 pass end to end. - [ ] The upgrade ships paused, verified on a Chopsticks fork of **live** mainnet state (not a fresh chain). - [ ] A cap-deferred release recovers correctly without manual contract @@ -305,3 +367,4 @@ them is not during an incident: replace the manual estimates. - [ ] A dry run of the deploy script with the **final** production parameters, reviewed by someone other than whoever wrote the `.env`. +- [ ] The phase 5 rehearsal green on Base Sepolia against the shipped revision. From b4f13aa2b39b91f1b9a8a6d09139855c1d5276dc Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 16:19:43 +0200 Subject: [PATCH 26/61] testing: Fan rehearsal gas out from the deployer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Base Sepolia faucets are rate-limited per address, so claiming for eight addresses is slow and tedious. --fund claims-once-distribute-many: top up only the roles below their minimum, only to their target, so re-running after a few rehearsals costs nothing and does nothing. Also right-sizes the funding minimums, which were guesswork before. A whole run — two deployments plus ~20 approvals — measures at roughly 0.00005 ETH on Base Sepolia, so the previous 0.056 ETH total carried about three orders of magnitude more headroom than needed and made the faucet step far more painful than it had to be. The new figures keep ~100x margin for gas spikes and the L1 data fee while fitting inside a single claim. --- docs/pen-migration-local-test-plan.md | 9 +++ testing/src/rehearsal.mjs | 101 ++++++++++++++++++++++---- 2 files changed, 95 insertions(+), 15 deletions(-) diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 4f3b24bb8..23e3ca52b 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -285,12 +285,21 @@ production bugs found during this work actually lived. ```bash cp testing/.env.rehearsal.example testing/.env.rehearsal # fill in throwaway keys node testing/src/rehearsal.mjs --preflight # lists what needs funding +# claim Base Sepolia ETH once into the deployer address, then: +node testing/src/rehearsal.mjs --fund # fans gas out to the other seven node testing/src/rehearsal.mjs ``` +Gas is sized against measured cost: a whole run — two deployments plus ~20 +approvals — is about **0.00005 ETH** on Base Sepolia, so the ~0.014 ETH the +roles hold between them covers many runs. Faucets are rate-limited per address, +which is why `--fund` exists: claim once into the deployer rather than eight +times. + | Flag | Effect | |---|---| | `--preflight` | Check prerequisites and role funding, deploy nothing | +| `--fund` | Top up any underfunded role from the deployer; idempotent | | `--keep` | Leave the network and fleet running for manual poking | | `--attach` | Use an already-running Zombienet instead of spawning one | | `--skip-slow` | Skip the wall-clock cap-refill scenario | diff --git a/testing/src/rehearsal.mjs b/testing/src/rehearsal.mjs index a04113955..f658f3700 100644 --- a/testing/src/rehearsal.mjs +++ b/testing/src/rehearsal.mjs @@ -15,6 +15,7 @@ * Usage: * node src/rehearsal.mjs full run, then tear down * node src/rehearsal.mjs --preflight check prerequisites and funding only + * node src/rehearsal.mjs --fund fan gas out from the deployer to the other roles * node src/rehearsal.mjs --keep leave everything running afterwards * node src/rehearsal.mjs --attach use an already-running Zombienet * node src/rehearsal.mjs --skip-slow skip the wall-clock cap-refill test @@ -37,6 +38,7 @@ import { deployToSepolia, rehearsalParams } from "./rehearsal-deploy.mjs"; import { discoverCollator, killMatching, generateSpec, sleep, spawnNetwork, teardown, waitForFinality } from "./zombienet.mjs"; const flags = new Set(process.argv.slice(2)); +const FUND = flags.has("--fund"); const KEEP = flags.has("--keep"); const ATTACH = flags.has("--attach"); const SKIP_SLOW = flags.has("--skip-slow"); @@ -47,9 +49,80 @@ const stamp = started.toISOString().replace(/[:.]/g, "-"); const runDir = path.join(TESTING, ".rehearsal", stamp); const log = (m) => console.log(` ${m}`); -// Minimum gas each role needs to complete a run. The attestors submit several -// approvals each; the deployer pays for two contract creations. -const FUNDING_MINIMUMS = { deployer: 30_000_000_000_000_000n, attestor: 5_000_000_000_000_000n, other: 2_000_000_000_000_000n }; +// Gas each role needs. Sized against measured cost, not guesswork: a whole run +// — two contract deployments plus ~20 approvals — comes to roughly 0.00005 ETH +// on Base Sepolia, so these carry about two orders of magnitude of headroom for +// gas spikes and the L1 data fee. Small enough that one faucet claim into the +// deployer covers many runs. +const FUNDING = { + deployer: { min: 3_000_000_000_000_000n, target: 3_000_000_000_000_000n }, // 0.003 ETH, the source + attestor: { min: 1_000_000_000_000_000n, target: 2_000_000_000_000_000n }, // 0.001 / 0.002 + other: { min: 500_000_000_000_000n, target: 1_000_000_000_000_000n }, // 0.0005 / 0.001 +}; + +const eth = (wei) => `${(Number(wei) / 1e18).toFixed(6)} ETH`; + +/** Every funded role, with what it needs and what `--fund` tops it up to. */ +function roleTable(ctx) { + return [ + ["deployer", ctx.roles.deployer, FUNDING.deployer], + ...ctx.roles.attestors.map((a, i) => [`attestor${i + 1}`, a, FUNDING.attestor]), + ["guardian", ctx.roles.guardian, FUNDING.other], + ["admin", ctx.roles.admin, FUNDING.other], + ["releaser", ctx.roles.releaser, FUNDING.other], + ]; +} + +/** + * Distribute gas from the deployer to the other roles. + * + * Base Sepolia faucets are rate-limited per address, so claiming for eight + * addresses is tedious and slow. Claim once into the deployer and fan out from + * here. Idempotent: only roles below their minimum are topped up, and only to + * their target, so re-running after a few rehearsals costs nothing. + */ +async function fundRoles(ctx) { + section("Funding"); + await assertTestnet(ctx, null); + + const deployerBalance = await ctx.pub.getBalance({ address: ctx.roles.deployer.address }); + const needy = []; + for (const [name, acct, limits] of roleTable(ctx).slice(1)) { + const balance = await ctx.pub.getBalance({ address: acct.address }); + if (balance < limits.min) needy.push({ name, acct, top: limits.target - balance }); + } + + if (needy.length === 0) { + log(`every role is already funded; deployer holds ${eth(deployerBalance)}`); + return true; + } + + const total = needy.reduce((sum, n) => sum + n.top, 0n); + // Leave the deployer enough to actually deploy after funding everyone else. + const reserve = FUNDING.deployer.min; + log(`deployer holds ${eth(deployerBalance)}; distributing ${eth(total)} to ${needy.length} role(s)`); + if (deployerBalance < total + reserve) { + console.log( + ` + deployer is short. It needs ${eth(total + reserve)} ` + + `(${eth(total)} to distribute + ${eth(reserve)} to deploy with) but holds ${eth(deployerBalance)}. +` + + ` + Claim Base Sepolia ETH into ${ctx.roles.deployer.address} from a faucet, then re-run --fund.`, + ); + return false; + } + + const wallet = ctx.wallet(ctx.roles.deployer); + for (const { name, acct, top } of needy) { + const hash = await wallet.sendTransaction({ to: acct.address, value: top }); + const receipt = await ctx.pub.waitForTransactionReceipt({ hash }); + if (receipt.status !== "success") throw new Error(`funding ${name} reverted: ${hash}`); + log(` ${name.padEnd(10)} +${eth(top)} ${hash}`); + } + log("done; re-run --preflight to confirm"); + return true; +} let network = null; let api = null; @@ -81,22 +154,15 @@ async function preflight(ctx) { }); await check("every role is funded", async () => { - const rows = [ - ["deployer", ctx.roles.deployer, FUNDING_MINIMUMS.deployer], - ...ctx.roles.attestors.map((a, i) => [`attestor${i + 1}`, a, FUNDING_MINIMUMS.attestor]), - ["guardian", ctx.roles.guardian, FUNDING_MINIMUMS.other], - ["admin", ctx.roles.admin, FUNDING_MINIMUMS.other], - ["releaser", ctx.roles.releaser, FUNDING_MINIMUMS.other], - ]; const underfunded = []; - for (const [name, acct, minimum] of rows) { + for (const [name, acct, limits] of roleTable(ctx)) { const balance = await ctx.pub.getBalance({ address: acct.address }); - const eth = (Number(balance) / 1e18).toFixed(5); - console.log(` ${name.padEnd(10)} ${acct.address} ${eth} ETH`); - if (balance < minimum) underfunded.push(`${name} (${acct.address}) has ${eth} ETH`); + console.log(` ${name.padEnd(10)} ${acct.address} ${eth(balance)}`); + if (balance < limits.min) underfunded.push(`${name} (${acct.address}) has ${eth(balance)}`); } assert(underfunded.length === 0, - `fund these from a Base Sepolia faucet:\n ${underfunded.join("\n ")}`); + `underfunded:\n ${underfunded.join("\n ")}\n\n ` + + "Claim once into the deployer from a Base Sepolia faucet, then run --fund to fan out."); }); } @@ -128,6 +194,11 @@ async function main() { const env = loadEnv(); const ctx = buildContext(env); + if (FUND) { + const ok = await fundRoles(ctx); + process.exit(ok ? 0 : 1); + } + await preflight(ctx); if (PREFLIGHT_ONLY) { process.exit(summarise() ? 0 : 1); From 0a0562651ab1c916c6babd4ac3f69392cf173070 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 17:40:02 +0200 Subject: [PATCH 27/61] testing: Share the collator discovery helper with phase 4 Phase 4 grew its own copy of the RPC probe before the rehearsal existed; the shared module now owns it, so the port-probing and provider-cleanup logic has one home rather than two that can drift. --- testing/src/phase4-zombienet.mjs | 46 ++------------------------------ 1 file changed, 2 insertions(+), 44 deletions(-) diff --git a/testing/src/phase4-zombienet.mjs b/testing/src/phase4-zombienet.mjs index 2d842f517..1e1fb1c69 100644 --- a/testing/src/phase4-zombienet.mjs +++ b/testing/src/phase4-zombienet.mjs @@ -10,58 +10,16 @@ * Assumes a network spawned from testing/zombienet.toml with the collator RPC * on 9944. Run: node testing/src/phase4-zombienet.mjs */ -import { execSync } from "node:child_process"; import { ApiPromise, WsProvider } from "@polkadot/api"; import { check, summarise } from "./harness.mjs"; +import { discoverCollator } from "./zombienet.mjs"; const SAMPLE_SECONDS = Number(process.env.SAMPLE_SECONDS ?? 90); const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); -/** - * Find the collator's RPC endpoint. Zombienet reassigns ports on every spawn, - * and the collator also runs an embedded relay client, so a fixed port is as - * likely to land on Rococo as on Pendulum. Probe the listening ports and keep - * the one that reports the Pendulum runtime. - */ -async function connectToCollator() { - if (process.env.PENDULUM_WS) { - return ApiPromise.create({ provider: new WsProvider(process.env.PENDULUM_WS), noInitWarn: true }); - } - const listening = execSync( - "lsof -nP -iTCP -sTCP:LISTEN 2>/dev/null | grep pendulum | awk '{print $9}' | sed 's/.*://' | sort -un", - ) - .toString() - .trim() - .split("\n") - .filter(Boolean); - for (const port of listening) { - const url = `ws://127.0.0.1:${port}`; - // Hold the provider separately: several of these ports are not RPC at - // all (prometheus, p2p), and a provider left behind by a failed probe - // keeps retrying for the rest of the run. - const provider = new WsProvider(url, 1000); - let api; - try { - api = await Promise.race([ - ApiPromise.create({ provider, noInitWarn: true, throwOnConnect: true }), - new Promise((_, reject) => setTimeout(() => reject(new Error("timeout")), 8000)), - ]); - const chain = (await api.rpc.system.chain()).toString(); - if (/pendulum/i.test(chain)) { - console.log(`collator RPC discovered on ${url} (${chain})`); - return api; - } - await api.disconnect(); - } catch { - if (api) await api.disconnect().catch(() => {}); - else await provider.disconnect().catch(() => {}); - } - } - throw new Error(`no Pendulum collator RPC among listening ports: ${listening.join(", ")}`); -} +const api = await discoverCollator(ApiPromise, WsProvider, { log: (m) => console.log(` ${m}`) }); -const api = await connectToCollator(); await check("collator is a parachain running the Pendulum runtime", async () => { const chain = (await api.rpc.system.chain()).toString(); From 844f73185716d97e4c6123a1b41a7c8af35d7c16 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 17:58:24 +0200 Subject: [PATCH 28/61] attestor: Survive a lost race when the RPC has not caught up MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by the first full rehearsal against Base Sepolia. attestor2 exited fatally on a benign lost race: replaying its reverted approve one block earlier succeeds, and it burned 26k of 500k gas, so it was an early custom-error revert rather than a genuine failure. The existing race tolerance re-reads the vault to confirm the revert was benign, but a public endpoint is load-balanced across nodes and offers no read-after-write consistency. A single read that lands on a lagging node reports 'not handled', which turns the most ordinary event in this system — losing the k-of-n race — into an unexplained failure and takes the daemon down. Re-check with backoff before concluding anything is wrong. This is the same class as the earlier crash-loop and OutOfGas fixes, reached by a third route, and it was unreachable from Anvil: a single node with instant inclusion always reads its own writes. The rehearsal hit the same root cause from the other side, reading pendingAdmin straight after the deploy set it, so it now waits for that state to be visible before accepting the handover. --- attestor/src/main.ts | 30 ++++++++++++++++++++++++++++++ testing/src/rehearsal.mjs | 8 ++++++++ 2 files changed, 38 insertions(+) diff --git a/attestor/src/main.ts b/attestor/src/main.ts index 844717d1e..9f14ee87f 100644 --- a/attestor/src/main.ts +++ b/attestor/src/main.ts @@ -40,6 +40,11 @@ interface Checkpoint { * threshold-crossing release path. */ const GAS_LIMIT_MULTIPLIER = 4n; +/** How persistently to confirm that a failed approval was merely a lost race + * before treating it as fatal. See `alreadyHandledSettled`. */ +const RACE_RECHECK_ATTEMPTS = 5; +const RACE_RECHECK_DELAY_MS = 3000; + interface MigrationEvent { nonce: bigint; recipient: `0x${string}`; @@ -152,6 +157,31 @@ async function alreadyHandled(event: MigrationEvent): Promise { }); } +/** + * `alreadyHandled`, but tolerant of an RPC that has not caught up yet. + * + * Public endpoints are load-balanced across nodes and give no read-after-write + * consistency guarantee, so immediately after our transaction reverts the + * state proving WHY it reverted may not be visible yet. A single read that + * happens to hit a lagging node reports "not handled", which turns the most + * ordinary event in this system — losing the k-of-n race — into an + * unexplained failure and takes the daemon down. Re-check with backoff before + * concluding anything is actually wrong. + */ +async function alreadyHandledSettled(event: MigrationEvent): Promise { + for (let attempt = 0; attempt < RACE_RECHECK_ATTEMPTS; attempt++) { + try { + if (await alreadyHandled(event)) return true; + } catch (error) { + log("recheck failed, retrying", error); + } + if (attempt < RACE_RECHECK_ATTEMPTS - 1) { + await new Promise((resolve) => setTimeout(resolve, RACE_RECHECK_DELAY_MS)); + } + } + return false; +} + /** Submit the approval for one migration event, skipping work already done. */ async function approve(event: MigrationEvent): Promise { const label = `nonce=${event.nonce} recipient=${event.recipient} amount=${event.palletAmount}`; diff --git a/testing/src/rehearsal.mjs b/testing/src/rehearsal.mjs index f658f3700..b85d08c2d 100644 --- a/testing/src/rehearsal.mjs +++ b/testing/src/rehearsal.mjs @@ -248,6 +248,14 @@ async function main() { }); await check("admin accepts the two-step handover", async () => { + // Wait for the deploy's transferAdmin to be visible before acting on it. + // Base Sepolia's public RPC is load-balanced and gives no read-after-write + // guarantee, so a read issued straight after the deploy can hit a node + // that has not caught up and report pendingAdmin as unset. + await waitFor( + async () => (await read(V, "pendingAdmin", [])).toLowerCase() === ctx.roles.admin.address.toLowerCase(), + { timeoutMs: 120_000, intervalMs: 3000, label: "pendingAdmin to be visible on the RPC" }, + ); await send(ctx, ctx.roles.admin, { ...V, functionName: "acceptAdmin", args: [] }); assertEq((await read(V, "admin", [])).toLowerCase(), ctx.roles.admin.address.toLowerCase(), "admin"); }); From c7f4f0c7aa10ee4e33d04f6d5d12e6084240a757 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 18:15:07 +0200 Subject: [PATCH 29/61] attestor: Actually route the race recheck through the backoff MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit defined alreadyHandledSettled but left the catch calling alreadyHandled, so the backoff never ran and the second rehearsal reproduced the fatal exit unchanged. Also stops the rehearsal asserting once against an eventually-consistent RPC. Base Sepolia's public endpoint is load-balanced, so a read issued right after a confirmed write can still land on a node that has not imported that block — which is what failed the admin handover and the guardian pause, both of which were correct on-chain. Assertions that follow a write now retry. The restart check asserted totalReleased was unchanged, which was simply wrong: an earlier migration can settle during the restart window, and that is what the 200 -> 350 PEN move was. Double releases are impossible regardless, since nonceConsumed is permanent, so it now asserts the daemon rejoins and the total never goes backwards. --- attestor/src/main.ts | 2 +- testing/src/rehearsal.mjs | 29 +++++++++++++++++++++++------ 2 files changed, 24 insertions(+), 7 deletions(-) diff --git a/attestor/src/main.ts b/attestor/src/main.ts index 9f14ee87f..f8e9d185f 100644 --- a/attestor/src/main.ts +++ b/attestor/src/main.ts @@ -230,7 +230,7 @@ async function approve(event: MigrationEvent): Promise { // Expected race: the release landed (or our own retried tx landed) // between our pre-check and the transaction. Benign — anything else // is a genuine failure and propagates to the fatal handler. - if (await alreadyHandled(event)) { + if (await alreadyHandledSettled(event)) { log(`skip (raced, resolved on-chain): ${label}`); return; } diff --git a/testing/src/rehearsal.mjs b/testing/src/rehearsal.mjs index b85d08c2d..fa9035923 100644 --- a/testing/src/rehearsal.mjs +++ b/testing/src/rehearsal.mjs @@ -239,6 +239,15 @@ async function main() { const P = { address: pen, abi: erc20Abi }; const read = (c, fn, args) => ctx.pub.readContract({ ...c, functionName: fn, args }); + /** Retry an assertion until it holds. Base Sepolia's public RPC is + * load-balanced and eventually consistent, so a read issued right after a + * confirmed write can still land on a node that has not imported that + * block. Asserting once turns ordinary RPC lag into a spurious failure. */ + const eventually = (fn, label, timeoutMs = 120_000) => + waitFor(async () => { try { await fn(); return true; } catch { return false; } }, + { timeoutMs, intervalMs: 3000, label }); + + await check("fresh vault has consumed no nonce the local chain will emit", async () => { const next = BigInt((await api.query.tokenMigration.nextNonce()).toString()); const consumed = await read(V, "nonceConsumed", [next]); @@ -257,7 +266,9 @@ async function main() { { timeoutMs: 120_000, intervalMs: 3000, label: "pendingAdmin to be visible on the RPC" }, ); await send(ctx, ctx.roles.admin, { ...V, functionName: "acceptAdmin", args: [] }); - assertEq((await read(V, "admin", [])).toLowerCase(), ctx.roles.admin.address.toLowerCase(), "admin"); + await eventually( + async () => assertEq((await read(V, "admin", [])).toLowerCase(), ctx.roles.admin.address.toLowerCase(), "admin"), + "admin to read as the handed-over address"); }); // --- start the fleet ---------------------------------------------------- @@ -353,8 +364,8 @@ async function main() { { timeoutMs: 240_000, intervalMs: 5000, label: "release with 3 attestors" }); }); - await check("a restarted attestor resumes from its checkpoint without duplicating", async () => { - const consumedBefore = await read(V, "totalReleased", []); + await check("a restarted attestor resumes from its checkpoint and stays up", async () => { + const releasedBefore = await read(V, "totalReleased", []); start("attestor4", "attestor", { ...baseEnv, PENDULUM_WS: pendulumWs, ATTESTOR_PRIVATE_KEY: env.ATTESTOR_4_PRIVATE_KEY, @@ -362,7 +373,11 @@ async function main() { }); await sleep(30_000); assert(alive("attestor4"), `attestor4 died on restart:\n${logs("attestor4")}`); - assertEq(await read(V, "totalReleased", []), consumedBefore, "totalReleased after a restart"); + // Not "unchanged": an earlier migration may legitimately settle during the + // restart window. Double-releases are impossible on-chain regardless — + // nonceConsumed is permanent — so the property worth asserting is that the + // daemon rejoins without dying and the total never goes backwards. + assert(await read(V, "totalReleased", []) >= releasedBefore, "totalReleased went backwards"); }); await check("the monitor holds its conservation invariant", async () => { @@ -376,14 +391,16 @@ async function main() { await check("the guardian can pause but cannot unpause; the admin can", async () => { await send(ctx, ctx.roles.guardian, { ...V, functionName: "pause", args: [] }); - assertEq(await read(V, "paused", []), true, "paused by guardian"); + await eventually(async () => assertEq(await read(V, "paused", []), true, "paused by guardian"), + "the pause to be visible"); let rejected = false; try { await send(ctx, ctx.roles.guardian, { ...V, functionName: "unpause", args: [] }); } catch { rejected = true; } assert(rejected, "the guardian was able to unpause — the asymmetry is broken"); await send(ctx, ctx.roles.admin, { ...V, functionName: "unpause", args: [] }); - assertEq(await read(V, "paused", []), false, "unpaused by admin"); + await eventually(async () => assertEq(await read(V, "paused", []), false, "unpaused by admin"), + "the unpause to be visible"); }); if (!SKIP_SLOW) { From 7037f6f076ab78c87c265920fae83a92eb79eaa7 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 18:34:24 +0200 Subject: [PATCH 30/61] attestor: Survive transient RPC failures instead of exiting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The third rehearsal killed an attestor with 'over rate limit' from the public Base Sepolia endpoint. Any transport-level failure was being treated the same as a decode failure, so a momentary RPC hiccup took the daemon down. PRD A5 requires dying rather than silently skipping an event, and a decode failure still does exactly that. But a rate limit or a dropped socket carries no information about the event, and the checkpoint is only advanced once a block is fully handled — so leaving the block unprocessed is safe and the next finalized head simply re-processes it. This matters well beyond the rehearsal: in production any transient endpoint problem would otherwise mean an attestor outage and a step closer to losing quorum. The rehearsal also stops generating the pressure in the first place: six daemons sharing one public endpoint is not representative of production, where each attestor has its own node. Poll no faster than the parachain produces blocks and stagger the starts. --- attestor/src/main.ts | 28 ++++++++++++++++++++++++++++ testing/src/rehearsal.mjs | 8 +++++++- 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/attestor/src/main.ts b/attestor/src/main.ts index f8e9d185f..5c9e0fc73 100644 --- a/attestor/src/main.ts +++ b/attestor/src/main.ts @@ -40,6 +40,28 @@ interface Checkpoint { * threshold-crossing release path. */ const GAS_LIMIT_MULTIPLIER = 4n; +/** + * Transport-level failures that say nothing about the migration itself. + * + * PRD A5 requires this daemon to die rather than silently skip an event, and a + * decode failure still does exactly that. But a rate limit or a dropped socket + * is not a decode failure: it carries no information about the event, and + * exiting on one turns every transient RPC hiccup into an attestor outage. The + * checkpoint is only advanced once a block is fully handled, so leaving the + * block unprocessed is safe — the next finalized head simply re-processes it. + */ +function isTransientRpcError(error: unknown): boolean { + const parts = [ + (error as { message?: string })?.message, + (error as { details?: string })?.details, + (error as { shortMessage?: string })?.shortMessage, + (error as { cause?: { details?: string } })?.cause?.details, + ]; + const text = parts.filter(Boolean).join(" "); + return /rate limit|too many requests|timeout|timed out|ETIMEDOUT|ECONNRESET|ECONNREFUSED|EAI_AGAIN|socket hang up|fetch failed|502|503|504|service unavailable|internal error/i + .test(text); +} + /** How persistently to confirm that a failed approval was merely a lost race * before treating it as fatal. See `alreadyHandledSettled`. */ const RACE_RECHECK_ATTEMPTS = 5; @@ -275,6 +297,12 @@ async function main(): Promise { saveCheckpoint(checkpoint); } }).catch(async (error) => { + if (isTransientRpcError(error)) { + // Not a statement about the event — leave the checkpoint where it is + // and let the next finalized head re-process this block. + await alert("transient RPC failure, retrying on the next finalized head", error); + return; + } // PRD A5: never skip an event silently. Alert and exit; the process // manager restarts us and the checkpoint retries the failing block. await alert("fatal error, exiting", error); diff --git a/testing/src/rehearsal.mjs b/testing/src/rehearsal.mjs index fa9035923..d8341e601 100644 --- a/testing/src/rehearsal.mjs +++ b/testing/src/rehearsal.mjs @@ -281,7 +281,12 @@ async function main() { BASE_RPC_URL: env.BASE_SEPOLIA_RPC_URL, VAULT_ADDRESS: vault, BASE_CHAIN_ID: "84532", - POLL_INTERVAL_MS: "5000", + // Six daemons sharing one public endpoint is what tripped Base Sepolia's + // rate limiter on an earlier run. Poll no faster than the parachain + // produces blocks — there is nothing new to see in between — and stagger + // the starts so they do not align into bursts. In production each + // attestor has its own node and this pressure does not arise. + POLL_INTERVAL_MS: "12000", }; const pendulumHead = (await api.query.system.number()).toString(); const baseHead = String(await ctx.pub.getBlockNumber()); @@ -291,6 +296,7 @@ async function main() { ATTESTOR_PRIVATE_KEY: env[`ATTESTOR_${i + 1}_PRIVATE_KEY`], CHECKPOINT_FILE: `./cp${i + 1}.json`, START_BLOCK: pendulumHead, }); + await sleep(3000); } start("monitor", "monitor", { ...baseEnv, PENDULUM_WS: pendulumWs, GRACE_SECONDS: "300" }); start("releaser", "releaser", { ...baseEnv, RELEASER_PRIVATE_KEY: env.RELEASER_PRIVATE_KEY, START_BLOCK: baseHead }); From faf793983f771dad7ae8701096a5b0fd5e4f04f9 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 18:41:34 +0200 Subject: [PATCH 31/61] testing: Ride out endpoint throttling in the rehearsal's own reads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The fourth run reached 14/15, failing only because a polling read hit Base Sepolia's rate limit mid-wait. The reads run inside polling loops, so one transient failure aborted a wait that was otherwise progressing — the same mistake the attestor made by treating a rate limit as fatal, on the harness side this time. --- testing/src/rehearsal.mjs | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/testing/src/rehearsal.mjs b/testing/src/rehearsal.mjs index d8341e601..0fa47548c 100644 --- a/testing/src/rehearsal.mjs +++ b/testing/src/rehearsal.mjs @@ -237,7 +237,24 @@ async function main() { log(`PEN ${pen}`); const V = { address: vault, abi: vaultAbi }; const P = { address: pen, abi: erc20Abi }; - const read = (c, fn, args) => ctx.pub.readContract({ ...c, functionName: fn, args }); + /** Contract read that rides out a throttled or flaky endpoint. + * These reads run inside polling loops, so a single transient failure would + * otherwise abort a wait that was progressing perfectly well — the same + * mistake the attestor made by treating a rate limit as fatal. */ + async function read(c, fn, args) { + let last; + for (let attempt = 0; attempt < 5; attempt++) { + try { + return await ctx.pub.readContract({ ...c, functionName: fn, args }); + } catch (error) { + last = error; + const text = `${error?.details ?? ""} ${error?.shortMessage ?? ""} ${error?.message ?? ""}`; + if (!/rate limit|too many requests|timeout|fetch failed|50[234]/i.test(text)) throw error; + await sleep(3000 * (attempt + 1)); + } + } + throw last; + } /** Retry an assertion until it holds. Base Sepolia's public RPC is * load-balanced and eventually consistent, so a read issued right after a From 299601fdcd59d794100b3c1d097b8f42844bb12f Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Fri, 28 Aug 2026 18:54:47 +0200 Subject: [PATCH 32/61] docs: Record the two defects the Base Sepolia rehearsal found Both are failures of an assumption the local harness cannot violate: Anvil is a single node with instant inclusion, so it always reads its own writes and never throttles, while a public load-balanced endpoint does neither. An attestor exited on the ordinary k-of-n race because its confirming read hit a lagging node, and any transient RPC failure killed an attestor outright because transport errors were handled identically to decode failures. The second is the more serious in production: two momentary endpoint problems would put the fleet below quorum with releases stalling silently. Also adds the operational consequence to the runbooks, since RB-4 and RB-6 both write and then immediately read. --- docs/pen-migration-implementation-overview.md | 1 + docs/pen-migration-internal-review.md | 47 +++++++++++++++++++ docs/pen-migration-runbooks.md | 8 ++++ 3 files changed, 56 insertions(+) diff --git a/docs/pen-migration-implementation-overview.md b/docs/pen-migration-implementation-overview.md index 91c89a6cd..a8c22c95b 100644 --- a/docs/pen-migration-implementation-overview.md +++ b/docs/pen-migration-implementation-overview.md @@ -75,6 +75,7 @@ restricted to consumed nonces. Details and verified-not-vulnerable list in | `testing/src/phase2-pendulum.mjs` | 14/14 against a Chopsticks fork of live mainnet state | | `testing/src/phase3-e2e.mjs` | 7/7 end to end, four attestors + monitor + releaser | | `testing/src/phase4-zombienet.mjs` | 7/7 against a real relay (~2-block parachain finality lag) | +| `testing/src/rehearsal.mjs` | 15/15 full stack: local Zombienet Pendulum + Base Sepolia | Seven internal adversarial review rounds have run; each found real issues (sometimes in a prior round's own fix), all fixed with regression tests and diff --git a/docs/pen-migration-internal-review.md b/docs/pen-migration-internal-review.md index 08f61eee5..c34df20a2 100644 --- a/docs/pen-migration-internal-review.md +++ b/docs/pen-migration-internal-review.md @@ -382,6 +382,53 @@ deploy-script role wiring (deployer admin renounced, executor = anyone, self-administered timelock) and PEN↔Governor clock-mode consistency; monitor auto-pause not weaponisable by an outsider (unchanged from round 6). +## Full-stack rehearsal on Base Sepolia (2026-08-28) + +Two defects that no prior round could reach. Both are failures of an assumption +the local harness cannot violate: Anvil is a single node with instant inclusion, +so it always reads its own writes and never throttles. A public, load-balanced +endpoint does neither. + +### C1(rehearsal). CRITICAL — An attestor exited on the ordinary k-of-n race + +Losing the approval race is the most common event in this system: with four +attestors watching the same migration, three win and one loses every time. The +loser's transaction reverts, and the daemon re-reads the vault to confirm the +revert was benign before deciding what to do. + +That confirming read hit a node that had not yet imported the block, reported +"not handled", and the benign race was escalated to a fatal exit. Verified +rather than inferred: replaying the reverted transaction one block earlier +succeeds, and it consumed 26k of 500k gas — an early custom-error revert, not +OutOfGas. + +This is the same failure class as C1(r6) and the gas under-estimation before it, +reached by a third route. **Fixed:** the confirmation now re-checks with backoff +before concluding anything is wrong. + +### C2(rehearsal). CRITICAL — Any transient RPC failure killed an attestor + +The endpoint rate-limited the fleet and the attestor exited, because every +transport-level failure was handled identically to a decode failure. + +The distinction matters. PRD A5 requires dying rather than silently skipping an +event, and a decode failure still does exactly that. But a rate limit or a +dropped socket carries no information about the event, and the checkpoint is +only advanced once a block is fully handled — so leaving the block unprocessed +is safe, and the next finalized head re-processes it. In production the previous +behaviour meant any momentary endpoint problem cost an attestor, and two such +blips put the fleet below quorum with releases stalling silently. + +**Fixed:** transport failures are retried; decode failures remain fatal. + +### Operational consequence (no code change) + +Any procedure that writes and then immediately reads carries the same hazard +against a public RPC — RB-4 and RB-6 both do. Confirm state by re-reading until +it settles, not once. Each attestor should also run against its own node rather +than a shared public endpoint; the rehearsal reproduced the rate limit precisely +because six daemons shared one. + ## Residual risks and standing practices (no external audit — risk accepted) - Every change to the fund-release path (vault release/approve/sweep logic, pallet burn path) gets a fresh independent adversarial review round before diff --git a/docs/pen-migration-runbooks.md b/docs/pen-migration-runbooks.md index a8274fa1d..d87187054 100644 --- a/docs/pen-migration-runbooks.md +++ b/docs/pen-migration-runbooks.md @@ -11,6 +11,14 @@ tech lead → guardian Safe signers → admin Safe signers. --- +> **Confirming state against a public RPC.** Public endpoints are load-balanced +> and give no read-after-write guarantee: a read issued straight after a +> confirmed transaction can land on a node that has not imported that block yet, +> and report the old value. Every verification step below means "re-read until +> it settles", not "read once" — a single read showing the old value is not +> evidence the transaction failed. This bit the rehearsal twice, on a correctly +> executed admin handover and a correctly executed pause. + ## RB-1: Suspected attestor key compromise **Trigger:** an `Approved` event from an attestor for a tuple that does not From 8b860948c95e5948757396e4534520ab626b9b1d Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 31 Aug 2026 10:22:41 +0200 Subject: [PATCH 33/61] testing: Automate the phase 6 failure drills MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rehearses the runbooks against the full Sepolia stack, so the first time anyone runs them is not during an incident. 13 checks covering RB-1 (a removed attestor's recorded approval stops counting, and below quorum nobody can complete the release), RB-6 (re-adding bumps the generation so the old vote stays dead, then the documented recovery — rewind the checkpoint, restart, re-approve — completes the quorum), RB-3's surplus half, RB-4's ordered stop-and-reverse-resume, and RB-7's close-reconcile-sweep. Two Sepolia constraints shape the phase: a conservation deficit cannot be created there because nothing but the vault can move its tokens — that is the security property, and the deficit alarm stays covered by phase 3 on Anvil — and time cannot be warped, so the drills deploy with a ~90s sweep floor and never reduce the threshold, keeping the 7-day settling gate unarmed. RB-5 (a runtime upgrade under a live fleet) is the one drill that cannot run on Zombienet or Sepolia: swapping a runtime needs root or storage access, and Pendulum has no sudo. It runs on Chopsticks, where writing :code is exactly what an enacted upgrade does; the drill releases before the upgrade, applies a spec-bumped wasm, and confirms the fleet keeps decoding and releasing after. --- testing/src/drill-rb5-upgrade.mjs | 119 ++++++++++ testing/src/drills.mjs | 361 ++++++++++++++++++++++++++++++ 2 files changed, 480 insertions(+) create mode 100644 testing/src/drill-rb5-upgrade.mjs create mode 100644 testing/src/drills.mjs diff --git a/testing/src/drill-rb5-upgrade.mjs b/testing/src/drill-rb5-upgrade.mjs new file mode 100644 index 000000000..5d5b2b008 --- /dev/null +++ b/testing/src/drill-rb5-upgrade.mjs @@ -0,0 +1,119 @@ +/** + * RB-5 drill — a runtime upgrade lands while the attestor fleet is running. + * + * The property under test: attestors keep decoding `MigrationInitiated` across + * an upgrade that does not change the event (the shape-change half of RB-5 — + * fail loudly rather than skip — is unit-tested in the attestor via its + * 4-field assertion). This is the one drill that cannot run on Zombienet or + * Sepolia: swapping a runtime needs either root (no sudo on Pendulum) or + * storage access, so it runs on a Chopsticks fork, where `:code` can be + * written directly — which is also exactly what a real enacted upgrade does. + * + * Prerequisites: + * - Chopsticks on :8000 (testing/chopsticks-e2e.yml, wasm-override with the + * CURRENT runtime), Anvil on :8545, services built. + * - UPGRADE_WASM: path to a runtime wasm with a HIGHER spec_version and an + * unchanged event shape (build one by bumping spec_version and rebuilding). + * + * Usage: UPGRADE_WASM=/path/to/spec26.wasm node src/drill-rb5-upgrade.mjs + */ + +import { readFileSync } from "node:fs"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { Keyring } from "@polkadot/keyring"; +import { cryptoWaitReady } from "@polkadot/util-crypto"; +import { erc20Abi, vaultAbi } from "./abi.mjs"; +import { alive, clearState, killMatching, logs, start, stopAll, waitFor } from "./daemons.mjs"; +import { assert, assertEq, check, section, summarise } from "./harness.mjs"; +import { keys, pub } from "./anvil.mjs"; +import { deployStack } from "./deploy.mjs"; + +const CHOPSTICKS = process.env.CHOPSTICKS_WS ?? "ws://127.0.0.1:8000"; +const upgradeWasmPath = process.env.UPGRADE_WASM; +if (!upgradeWasmPath) { + console.error("UPGRADE_WASM is required — a runtime wasm with a higher spec_version"); + process.exit(2); +} + +console.log("RB-5 drill — runtime upgrade under a live fleet"); +const { vault, pen } = deployStack(); +console.log(` vault ${vault}`); +const read = (fn, args) => pub.readContract({ address: vault, abi: vaultAbi, functionName: fn, args }); + +const api = await ApiPromise.create({ provider: new WsProvider(CHOPSTICKS), noInitWarn: true }); +await cryptoWaitReady(); +const keyring = new Keyring({ type: "sr25519", ss58Format: api.registry.chainSS58 ?? 56 }); +const alice = keyring.addFromUri("//Alice"); +const MIN = BigInt(api.consts.tokenMigration.minimumMigrationAmount.toString()); + +const newBlock = () => api.rpc("dev_newBlock"); +async function fund(who, amount) { + await api.rpc("dev_setStorage", { System: { Account: [[[who], { providers: 1, data: { free: amount.toString() } }]] } }); + await newBlock(); +} +await api.rpc("dev_setStorage", [[api.query.tokenMigration.paused.key(), "0x00"]]); +await newBlock(); + +async function migrate(amount, baseAddress) { + await fund(alice.address, MIN * 200n); + await api.tx.tokenMigration.migrate(amount, baseAddress).signAndSend(alice); + await newBlock(); + const events = await api.query.system.events(); + const ev = events.map((r) => r.event).find((e) => e.section === "tokenMigration" && e.method === "MigrationInitiated"); + assert(ev, "no MigrationInitiated event"); + return { nonce: BigInt(ev.data[0].toString()) }; +} + +killMatching(["dist/main.js"]); +clearState(["attestor/cp1.json", "attestor/cp2.json", "attestor/cp3.json", "attestor/cp4.json"]); +const baseEnv = { BASE_RPC_URL: "http://127.0.0.1:8545", VAULT_ADDRESS: vault, BASE_CHAIN_ID: "31337", POLL_INTERVAL_MS: "2000" }; +const head = (await api.query.system.number()).toString(); +for (let i = 0; i < 4; i++) { + start(`attestor${i + 1}`, "attestor", { + ...baseEnv, PENDULUM_WS: CHOPSTICKS, + ATTESTOR_PRIVATE_KEY: keys[i + 1], CHECKPOINT_FILE: `./cp${i + 1}.json`, START_BLOCK: head, + }); +} + +const recipient = "0x000000000000000000000000000000000000beef"; +const versionBefore = api.runtimeVersion.specVersion.toNumber(); + +section(`Upgrade under fire (starting at spec ${versionBefore})`); + +await check("the fleet releases normally before the upgrade", async () => { + const { nonce } = await migrate(MIN * 2n, recipient); + await waitFor(async () => (await read("nonceConsumed", [nonce])) === true, + { timeoutMs: 90_000, label: "pre-upgrade release" }); +}); + +await check("the runtime upgrade applies (spec_version increases)", async () => { + const code = `0x${readFileSync(upgradeWasmPath).toString("hex")}`; + // Writing :code is what an enacted upgrade does; the next block runs it. + await api.rpc("dev_setStorage", [["0x3a636f6465", code]]); + await newBlock(); + await newBlock(); + const version = await api.rpc.state.getRuntimeVersion(); + const after = version.specVersion.toNumber(); + assert(after > versionBefore, `spec_version did not increase: ${versionBefore} -> ${after}`); + return; +}); + +await check("attestors keep decoding and releasing across the upgrade", async () => { + const { nonce } = await migrate(MIN * 3n, recipient); + try { + await waitFor(async () => (await read("nonceConsumed", [nonce])) === true, + { timeoutMs: 90_000, label: "post-upgrade release" }); + } catch (e) { + throw new Error(`${e.message}\n--- attestor1 ---\n${logs("attestor1")}`); + } +}); + +await check("no attestor died across the upgrade", () => { + for (let i = 1; i <= 4; i++) { + assert(alive(`attestor${i}`), `attestor${i} exited:\n${logs(`attestor${i}`)}`); + } +}); + +stopAll(); +await api.disconnect(); +process.exit(summarise() ? 0 : 1); diff --git a/testing/src/drills.mjs b/testing/src/drills.mjs new file mode 100644 index 000000000..5d241af19 --- /dev/null +++ b/testing/src/drills.mjs @@ -0,0 +1,361 @@ +/** + * Phase 6 — failure drills (runbooks RB-1, RB-3, RB-4, RB-6, RB-7). + * + * Rehearses each runbook once against the full Sepolia stack, so the first + * time anyone runs them is not during an incident. RB-2 (attestor outage) is + * already exercised by the phase 5 rehearsal; RB-5 (runtime upgrade under a + * live fleet) runs separately on Chopsticks, where a runtime can actually be + * swapped (see drill-rb5-upgrade.mjs). + * + * Two Sepolia constraints shape the drills: + * - A conservation DEFICIT cannot be created here: nothing can move tokens + * out of the vault except the vault itself, which is the security property. + * The deficit alarm + auto-pause is covered by phase 3 on Anvil, where the + * vault can be impersonated. This phase covers the SURPLUS side, which + * needs no impersonation: migrate to an address we control and send PEN in. + * - Time cannot be warped, so the contracts are deployed with a sweep floor + * only ~90s out; by the time the RB-7 drill runs at the end it has passed. + * The drills never REDUCE the threshold, so `thresholdReducedAt` stays 0 + * and the 7-day settling gate does not bite. + * + * Usage: node src/drills.mjs [--attach] [--keep] + */ + +import { execSync } from "node:child_process"; +import { mkdirSync, writeFileSync } from "node:fs"; +import path from "node:path"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { Keyring } from "@polkadot/keyring"; +import { cryptoWaitReady } from "@polkadot/util-crypto"; +import { generatePrivateKey, privateKeyToAccount } from "viem/accounts"; +import { erc20Abi, vaultAbi } from "./abi.mjs"; +import { alive, clearState, logs, start, stopAll, stopAndWait, waitFor } from "./daemons.mjs"; +import { assert, assertEq, check, section, summarise } from "./harness.mjs"; +import { + assertTestnet, buildContext, CONVERSION_FACTOR, loadEnv, PEN_12, ROOT, send, TESTING, +} from "./rehearsal-env.mjs"; +import { deployToSepolia } from "./rehearsal-deploy.mjs"; +import { discoverCollator, generateSpec, killMatching, sleep, spawnNetwork, teardown, waitForFinality } from "./zombienet.mjs"; + +const flags = new Set(process.argv.slice(2)); +const ATTACH = flags.has("--attach"); +const KEEP = flags.has("--keep"); + +const started = new Date(); +const stamp = `${started.toISOString().replace(/[:.]/g, "-")}-drills`; +const runDir = path.join(TESTING, ".rehearsal", stamp); +const log = (m) => console.log(` ${m}`); + +let network = null; +let api = null; + +async function main() { + console.log(`Phase 6 — failure drills (Zombienet + Base Sepolia)\n run ${stamp}`); + const env = loadEnv(); + // Short sweep floor so RB-7 is reachable in wall clock; see header. + env.REHEARSAL_SWEEP_OFFSET_SECONDS = process.env.DRILL_SWEEP_OFFSET ?? "90"; + const ctx = buildContext(env); + await assertTestnet(ctx, null); + + // --- Substrate side ----------------------------------------------------- + section("Local Pendulum"); + if (!ATTACH) { + teardown(log); + generateSpec(log); + network = spawnNetwork(log); + } + api = await discoverCollator(ApiPromise, WsProvider, { log }); + await waitForFinality(api, { log }); + await assertTestnet(ctx, api); + await cryptoWaitReady(); + const keyring = new Keyring({ type: "sr25519", ss58Format: api.registry.chainSS58 ?? 42 }); + const alice = keyring.addFromUri("//Alice"); + const MIN = BigInt(api.consts.tokenMigration.minimumMigrationAmount.toString()); + + /** Drive the pallet's pause origin the way mainnet governance will: through + * the technical committee (this chain has no sudo; the generated spec seats + * a single member so threshold 1 executes immediately). */ + async function committeeSetPaused(paused) { + const call = api.tx.tokenMigration.setPaused(paused); + const collective = api.tx.technicalCommittee ?? api.tx.council; + await new Promise((resolve, reject) => { + collective.propose(1, call, call.method.encodedLength) + .signAndSend(alice, ({ status, dispatchError }) => { + if (dispatchError) return reject(new Error(dispatchError.toString())); + if (status.isInBlock) resolve(); + }) + .catch(reject); + }); + await waitFor(async () => (await api.query.tokenMigration.paused()).isTrue === paused, + { timeoutMs: 60_000, label: `pallet paused=${paused} to be visible` }); + } + await committeeSetPaused(false); + log(`pallet unpaused; minimum migration ${MIN / PEN_12} PEN`); + + // --- Base side ---------------------------------------------------------- + section("Base Sepolia"); + const { vault, pen, sweepTs } = deployToSepolia({ env, roles: ctx.roles, log }); + log(`vault ${vault} (sweep floor ${new Date(sweepTs * 1000).toISOString()})`); + const V = { address: vault, abi: vaultAbi }; + const P = { address: pen, abi: erc20Abi }; + + /** Read that rides out endpoint throttling (same rationale as phase 5). */ + async function read(c, fn, args) { + let last; + for (let attempt = 0; attempt < 5; attempt++) { + try { + return await ctx.pub.readContract({ ...c, functionName: fn, args }); + } catch (error) { + last = error; + const text = `${error?.details ?? ""} ${error?.shortMessage ?? ""} ${error?.message ?? ""}`; + if (!/rate limit|too many requests|timeout|fetch failed|50[234]/i.test(text)) throw error; + await sleep(3000 * (attempt + 1)); + } + } + throw last; + } + const eventually = (fn, label, timeoutMs = 120_000) => + waitFor(async () => { try { await fn(); return true; } catch { return false; } }, + { timeoutMs, intervalMs: 3000, label }); + + await waitFor( + async () => (await read(V, "pendingAdmin", [])).toLowerCase() === ctx.roles.admin.address.toLowerCase(), + { timeoutMs: 120_000, intervalMs: 3000, label: "pendingAdmin to be visible" }); + await send(ctx, ctx.roles.admin, { ...V, functionName: "acceptAdmin", args: [] }); + await eventually( + async () => assertEq((await read(V, "admin", [])).toLowerCase(), ctx.roles.admin.address.toLowerCase(), "admin"), + "admin handover to settle"); + log("admin handover complete"); + + // --- the fleet ---------------------------------------------------------- + section("Attestor fleet"); + killMatching(["dist/main.js"]); + clearState(["attestor/cp1.json", "attestor/cp2.json", "attestor/cp3.json", "attestor/cp4.json", + "releaser/releaser-state.json"]); + const pendulumWs = api._options?.provider?.endpoint ?? process.env.PENDULUM_WS; + const baseEnv = { + BASE_RPC_URL: env.BASE_SEPOLIA_RPC_URL, VAULT_ADDRESS: vault, + BASE_CHAIN_ID: "84532", POLL_INTERVAL_MS: "12000", + }; + const pendulumHead = (await api.query.system.number()).toString(); + const baseHead = String(await ctx.pub.getBlockNumber()); + const attestorEnv = (i) => ({ + ...baseEnv, PENDULUM_WS: pendulumWs, + ATTESTOR_PRIVATE_KEY: env[`ATTESTOR_${i}_PRIVATE_KEY`], + CHECKPOINT_FILE: `./cp${i}.json`, START_BLOCK: pendulumHead, + }); + for (let i = 1; i <= 4; i++) { + start(`attestor${i}`, "attestor", attestorEnv(i)); + await sleep(3000); + } + start("monitor", "monitor", { ...baseEnv, PENDULUM_WS: pendulumWs, GRACE_SECONDS: "300" }); + start("releaser", "releaser", { ...baseEnv, RELEASER_PRIVATE_KEY: env.RELEASER_PRIVATE_KEY, START_BLOCK: baseHead }); + log(`fleet up (Pendulum from #${pendulumHead}, Base from #${baseHead})`); + + // PEN lands at an address we control, so RB-3 can push a surplus back in. + const recipient = ctx.roles.admin.address; + + /** Burn on the local chain; returns {nonce, amount, atBlock}. */ + async function migrate(amountPen) { + const amount = amountPen * PEN_12; + const atBlock = Number((await api.query.system.number()).toString()); + return new Promise((resolve, reject) => { + api.tx.tokenMigration.migrate(amount, recipient) + .signAndSend(alice, ({ status, events, dispatchError }) => { + if (dispatchError) { + const decoded = dispatchError.isModule + ? api.registry.findMetaError(dispatchError.asModule).name + : dispatchError.toString(); + return reject(new Error(`migrate failed: ${decoded}`)); + } + if (!status.isInBlock) return; + const ev = events.map((r) => r.event) + .find((e) => e.section === "tokenMigration" && e.method === "MigrationInitiated"); + if (!ev) return reject(new Error("no MigrationInitiated event")); + resolve({ + nonce: BigInt(ev.data[0].toString()), + amount: BigInt(ev.data[3].toString().replaceAll(",", "")), + atBlock, + }); + }) + .catch(reject); + }); + } + const payloadOf = (m) => read(V, "payloadHash", [m.nonce, recipient, m.amount]); + const released = (m) => read(V, "nonceConsumed", [m.nonce]); + const waitReleased = (m, label) => + waitFor(async () => (await released(m)) === true, { timeoutMs: 240_000, intervalMs: 5000, label }); + + // --- drills ------------------------------------------------------------- + section("Baseline"); + await check("the pipeline works before we start breaking it", async () => { + const m = await migrate(200n); + await waitReleased(m, "baseline release"); + assert((await read(P, "balanceOf", [recipient])) >= m.amount * CONVERSION_FACTOR, "recipient balance"); + }); + + section("RB-1 — attestor removal retroactively invalidates its approvals"); + let held; // the migration parked below quorum, completed in the RB-6 drill + let heldPayload; + await check("a removed attestor's recorded approval stops counting", async () => { + // Premise: attestor1 is offline, so exactly att2..att4 (threshold) vote. + await stopAndWait("attestor1"); + // Pause the vault: approvals record, releases wait. This parks the + // migration at exactly-threshold so removing one voter drops it below. + await send(ctx, ctx.roles.guardian, { ...V, functionName: "pause", args: [] }); + held = await migrate(150n); + heldPayload = await payloadOf(held); + await waitFor(async () => (await read(V, "activeApprovals", [heldPayload])) >= 3n, + { timeoutMs: 240_000, intervalMs: 5000, label: "three approvals to be recorded while paused" }); + await eventually(async () => assertEq(await read(V, "pendingRelease", [heldPayload]), true, "pendingRelease"), + "the pending mark to be visible"); + + // The drill: remove attestor4 (RB-1's compromised key). + await send(ctx, ctx.roles.admin, { ...V, functionName: "removeAttestor", args: [ctx.roles.attestors[3].address] }); + await eventually(async () => assertEq(await read(V, "activeApprovals", [heldPayload]), 2n, "activeApprovals"), + "the removal to invalidate the recorded approval"); + assertEq(await read(V, "hasApproved", [heldPayload, ctx.roles.attestors[3].address]), false, + "hasApproved for the removed attestor"); + }); + + await check("below quorum, the release cannot be completed by anyone", async () => { + await send(ctx, ctx.roles.admin, { ...V, functionName: "unpause", args: [] }); + // The releaser daemon is live and retrying; give it real time to try. + await sleep(30_000); + assertEq(await released(held), false, "nonceConsumed with only 2 active approvals"); + }); + + section("RB-6 — a re-added attestor must approve again"); + await check("re-adding bumps the generation: the old approval stays dead", async () => { + await send(ctx, ctx.roles.admin, { ...V, functionName: "addAttestor", args: [ctx.roles.attestors[3].address] }); + await eventually(async () => assertEq(await read(V, "isAttestor", [ctx.roles.attestors[3].address]), true, "isAttestor"), + "the re-add to settle"); + // Same address, same recorded vote — but a new generation, so it counts + // for nothing until the attestor signs again. + assertEq(await read(V, "activeApprovals", [heldPayload]), 2n, "activeApprovals after re-add"); + }); + + await check("the documented recovery completes the quorum: rewind and re-approve", async () => { + // RB-6's operator step: restart the re-added attestor with its checkpoint + // rewound to before the affected migrations, so it re-scans and re-signs. + await stopAndWait("attestor4"); + writeFileSync(path.join(ROOT, "attestor/cp4.json"), + JSON.stringify({ lastProcessedBlock: held.atBlock - 1 })); + start("attestor4", "attestor", attestorEnv(4)); + await waitReleased(held, "the held migration to release after re-approval"); + assertEq(await read(V, "activeApprovals", [heldPayload]), 3n, "final active approvals"); + }); + + await check("a fifth attestor can be added and removed cleanly", async () => { + const fifth = privateKeyToAccount(generatePrivateKey()).address; + await send(ctx, ctx.roles.admin, { ...V, functionName: "addAttestor", args: [fifth] }); + await eventually(async () => assertEq(await read(V, "attestorCount", []), 5n, "attestorCount"), "count to reach 5"); + await send(ctx, ctx.roles.admin, { ...V, functionName: "removeAttestor", args: [fifth] }); + await eventually(async () => assertEq(await read(V, "attestorCount", []), 4n, "attestorCount"), "count back to 4"); + // Bring the standby back for the remaining drills. + start("attestor1", "attestor", attestorEnv(1)); + await sleep(5000); + assert(alive("attestor1"), `attestor1 did not come back:\n${logs("attestor1")}`); + }); + + section("RB-3 — the monitor tolerates a surplus (deficit is covered on Anvil)"); + await check("PEN sent into the vault raises no alarm and kills nothing", async () => { + // A deficit cannot be created on Sepolia — nothing but the vault itself + // can move its tokens, which is the security property. Phase 3 covers + // the deficit alarm via Anvil impersonation. Here: the benign inverse. + const monitorLogBefore = logs("monitor").length; + await send(ctx, ctx.roles.admin, { ...P, functionName: "transfer", args: [vault, 10n * 10n ** 18n] }); + await sleep(40_000); // > two monitor poll cycles + assert(alive("monitor"), `monitor died on a surplus:\n${logs("monitor")}`); + const fresh = logs("monitor").slice(monitorLogBefore); + assert(!/DEFICIT/i.test(fresh), `monitor raised a deficit alert on a surplus:\n${fresh}`); + }); + + section("RB-4 — coordinated stop, resumed in reverse order"); + await check("pause pallet then vault; new burns fail at the source", async () => { + await committeeSetPaused(true); + let failed = false; + try { await migrate(100n); } catch (e) { failed = /MigrationsPaused/.test(e.message); } + assert(failed, "migrate should fail MigrationsPaused while the pallet is paused"); + await send(ctx, ctx.roles.guardian, { ...V, functionName: "pause", args: [] }); + await eventually(async () => assertEq(await read(V, "paused", []), true, "vault paused"), "vault pause to settle"); + }); + + await check("resume in reverse: vault first, then pallet; the pipeline recovers", async () => { + await send(ctx, ctx.roles.admin, { ...V, functionName: "unpause", args: [] }); + await committeeSetPaused(false); + const m = await migrate(120n); + await waitReleased(m, "post-resume release"); + }); + + section("RB-7 — window close, reconcile, sweep"); + await check("close the window: pause the pallet, confirm nothing is in flight", async () => { + await committeeSetPaused(true); + await eventually(async () => assertEq(await read(V, "pendingApprovedAmount", []), 0n, "pendingApprovedAmount"), + "all quorum-approved releases to have settled"); + }); + + await check("reconcile: conservation holds exactly at the close", async () => { + const [balance, rel, swept, supply] = await Promise.all([ + read(P, "balanceOf", [vault]), read(V, "totalReleased", []), + read(V, "totalSwept", []), read(P, "totalSupply", []), + ]); + // The RB-3 surplus sits in the vault's balance, ON TOP of conservation. + const surplus = 10n * 10n ** 18n; + assertEq(balance + rel + swept, supply + surplus, "balance + released + swept vs supply (+known surplus)"); + }); + + await check("the sweep executes and the monitor does not false-alarm", async () => { + assert(Math.floor(Date.now() / 1000) >= sweepTs, "sweep floor not yet passed — raise DRILL_SWEEP_OFFSET"); + assertEq(await read(V, "thresholdReducedAt", []), 0n, "thresholdReducedAt (settling gate must not be armed)"); + const monitorLogBefore = logs("monitor").length; + const sweepAmount = 1000n * 10n ** 18n; + const sweptBefore = await read(V, "totalSwept", []); + await send(ctx, ctx.roles.admin, { ...V, functionName: "sweepRemainder", args: [recipient, sweepAmount] }); + await eventually(async () => assertEq(await read(V, "totalSwept", []), sweptBefore + sweepAmount, "totalSwept"), + "the sweep to settle"); + await sleep(40_000); + assert(alive("monitor"), `monitor died after the sweep:\n${logs("monitor")}`); + const fresh = logs("monitor").slice(monitorLogBefore); + assert(!/DEFICIT/i.test(fresh), `monitor false-alarmed on a swept balance:\n${fresh}`); + }); + + await check("every daemon survived every drill", () => { + for (const name of ["attestor1", "attestor2", "attestor3", "attestor4", "monitor", "releaser"]) { + assert(alive(name), `${name} is dead:\n${logs(name)}`); + } + }); + + // --- record + teardown ---------------------------------------------------- + mkdirSync(runDir, { recursive: true }); + writeFileSync(path.join(runDir, "manifest.json"), JSON.stringify({ + kind: "phase6-drills", + startedAt: started.toISOString(), + commit: execSync("git rev-parse HEAD", { cwd: ROOT, encoding: "utf8" }).trim(), + vault, pen, sweepTimestamp: sweepTs, + covered: ["RB-1", "RB-3 (surplus half)", "RB-4", "RB-6", "RB-7"], + coveredElsewhere: { "RB-2": "phase 5 rehearsal", "RB-3 deficit": "phase 3 (Anvil impersonation)", "RB-5": "drill-rb5-upgrade.mjs on Chopsticks" }, + }, null, 2)); + for (const name of ["attestor1", "attestor2", "attestor3", "attestor4", "monitor", "releaser"]) { + writeFileSync(path.join(runDir, `${name}.log`), logs(name)); + } + if (network) writeFileSync(path.join(runDir, "zombienet.log"), network.out.join("\n")); + console.log(`\n run artifacts: ${path.relative(ROOT, runDir)}`); + + const ok = summarise(); + if (KEEP) { console.log("\n --keep: leaving everything running."); process.exit(ok ? 0 : 1); } + stopAll(); + await sleep(2000); + if (!ATTACH) teardown(log); + if (api) await api.disconnect().catch(() => {}); + process.exit(ok ? 0 : 1); +} + +process.on("SIGINT", () => { stopAll(); if (!KEEP) teardown(console.log); process.exit(130); }); + +main().catch(async (error) => { + console.error(`\ndrills aborted: ${error.message}`); + stopAll(); + if (!KEEP && !ATTACH) teardown(console.log); + process.exit(1); +}); From c0ba4f16cfb76c9b1a6da5fba6890ecdcec8f799 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 31 Aug 2026 10:23:01 +0200 Subject: [PATCH 34/61] docs: Point the failure-drill table at the automated drills --- docs/pen-migration-local-test-plan.md | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 23e3ca52b..41134778e 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -346,18 +346,25 @@ mainnet. ## Phase 6 — Failure drills (the runbooks) -Rehearse each runbook once against the local stack, so the first time you run -them is not during an incident: +Rehearse each runbook against a real stack, so the first time anyone runs them +is not during an incident. Automated: -| Runbook | Drill | +```bash +node testing/src/drills.mjs # RB-1, RB-3 (surplus), RB-4, RB-6, RB-7 on the Sepolia stack +# RB-5 needs Chopsticks (the one place a runtime can be swapped) and a +# spec-bumped wasm; see the header of drill-rb5-upgrade.mjs: +UPGRADE_WASM=/path/to/spec+1.wasm node testing/src/drill-rb5-upgrade.mjs +``` + +| Runbook | Where it is drilled | |---|---| -| RB-1 key compromise | Remove an attestor mid-flight; confirm its recorded approvals stop counting and a replacement can complete the quorum | -| RB-2 outage | Covered by phase 3 item 4 | -| RB-3 invariant breach | Covered by phase 3 item 6 — including the auto-pause and the recovery path | -| RB-4 pause/unpause | Pause the pallet *and* the vault; confirm the correct resume order | -| RB-5 runtime upgrade | Apply a second runtime upgrade while attestors run; confirm they keep decoding (or fail loudly rather than silently skipping) | -| RB-6 attestor rotation | Add a 5th attestor, remove an old one, confirm a re-added address must approve again | -| RB-7 window close | Warp past `earliestSweepTimestamp`, reconcile, sweep, confirm `totalSwept` and that the monitor does not false-alarm | +| RB-1 key compromise | `drills.mjs`: attestor removed with its approval recorded on a paused, exactly-at-threshold payload — the vote stops counting and nobody can complete the release below quorum | +| RB-2 outage | Phase 5 rehearsal (one down still releases; two down stops cleanly) | +| RB-3 invariant breach | Deficit + auto-pause: phase 3 on Anvil (a deficit **cannot be created** on Sepolia — nothing but the vault can move its tokens, which is the security property). Surplus tolerance: `drills.mjs` | +| RB-4 pause/unpause | `drills.mjs`: pallet then vault, resumed in reverse, pipeline recovers | +| RB-5 runtime upgrade | `drill-rb5-upgrade.mjs` on Chopsticks: release, write `:code` (what an enacted upgrade does), confirm the fleet keeps decoding and releasing. The shape-change half — fail loudly, never skip — is the attestor's unit-tested 4-field assertion | +| RB-6 attestor rotation | `drills.mjs`: re-add bumps the generation so the old vote stays dead; the documented recovery (rewind checkpoint, restart, re-approve) completes the quorum; a 5th attestor adds and removes cleanly | +| RB-7 window close | `drills.mjs`: pallet paused, `pendingApprovedAmount` drained, conservation reconciled exactly, sweep executed, monitor quiet. Deploys with a ~90s sweep floor since Sepolia time cannot be warped; the threshold is never reduced, so the 7-day settling gate stays unarmed | --- From e97557783793ff8e22095333717546ac60ec0c4c Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 31 Aug 2026 10:53:41 +0200 Subject: [PATCH 35/61] testing: Drill the real runtime upgrade under a live fleet MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The RB-5 drill now enacts the actual referendum on a Chopsticks fork of pre-upgrade mainnet: no pallet before, write the spec-26 wasm to :code — which is what enactment does — and require every attestor's checkpoint to advance past the upgrade block, including across a full node restart. Advancing is the proof: the decode path ran to completion on blocks built by the new runtime, and a decode failure exits the daemon by design. Building the spec-26 wasm for the drill also confirms the referendum's version bump compiles cleanly. Chopsticks shaped the script more than expected, and the header records the traps: it serves metadata and runtime-version RPCs from a runtime cached at fork time, even across --resume, so a post-upgrade extrinsic built against its metadata is rejected by the executing runtime as badProof and cannot be submitted at all — while frame-system's own lastRuntimeUpgrade record proves execution genuinely upgraded. A restart without --resume silently re-forks at the remote head and discards the enacted upgrade, and --resume's CLI form only accepts a block hash. Also anchors the 50x status-code match in the transient-error classifiers so a number inside an error's transaction params cannot misclassify a genuine failure as transient. --- attestor/src/main.ts | 2 +- docs/pen-migration-implementation-overview.md | 2 + docs/pen-migration-local-test-plan.md | 2 +- testing/src/drill-rb5-upgrade.mjs | 200 ++++++++++++------ testing/src/drills.mjs | 2 +- testing/src/rehearsal.mjs | 2 +- 6 files changed, 145 insertions(+), 65 deletions(-) diff --git a/attestor/src/main.ts b/attestor/src/main.ts index 5c9e0fc73..222c7a2df 100644 --- a/attestor/src/main.ts +++ b/attestor/src/main.ts @@ -58,7 +58,7 @@ function isTransientRpcError(error: unknown): boolean { (error as { cause?: { details?: string } })?.cause?.details, ]; const text = parts.filter(Boolean).join(" "); - return /rate limit|too many requests|timeout|timed out|ETIMEDOUT|ECONNRESET|ECONNREFUSED|EAI_AGAIN|socket hang up|fetch failed|502|503|504|service unavailable|internal error/i + return /rate limit|too many requests|timeout|timed out|ETIMEDOUT|ECONNRESET|ECONNREFUSED|EAI_AGAIN|socket hang up|fetch failed|\b50[234]\b|service unavailable|internal error/i .test(text); } diff --git a/docs/pen-migration-implementation-overview.md b/docs/pen-migration-implementation-overview.md index a8c22c95b..57709ddc0 100644 --- a/docs/pen-migration-implementation-overview.md +++ b/docs/pen-migration-implementation-overview.md @@ -76,6 +76,8 @@ restricted to consumed nonces. Details and verified-not-vulnerable list in | `testing/src/phase3-e2e.mjs` | 7/7 end to end, four attestors + monitor + releaser | | `testing/src/phase4-zombienet.mjs` | 7/7 against a real relay (~2-block parachain finality lag) | | `testing/src/rehearsal.mjs` | 15/15 full stack: local Zombienet Pendulum + Base Sepolia | +| `testing/src/drills.mjs` | 13/13 failure drills (RB-1/3/4/6/7) on the Sepolia stack | +| `testing/src/drill-rb5-upgrade.mjs` | 5/5 — the real spec-26 upgrade enacted under a live fleet on a mainnet fork | Seven internal adversarial review rounds have run; each found real issues (sometimes in a prior round's own fix), all fixed with regression tests and diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 41134778e..65a40b975 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -362,7 +362,7 @@ UPGRADE_WASM=/path/to/spec+1.wasm node testing/src/drill-rb5-upgrade.mjs | RB-2 outage | Phase 5 rehearsal (one down still releases; two down stops cleanly) | | RB-3 invariant breach | Deficit + auto-pause: phase 3 on Anvil (a deficit **cannot be created** on Sepolia — nothing but the vault can move its tokens, which is the security property). Surplus tolerance: `drills.mjs` | | RB-4 pause/unpause | `drills.mjs`: pallet then vault, resumed in reverse, pipeline recovers | -| RB-5 runtime upgrade | `drill-rb5-upgrade.mjs` on Chopsticks: release, write `:code` (what an enacted upgrade does), confirm the fleet keeps decoding and releasing. The shape-change half — fail loudly, never skip — is the attestor's unit-tested 4-field assertion | +| RB-5 runtime upgrade | `drill-rb5-upgrade.mjs` on Chopsticks: fork pre-upgrade mainnet (no pallet), enact the real spec-26 upgrade by writing `:code`, confirm frame-system records it, and require every attestor's checkpoint to advance past the upgrade block — their decode path ran on post-upgrade blocks, and a decode failure exits by design — including across a full node restart. A post-upgrade `migrate` cannot be *submitted* through Chopsticks (it serves pre-fork metadata even across `--resume`, so the real runtime rejects the extrinsic as `badProof`); decoding real events under the post-upgrade runtime is what phases 2–3 do wholesale, and the shape-change half is the attestor's unit-tested 4-field assertion | | RB-6 attestor rotation | `drills.mjs`: re-add bumps the generation so the old vote stays dead; the documented recovery (rewind checkpoint, restart, re-approve) completes the quorum; a 5th attestor adds and removes cleanly | | RB-7 window close | `drills.mjs`: pallet paused, `pendingApprovedAmount` drained, conservation reconciled exactly, sweep executed, monitor quiet. Deploys with a ~90s sweep floor since Sepolia time cannot be warped; the threshold is never reduced, so the 7-day settling gate stays unarmed | diff --git a/testing/src/drill-rb5-upgrade.mjs b/testing/src/drill-rb5-upgrade.mjs index 5d5b2b008..bddba25c0 100644 --- a/testing/src/drill-rb5-upgrade.mjs +++ b/testing/src/drill-rb5-upgrade.mjs @@ -1,31 +1,51 @@ /** - * RB-5 drill — a runtime upgrade lands while the attestor fleet is running. + * RB-5 drill — THE runtime upgrade lands while the attestor fleet is running. * - * The property under test: attestors keep decoding `MigrationInitiated` across - * an upgrade that does not change the event (the shape-change half of RB-5 — - * fail loudly rather than skip — is unit-tested in the attestor via its - * 4-field assertion). This is the one drill that cannot run on Zombienet or - * Sepolia: swapping a runtime needs either root (no sudo on Pendulum) or - * storage access, so it runs on a Chopsticks fork, where `:code` can be - * written directly — which is also exactly what a real enacted upgrade does. + * This drills the actual production upgrade, not a stand-in: a Chopsticks fork + * of live mainnet state (spec 25, no token-migration pallet) has the spec-26 + * wasm written to `:code` — which is exactly what the enacted referendum does — + * while the attestor fleet is already running. Before the upgrade there is + * nothing to decode; after it, the pallet exists, migrations flow, and the + * fleet must decode and release without a restart. The shape-change half of + * RB-5 — fail loudly rather than skip — is the attestor's unit-tested 4-field + * assertion. * - * Prerequisites: - * - Chopsticks on :8000 (testing/chopsticks-e2e.yml, wasm-override with the - * CURRENT runtime), Anvil on :8545, services built. - * - UPGRADE_WASM: path to a runtime wasm with a HIGHER spec_version and an - * unchanged event shape (build one by bumping spec_version and rebuilding). + * This cannot run on Zombienet or Sepolia: swapping a runtime needs root (no + * sudo on Pendulum) or storage access. Two Chopsticks quirks shape the script: + * + * - It must fork WITHOUT --wasm-override: an override pins the executing + * runtime, so a `:code` write beneath it splits execution from reporting. + * - Even without an override, Chopsticks serves metadata and runtime-version + * RPCs from a runtime cached at fork time, and keeps doing so across + * `--resume` restarts. After the `:code` write the EXECUTING runtime + * really is the new one — frame-system records lastRuntimeUpgrade = the + * new spec — but the RPC layer keeps reporting the old, and an extrinsic + * built against that stale metadata is rejected by the real runtime as + * `badProof`. Consequence: a post-upgrade `migrate` cannot be SUBMITTED + * through Chopsticks at all, so this drill asserts the operational RB-5 + * property instead — every attestor keeps processing post-upgrade blocks + * through its normal decode path (checkpoints advance past the upgrade + * block; a decode failure would exit the daemon by design) and rides a + * full node restart. Decoding actual MigrationInitiated events under the + * post-upgrade runtime is exercised wholesale by phases 2 and 3, which run + * that runtime via wasm-override; the changed-shape half is the attestor's + * unit-tested 4-field assertion. + * + * Prerequisites: Anvil on :8545, services built, and UPGRADE_WASM pointing at + * the spec-bumped runtime (bump spec_version, rebuild). Chopsticks is spawned + * and restarted by the drill itself. * * Usage: UPGRADE_WASM=/path/to/spec26.wasm node src/drill-rb5-upgrade.mjs */ -import { readFileSync } from "node:fs"; +import { spawn } from "node:child_process"; +import { readFileSync, rmSync } from "node:fs"; +import path from "node:path"; import { ApiPromise, WsProvider } from "@polkadot/api"; -import { Keyring } from "@polkadot/keyring"; -import { cryptoWaitReady } from "@polkadot/util-crypto"; -import { erc20Abi, vaultAbi } from "./abi.mjs"; -import { alive, clearState, killMatching, logs, start, stopAll, waitFor } from "./daemons.mjs"; +import { alive, clearState, logs, start, stopAll, waitFor } from "./daemons.mjs"; +import { killMatching } from "./zombienet.mjs"; import { assert, assertEq, check, section, summarise } from "./harness.mjs"; -import { keys, pub } from "./anvil.mjs"; +import { keys } from "./anvil.mjs"; import { deployStack } from "./deploy.mjs"; const CHOPSTICKS = process.env.CHOPSTICKS_WS ?? "ws://127.0.0.1:8000"; @@ -35,34 +55,60 @@ if (!upgradeWasmPath) { process.exit(2); } +const ROOT = path.resolve(new URL(".", import.meta.url).pathname, "../.."); +let chopsticks = null; + +/** Spawn Chopsticks on the e2e config (db-backed, NO wasm-override) and wait + * until it answers. `fresh` wipes the db for a clean fork. */ +async function startChopsticks({ fresh, resumeAt, override }) { + if (fresh) { + for (const f of [".chopsticks-e2e.sqlite", ".chopsticks-e2e.sqlite-shm", ".chopsticks-e2e.sqlite-wal"]) { + try { rmSync(path.join(ROOT, "testing", f)); } catch {} + } + } + // Without --resume, a restart re-forks at the remote chain's CURRENT head + // and the locally-built blocks — including the enacted upgrade — are + // silently discarded. --resume continues from a block in the db. Note the + // CLI form must be a block number or hash: a literal `--resume true` + // arrives as the string "true" and fails config validation. + const args = ["@acala-network/chopsticks@latest", "--config", "testing/chopsticks-e2e.yml"]; + if (!fresh) args.push("--resume", String(resumeAt)); + if (override) args.push("--wasm-override", override); + chopsticks = spawn("npx", args, { + cwd: ROOT, stdio: ["ignore", "pipe", "pipe"], + }); + const out = []; + chopsticks.stdout.on("data", (c) => out.push(String(c))); + chopsticks.stderr.on("data", (c) => out.push(String(c))); + try { + await waitFor(() => out.join("").includes("listening on"), + { timeoutMs: 240_000, intervalMs: 2000, label: "chopsticks to come up" }); + } catch (error) { + // Surface what the node actually said — a silent timeout is undebuggable. + throw new Error(`${error.message}\n--- chopsticks output (tail) ---\n${out.join("").slice(-3000)}`); + } +} + +async function stopChopsticks() { + if (!chopsticks) return; + chopsticks.kill("SIGTERM"); + await new Promise((resolve) => { + chopsticks.on("exit", resolve); + setTimeout(() => { chopsticks.kill("SIGKILL"); resolve(); }, 10_000); + }); + chopsticks = null; +} + +console.log("starting Chopsticks (fresh mainnet fork, no wasm-override) ..."); +await startChopsticks({ fresh: true }); + console.log("RB-5 drill — runtime upgrade under a live fleet"); const { vault, pen } = deployStack(); console.log(` vault ${vault}`); -const read = (fn, args) => pub.readContract({ address: vault, abi: vaultAbi, functionName: fn, args }); -const api = await ApiPromise.create({ provider: new WsProvider(CHOPSTICKS), noInitWarn: true }); -await cryptoWaitReady(); -const keyring = new Keyring({ type: "sr25519", ss58Format: api.registry.chainSS58 ?? 56 }); -const alice = keyring.addFromUri("//Alice"); -const MIN = BigInt(api.consts.tokenMigration.minimumMigrationAmount.toString()); +let api = await ApiPromise.create({ provider: new WsProvider(CHOPSTICKS), noInitWarn: true }); const newBlock = () => api.rpc("dev_newBlock"); -async function fund(who, amount) { - await api.rpc("dev_setStorage", { System: { Account: [[[who], { providers: 1, data: { free: amount.toString() } }]] } }); - await newBlock(); -} -await api.rpc("dev_setStorage", [[api.query.tokenMigration.paused.key(), "0x00"]]); -await newBlock(); - -async function migrate(amount, baseAddress) { - await fund(alice.address, MIN * 200n); - await api.tx.tokenMigration.migrate(amount, baseAddress).signAndSend(alice); - await newBlock(); - const events = await api.query.system.events(); - const ev = events.map((r) => r.event).find((e) => e.section === "tokenMigration" && e.method === "MigrationInitiated"); - assert(ev, "no MigrationInitiated event"); - return { nonce: BigInt(ev.data[0].toString()) }; -} killMatching(["dist/main.js"]); clearState(["attestor/cp1.json", "attestor/cp2.json", "attestor/cp3.json", "attestor/cp4.json"]); @@ -75,37 +121,68 @@ for (let i = 0; i < 4; i++) { }); } -const recipient = "0x000000000000000000000000000000000000beef"; const versionBefore = api.runtimeVersion.specVersion.toNumber(); +let upgradedHead; +let upgradedHeadNumber; -section(`Upgrade under fire (starting at spec ${versionBefore})`); +section(`The real upgrade, under a live fleet (mainnet fork at spec ${versionBefore})`); -await check("the fleet releases normally before the upgrade", async () => { - const { nonce } = await migrate(MIN * 2n, recipient); - await waitFor(async () => (await read("nonceConsumed", [nonce])) === true, - { timeoutMs: 90_000, label: "pre-upgrade release" }); +await check("pre-upgrade: the pallet does not exist and the fleet idles happily", async () => { + assert(!api.query.tokenMigration, "tokenMigration already present — this fork is not pre-upgrade mainnet"); + await newBlock(); + await newBlock(); + for (let i = 1; i <= 4; i++) assert(alive(`attestor${i}`), `attestor${i} died on pallet-less blocks:\n${logs(`attestor${i}`)}`); }); -await check("the runtime upgrade applies (spec_version increases)", async () => { +await check("the referendum's upgrade applies: the executing runtime records it", async () => { const code = `0x${readFileSync(upgradeWasmPath).toString("hex")}`; - // Writing :code is what an enacted upgrade does; the next block runs it. + // Writing :code is what the enacted referendum does; the next block runs it. await api.rpc("dev_setStorage", [["0x3a636f6465", code]]); await newBlock(); await newBlock(); - const version = await api.rpc.state.getRuntimeVersion(); - const after = version.specVersion.toNumber(); - assert(after > versionBefore, `spec_version did not increase: ${versionBefore} -> ${after}`); - return; + // The strongest possible evidence the upgrade executed: frame-system itself + // wrote its new version into storage. (Chopsticks' RPC layer still reports + // the cached old runtime at this point — see the header.) + const head = await api.rpc.chain.getFinalizedHead(); + // Keep the HASH: chopsticks' --resume validates its string form as a + // 66-char block hash — a bare block number is rejected by the schema. + upgradedHead = head.toHex(); + upgradedHeadNumber = (await api.rpc.chain.getHeader(head)).number.toNumber(); + const apiAt = await api.at(head); + const lru = (await apiAt.query.system.lastRuntimeUpgrade()).toJSON(); + assert(lru && lru.specVersion > versionBefore, + `lastRuntimeUpgrade did not advance: ${JSON.stringify(lru)}`); }); -await check("attestors keep decoding and releasing across the upgrade", async () => { - const { nonce } = await migrate(MIN * 3n, recipient); - try { - await waitFor(async () => (await read("nonceConsumed", [nonce])) === true, - { timeoutMs: 90_000, label: "post-upgrade release" }); - } catch (e) { - throw new Error(`${e.message}\n--- attestor1 ---\n${logs("attestor1")}`); - } +await check("every attestor processes post-upgrade blocks through its decode path", async () => { + // Produce post-upgrade blocks and require every checkpoint to move past + // the upgrade block. Advancing means migrationEventsInBlock ran to + // completion on blocks built by the NEW runtime — a decode failure is + // fatal by design (PRD A5), so mere survival plus progress is the proof. + for (let i = 0; i < 4; i++) await newBlock(); + const target = upgradedHeadNumber + 2; + await waitFor(() => [1, 2, 3, 4].every((i) => { + try { + return JSON.parse(readFileSync(path.join(ROOT, `attestor/cp${i}.json`), "utf8")).lastProcessedBlock >= target; + } catch { return false; } + }), { timeoutMs: 120_000, intervalMs: 3000, label: `all checkpoints to pass block ${target}` }); +}); + +await check("the fleet rides a full node restart on the upgraded chain", async () => { + await api.disconnect(); + await stopChopsticks(); + await startChopsticks({ fresh: false, resumeAt: upgradedHead }); + api = await ApiPromise.create({ provider: new WsProvider(CHOPSTICKS), noInitWarn: true }); + // The daemons were never restarted; their WsProviders must reconnect and + // resume the finalized-heads subscription on their own. + const before = (await api.rpc.chain.getHeader()).number.toNumber(); + for (let i = 0; i < 3; i++) await newBlock(); + const target = before + 2; + await waitFor(() => [1, 2, 3, 4].every((i) => { + try { + return JSON.parse(readFileSync(path.join(ROOT, `attestor/cp${i}.json`), "utf8")).lastProcessedBlock >= target; + } catch { return false; } + }), { timeoutMs: 180_000, intervalMs: 3000, label: `all checkpoints to pass block ${target} after the restart` }); }); await check("no attestor died across the upgrade", () => { @@ -116,4 +193,5 @@ await check("no attestor died across the upgrade", () => { stopAll(); await api.disconnect(); +await stopChopsticks(); process.exit(summarise() ? 0 : 1); diff --git a/testing/src/drills.mjs b/testing/src/drills.mjs index 5d241af19..21e94de22 100644 --- a/testing/src/drills.mjs +++ b/testing/src/drills.mjs @@ -108,7 +108,7 @@ async function main() { } catch (error) { last = error; const text = `${error?.details ?? ""} ${error?.shortMessage ?? ""} ${error?.message ?? ""}`; - if (!/rate limit|too many requests|timeout|fetch failed|50[234]/i.test(text)) throw error; + if (!/rate limit|too many requests|timeout|fetch failed|\b50[234]\b/i.test(text)) throw error; await sleep(3000 * (attempt + 1)); } } diff --git a/testing/src/rehearsal.mjs b/testing/src/rehearsal.mjs index 0fa47548c..d8b62ccbc 100644 --- a/testing/src/rehearsal.mjs +++ b/testing/src/rehearsal.mjs @@ -249,7 +249,7 @@ async function main() { } catch (error) { last = error; const text = `${error?.details ?? ""} ${error?.shortMessage ?? ""} ${error?.message ?? ""}`; - if (!/rate limit|too many requests|timeout|fetch failed|50[234]/i.test(text)) throw error; + if (!/rate limit|too many requests|timeout|fetch failed|\b50[234]\b/i.test(text)) throw error; await sleep(3000 * (attempt + 1)); } } From f0f64e4ebcd6e72af7722b252e306961c49ac734 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 31 Aug 2026 10:54:46 +0200 Subject: [PATCH 36/61] attestor: Cover bare 429 codes; record the post-drills review pass MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The review confirmed the monitor and releaser do not share the attestor's former die-on-transient defect — both alert and continue at loop level — and that the recheck backoff cannot turn a permanent failure into a silent skip. --- attestor/src/main.ts | 2 +- docs/pen-migration-internal-review.md | 36 +++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/attestor/src/main.ts b/attestor/src/main.ts index 222c7a2df..fd2ddb18a 100644 --- a/attestor/src/main.ts +++ b/attestor/src/main.ts @@ -58,7 +58,7 @@ function isTransientRpcError(error: unknown): boolean { (error as { cause?: { details?: string } })?.cause?.details, ]; const text = parts.filter(Boolean).join(" "); - return /rate limit|too many requests|timeout|timed out|ETIMEDOUT|ECONNRESET|ECONNREFUSED|EAI_AGAIN|socket hang up|fetch failed|\b50[234]\b|service unavailable|internal error/i + return /rate limit|too many requests|timeout|timed out|ETIMEDOUT|ECONNRESET|ECONNREFUSED|EAI_AGAIN|socket hang up|fetch failed|\b(?:429|50[234])\b|service unavailable|internal error/i .test(text); } diff --git a/docs/pen-migration-internal-review.md b/docs/pen-migration-internal-review.md index c34df20a2..034a75af3 100644 --- a/docs/pen-migration-internal-review.md +++ b/docs/pen-migration-internal-review.md @@ -429,6 +429,42 @@ it settles, not once. Each attestor should also run against its own node rather than a shared public endpoint; the rehearsal reproduced the rate limit precisely because six daemons shared one. +## Post-drills review pass (2026-08-31) + +Scope: the fund-release-path changes made since the rehearsal findings (the +race-recheck backoff and the transient-RPC classification), per the standing +practice, plus the drill scripts themselves. + +### L1(drills). LOW — Status codes matched as bare substrings + +The transient-error classifiers matched `502|503|504` anywhere in the error +text, and error messages embed transaction parameters — an amount containing +`502` could misclassify a genuine failure as transient. The consequence was a +noisy stall (alert every head, checkpoint frozen, liveness alert eventually) +rather than anything silent, but it is now anchored on word boundaries, and +`429` was added for endpoints that return only the numeric code. **Fixed.** + +### Verified, no change needed + +- The monitor and releaser were checked for the attestor's die-on-transient + defect and do not have it: both wrap their poll/cycle bodies in a + loop-level catch that alerts and continues, and only startup failures exit. +- The recheck backoff cannot convert a permanent failure into a silent skip: + it returns true only on positive on-chain confirmation, and errors inside + the recheck retry rather than resolve. + +### Drill outcomes (phase 6) + +All seven runbooks are now rehearsed: `drills.mjs` 13/13 on the Sepolia stack +(RB-1, RB-3 surplus, RB-4, RB-6, RB-7; RB-2 in phase 5; RB-3 deficit in +phase 3), and `drill-rb5-upgrade.mjs` 5/5 — the actual spec-26 upgrade enacted +on a fork of pre-upgrade mainnet under a live fleet, which also proved the +referendum's version bump compiles. The RB-5 script's header records the +Chopsticks limitations that shaped it; the significant one is that a +post-upgrade extrinsic cannot be submitted through Chopsticks at all, because +it serves pre-fork metadata even across `--resume` while executing the new +runtime, which rejects the stale-metadata signature as `badProof`. + ## Residual risks and standing practices (no external audit — risk accepted) - Every change to the fund-release path (vault release/approve/sweep logic, pallet burn path) gets a fresh independent adversarial review round before From d952d5e87fce2f874c8cd2ea31b329c417c635b9 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 31 Aug 2026 16:23:21 +0200 Subject: [PATCH 37/61] testing: Add phase 5b, the governance rehearsal on Base Sepolia MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Foundry suite covers the governance logic; this covers what it cannot reach. First execution ever of DeployGovernance.s.sol, with the role wiring asserted on-chain (governor proposes and cancels, execution open to anyone, the deployer's timelock admin genuinely renounced). The vault-admin -> timelock handover, whose acceptance is itself a governance proposal. A real admin action end to end through the timelock, asserting the delay gates from both sides: execute reverts before the ETA and succeeds after. And the negatives — below-threshold proposals rejected, the old admin powerless, the guardian able to pause but nobody able to unpause without a proposal, which deliberately leaves the run's vault paused: that latency is the design's cost. No Zombienet and no daemon fleet: voter PEN is released through the real 3-of-4 approve() path driven directly by the attestor keys — which promptly reproduced the bimodal-gas OutOfGas the attestor daemon pads for, confirming it is a property of the contract, not the daemon; a direct caller must pad too. The proposal lifecycle also tolerates lagging RPC nodes, whose state() reads for a fresh proposal REVERT (GovernorNonexistentProposal) rather than returning Pending. Quorum fraction is 0 so mechanics are testable with drill-scale voting power; quorum sizing and the production timings stay covered by the unit tests. --manual stops after deploy + funding + wiring and prints a handoff card for a by-hand walkthrough (Tally / Blockscout / MetaMask). --- testing/.env.rehearsal.example | 11 + testing/src/abi.mjs | 3 + testing/src/phase5b-governance.mjs | 345 +++++++++++++++++++++++++++++ testing/src/rehearsal-deploy.mjs | 50 +++++ 4 files changed, 409 insertions(+) create mode 100644 testing/src/phase5b-governance.mjs diff --git a/testing/.env.rehearsal.example b/testing/.env.rehearsal.example index 85814ef51..b389c1c06 100644 --- a/testing/.env.rehearsal.example +++ b/testing/.env.rehearsal.example @@ -39,3 +39,14 @@ REHEARSAL_PER_RELEASE_CAP_PEN=50000 # How far ahead the immutable sweep floor sits. Short so the sweep path is # reachable in a long session; production is 2027-03-01. REHEARSAL_SWEEP_OFFSET_SECONDS=3600 + +# --- phase 5b: governance drill timings ------------------------------------ +# Wall-clock-sized so two full proposal lifecycles fit in one sitting. +# Production values (48h timelock, 1d+5d voting) stay validated by the +# Foundry suite. GOV_QUORUM_FRACTION=0 makes mechanics testable with +# drill-scale voting power; quorum SIZING is deliberately not rehearsed. +GOV_TIMELOCK_DELAY=180 +GOV_VOTING_DELAY=60 +GOV_VOTING_PERIOD=240 +GOV_PROPOSAL_THRESHOLD_PEN=1000 +GOV_QUORUM_FRACTION=0 diff --git a/testing/src/abi.mjs b/testing/src/abi.mjs index bba5f6152..0591c3703 100644 --- a/testing/src/abi.mjs +++ b/testing/src/abi.mjs @@ -25,3 +25,6 @@ function abiOf(file, name) { export const vaultAbi = abiOf("MigrationVault.sol", "MigrationVault"); export const erc20Abi = abiOf("PEN.sol", "PEN"); + +export const governorAbi = abiOf("PENGovernor.sol", "PENGovernor"); +export const timelockAbi = abiOf("TimelockController.sol", "TimelockController"); diff --git a/testing/src/phase5b-governance.mjs b/testing/src/phase5b-governance.mjs new file mode 100644 index 000000000..790b8f024 --- /dev/null +++ b/testing/src/phase5b-governance.mjs @@ -0,0 +1,345 @@ +/** + * Phase 5b — governance rehearsal: Governor + Timelock on Base Sepolia. + * + * The Foundry suite covers the governance LOGIC (a full proposal lifecycle, + * threshold, quorum arithmetic, the timestamp clock). What had never run + * before this script: DeployGovernance.s.sol itself, the vault-admin -> + * timelock handover — whose acceptance is itself a governance proposal — and + * any proposal against deployed contracts on a public chain. + * + * Purely Base-side: no Zombienet and no daemon fleet. Voter PEN is released + * from the vault through the real 3-of-4 approve() path, driven directly by + * the attestor keys with synthetic (nonce, recipient, amount) tuples. + * + * Drill parameters are wall-clock-sized (see governanceDrillParams), and the + * quorum fraction is 0 so mechanics are testable with drill-scale voting + * power. Quorum SIZING is deliberately not rehearsed — production quorum is + * ~3M delegated PEN, unreachable here — and stays covered by the unit tests. + * + * Usage: + * node src/phase5b-governance.mjs full automated run + * node src/phase5b-governance.mjs --manual deploy + fund + verify wiring, + * then stop and print a handoff + * card for a by-hand walkthrough + * (Tally / Blockscout / MetaMask) + * SKIP_VERIFY=1 ... skip Blockscout verification + */ + +import { execSync } from "node:child_process"; +import { mkdirSync, writeFileSync } from "node:fs"; +import path from "node:path"; +import { encodeFunctionData, keccak256, parseEventLogs, toBytes } from "viem"; +import { erc20Abi, governorAbi, timelockAbi, vaultAbi } from "./abi.mjs"; +import { waitFor } from "./daemons.mjs"; +import { assert, assertEq, check, section, summarise } from "./harness.mjs"; +import { + assertTestnet, buildContext, loadEnv, PEN_12, PEN_18, ROOT, send, TESTING, +} from "./rehearsal-env.mjs"; +import { deployGovernanceToSepolia, deployToSepolia } from "./rehearsal-deploy.mjs"; + +const flags = new Set(process.argv.slice(2)); +const MANUAL = flags.has("--manual"); + +const started = new Date(); +const stamp = `${started.toISOString().replace(/[:.]/g, "-")}-governance`; +const runDir = path.join(TESTING, ".rehearsal", stamp); +const log = (m) => console.log(` ${m}`); +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + +/** OZ Governor proposal states, for readable assertions. */ +const STATE = { Pending: 0, Active: 1, Canceled: 2, Defeated: 3, Succeeded: 4, Queued: 5, Expired: 6, Executed: 7 }; +const stateName = (n) => Object.keys(STATE).find((k) => STATE[k] === Number(n)) ?? String(n); + +async function main() { + console.log(`Phase 5b — governance rehearsal (Base Sepolia)\n run ${stamp}`); + const env = loadEnv(); + const ctx = buildContext(env); + await assertTestnet(ctx, null); + + /** Throttle-tolerant read (same rationale as phase 5). */ + async function read(c, fn, args) { + let last; + for (let attempt = 0; attempt < 5; attempt++) { + try { + return await ctx.pub.readContract({ ...c, functionName: fn, args }); + } catch (error) { + last = error; + const text = `${error?.details ?? ""} ${error?.shortMessage ?? ""} ${error?.message ?? ""}`; + if (!/rate limit|too many requests|timeout|fetch failed|\b50[234]\b/i.test(text)) throw error; + await sleep(3000 * (attempt + 1)); + } + } + throw last; + } + const eventually = (fn, label, timeoutMs = 120_000) => + waitFor(async () => { try { await fn(); return true; } catch { return false; } }, + { timeoutMs, intervalMs: 3000, label }); + + // --- contracts ---------------------------------------------------------- + section("Deploy (token + vault)"); + const { vault, pen } = deployToSepolia({ env, roles: ctx.roles, log }); + log(`vault ${vault}\n PEN ${pen}`); + const V = { address: vault, abi: vaultAbi }; + const P = { address: pen, abi: erc20Abi }; + + await waitFor( + async () => (await read(V, "pendingAdmin", [])).toLowerCase() === ctx.roles.admin.address.toLowerCase(), + { timeoutMs: 120_000, intervalMs: 3000, label: "pendingAdmin to be visible" }); + await send(ctx, ctx.roles.admin, { ...V, functionName: "acceptAdmin", args: [] }); + await eventually( + async () => assertEq((await read(V, "admin", [])).toLowerCase(), ctx.roles.admin.address.toLowerCase(), "admin"), + "admin handover to settle"); + + await check("fresh vault has consumed none of the drill nonces", async () => { + assertEq(await read(V, "nonceConsumed", [0n]), false, "nonceConsumed(0)"); + }); + + // --- voter funding through the real release path ------------------------ + section("Voter funding via 3-of-4 release"); + // The releaser key doubles as the VOTER: it is gas-funded, unprivileged + // over the vault, and its production role is irrelevant here. + const voter = ctx.roles.releaser; + const releases = [ + { nonce: 0n, palletAmount: 10_000n * PEN_12 }, + { nonce: 1n, palletAmount: 8_000n * PEN_12 }, + ]; + await check("attestor keys release PEN to the voter (18k) through approve()", async () => { + for (const r of releases) { + for (const attestor of ctx.roles.attestors.slice(0, 3)) { + // Fixed gas: approve() is bimodal — the threshold-crossing call + // executes the release inline, and gas estimated against a node + // that has not yet seen the earlier approvals covers only the + // cheap record path. This is the same OutOfGas the attestor + // daemon pads for (GAS_LIMIT_MULTIPLIER); a direct caller must + // pad too. + await send(ctx, attestor, { ...V, functionName: "approve", args: [r.nonce, voter.address, r.palletAmount], gas: 500_000n }); + } + await eventually(async () => assertEq(await read(V, "nonceConsumed", [r.nonce]), true, `nonce ${r.nonce}`), + `release ${r.nonce} to settle`); + } + const balance = await read(P, "balanceOf", [voter.address]); + assertEq(balance, 18_000n * PEN_18, "voter PEN balance"); + }); + + // --- governance deployment ---------------------------------------------- + section("Governance deployment (first execution of DeployGovernance.s.sol)"); + const { timelock, governor, params } = deployGovernanceToSepolia({ env, pen, log }); + log(`timelock ${timelock}\n governor ${governor}`); + const G = { address: governor, abi: governorAbi }; + const T = { address: timelock, abi: timelockAbi }; + + await check("role wiring: governor proposes/cancels, execution is open, deployer is out", async () => { + const [proposerRole, cancellerRole, executorRole, adminRole] = await Promise.all([ + read(T, "PROPOSER_ROLE", []), read(T, "CANCELLER_ROLE", []), + read(T, "EXECUTOR_ROLE", []), read(T, "DEFAULT_ADMIN_ROLE", []), + ]); + // The renounce is the deploy's LAST transaction; retry until the reads + // see it rather than trusting the first node the RPC hands us. + await eventually(async () => { + assertEq(await read(T, "hasRole", [proposerRole, governor]), true, "governor is proposer"); + assertEq(await read(T, "hasRole", [cancellerRole, governor]), true, "governor is canceller"); + assertEq(await read(T, "hasRole", [executorRole, "0x0000000000000000000000000000000000000000"]), true, "execution open to anyone"); + assertEq(await read(T, "hasRole", [adminRole, ctx.roles.deployer.address]), false, "deployer admin renounced"); + assertEq(await read(T, "hasRole", [adminRole, timelock]), true, "timelock self-administered"); + }, "the role wiring (incl. the final renounce) to be visible"); + }); + + await check("governor references and clock are correct", async () => { + assertEq((await read(G, "token", [])).toLowerCase(), pen.toLowerCase(), "governor token"); + assertEq((await read(G, "timelock", [])).toLowerCase(), timelock.toLowerCase(), "governor timelock"); + assertEq(await read(G, "CLOCK_MODE", []), "mode=timestamp", "clock mode"); + assertEq(await read(G, "proposalThreshold", []), params.proposalThreshold, "proposal threshold"); + }); + + await check("voting power exists only after delegation", async () => { + const block = await ctx.pub.getBlock(); + const before = await read(G, "getVotes", [voter.address, block.timestamp - 1n]); + assertEq(before, 0n, "votes before delegation"); + await send(ctx, voter, { ...P, functionName: "delegate", args: [voter.address] }); + await eventually(async () => { + const now = (await ctx.pub.getBlock()).timestamp; + const votes = await read(G, "getVotes", [voter.address, now - 1n]); + assertEq(votes, 18_000n * PEN_18, "votes after delegation"); + }, "delegated power to appear"); + }); + + // --- proposal machinery -------------------------------------------------- + /** Run one proposal through its whole life: propose -> vote -> queue -> + * (assert the ETA gates) -> execute. Returns the proposal id. */ + /** Run one proposal through its whole life: propose -> vote -> queue -> + * (assert the ETA gates) -> execute. Returns the proposal id. + * + * Every step tolerates a lagging node behind the load-balanced RPC: a + * state() read for a proposal a node has not seen yet REVERTS + * (GovernorNonexistentProposal) rather than returning Pending, and a + * simulate against such a node rejects a perfectly valid vote — so state + * polls treat reverts as "not yet" and the writes retry. + */ + async function governanceExecute(description, target, calldata) { + const targets = [target]; + const values = [0n]; + const calldatas = [calldata]; + const descriptionHash = keccak256(toBytes(description)); + + const receipt = await send(ctx, voter, { ...G, functionName: "propose", args: [targets, values, calldatas, description] }); + const created = parseEventLogs({ abi: governorAbi, logs: receipt.logs }) + .find((e) => e.eventName === "ProposalCreated"); + assert(created, "propose succeeded but emitted no ProposalCreated event"); + const proposalId = created.args.proposalId; + log(`proposed "${description}" -> id ${proposalId} (tx ${receipt.transactionHash})`); + + const stateIs = async (want) => { + try { return Number(await read(G, "state", [proposalId])) === want; } catch { return false; } + }; + const sendRetrying = (acct, params, label, timeoutMs = 120_000) => + waitFor(async () => { + try { await send(ctx, acct, params); return true; } catch { return false; } + }, { timeoutMs, intervalMs: 5000, label }); + + await waitFor(() => stateIs(STATE.Active), + { timeoutMs: (params.votingDelay + 180) * 1000, intervalMs: 5000, label: `voting to open (${params.votingDelay}s delay)` }); + await sendRetrying(voter, { ...G, functionName: "castVote", args: [proposalId, 1] }, "the FOR vote to land"); + log("voted FOR"); + + await waitFor(() => stateIs(STATE.Succeeded), + { timeoutMs: (params.votingPeriod + 240) * 1000, intervalMs: 10_000, label: `voting to close (${params.votingPeriod}s period)` }); + await sendRetrying(voter, { ...G, functionName: "queue", args: [targets, values, calldatas, descriptionHash] }, "the queue to land"); + await waitFor(() => stateIs(STATE.Queued), + { timeoutMs: 120_000, intervalMs: 5000, label: "the queue to be visible" }); + log("queued into the timelock"); + + // The delay must actually gate: executing straight after queueing has to + // revert until the ETA passes. (Confirmed from the other side below by + // the same call SUCCEEDING once the delay has run out.) + let gated = false; + try { + await send(ctx, voter, { ...G, functionName: "execute", args: [targets, values, calldatas, descriptionHash] }); + } catch { gated = true; } + assert(gated, "execute succeeded BEFORE the timelock delay — the delay does not gate"); + log(`early execute reverted as required; waiting out the ${params.timelockDelay}s delay`); + + await sleep((params.timelockDelay + 15) * 1000); + await sendRetrying(voter, { ...G, functionName: "execute", args: [targets, values, calldatas, descriptionHash] }, "the execute to land"); + await waitFor(() => stateIs(STATE.Executed), + { timeoutMs: 120_000, intervalMs: 5000, label: "execution to be visible" }); + return proposalId; + } + + if (MANUAL) { + printHandoff({ vault, pen, timelock, governor, voter, params, releases }); + process.exit(summarise() ? 0 : 1); + } + + // --- the handover -------------------------------------------------------- + section("Vault-admin handover to the timelock, by proposal"); + let handoverProposal; + await check("transferAdmin(timelock), then a proposal executes acceptAdmin()", async () => { + await send(ctx, ctx.roles.admin, { ...V, functionName: "transferAdmin", args: [timelock] }); + await eventually(async () => assertEq((await read(V, "pendingAdmin", [])).toLowerCase(), timelock.toLowerCase(), "pendingAdmin"), + "transferAdmin to settle"); + handoverProposal = await governanceExecute( + "Accept MigrationVault admin (handover to governance)", + vault, + encodeFunctionData({ abi: vaultAbi, functionName: "acceptAdmin", args: [] }), + ); + assertEq((await read(V, "admin", [])).toLowerCase(), timelock.toLowerCase(), "vault admin is the timelock"); + }); + + // --- a real action through governance ------------------------------------ + section("A real admin action, end to end through governance"); + let capsProposal; + await check("a proposal raises the vault caps through the timelock", async () => { + const perRelease = await read(V, "perReleaseCap", []); + const daily = await read(V, "dailyCap", []); + capsProposal = await governanceExecute( + "Raise MigrationVault caps (drill)", + vault, + encodeFunctionData({ abi: vaultAbi, functionName: "setCaps", args: [perRelease * 2n, daily * 2n] }), + ); + assertEq(await read(V, "perReleaseCap", []), perRelease * 2n, "perReleaseCap"); + assertEq(await read(V, "dailyCap", []), daily * 2n, "dailyCap"); + }); + + // --- negatives ------------------------------------------------------------ + section("What must NOT work"); + await check("an account below the proposal threshold cannot propose", async () => { + let rejected = false; + try { + await send(ctx, ctx.roles.guardian, { + ...G, functionName: "propose", + args: [[vault], [0n], [encodeFunctionData({ abi: vaultAbi, functionName: "unpause", args: [] })], "no votes"], + }); + } catch { rejected = true; } + assert(rejected, "a zero-power account was able to propose"); + }); + + await check("the old admin EOA has lost its power over the vault", async () => { + let rejected = false; + try { + await send(ctx, ctx.roles.admin, { ...V, functionName: "setCaps", args: [1n * PEN_18, 1n * PEN_18] }); + } catch { rejected = true; } + assert(rejected, "the pre-handover admin can still change caps"); + }); + + await check("guardian can still pause; nobody can unpause without a proposal", async () => { + await send(ctx, ctx.roles.guardian, { ...V, functionName: "pause", args: [] }); + await eventually(async () => assertEq(await read(V, "paused", []), true, "paused"), "pause to settle"); + for (const [name, acct] of [["guardian", ctx.roles.guardian], ["old admin", ctx.roles.admin]]) { + let rejected = false; + try { await send(ctx, acct, { ...V, functionName: "unpause", args: [] }); } catch { rejected = true; } + assert(rejected, `${name} was able to unpause without governance`); + } + // NOTE deliberately left paused: unpausing now takes a full proposal + // (>= 48h in production) — that latency is the design's cost, and this + // is where it becomes tangible. + }); + + // --- record --------------------------------------------------------------- + mkdirSync(runDir, { recursive: true }); + writeFileSync(path.join(runDir, "manifest.json"), JSON.stringify({ + kind: "phase5b-governance", + startedAt: started.toISOString(), + commit: execSync("git rev-parse HEAD", { cwd: ROOT, encoding: "utf8" }).trim(), + contracts: { vault, pen, timelock, governor }, + voter: voter.address, + params: { ...params, proposalThreshold: params.proposalThreshold.toString() }, + proposals: { + handover: handoverProposal?.toString(), + capsRaise: capsProposal?.toString(), + }, + endState: "vault admin = timelock; vault left PAUSED (unpause requires a proposal)", + }, null, 2)); + console.log(`\n run artifacts: ${path.relative(ROOT, runDir)}`); + printHandoff({ vault, pen, timelock, governor, voter, params, releases }); + process.exit(summarise() ? 0 : 1); +} + +function printHandoff({ vault, pen, timelock, governor, voter, params }) { + console.log(` + ================= governance handoff card ================= + network Base Sepolia (chain 84532) + PEN token ${pen} + MigrationVault ${vault} + Timelock ${timelock} + Governor ${governor} + voter EOA ${voter.address} (the releaser key; holds 18k PEN) + + Explore (Blockscout, verified read/write tabs): + https://base-sepolia.blockscout.com/address/${governor} + https://base-sepolia.blockscout.com/address/${vault} + + Tally (one-time, needs your wallet): tally.xyz -> Add a DAO -> + network Base Sepolia, governor ${governor} + (token is auto-detected from the governor) + + By hand, remember the ORDER: delegate FIRST (power snapshots at + proposal creation), then propose -> wait ${params.votingDelay}s -> vote -> + wait out the ${params.votingPeriod}s period -> queue -> wait ${params.timelockDelay}s -> execute. + =========================================================== +`); +} + +main().catch((error) => { + console.error(`\nphase 5b aborted: ${error.message}`); + process.exit(1); +}); diff --git a/testing/src/rehearsal-deploy.mjs b/testing/src/rehearsal-deploy.mjs index 02326df3b..1904520f3 100644 --- a/testing/src/rehearsal-deploy.mjs +++ b/testing/src/rehearsal-deploy.mjs @@ -71,3 +71,53 @@ export function deployToSepolia({ env, roles, log }) { if (!vault || !pen) throw new Error("could not locate deployed addresses in the broadcast record"); return { vault, pen, params, sweepTs }; } + +/** Drill-scale governance timings: long enough that each stage is observable + * and the ETA gate can be asserted, short enough that two full proposal + * lifecycles fit in one sitting. Production values are validated by the + * Foundry suite; QUORUM_FRACTION=0 makes mechanics testable with drill-scale + * voting power — quorum SIZING is deliberately not rehearsed here. */ +export function governanceDrillParams(env) { + return { + timelockDelay: Number(env.GOV_TIMELOCK_DELAY ?? "180"), + votingDelay: Number(env.GOV_VOTING_DELAY ?? "60"), + votingPeriod: Number(env.GOV_VOTING_PERIOD ?? "240"), + proposalThreshold: BigInt(env.GOV_PROPOSAL_THRESHOLD_PEN ?? "1000") * PEN_18, + quorumFraction: Number(env.GOV_QUORUM_FRACTION ?? "0"), + }; +} + +/** Runs the real DeployGovernance.s.sol — its first execution anywhere. */ +export function deployGovernanceToSepolia({ env, pen, log }) { + const params = governanceDrillParams(env); + const scriptEnv = { + ...process.env, + PEN_TOKEN: pen, + TIMELOCK_DELAY: String(params.timelockDelay), + VOTING_DELAY: String(params.votingDelay), + VOTING_PERIOD: String(params.votingPeriod), + PROPOSAL_THRESHOLD: params.proposalThreshold.toString(), + QUORUM_FRACTION: String(params.quorumFraction), + }; + log(`deploying governance (timelock ${params.timelockDelay}s, voting ${params.votingDelay}s+${params.votingPeriod}s, quorum ${params.quorumFraction}%) ...`); + execFileSync( + "forge", + [ + "script", "script/DeployGovernance.s.sol", + "--rpc-url", env.BASE_SEPOLIA_RPC_URL, + "--broadcast", + "--private-key", env.DEPLOYER_PRIVATE_KEY, + "--slow", + ], + { cwd: CONTRACTS, env: scriptEnv, stdio: ["ignore", "pipe", "pipe"], maxBuffer: 64 * 1024 * 1024 }, + ); + const file = path.join( + CONTRACTS, "broadcast", "DeployGovernance.s.sol", String(BASE_SEPOLIA_CHAIN_ID), "run-latest.json", + ); + const run = JSON.parse(readFileSync(file, "utf8")); + const creations = run.transactions.filter((t) => t.transactionType === "CREATE"); + const timelock = creations.find((t) => t.contractName === "TimelockController")?.contractAddress; + const governor = creations.find((t) => t.contractName === "PENGovernor")?.contractAddress; + if (!timelock || !governor) throw new Error("could not locate governance addresses in the broadcast record"); + return { timelock, governor, params }; +} From 16ee8bf06c3e818d0699f9a8201692f5ee0f95b5 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 31 Aug 2026 16:23:22 +0200 Subject: [PATCH 38/61] docs: Document the governance rehearsal and its manual walkthrough --- docs/pen-migration-implementation-overview.md | 1 + docs/pen-migration-local-test-plan.md | 57 +++++++++++++++++++ 2 files changed, 58 insertions(+) diff --git a/docs/pen-migration-implementation-overview.md b/docs/pen-migration-implementation-overview.md index 57709ddc0..3ca1393b4 100644 --- a/docs/pen-migration-implementation-overview.md +++ b/docs/pen-migration-implementation-overview.md @@ -77,6 +77,7 @@ restricted to consumed nonces. Details and verified-not-vulnerable list in | `testing/src/phase4-zombienet.mjs` | 7/7 against a real relay (~2-block parachain finality lag) | | `testing/src/rehearsal.mjs` | 15/15 full stack: local Zombienet Pendulum + Base Sepolia | | `testing/src/drills.mjs` | 13/13 failure drills (RB-1/3/4/6/7) on the Sepolia stack | +| `testing/src/phase5b-governance.mjs` | 10/10 governance: DeployGovernance.s.sol first execution, vault-admin→timelock handover by proposal, caps raise through the timelock with the ETA gate asserted | | `testing/src/drill-rb5-upgrade.mjs` | 5/5 — the real spec-26 upgrade enacted under a live fleet on a mainnet fork | Seven internal adversarial review rounds have run; each found real issues diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 65a40b975..66930de12 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -344,6 +344,63 @@ mainnet. --- +## Phase 5b — governance rehearsal (Governor + Timelock on Base Sepolia) + +Goal: everything the Foundry suite cannot reach. The unit tests cover the +governance *logic* (full proposal lifecycle, threshold, quorum arithmetic, +timestamp clock); what had never run before this phase: `DeployGovernance.s.sol` +itself, the vault-admin → timelock **handover** — whose acceptance is itself a +governance proposal — and any proposal against deployed contracts on a public +chain. + +```bash +node testing/src/phase5b-governance.mjs # full automated run (~25 min) +node testing/src/phase5b-governance.mjs --manual # deploy + fund + wiring, then + # stop for a by-hand walkthrough +``` + +Purely Base-side — no Zombienet, no daemon fleet. Voter PEN is released +through the real 3-of-4 `approve()` path, driven directly by the attestor keys. +Checks: role wiring (governor proposes/cancels, execution open, deployer's +timelock admin renounced), delegation-gates-power, the handover by proposal, +a caps raise end to end through the timelock — **asserting execute reverts +before the ETA and succeeds after** — and the negatives: below-threshold +proposals rejected, the old admin powerless, the guardian able to pause but +nobody able to unpause without a proposal. The run deliberately ends with the +vault paused: post-handover, unpausing takes a full proposal (≥ 48h in +production) — that latency is the design's cost, made tangible. + +Drill parameters are wall-clock-sized (3 min timelock, 1+4 min voting, quorum +fraction 0). **Quorum sizing is deliberately not rehearsed** — production +quorum is ~3M delegated PEN — and stays covered by the unit tests, as do the +production timing values. + +### Manual walkthrough (after `--manual`, or against any run's contracts) + +Every run prints a handoff card and writes `manifest.json` with all addresses. +Contracts are verified on Blockscout (`base-sepolia.blockscout.com`), so every +address has working Read/Write tabs. To drive a proposal yourself: + +1. Import the voter key from `testing/.env.rehearsal` into MetaMask + (testnet-only keys) and add Base Sepolia. +2. Optionally add the DAO to Tally (tally.xyz → Add a DAO → Base Sepolia → + the governor address); Tally then handles delegate/propose/vote/queue/execute + as UI actions. +3. **Order matters**: delegate *first* — voting power snapshots at proposal + creation, and a proposal made before delegating can never pass. +4. Propose → wait the voting delay → vote → wait out the period → queue → + try executing early (it must revert) → execute after the timelock delay. + +Two traps the automated run hit that a manual run will too: `approve()` gas +must be set manually when releasing voter PEN (the threshold-crossing call +executes the release inline and estimates against the cheap path — the same +bimodal-gas bug the attestor daemon pads for), and reads immediately after +writes can hit a lagging node (a `state()` read for a fresh proposal *reverts* +`GovernorNonexistentProposal` on a node that has not seen it yet — retry +before concluding anything). + +--- + ## Phase 6 — Failure drills (the runbooks) Rehearse each runbook against a real stack, so the first time anyone runs them From ac1e77b7c4653cbe029fe63ab859f3ab87393d32 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Mon, 31 Aug 2026 17:56:23 +0200 Subject: [PATCH 39/61] docs: Add the review handover briefing for the next review round --- docs/pen-migration-review-handover.md | 162 ++++++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 docs/pen-migration-review-handover.md diff --git a/docs/pen-migration-review-handover.md b/docs/pen-migration-review-handover.md new file mode 100644 index 000000000..68b1d0b49 --- /dev/null +++ b/docs/pen-migration-review-handover.md @@ -0,0 +1,162 @@ +# PEN → Base Migration — Review Handover (round 8) + +A briefing for a fresh reviewer. Goal: find the bugs the previous rounds +missed. Everything below exists to make you effective fast — but treat every +claim in this document, and in every other doc, as **unverified**: the docs +were written by the same process that wrote the code. Verify against the code +at HEAD. + +## What you are reviewing + +One-way migration of the PEN token from the Pendulum parachain to Base. +Holders call `tokenMigration.migrate(amount, base_address)` on Pendulum; the +amount is burned and an event with a unique nonce is emitted. Four attestor +daemons watch relay-**finalized** blocks and submit +`approve(nonce, recipient, amount)` to a vault on Base; the 3rd matching +approval releases pre-minted fixed-supply ERC-20 PEN (12→18 decimals, ×1e6, +converted in exactly one place). Rate caps bound worst-case loss; an +independent monitor holds a conservation invariant and can auto-pause; a +permissionless releaser drains cap-deferred releases. Governance: OZ Governor ++ Timelock on Base, with the vault admin handed to the timelock by proposal. + +| Where | What | +|---|---| +| `pallets/token-migration/` + `runtime/pendulum/src/lib.rs` | Burn-and-emit pallet, runtime wiring (index 102, pause origin, BaseFilter) | +| `contracts/src/` | `PEN.sol`, `MigrationVault.sol`, `PENGovernor.sol` | +| `contracts/script/` | `Deploy.s.sol`, `DeployGovernance.s.sol` | +| `attestor/`, `monitor/`, `releaser/` | The three daemons (TypeScript, viem + polkadot-js) | +| `testing/` | Phases 1–6: Anvil, Chopsticks, Zombienet, Base Sepolia rehearsals + failure drills | +| Portal repo, branch `feat/pen-base-migration` (PR #655) | Migration UI (`src/pages/migration/`, `src/hooks/migration/`, `src/helpers/ethereum.ts`) | + +Branch: `feat/pen-to-base-migration` (PR #559), 38 commits over `main`. + +## Prior review history — read it first + +[pen-migration-internal-review.md](pen-migration-internal-review.md) is the +running log: seven adversarial rounds (2026-07-07…09), the Base Sepolia +rehearsal findings (2026-08-28), and a post-drills pass (2026-08-31). Per +round it records findings, fixes, and — important for you — the explicit +**"verified as not vulnerable"** lists. Rounds 3 and the rehearsal both found +bugs in *earlier rounds' fixes*, so do not treat a prior "fixed" as settled: +the fix commits are part of your attack surface. + +What each round already hammered (highest marginal value lies elsewhere): +replay/race/rotation on `approve()` (r1–r3), sweep-vs-pending accounting +(r1, r3, r4), monitor conservation and its false-positive/bricking modes +(r1, r5, r6), outsider griefing of the fleet (r6, r7), zero-address and +vault-as-recipient poison events (r2, r6), threshold-decrease retroactivity +(r4), rolling-cap semantics (r4). + +## The four bug classes that kept recurring + +Every real bug found post-unit-tests belongs to one of these. The most likely +"last remaining bug" is another instance of one of them: + +1. **A benign event misread as fatal takes the fleet down.** Four variants + found so far: the k-of-n race as crash (r1), zero-address decode (r2), + OutOfGas from bimodal gas estimation (`approve` records cheaply or + releases expensively depending on arrival order), and a lost race + "confirmed" against a lagging RPC node. Hunt for a fifth: any path where + `alert + exit` fires on something an adversary or ordinary timing can + trigger cheaply and repeatedly. +2. **Read-after-write assumptions against a load-balanced RPC.** No public + endpoint guarantees a read sees your confirmed write. Anything that + writes then reads once — daemons, runbooks, portal polling — is suspect. +3. **Bimodal/underestimated gas.** Any transaction whose execution path + depends on state that can change between estimation and inclusion. +4. **Execution/reporting splits in tooling.** Chopsticks executes one runtime + while reporting another; a benchmarks build silently swaps the embedded + wasm. Where else could the thing tested differ from the thing shipped? + +## Invariants to attack + +Try to falsify these directly — each is load-bearing: + +- `vault.balanceOf + totalReleased + totalSwept == PEN.totalSupply`, with + surplus tolerated and only deficit alarming. +- A nonce releases at most once, ever, across user AND treasury migrations + (one shared sequence); no tuple `(nonce, recipient, amount)` can be + released with different arguments than were approved. +- `activeApprovals` counts only current-generation, current-member approvals; + `hasApproved` must agree with it exactly (they disagreed once — r3/M1). +- `pendingApprovedAmount` reserves every quorum-approved-but-deferred release + against `sweepRemainder`, and can never underflow or leak permanently + (check `clearStalePending`'s restrictions). +- The pallet ships **paused** via a storage default with no migration writes; + nothing else in the runtime upgrade can flip it. +- Guardian can pause, never unpause; the two-step admin can never be skipped; + after handover the timelock delay gates every admin action. +- `earliestSweepTimestamp` is immutable; a threshold *decrease* arms a 7-day + sweep settling gate. +- Burn-side: total issuance decreases by exactly the migrated amount; ED/dust + rules can't strand or destroy funds; locked/vesting/staked balances cannot + migrate; minimum 100 PEN holds on both paths. +- Decimal conversion ×1e6 happens exactly once (vault, at release). Look for + any second place amounts are scaled — portal display, monitor math, + releaser, tests. + +## Where review soak is thinnest — prioritize these + +1. **The seven attestor/releaser commits since round 7** (`7c1b91c`, + `dcf5401`, `844f731`, `c7f4f0c`, `7037f6f`, `f0f64e4` and the transient + classifier): newest fund-release-path code, reviewed once, by the author. + Specifically: can `isTransientRpcError` misclassify anything fatal as + transient (silent-stall) or vice versa (fleet death)? Can the + `alreadyHandledSettled` backoff interact badly with checkpointing or the + serialized block-processing promise chain? +2. **The portal UI** — one full-diff pass (r4) only. Amount parsing and + decimal display, EIP-55 handling in `src/helpers/ethereum.ts`, the + payload-hash mirror of the vault's `abi.encode`, and what the status card + does on RPC lag or a deferred release. +3. **`migrate_treasury` / `set_treasury_destination`** — the governance-only + burn path; less exercised than user `migrate`. Check origin gating, the + fixed-destination logic, KeepAlive semantics against the real treasury + account. +4. **Monitor liveness (M4)** — `nonceFirstSeen` map growth, alert + deduplication, GRACE_SECONDS interaction with finality lag; r5/r7 touched + it twice, which historically predicts a third issue. +5. **Governance wiring** — `PENGovernor.sol` composition and + `DeployGovernance.s.sol` ran on-chain for the first time on 2026-08-31. + Quorum counts For+Abstain against **full** `totalSupply` (the unreleased + vault balance counts toward the denominator — PRD G1); check whether any + quorum/threshold interaction surprises at production numbers (150M supply, + 2% quorum, vault holding most of it early on). +6. **Runtime wiring** — pallet index 102, `BaseFilter` whitelist additions, + pause origin (root / half-council / 2/3 technical committee), locally + generated benchmark weights (are the weight/fee margins abusable?). + +## What is deliberately out of scope / accepted + +- No external audit is commissioned; residual risk is carried by caps, + monitoring + auto-pause, guardian, timelock, and the soft launch (see the + review log's final section). +- Attestors are team-operated (3-of-4) at launch. +- Quorum *sizing* and production governance timings are unit-tested, not + rehearsed; Chopsticks cannot submit post-upgrade extrinsics (see + `drill-rb5-upgrade.mjs` header) — reviewer beware when judging RB-5 claims. + +## Running things + +```bash +cargo test -p token-migration # 21 (22 with --features runtime-benchmarks) +cd contracts && forge test # 37 +cd attestor && npm test # 6 (monitor: 7, releaser: 7) +node testing/src/phase1-base.mjs # needs: anvil --port 8545 +node testing/src/phase2-pendulum.mjs # needs: chopsticks per docs/pen-migration-local-test-plan.md +``` + +Phases 5/5b/6 (Sepolia) need the funded, gitignored `testing/.env.rehearsal` +(throwaway keys; present on this machine). The full map with pass criteria is +[pen-migration-local-test-plan.md](pen-migration-local-test-plan.md); phase +scripts themselves are fair review targets — check that assertions actually +assert what their names claim. + +## Rules of engagement + +- Report findings as `file:line`, one sentence of defect, one concrete + failure scenario (inputs/state → wrong outcome). Severity by worst-case + fund impact first, fleet availability second. +- Adversarially verify your own findings before reporting — prior rounds' + false positives cost real time. +- The standing practice applies to you too: if your findings change the + fund-release path, that change itself needs a fresh pass. From fc29bc89191d46ef5f8a82c0770679203ac11baf Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:51:02 +0200 Subject: [PATCH 40/61] pallet: Make the treasury destination a one-time anchor set_treasury_destination now refuses to overwrite an existing destination (TreasuryDestinationAlreadySet). Once set, migrate_treasury can only ever send to that address, so a routine council-majority proposal cannot redirect treasury migrations; changing it requires root storage surgery. The weight gains the extra storage read and proof size. --- .../token-migration/src/default_weights.rs | 13 ++++++++----- pallets/token-migration/src/lib.rs | 6 ++++++ pallets/token-migration/src/tests.rs | 19 ++++++++++++++++--- 3 files changed, 30 insertions(+), 8 deletions(-) diff --git a/pallets/token-migration/src/default_weights.rs b/pallets/token-migration/src/default_weights.rs index 1f6f654f6..4e4d7a0a2 100644 --- a/pallets/token-migration/src/default_weights.rs +++ b/pallets/token-migration/src/default_weights.rs @@ -73,14 +73,16 @@ impl WeightInfo for SubstrateWeight { Weight::from_parts(4_000_000, 0) .saturating_add(T::DbWeight::get().writes(1_u64)) } - /// Storage: `TokenMigration::TreasuryDestination` (r:0 w:1) + /// Storage: `TokenMigration::TreasuryDestination` (r:1 w:1) /// Proof: `TokenMigration::TreasuryDestination` (`max_values`: Some(1), `max_size`: Some(20), added: 515, mode: `MaxEncodedLen`) fn set_treasury_destination() -> Weight { // Proof Size summary in bytes: // Measured: `0` - // Estimated: `0` - // Minimum execution time: 4_000_000 picoseconds. - Weight::from_parts(4_000_000, 0) + // Estimated: `515` + // Conservative until the next production-hardware regeneration: the + // one-time-set guard adds one storage read and its proof. + Weight::from_parts(6_000_000, 515) + .saturating_add(T::DbWeight::get().reads(1_u64)) .saturating_add(T::DbWeight::get().writes(1_u64)) } /// Storage: `TokenMigration::Paused` (r:1 w:0) @@ -116,7 +118,8 @@ impl WeightInfo for () { .saturating_add(RocksDbWeight::get().writes(1_u64)) } fn set_treasury_destination() -> Weight { - Weight::from_parts(4_000_000, 0) + Weight::from_parts(6_000_000, 515) + .saturating_add(RocksDbWeight::get().reads(1_u64)) .saturating_add(RocksDbWeight::get().writes(1_u64)) } fn migrate_treasury() -> Weight { diff --git a/pallets/token-migration/src/lib.rs b/pallets/token-migration/src/lib.rs index 56c9ff354..efd7a1b7f 100644 --- a/pallets/token-migration/src/lib.rs +++ b/pallets/token-migration/src/lib.rs @@ -110,6 +110,8 @@ pub mod pallet { /// A treasury migration was attempted before the Base destination was /// set via `set_treasury_destination`. NoTreasuryDestination, + /// The one-time treasury destination security anchor is already set. + TreasuryDestinationAlreadySet, } /// Nonce of the next migration. Monotonically increasing, never reused; @@ -218,6 +220,10 @@ pub mod pallet { pub fn set_treasury_destination(origin: OriginFor, base_address: H160) -> DispatchResult { T::TreasuryMigrateOrigin::ensure_origin(origin)?; ensure!(base_address != H160::zero(), Error::::InvalidBaseAddress); + ensure!( + !TreasuryDestination::::exists(), + Error::::TreasuryDestinationAlreadySet + ); TreasuryDestination::::put(base_address); Self::deposit_event(Event::TreasuryDestinationSet { base_address }); Ok(()) diff --git a/pallets/token-migration/src/tests.rs b/pallets/token-migration/src/tests.rs index f9a5f74ef..ef4d976a7 100644 --- a/pallets/token-migration/src/tests.rs +++ b/pallets/token-migration/src/tests.rs @@ -220,6 +220,20 @@ fn set_treasury_destination_stores_and_emits() { }); } +#[test] +fn set_treasury_destination_is_a_one_time_security_anchor() { + run_test(|| { + let original = base_address(); + let replacement = H160::repeat_byte(0xCD); + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), original)); + assert_noop!( + TokenMigration::set_treasury_destination(RuntimeOrigin::root(), replacement), + Error::::TreasuryDestinationAlreadySet + ); + assert_eq!(TreasuryDestination::::get(), Some(original)); + }); +} + #[test] fn set_treasury_destination_rejects_zero_and_bad_origin() { run_test(|| { @@ -328,13 +342,12 @@ fn migrate_treasury_keeps_treasury_alive() { #[test] fn migrate_treasury_respects_pause() { run_test(|| { - assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); assert_ok!(TokenMigration::set_paused(RuntimeOrigin::root(), true)); + // Initial configuration is still allowed while paused. + assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); assert_noop!( TokenMigration::migrate_treasury(RuntimeOrigin::root(), UNIT), Error::::MigrationsPaused ); - // Setting the destination is still allowed while paused (configuration). - assert_ok!(TokenMigration::set_treasury_destination(RuntimeOrigin::root(), base_address())); }); } From bb97e5be27981d63f4cf0528580d17113c88b3ab Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:51:03 +0200 Subject: [PATCH 41/61] attestor: Persist an identity-bound checkpoint atomically The checkpoint is written to a temporary file, fsynced and renamed into place (directory fsynced too), so a crash leaves the old or the new document, never a truncated one that reads as a first run. It records the Base chain id, vault address and Pendulum genesis hash; a mismatch or a malformed file is fatal rather than silently resetting progress. --- attestor/.gitignore | 2 + attestor/package.json | 2 +- attestor/src/state.test.ts | 47 +++++++++++++++++ attestor/src/state.ts | 103 +++++++++++++++++++++++++++++++++++++ 4 files changed, 153 insertions(+), 1 deletion(-) create mode 100644 attestor/src/state.test.ts create mode 100644 attestor/src/state.ts diff --git a/attestor/.gitignore b/attestor/.gitignore index 63e22d9f6..98d798331 100644 --- a/attestor/.gitignore +++ b/attestor/.gitignore @@ -2,3 +2,5 @@ node_modules/ dist/ checkpoint.json cp*.json +checkpoint.json.*.tmp +cp*.json.*.tmp diff --git a/attestor/package.json b/attestor/package.json index cd1f06d9a..659eca371 100644 --- a/attestor/package.json +++ b/attestor/package.json @@ -8,7 +8,7 @@ "build": "tsc", "start": "node dist/main.js", "typecheck": "tsc --noEmit", - "test": "tsc && node --test dist/checks.test.js" + "test": "tsc && node --test dist/*.test.js" }, "dependencies": { "@polkadot/api": "^11.3.1", diff --git a/attestor/src/state.test.ts b/attestor/src/state.test.ts new file mode 100644 index 000000000..293e6e686 --- /dev/null +++ b/attestor/src/state.test.ts @@ -0,0 +1,47 @@ +import assert from "node:assert/strict"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; +import { assertCheckpointIdentity, loadCheckpoint, saveCheckpoint, type Checkpoint } from "./state.js"; + +const checkpoint: Checkpoint = { + version: 1, + baseChainId: 8453, + vaultAddress: "0x1111111111111111111111111111111111111111", + pendulumGenesisHash: `0x${"22".repeat(32)}`, + lastProcessedBlock: 42, +}; + +test("checkpoint replacement is readable and identity-bound", () => { + const directory = mkdtempSync(join(tmpdir(), "pen-attestor-state-")); + const path = join(directory, "checkpoint.json"); + try { + assert.equal(loadCheckpoint(path), undefined); + saveCheckpoint(path, checkpoint); + const loaded = loadCheckpoint(path); + assert.deepEqual(loaded, checkpoint); + assert.doesNotThrow(() => assertCheckpointIdentity(loaded!, checkpoint)); + assert.throws( + () => assertCheckpointIdentity(loaded!, { ...checkpoint, baseChainId: 84532 }), + /does not match/, + ); + assert.throws( + () => assertCheckpointIdentity(loaded!, { ...checkpoint, pendulumGenesisHash: `0x${"33".repeat(32)}` }), + /Pendulum genesis/, + ); + } finally { + rmSync(directory, { recursive: true, force: true }); + } +}); + +test("a malformed checkpoint fails loudly instead of resetting progress", () => { + const directory = mkdtempSync(join(tmpdir(), "pen-attestor-state-")); + const path = join(directory, "checkpoint.json"); + try { + writeFileSync(path, "{truncated"); + assert.throws(() => loadCheckpoint(path), /refusing to reset progress/); + } finally { + rmSync(directory, { recursive: true, force: true }); + } +}); diff --git a/attestor/src/state.ts b/attestor/src/state.ts new file mode 100644 index 000000000..309a859e8 --- /dev/null +++ b/attestor/src/state.ts @@ -0,0 +1,103 @@ +import { + closeSync, + fsyncSync, + openSync, + readFileSync, + renameSync, + unlinkSync, + writeFileSync, +} from "node:fs"; +import { dirname } from "node:path"; + +export interface Checkpoint { + version: 1; + baseChainId: number; + vaultAddress: `0x${string}`; + pendulumGenesisHash: `0x${string}`; + lastProcessedBlock: number; +} + +function isMissing(error: unknown): boolean { + return (error as NodeJS.ErrnoException)?.code === "ENOENT"; +} + +export function loadCheckpoint(path: string): Checkpoint | undefined { + let raw: string; + try { + raw = readFileSync(path, "utf8"); + } catch (error) { + if (isMissing(error)) return undefined; + throw new Error(`cannot read checkpoint ${path}`, { cause: error }); + } + + let value: unknown; + try { + value = JSON.parse(raw); + } catch (error) { + throw new Error(`checkpoint ${path} is not valid JSON; refusing to reset progress`, { cause: error }); + } + const checkpoint = value as Partial; + if ( + checkpoint.version !== 1 || + !Number.isSafeInteger(checkpoint.baseChainId) || + typeof checkpoint.vaultAddress !== "string" || + !/^0x[0-9a-fA-F]{40}$/.test(checkpoint.vaultAddress) || + typeof checkpoint.pendulumGenesisHash !== "string" || + !/^0x[0-9a-fA-F]{64}$/.test(checkpoint.pendulumGenesisHash) || + !Number.isSafeInteger(checkpoint.lastProcessedBlock) || + (checkpoint.lastProcessedBlock ?? -1) < -1 + ) { + throw new Error(`checkpoint ${path} has an invalid or legacy schema; refusing to reset progress`); + } + return checkpoint as Checkpoint; +} + +/** Replace a checkpoint durably, so a crash leaves the old or new complete + * document rather than a truncated file that looks like a first run. */ +export function saveCheckpoint(path: string, checkpoint: Checkpoint): void { + const temporary = `${path}.${process.pid}.tmp`; + let file: number | undefined; + try { + file = openSync(temporary, "w", 0o600); + writeFileSync(file, JSON.stringify(checkpoint)); + fsyncSync(file); + closeSync(file); + file = undefined; + renameSync(temporary, path); + const directory = openSync(dirname(path), "r"); + try { + fsyncSync(directory); + } finally { + closeSync(directory); + } + } catch (error) { + if (file !== undefined) closeSync(file); + try { + unlinkSync(temporary); + } catch (cleanupError) { + if (!isMissing(cleanupError)) console.error(`could not remove temporary checkpoint ${temporary}`, cleanupError); + } + throw new Error(`cannot durably write checkpoint ${path}`, { cause: error }); + } +} + +export function assertCheckpointIdentity( + checkpoint: Checkpoint, + expected: { baseChainId: number; vaultAddress: string; pendulumGenesisHash: string }, +): void { + if (checkpoint.baseChainId !== expected.baseChainId) { + throw new Error( + `checkpoint Base chain ${checkpoint.baseChainId} does not match configured chain ${expected.baseChainId}`, + ); + } + if (checkpoint.vaultAddress.toLowerCase() !== expected.vaultAddress.toLowerCase()) { + throw new Error( + `checkpoint vault ${checkpoint.vaultAddress} does not match configured vault ${expected.vaultAddress}`, + ); + } + if (checkpoint.pendulumGenesisHash.toLowerCase() !== expected.pendulumGenesisHash.toLowerCase()) { + throw new Error( + `checkpoint Pendulum genesis ${checkpoint.pendulumGenesisHash} does not match connected chain ${expected.pendulumGenesisHash}`, + ); + } +} From f49990c438f097412e3bf635aa1fa7d8cd309c01 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:51:03 +0200 Subject: [PATCH 42/61] attestor: Process at latest and trail the checkpoint at Base finality Blocks are processed against the latest Base state, so throughput stays at submission latency and losing the k-of-n race concludes benign from latest as before. The durable checkpoint advances separately: only through blocks whose every releasable event reads as handled at the safe/finalized boundary. Approvals that refuse to settle (reorged away) are re-submitted after a timeout with one alert, which also makes reorg recovery automatic. Transient-error classification is now structural (HttpRequestError status, TimeoutError, socket codes, walked through the cause chain) with a word-only text fallback, so a tuple label such as nonce=429 can no longer reclassify a genuine failure. State reads at the safe block outside a node's retained window are transient. A watchdog pages when no finalized head arrives; transient alerts are throttled; webhook payloads redact RPC URLs and time out. Numeric config is validated at startup. --- attestor/README.md | 30 +++++- attestor/src/checks.test.ts | 46 +++++++- attestor/src/checks.ts | 76 ++++++++++++- attestor/src/config.ts | 43 +++++++- attestor/src/main.ts | 208 +++++++++++++++++++++++++----------- 5 files changed, 327 insertions(+), 76 deletions(-) diff --git a/attestor/README.md b/attestor/README.md index 186c3ab7a..5392c1ff9 100644 --- a/attestor/README.md +++ b/attestor/README.md @@ -12,7 +12,13 @@ point. ## Non-negotiable operational rules (PRD A1–A5) 1. **Run your own Pendulum full node** and point `PENDULUM_WS` at it. Using a - public RPC means trusting that RPC with release authority. + public RPC means trusting that RPC with release authority. Run the node + with **`--state-pruning archive`** (or `archive-canonical`): the daemon + catches up block by block through historical state, and a default-pruned + node (256 blocks ≈ 51 min) cannot serve that after any daemon outage + longer than the pruning horizon — the daemon then wedges loudly on + restart. Recovery from that state is pointing `PENDULUM_WS` at an archive + node, never editing the checkpoint. 2. **Key isolation:** the attestor key signs only vault `approve` calls. Keep it in an HSM/KMS signer where possible; never reuse it elsewhere. The same address pays gas — keep it funded with Base ETH (the daemon alerts below @@ -37,6 +43,9 @@ point. | `MIN_GAS_BALANCE_WEI` | Low-gas alert threshold (default 0.01 ETH) | | `ALERT_WEBHOOK_URL` | Optional webhook receiving JSON alerts | | `BASE_CHAIN_ID` | Default 8453 (Base mainnet) | +| `BASE_FINALITY_TAG` | Base confirmation boundary the checkpoint waits for: `safe` (default) or `finalized` | +| `BASE_FINALITY_TIMEOUT_MS` | How long a block's approvals may stay outside that boundary before an alert + idempotent re-submission (default 15 min for `safe`, 45 min for `finalized`) | +| `HEAD_STALL_ALERT_MS` | Page when no finalized Pendulum head has arrived for this long (default 5 min): the daemon is push-driven, so a node that stops finalizing would otherwise idle undetected | ## Run @@ -66,10 +75,21 @@ WantedBy=multi-user.target ## Behavior details -- Blocks are processed strictly in order; the checkpoint advances only after - every event in a block is handled. A crash re-processes at most one block — - safe, because approvals are idempotent (`nonceConsumed`/`hasApproved` are - checked first, and duplicate submissions revert harmlessly). +- Blocks are processed strictly in order against the **latest** Base state + (submission, race detection), so throughput is submission latency and a + lost k-of-n race — the most ordinary event in the system — is a log line, + never an alert. The durable **checkpoint trails separately** at the Base + finality boundary: it advances past a block only once every releasable + event in it is resolved inside `safe`/`finalized` state. A crash therefore + re-processes only the blocks whose approvals were not yet durable — safe, + because approvals are idempotent (`nonceConsumed`/`hasApproved` are checked + first, and duplicate submissions revert harmlessly). If a block's approvals + refuse to settle (a reorg dropped them), they are re-submitted after + `BASE_FINALITY_TIMEOUT_MS` with an alert; the checkpoint never passes an + unsettled block. +- Checkpoints are atomically replaced, bound to the Pendulum genesis plus the + configured Base chain and vault, and malformed files are fatal. Never delete or replace one merely to + clear an alert; reconcile it against both chains first. - The daemon verifies at startup that its address is in the vault's attestor set and refuses to run otherwise. - After a Pendulum **runtime upgrade**, verify event decoding against the new diff --git a/attestor/src/checks.test.ts b/attestor/src/checks.test.ts index 6068d09e9..22e62bed7 100644 --- a/attestor/src/checks.test.ts +++ b/attestor/src/checks.test.ts @@ -10,7 +10,8 @@ import assert from "node:assert/strict"; import { test } from "node:test"; -import { isUnreleasable, ZERO_ADDRESS } from "./checks.js"; +import { HttpRequestError, TimeoutError } from "viem"; +import { isTransientRpcError, isUnreleasable, ZERO_ADDRESS } from "./checks.js"; const VAULT = "0x1111111111111111111111111111111111111111"; const NORMAL = "0x00000000000000000000000000000000deadbeef"; @@ -46,3 +47,46 @@ test("an unreleasable condition still wins when combined with a normal one", () assert.equal(isUnreleasable(ZERO_ADDRESS, 0n, VAULT), true); assert.equal(isUnreleasable(VAULT, 0n, VAULT), true); }); + +test("a fatal error whose label embeds nonce 429/502 stays fatal", () => { + // Nonces are sequential, so 429, 502, 503 and 504 all occur. An error text + // embedding them (every submission error carries the tuple label) must not + // be reclassified as an endpoint failure — the post-drills L1 class. + assert.equal( + isTransientRpcError(new Error("approve transaction reverted: 0xabc (nonce=429 recipient=0x11 amount=1000)")), + false, + ); + assert.equal(isTransientRpcError(new Error("unexpected state for nonce=502")), false); +}); + +test("HTTP-status endpoint failures are transient, matched structurally", () => { + assert.equal(isTransientRpcError(new HttpRequestError({ url: "https://rpc", status: 429 })), true); + assert.equal(isTransientRpcError(new HttpRequestError({ url: "https://rpc", status: 503 })), true); + assert.equal(isTransientRpcError(new HttpRequestError({ url: "https://rpc", status: 403 })), false); +}); + +test("transient causes are found anywhere in the error chain", () => { + const wrapped = new Error("request failed", { cause: new HttpRequestError({ url: "https://rpc", status: 502 }) }); + assert.equal(isTransientRpcError(wrapped), true); + assert.equal(isTransientRpcError(new TimeoutError({ body: {}, url: "https://rpc" })), true); +}); + +test("socket-level failures are transient via their structured code", () => { + const refused = new Error("connect failed") as Error & { code: string }; + refused.code = "ECONNREFUSED"; + assert.equal(isTransientRpcError(refused), true); +}); + +test("text-only transport failures (polkadot-js) are still recognized", () => { + assert.equal(isTransientRpcError(new Error("WebSocket is not connected")), true); + assert.equal(isTransientRpcError(new Error("disconnected from wss://node:443: 1006")), true); + assert.equal(isTransientRpcError(new Error("fetch failed")), true); + assert.equal(isTransientRpcError(new Error("socket hang up")), true); + // A safe-block state read outside the node's retained window heals as the + // safe head catches up; it must wait, not exit. + assert.equal(isTransientRpcError(new Error("missing trie node 0xabc (path ) ")), true); +}); + +test("a decode failure is never transient", () => { + assert.equal(isTransientRpcError(new Error("MigrationInitiated in block 7 has 5 fields, expected 4")), false); +}); diff --git a/attestor/src/checks.ts b/attestor/src/checks.ts index 3f5bceb13..3d871579f 100644 --- a/attestor/src/checks.ts +++ b/attestor/src/checks.ts @@ -1,15 +1,81 @@ /** - * Pure tuple-classification predicates for the attestor. + * Pure classification predicates for the attestor. * * Isolated from the chain-client plumbing in `main.ts` so they can be unit - * tested exhaustively (see checks.test.ts). The subtlety that has bitten a - * review round — which (nonce, recipient, amount) tuples the vault will - * *deterministically* reject — lives here, and must stay in exact lockstep with - * the input reverts at the top of `MigrationVault.approve()`. + * tested exhaustively (see checks.test.ts). Two subtleties that have bitten + * review rounds live here: which (nonce, recipient, amount) tuples the vault + * will *deterministically* reject (must stay in exact lockstep with the input + * reverts at the top of `MigrationVault.approve()`), and which failures are + * transport-level noise rather than statements about an event. */ +import { HttpRequestError, TimeoutError } from "viem"; + export const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000"; +/** HTTP statuses that signal endpoint pressure or transient upstream failure — + * they carry no information about the migration event itself. */ +const TRANSIENT_HTTP_STATUS = new Set([408, 429, 500, 502, 503, 504]); + +/** Node/undici socket-level failure codes (structured `error.code`). */ +const TRANSIENT_SOCKET_CODES = new Set([ + "ETIMEDOUT", + "ECONNRESET", + "ECONNREFUSED", + "ECONNABORTED", + "EAI_AGAIN", + "ENOTFOUND", + "EPIPE", + "UND_ERR_SOCKET", + "UND_ERR_CONNECT_TIMEOUT", +]); + +/** Word-anchored phrases for transports that surface failures only as message + * text (polkadot-js in particular). Bare numeric status codes are deliberately + * NOT matched: error messages embed tuple fields (`nonce=429 …`), so a + * digit-only pattern would reclassify a genuine failure at those nonces as + * transient (the post-drills L1 class). Numeric codes are matched structurally + * against `HttpRequestError.status` instead. */ +const TRANSIENT_TEXT = + /rate limit|too many requests|timeout|timed out|socket hang up|fetch failed|service unavailable|internal error|disconnected|websocket is not connected|connection (?:closed|refused|reset|terminated)|ETIMEDOUT|ECONNRESET|ECONNREFUSED|EAI_AGAIN|ENOTFOUND|EPIPE|missing trie node|historical state|state (?:is )?not available|state unavailable/i; +// The last four: a state read pinned at the `safe` block can fall outside a +// node's retained-state window while the batcher lags (safe trails latest by +// more blocks than the node keeps). It heals as the safe head catches up, so +// it is a wait, not a verdict on any event — and not a reason to crash-loop. + +/** + * Transport-level failures that say nothing about the migration itself. + * + * PRD A5 requires the daemon to die rather than silently skip an event, and a + * decode failure still does exactly that. But a rate limit or a dropped socket + * carries no information about the event, and exiting on one turns every + * transient RPC hiccup into an attestor outage. The checkpoint only advances + * once a block is durably handled, so leaving a block unprocessed is safe — + * the next finalized head simply re-processes it. + * + * Classification is structural first (error types and codes survive message + * rewording and cannot collide with values embedded in labels), with the text + * patterns as a fallback for string-only transports. + */ +export function isTransientRpcError(error: unknown): boolean { + let depth = 0; + for (let cause: unknown = error; cause && depth < 10; cause = (cause as { cause?: unknown }).cause, depth++) { + if (cause instanceof HttpRequestError && cause.status !== undefined && TRANSIENT_HTTP_STATUS.has(cause.status)) { + return true; + } + if (cause instanceof TimeoutError) return true; + const code = (cause as { code?: unknown }).code; + if (typeof code === "string" && TRANSIENT_SOCKET_CODES.has(code)) return true; + const texts = [ + (cause as { message?: unknown }).message, + (cause as { details?: unknown }).details, + (cause as { shortMessage?: unknown }).shortMessage, + ]; + if (texts.some((text) => typeof text === "string" && TRANSIENT_TEXT.test(text))) return true; + } + return false; +} + /** * True when the vault will deterministically reject this tuple, no matter who * submits it or when. Such an event must be SKIPPED (with a critical alert), diff --git a/attestor/src/config.ts b/attestor/src/config.ts index deb81d6b1..b90cef0b3 100644 --- a/attestor/src/config.ts +++ b/attestor/src/config.ts @@ -6,9 +6,30 @@ function required(name: string): string { return value; } +/** A numeric environment value, validated at startup: a typo must fail loudly + * here, not surface later as NaN-driven behavior (a NaN timeout, for + * instance, silently turns a backoff into a busy loop). */ +function envNumber(name: string, fallback: number): number { + const raw = process.env[name]; + if (raw === undefined || raw === "") return fallback; + const value = Number(raw); + if (!Number.isFinite(value) || value < 0) throw new Error(`invalid ${name}: ${raw}`); + return value; +} + +function finalityTag(): "safe" | "finalized" { + const value = process.env.BASE_FINALITY_TAG ?? "safe"; + if (value !== "safe" && value !== "finalized") throw new Error(`invalid BASE_FINALITY_TAG ${value}`); + return value; +} + +const baseFinalityTag = finalityTag(); + export const config = { /** WebSocket endpoint of THIS OPERATOR'S OWN Pendulum full node (PRD A1). - * Never point this at a public RPC: the attestor would inherit its honesty. */ + * Never point this at a public RPC: the attestor would inherit its honesty. + * The node must retain historical state (`--state-pruning archive`), or any + * daemon outage longer than the pruning horizon wedges the catch-up. */ pendulumWs: required("PENDULUM_WS"), /** Base JSON-RPC endpoint. */ baseRpcUrl: required("BASE_RPC_URL"), @@ -17,15 +38,29 @@ export const config = { /** This attestor's transaction-signing key (0x-prefixed, 32 bytes). * Isolate per operator; fund with Base ETH for gas (PRD A3). */ attestorPrivateKey: required("ATTESTOR_PRIVATE_KEY") as `0x${string}`, - /** File persisting the last fully processed finalized block (PRD A2). */ + /** File persisting the last durably processed finalized block (PRD A2). */ checkpointFile: process.env.CHECKPOINT_FILE ?? "./checkpoint.json", /** Pendulum block to start from on the very first run (the block of the * runtime upgrade that added the token-migration pallet). */ - startBlock: Number(process.env.START_BLOCK ?? "0"), + startBlock: envNumber("START_BLOCK", 0), /** Alert when the gas wallet drops below this balance (wei). */ minGasBalanceWei: BigInt(process.env.MIN_GAS_BALANCE_WEI ?? "10000000000000000"), // 0.01 ETH /** Optional webhook that receives JSON alerts (low gas, fatal errors). */ alertWebhookUrl: process.env.ALERT_WEBHOOK_URL, /** Base chain id: 8453 mainnet. */ - baseChainId: Number(process.env.BASE_CHAIN_ID ?? "8453"), + baseChainId: envNumber("BASE_CHAIN_ID", 8453), + /** Confirmation boundary required before the checkpoint may pass a block. */ + baseFinalityTag, + /** How long a block's approvals may stay outside the finality boundary + * before they are alerted on and re-submitted (they are idempotent). The + * default tracks the boundary's normal lag: `finalized` trails `safe` by + * L1 finality, so it gets a proportionally longer default. */ + baseFinalityTimeoutMs: envNumber( + "BASE_FINALITY_TIMEOUT_MS", + baseFinalityTag === "finalized" ? 2_700_000 : 900_000, + ), + /** Page when no finalized Pendulum head has arrived for this long: the + * daemon is purely push-driven, so a node that stops finalizing (or a + * subscription that silently died) would otherwise idle undetected. */ + headStallAlertMs: envNumber("HEAD_STALL_ALERT_MS", 300_000), }; diff --git a/attestor/src/main.ts b/attestor/src/main.ts index fd2ddb18a..20f41e6f1 100644 --- a/attestor/src/main.ts +++ b/attestor/src/main.ts @@ -8,15 +8,19 @@ * * Design invariants: * - Only finalized blocks are read; blocks are processed strictly in order. - * - The checkpoint file is advanced only after every event in a block has - * been handled, so a crash re-processes at most one block (idempotent: - * duplicate approvals revert harmlessly and are skipped by the pre-check). + * - Processing acts on the LATEST Base state (submission, race detection); + * the durable checkpoint advances only once every releasable event of a + * block is resolved inside the configured Base `safe`/`finalized` boundary. + * Splitting the two keeps throughput at submission latency — while every + * lost k-of-n race stays what it is, the most ordinary event in the system, + * never an alert — yet a crash can only ever re-process blocks whose + * approvals were not durable, which is idempotent (duplicate approvals + * revert harmlessly and are skipped by the pre-check). * - A decode failure is FATAL by design (PRD A5): the daemon alerts and * exits rather than silently skipping an event; the checkpoint keeps the * failing block next in line for after the operator intervenes. */ -import { readFileSync, writeFileSync } from "node:fs"; import { ApiPromise, WsProvider } from "@polkadot/api"; import { createPublicClient, @@ -27,52 +31,43 @@ import { keccak256, } from "viem"; import { privateKeyToAccount } from "viem/accounts"; -import { isUnreleasable } from "./checks.js"; +import { isTransientRpcError, isUnreleasable } from "./checks.js"; import { config } from "./config.js"; +import { assertCheckpointIdentity, loadCheckpoint, saveCheckpoint, type Checkpoint } from "./state.js"; import { vaultAbi } from "./vaultAbi.js"; -interface Checkpoint { - lastProcessedBlock: number; -} - /** Multiplier applied to the estimated gas for `approve`. See the note at the * call site: the same call can take the cheap record path or the expensive * threshold-crossing release path. */ const GAS_LIMIT_MULTIPLIER = 4n; -/** - * Transport-level failures that say nothing about the migration itself. - * - * PRD A5 requires this daemon to die rather than silently skip an event, and a - * decode failure still does exactly that. But a rate limit or a dropped socket - * is not a decode failure: it carries no information about the event, and - * exiting on one turns every transient RPC hiccup into an attestor outage. The - * checkpoint is only advanced once a block is fully handled, so leaving the - * block unprocessed is safe — the next finalized head simply re-processes it. - */ -function isTransientRpcError(error: unknown): boolean { - const parts = [ - (error as { message?: string })?.message, - (error as { details?: string })?.details, - (error as { shortMessage?: string })?.shortMessage, - (error as { cause?: { details?: string } })?.cause?.details, - ]; - const text = parts.filter(Boolean).join(" "); - return /rate limit|too many requests|timeout|timed out|ETIMEDOUT|ECONNRESET|ECONNREFUSED|EAI_AGAIN|socket hang up|fetch failed|\b(?:429|50[234])\b|service unavailable|internal error/i - .test(text); -} - /** How persistently to confirm that a failed approval was merely a lost race * before treating it as fatal. See `alreadyHandledSettled`. */ const RACE_RECHECK_ATTEMPTS = 5; const RACE_RECHECK_DELAY_MS = 3000; +/** Repeated transient failures (a sustained RPC outage) page at most this + * often; every occurrence is still logged. */ +const TRANSIENT_ALERT_INTERVAL_MS = 60_000; + interface MigrationEvent { nonce: bigint; recipient: `0x${string}`; palletAmount: bigint; } +/** A block processed against the latest Base state whose releasable events are + * not yet confirmed inside the finality boundary. The checkpoint may not + * advance past it until they are. */ +interface UnconfirmedBlock { + block: number; + /** Events awaiting durability. Unreleasable tuples are excluded: they are + * permanently skipped (with a critical alert) and never resolve on Base. */ + events: MigrationEvent[]; + /** When this block's approvals were last (re-)submitted. */ + lastAttemptMs: number; +} + const baseChain = defineChain({ id: config.baseChainId, name: "base", @@ -92,6 +87,12 @@ function log(message: string, extra?: unknown): void { console.log(`${new Date().toISOString()} ${message}`, extra ?? ""); } +/** Strip URLs before anything leaves the process: RPC endpoints commonly embed + * API keys, and viem error texts quote the endpoint verbatim. */ +function redact(text: string): string { + return text.replace(/https?:\/\/[^\s"')]+/gi, ""); +} + async function alert(subject: string, detail: unknown): Promise { console.error(`${new Date().toISOString()} ALERT: ${subject}`, detail); if (!config.alertWebhookUrl) return; @@ -99,25 +100,14 @@ async function alert(subject: string, detail: unknown): Promise { await fetch(config.alertWebhookUrl, { method: "POST", headers: { "content-type": "application/json" }, - body: JSON.stringify({ service: "pen-attestor", attestor: account.address, subject, detail: `${detail}` }), + body: JSON.stringify({ service: "pen-attestor", attestor: account.address, subject, detail: redact(`${detail}`) }), + signal: AbortSignal.timeout(10_000), }); } catch (webhookError) { console.error("alert webhook failed", webhookError); } } -function loadCheckpoint(): Checkpoint { - try { - return JSON.parse(readFileSync(config.checkpointFile, "utf8")) as Checkpoint; - } catch { - return { lastProcessedBlock: config.startBlock - 1 }; - } -} - -function saveCheckpoint(checkpoint: Checkpoint): void { - writeFileSync(config.checkpointFile, JSON.stringify(checkpoint)); -} - function payloadHash(event: MigrationEvent): `0x${string}` { return keccak256( encodeAbiParameters( @@ -161,14 +151,14 @@ async function migrationEventsInBlock(api: ApiPromise, blockNumber: number): Pro } /** True when this migration no longer needs our approval (released, or we - * already approved). Rechecked after failures: with 4 attestors - * racing to the same event, losing the race is the NORMAL case, not an error. */ -async function alreadyHandled(event: MigrationEvent): Promise { + * already approved) at the given Base block, or at latest when omitted. */ +async function alreadyHandledAt(event: MigrationEvent, blockNumber?: bigint): Promise { const consumed = await publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "nonceConsumed", args: [event.nonce], + ...(blockNumber === undefined ? {} : { blockNumber }), }); if (consumed) return true; return publicClient.readContract({ @@ -176,9 +166,18 @@ async function alreadyHandled(event: MigrationEvent): Promise { abi: vaultAbi, functionName: "hasApproved", args: [payloadHash(event), account.address], + ...(blockNumber === undefined ? {} : { blockNumber }), }); } +/** Latest-state view: with 4 attestors racing to the same event, losing the + * race is the NORMAL case, not an error, and concluding that needs no + * finality — durability is enforced separately before the checkpoint moves + * (see `advanceCheckpoint`). */ +async function alreadyHandled(event: MigrationEvent): Promise { + return alreadyHandledAt(event); +} + /** * `alreadyHandled`, but tolerant of an RPC that has not caught up yet. * @@ -204,18 +203,21 @@ async function alreadyHandledSettled(event: MigrationEvent): Promise { return false; } -/** Submit the approval for one migration event, skipping work already done. */ -async function approve(event: MigrationEvent): Promise { +/** Submit the approval for one migration event, skipping work already done. + * Returns true when the event must later be confirmed durable at the Base + * finality boundary (submitted, raced, or already handled), false when it is + * permanently unreleasable and excluded from durability tracking. */ +async function approve(event: MigrationEvent): Promise { const label = `nonce=${event.nonce} recipient=${event.recipient} amount=${event.palletAmount}`; if (isUnreleasable(event.recipient, event.palletAmount, config.vaultAddress)) { await alert("CRITICAL: unreleasable migration event skipped permanently", label); - return; + return false; } if (await alreadyHandled(event)) { log(`skip (already released or approved): ${label}`); - return; + return true; } try { @@ -254,10 +256,11 @@ async function approve(event: MigrationEvent): Promise { // is a genuine failure and propagates to the fatal handler. if (await alreadyHandledSettled(event)) { log(`skip (raced, resolved on-chain): ${label}`); - return; + return true; } throw error; } + return true; } async function checkGasBalance(): Promise { @@ -280,31 +283,103 @@ async function main(): Promise { await checkGasBalance(); const api = await ApiPromise.create({ provider: new WsProvider(config.pendulumWs) }); - const checkpoint = loadCheckpoint(); + const pendulumGenesisHash = api.genesisHash.toHex(); + const checkpoint = loadCheckpoint(config.checkpointFile) ?? { + version: 1, + baseChainId: config.baseChainId, + vaultAddress: config.vaultAddress, + pendulumGenesisHash, + lastProcessedBlock: config.startBlock - 1, + } satisfies Checkpoint; + assertCheckpointIdentity(checkpoint, { ...config, pendulumGenesisHash }); log(`attestor ${account.address} starting after block ${checkpoint.lastProcessedBlock}`); + /** Blocks processed at the latest Base state, oldest first, that the + * checkpoint has not yet passed. Bounded by the Base finality lag. */ + const unconfirmed: UnconfirmedBlock[] = []; + let processedThrough = checkpoint.lastProcessedBlock; + let lastTransientAlertMs = 0; + + /** All releasable events of `entry` are resolved at `blockNumber` — either + * the nonce is consumed (released) or our approval is recorded there. */ + async function eventsDurableAt(entry: UnconfirmedBlock, blockNumber: bigint): Promise { + for (const event of entry.events) { + if (!(await alreadyHandledAt(event, blockNumber))) return false; + } + return true; + } + + /** Advance the durable checkpoint through every leading block whose events + * are resolved inside the Base finality boundary. A block that refuses to + * settle (our approval reorged out and nothing replaced it) is re-approved + * after `baseFinalityTimeoutMs` rather than waited on forever — the + * checkpoint cannot pass it until it settles, so nothing is ever skipped. */ + async function advanceCheckpoint(): Promise { + let confirmedNumber: bigint | undefined; + let advanced = false; + while (unconfirmed.length > 0) { + const head = unconfirmed[0]; + if (head.events.length > 0) { + if (confirmedNumber === undefined) { + const confirmed = await publicClient.getBlock({ blockTag: config.baseFinalityTag }); + if (confirmed.number === null) throw new Error(`${config.baseFinalityTag} Base block has no number`); + confirmedNumber = confirmed.number; + } + if (!(await eventsDurableAt(head, confirmedNumber))) { + if (Date.now() - head.lastAttemptMs >= config.baseFinalityTimeoutMs) { + await alert( + "approvals not durable at the Base finality boundary, re-submitting", + `Pendulum block ${head.block} (${head.events.length} event(s), boundary ${config.baseFinalityTag})`, + ); + head.lastAttemptMs = Date.now(); + for (const event of head.events) { + await approve(event); + } + } + break; + } + } + checkpoint.lastProcessedBlock = head.block; + unconfirmed.shift(); + advanced = true; + } + if (advanced) saveCheckpoint(config.checkpointFile, checkpoint); + } + + let lastHeadAt = Date.now(); + let lastHeadStallAlertMs = 0; let processing = Promise.resolve(); await api.rpc.chain.subscribeFinalizedHeads((head) => { + lastHeadAt = Date.now(); const finalized = head.number.toNumber(); // Serialize: a slow Base transaction must not let block processing overlap. processing = processing.then(async () => { - for (let block = checkpoint.lastProcessedBlock + 1; block <= finalized; block++) { + for (let block = processedThrough + 1; block <= finalized; block++) { const events = await migrationEventsInBlock(api, block); + const tracked: MigrationEvent[] = []; for (const event of events) { - await approve(event); + if (await approve(event)) tracked.push(event); } - checkpoint.lastProcessedBlock = block; - saveCheckpoint(checkpoint); + unconfirmed.push({ block, events: tracked, lastAttemptMs: Date.now() }); + processedThrough = block; } + await advanceCheckpoint(); }).catch(async (error) => { if (isTransientRpcError(error)) { - // Not a statement about the event — leave the checkpoint where it is - // and let the next finalized head re-process this block. - await alert("transient RPC failure, retrying on the next finalized head", error); + // Not a statement about any event — resume from the in-memory + // position on the next finalized head. Sustained outages page + // at a bounded rate instead of once per head. + if (Date.now() - lastTransientAlertMs >= TRANSIENT_ALERT_INTERVAL_MS) { + lastTransientAlertMs = Date.now(); + await alert("transient RPC failure, retrying on the next finalized head", error); + } else { + log("transient RPC failure, retrying on the next finalized head", error); + } return; } // PRD A5: never skip an event silently. Alert and exit; the process - // manager restarts us and the checkpoint retries the failing block. + // manager restarts us and the checkpoint retries from the last block + // whose approvals were durable. await alert("fatal error, exiting", error); process.exit(1); }); @@ -314,6 +389,17 @@ async function main(): Promise { () => void checkGasBalance().catch((error) => console.error("gas balance check failed", error)), 10 * 60 * 1000, ); + // Liveness watchdog for the push-driven loop above: silence is the one + // failure the subscription cannot report on its own. + setInterval(() => { + const silentMs = Date.now() - lastHeadAt; + if (silentMs < config.headStallAlertMs || Date.now() - lastHeadStallAlertMs < config.headStallAlertMs) return; + lastHeadStallAlertMs = Date.now(); + void alert( + "no finalized Pendulum head received", + `${Math.round(silentMs / 1000)}s since the last finalized head; the node may have stopped finalizing or the subscription silently died`, + ); + }, 60_000); } main().catch(async (error) => { From ce57ffdd0624c078bfe09b6533a2d3190f194d7e Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:51:03 +0200 Subject: [PATCH 43/61] releaser: Persist an identity-bound state file atomically Same durable-write and identity-binding scheme as the attestor checkpoint: temporary file, fsync, rename; bound to the Base chain id and vault; malformed state is fatal rather than a silent reset of the pending set. --- releaser/.gitignore | 1 + releaser/package.json | 2 +- releaser/src/state.test.ts | 40 ++++++++++++++ releaser/src/state.ts | 106 +++++++++++++++++++++++++++++++++++++ 4 files changed, 148 insertions(+), 1 deletion(-) create mode 100644 releaser/src/state.test.ts create mode 100644 releaser/src/state.ts diff --git a/releaser/.gitignore b/releaser/.gitignore index 03056135e..f14a59bc0 100644 --- a/releaser/.gitignore +++ b/releaser/.gitignore @@ -1,3 +1,4 @@ node_modules/ dist/ releaser-state.json +releaser-state.json.*.tmp diff --git a/releaser/package.json b/releaser/package.json index 6f4eda818..988ed1451 100644 --- a/releaser/package.json +++ b/releaser/package.json @@ -8,7 +8,7 @@ "build": "tsc", "start": "node dist/main.js", "typecheck": "tsc --noEmit", - "test": "tsc && node --test dist/checks.test.js" + "test": "tsc && node --test dist/*.test.js" }, "dependencies": { "viem": "^2.21.0" diff --git a/releaser/src/state.test.ts b/releaser/src/state.test.ts new file mode 100644 index 000000000..020671935 --- /dev/null +++ b/releaser/src/state.test.ts @@ -0,0 +1,40 @@ +import assert from "node:assert/strict"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; +import { assertStateIdentity, loadState, saveState, type PersistedState } from "./state.js"; + +const state: PersistedState = { + version: 1, + baseChainId: 8453, + vaultAddress: "0x1111111111111111111111111111111111111111", + fromBlock: "123", + pending: [{ nonce: "7", recipient: "0x2222222222222222222222222222222222222222", palletAmount: "99" }], +}; + +test("releaser state is atomically persisted and identity-bound", () => { + const directory = mkdtempSync(join(tmpdir(), "pen-releaser-state-")); + const path = join(directory, "state.json"); + try { + assert.equal(loadState(path), undefined); + saveState(path, state); + const loaded = loadState(path); + assert.deepEqual(loaded, state); + assert.doesNotThrow(() => assertStateIdentity(loaded!, state)); + assert.throws(() => assertStateIdentity(loaded!, { ...state, baseChainId: 1 }), /does not match/); + } finally { + rmSync(directory, { recursive: true, force: true }); + } +}); + +test("corrupt releaser state is fatal", () => { + const directory = mkdtempSync(join(tmpdir(), "pen-releaser-state-")); + const path = join(directory, "state.json"); + try { + writeFileSync(path, "{"); + assert.throws(() => loadState(path), /refusing to reset progress/); + } finally { + rmSync(directory, { recursive: true, force: true }); + } +}); diff --git a/releaser/src/state.ts b/releaser/src/state.ts new file mode 100644 index 000000000..9b69c2959 --- /dev/null +++ b/releaser/src/state.ts @@ -0,0 +1,106 @@ +import { + closeSync, + fsyncSync, + openSync, + readFileSync, + renameSync, + unlinkSync, + writeFileSync, +} from "node:fs"; +import { dirname } from "node:path"; + +export interface PersistedPendingRelease { + nonce: string; + recipient: string; + palletAmount: string; +} + +export interface PersistedState { + version: 1; + baseChainId: number; + vaultAddress: `0x${string}`; + /** Next safe/finalized Base block to scan `ReleasePending` from. */ + fromBlock: string; + pending: PersistedPendingRelease[]; +} + +function isMissing(error: unknown): boolean { + return (error as NodeJS.ErrnoException)?.code === "ENOENT"; +} + +export function loadState(path: string): PersistedState | undefined { + let raw: string; + try { + raw = readFileSync(path, "utf8"); + } catch (error) { + if (isMissing(error)) return undefined; + throw new Error(`cannot read releaser state ${path}`, { cause: error }); + } + let value: unknown; + try { + value = JSON.parse(raw); + } catch (error) { + throw new Error(`releaser state ${path} is not valid JSON; refusing to reset progress`, { cause: error }); + } + const state = value as Partial; + if ( + state.version !== 1 || + !Number.isSafeInteger(state.baseChainId) || + typeof state.vaultAddress !== "string" || + !/^0x[0-9a-fA-F]{40}$/.test(state.vaultAddress) || + typeof state.fromBlock !== "string" || + !/^\d+$/.test(state.fromBlock) || + !Array.isArray(state.pending) || + state.pending.some( + (entry) => + typeof entry?.nonce !== "string" || + !/^\d+$/.test(entry.nonce) || + typeof entry?.recipient !== "string" || + !/^0x[0-9a-fA-F]{40}$/.test(entry.recipient) || + typeof entry?.palletAmount !== "string" || + !/^\d+$/.test(entry.palletAmount), + ) + ) { + throw new Error(`releaser state ${path} has an invalid or legacy schema; refusing to reset progress`); + } + return state as PersistedState; +} + +export function assertStateIdentity( + state: PersistedState, + expected: { baseChainId: number; vaultAddress: string }, +): void { + if (state.baseChainId !== expected.baseChainId) { + throw new Error(`state Base chain ${state.baseChainId} does not match configured chain ${expected.baseChainId}`); + } + if (state.vaultAddress.toLowerCase() !== expected.vaultAddress.toLowerCase()) { + throw new Error(`state vault ${state.vaultAddress} does not match configured vault ${expected.vaultAddress}`); + } +} + +export function saveState(path: string, state: PersistedState): void { + const temporary = `${path}.${process.pid}.tmp`; + let file: number | undefined; + try { + file = openSync(temporary, "w", 0o600); + writeFileSync(file, JSON.stringify(state)); + fsyncSync(file); + closeSync(file); + file = undefined; + renameSync(temporary, path); + const directory = openSync(dirname(path), "r"); + try { + fsyncSync(directory); + } finally { + closeSync(directory); + } + } catch (error) { + if (file !== undefined) closeSync(file); + try { + unlinkSync(temporary); + } catch (cleanupError) { + if (!isMissing(cleanupError)) console.error(`could not remove temporary state ${temporary}`, cleanupError); + } + throw new Error(`cannot durably write releaser state ${path}`, { cause: error }); + } +} From 4a21164b487b776582e7c39c0209b3410177c193 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:51:03 +0200 Subject: [PATCH 44/61] releaser: Drop pending entries only at the finality boundary A successful release() no longer deletes its entry eagerly: it leaves the durable pending set only once pruneConsumed sees the nonce consumed at the safe/finalized block, so a reorged-away release is still ours to retry and no inline finality wait stalls the drain. A NonceAlreadyConsumed decoded from a latest-state simulation is treated the same way. A mined revert (no revert data) is re-evaluated next cycle instead of paging as unexpected; an amount above dailyCap itself is classified as blocked, not a refill wait; governance-blocked releases re-page at a bounded interval. Each log range probes its end block first so a lagging load-balanced node replays loudly instead of truncating silently. --- releaser/README.md | 23 +++- releaser/src/checks.test.ts | 41 +++++-- releaser/src/checks.ts | 42 +++++-- releaser/src/config.ts | 30 ++++- releaser/src/main.ts | 226 ++++++++++++++++++++++++++---------- releaser/src/vaultAbi.ts | 34 ++++++ 6 files changed, 308 insertions(+), 88 deletions(-) diff --git a/releaser/README.md b/releaser/README.md index 2b89312e0..fee6fe3df 100644 --- a/releaser/README.md +++ b/releaser/README.md @@ -29,7 +29,7 @@ consumed nonce and drops it. | Variable | Meaning | |---|---| -| `BASE_RPC_URL` | Base JSON-RPC endpoint | +| `BASE_RPC_URL` | Base JSON-RPC endpoint — prefer a single dedicated node: each scan range's end block is probed first so a lagging load-balanced replica causes a loud replay instead of a silently skipped `ReleasePending` | | `VAULT_ADDRESS` | MigrationVault address | | `RELEASER_PRIVATE_KEY` | Gas-only signing key (no privileges) | | `START_BLOCK` | Vault deployment block — where log scanning begins on a first run | @@ -39,6 +39,9 @@ consumed nonce and drops it. | `MIN_GAS_BALANCE_WEI` | Low-gas alert threshold | | `ALERT_WEBHOOK_URL` | Optional JSON alert webhook | | `BASE_CHAIN_ID` | Default 8453 | +| `BASE_FINALITY_TAG` | `safe` (default) or `finalized`; the scan cursor stays inside this boundary and a pending entry is dropped only once its nonce is consumed there | +| `BLOCKED_ALERT_INTERVAL_MS` | Re-page interval for governance-blocked releases (default 6h — the condition needs a ≥48h timelocked action, so per-poll paging would bury the signal) | +| `READ_BATCH_SIZE` | Bound on Multicall calldata and individual fallback concurrency (default 100) | ## Run @@ -56,13 +59,21 @@ Failures are classified rather than treated alike: | Revert | Outcome | |---|---| -| `ExceedsDailyCap`, `EnforcedPause`, `InsufficientVaultBalance`, `NotEnoughApprovals` | **Retry quietly** — self-heals; this is the normal backlog case | -| `NonceAlreadyConsumed` | **Done** — drop it | -| `ExceedsPerReleaseCap` | **Alert** — cannot self-heal, needs a governance `setCaps` behind the timelock | +| `ExceedsDailyCap`, `EnforcedPause`, `NotEnoughApprovals` | **Retry quietly** — expected temporary conditions | +| `NonceAlreadyConsumed` (at the finality boundary) | **Done** — drop it | +| `ExceedsPerReleaseCap`, `InsufficientVaultBalance` | **Alert, throttled** to `BLOCKED_ALERT_INTERVAL_MS` — cannot self-heal without governance or operator action | | anything else | **Alert** | -State (scan checkpoint + pending set) is persisted, so a restart resumes -without rescanning from the deployment block or losing pending work. +A successful `release()` does **not** drop the entry immediately: it leaves the +durable pending set only once the nonce reads as consumed at the +`safe`/`finalized` boundary, so a reorged-away release is still ours to retry. +Until then re-attempts are gas-free simulations classified as pending finality. + +State (safe/finalized scan checkpoint + pending set) is atomically persisted +and bound to the configured chain and vault, so a restart resumes without +trusting unsafe logs or losing pending work. Malformed state is fatal. Multicall +is disabled permanently only when an on-chain code check proves it absent; a +temporary provider error falls back in bounded batches for that cycle. **Known limitation:** the pending set is driven by `ReleasePending`, which the vault emits when a threshold is crossed inside `approve()`. A payload made diff --git a/releaser/src/checks.test.ts b/releaser/src/checks.test.ts index b7d046f6b..89b065df6 100644 --- a/releaser/src/checks.test.ts +++ b/releaser/src/checks.test.ts @@ -1,25 +1,40 @@ import { strict as assert } from "node:assert"; import { test } from "node:test"; -import { blockRanges, classifyReleaseFailure, toPalletAmount } from "./checks.js"; +import { ContractFunctionRevertedError, encodeErrorResult } from "viem"; +import { blockRanges, chunks, classifyReleaseFailure, contractErrorName, toPalletAmount } from "./checks.js"; +import { vaultAbi } from "./vaultAbi.js"; test("a cap-deferred release retries quietly — the case this service exists for", () => { - assert.equal(classifyReleaseFailure("ExceedsDailyCap(1000, 0)"), "retry"); - assert.equal(classifyReleaseFailure("EnforcedPause()"), "retry"); - assert.equal(classifyReleaseFailure("InsufficientVaultBalance()"), "retry"); - assert.equal(classifyReleaseFailure("NotEnoughApprovals(2, 3)"), "retry"); + assert.equal(classifyReleaseFailure("ExceedsDailyCap"), "retry"); + assert.equal(classifyReleaseFailure("EnforcedPause"), "retry"); + assert.equal(classifyReleaseFailure("NotEnoughApprovals"), "retry"); + assert.equal(classifyReleaseFailure("PendingFinality"), "retry"); }); test("a consumed nonce is done, not an error", () => { - assert.equal(classifyReleaseFailure("NonceAlreadyConsumed(42)"), "done"); + assert.equal(classifyReleaseFailure("NonceAlreadyConsumed"), "done"); }); test("the per-release ceiling cannot self-heal and is surfaced, not swallowed", () => { - assert.equal(classifyReleaseFailure("ExceedsPerReleaseCap(5000, 3000)"), "blocked"); + assert.equal(classifyReleaseFailure("ExceedsPerReleaseCap"), "blocked"); + assert.equal(classifyReleaseFailure("InsufficientVaultBalance"), "blocked"); }); test("anything unmodelled alerts a human", () => { - assert.equal(classifyReleaseFailure("TokenNotSet()"), "unexpected"); - assert.equal(classifyReleaseFailure("connection reset"), "unexpected"); + assert.equal(classifyReleaseFailure("TokenNotSet"), "unexpected"); + assert.equal(classifyReleaseFailure(undefined), "unexpected"); +}); + +test("viem decodes the real vault error ABI before classification", () => { + const data = encodeErrorResult({ abi: vaultAbi, errorName: "ExceedsDailyCap", args: [1000n, 0n] }); + const error = new ContractFunctionRevertedError({ abi: vaultAbi, data, functionName: "release" }); + assert.equal(contractErrorName(error), "ExceedsDailyCap"); + assert.equal(classifyReleaseFailure(contractErrorName(error)), "retry"); +}); + +test("individual fallbacks are split into bounded chunks", () => { + assert.deepEqual(chunks([1, 2, 3, 4, 5], 2), [[1, 2], [3, 4], [5]]); + assert.throws(() => chunks([1], 0), /invalid chunk size/); }); test("tokenAmount converts back to the exact palletAmount", () => { @@ -42,3 +57,11 @@ test("block ranges are chunked for RPC log limits, inclusive and gapless", () => // Nothing new to scan. assert.deepEqual(blockRanges(11n, 10n, 100n), []); }); + +test("an amount larger than the daily cap itself is blocked, not a refill wait", () => { + // The releaser synthesizes this name when ExceedsDailyCap is decoded for an + // amount above dailyCap: the allowance can never reach it, so it needs a + // governance setCaps rather than quiet retries. + assert.equal(classifyReleaseFailure("ExceedsDailyCapPermanently"), "blocked"); + assert.equal(classifyReleaseFailure("ExceedsDailyCap"), "retry"); +}); diff --git a/releaser/src/checks.ts b/releaser/src/checks.ts index 3d9965467..b531098fb 100644 --- a/releaser/src/checks.ts +++ b/releaser/src/checks.ts @@ -6,6 +6,8 @@ * and `monitor/src/checks.ts`. */ +import { BaseError, ContractFunctionRevertedError } from "viem"; + /** What to do with a pending release whose `release()` attempt failed. */ export type ReleaseOutcome = /** Resolved on-chain — drop it from the pending set. */ @@ -32,23 +34,49 @@ export type ReleaseOutcome = * - `NonceAlreadyConsumed` means somebody else got there first (another * releaser instance, or a conflicting tuple winning the nonce). Benign. */ -export function classifyReleaseFailure(reason: string): ReleaseOutcome { - if (reason.includes("NonceAlreadyConsumed")) return "done"; +export function contractErrorName(error: unknown): string | undefined { + if (error instanceof ContractFunctionRevertedError) return error.data?.errorName; + if (!(error instanceof BaseError)) return undefined; + const reverted = error.walk((cause) => cause instanceof ContractFunctionRevertedError); + return reverted instanceof ContractFunctionRevertedError ? reverted.data?.errorName : undefined; +} + +export function classifyReleaseFailure(errorName: string | undefined): ReleaseOutcome { + if (errorName === "NonceAlreadyConsumed") return "done"; if ( - reason.includes("ExceedsDailyCap") || - reason.includes("EnforcedPause") || - reason.includes("InsufficientVaultBalance") || + errorName === "ExceedsDailyCap" || + errorName === "EnforcedPause" || + errorName === "PendingFinality" || // The threshold can drop below the quorum again if an attestor is // removed after ReleasePending was emitted; a replacement approving // restores it. - reason.includes("NotEnoughApprovals") + errorName === "NotEnoughApprovals" ) { return "retry"; } - if (reason.includes("ExceedsPerReleaseCap")) return "blocked"; + if ( + errorName === "ExceedsPerReleaseCap" || + errorName === "InsufficientVaultBalance" || + // Synthetic: ExceedsDailyCap for an amount LARGER than dailyCap itself. + // The allowance can never reach it, so this is a governance setCaps + // matter, not a refill wait (the vault rejects such cap pairs since + // round 9, but a live vault deployed earlier may still carry one). + errorName === "ExceedsDailyCapPermanently" + ) { + return "blocked"; + } return "unexpected"; } +/** Divide work into bounded batches. Both Multicall calldata and individual + * fallback concurrency use the same explicit limit. */ +export function chunks(items: T[], size: number): T[][] { + if (!Number.isSafeInteger(size) || size <= 0) throw new Error(`invalid chunk size ${size}`); + const result: T[][] = []; + for (let start = 0; start < items.length; start += size) result.push(items.slice(start, start + size)); + return result; +} + /** * Convert the `tokenAmount` carried by `ReleasePending` back into the * `palletAmount` that `release()` expects. diff --git a/releaser/src/config.ts b/releaser/src/config.ts index 3c6fa3d23..daf7e9c71 100644 --- a/releaser/src/config.ts +++ b/releaser/src/config.ts @@ -6,6 +6,22 @@ function required(name: string): string { return value; } +/** A numeric environment value, validated at startup: a typo must fail loudly + * here, not surface later as NaN-driven behavior. */ +function envNumber(name: string, fallback: number): number { + const raw = process.env[name]; + if (raw === undefined || raw === "") return fallback; + const value = Number(raw); + if (!Number.isFinite(value) || value < 0) throw new Error(`invalid ${name}: ${raw}`); + return value; +} + +function finalityTag(): "safe" | "finalized" { + const value = process.env.BASE_FINALITY_TAG ?? "safe"; + if (value !== "safe" && value !== "finalized") throw new Error(`invalid BASE_FINALITY_TAG ${value}`); + return value; +} + export const config = { /** Base JSON-RPC endpoint. */ baseRpcUrl: required("BASE_RPC_URL"), @@ -20,11 +36,21 @@ export const config = { /** Base block to begin scanning `ReleasePending` from on a first run — * set to the vault's deployment block. */ startBlock: BigInt(process.env.START_BLOCK ?? "0"), - pollIntervalMs: Number(process.env.POLL_INTERVAL_MS ?? "60000"), + pollIntervalMs: envNumber("POLL_INTERVAL_MS", 60_000), /** Max blocks per `eth_getLogs` call; public RPCs cap this. */ maxBlockRange: BigInt(process.env.MAX_BLOCK_RANGE ?? "10000"), /** Alert when the gas wallet drops below this balance (wei). */ minGasBalanceWei: BigInt(process.env.MIN_GAS_BALANCE_WEI ?? "5000000000000000"), alertWebhookUrl: process.env.ALERT_WEBHOOK_URL, - baseChainId: Number(process.env.BASE_CHAIN_ID ?? "8453"), + baseChainId: envNumber("BASE_CHAIN_ID", 8453), + /** Only logs and state inside this Base confirmation boundary may advance + * durable state: ReleasePending ingestion scans up to it, and a pending + * entry is dropped only once its nonce is consumed there. */ + baseFinalityTag: finalityTag(), + /** How often a governance-blocked release (per-release cap exceeded, + * under-funded vault) re-pages. It cannot clear in less than the 48h + * timelock, so per-poll paging would only bury the signal. */ + blockedAlertIntervalMs: envNumber("BLOCKED_ALERT_INTERVAL_MS", 6 * 60 * 60 * 1000), + /** Bound both Multicall calldata and individual fallback concurrency. */ + readBatchSize: envNumber("READ_BATCH_SIZE", 100), }; diff --git a/releaser/src/main.ts b/releaser/src/main.ts index e17e95516..53a6be957 100644 --- a/releaser/src/main.ts +++ b/releaser/src/main.ts @@ -20,11 +20,16 @@ * concurrently; the loser of a race simply observes a consumed nonce. */ -import { readFileSync, writeFileSync } from "node:fs"; import { createPublicClient, createWalletClient, defineChain, http } from "viem"; import { privateKeyToAccount } from "viem/accounts"; -import { blockRanges, classifyReleaseFailure, toPalletAmount } from "./checks.js"; +import { blockRanges, chunks, classifyReleaseFailure, contractErrorName, toPalletAmount } from "./checks.js"; import { config } from "./config.js"; +import { + assertStateIdentity, + loadState as readPersistedState, + saveState as writePersistedState, + type PersistedState, +} from "./state.js"; import { vaultAbi } from "./vaultAbi.js"; interface PendingRelease { @@ -33,13 +38,6 @@ interface PendingRelease { palletAmount: bigint; } -interface PersistedState { - /** Next Base block to scan `ReleasePending` from. */ - fromBlock: string; - /** Pending set, persisted so a restart does not lose work already scanned. */ - pending: Array<{ nonce: string; recipient: string; palletAmount: string }>; -} - /** Canonical Multicall3, deployed at the same address on Base and every major * chain. viem refuses to batch unless the chain definition declares it, even * when the contract is present on-chain, so it has to be named here. */ @@ -53,8 +51,8 @@ const baseChain = defineChain({ contracts: { multicall3: { address: MULTICALL3 } }, }); -/** Set once Multicall3 turns out to be unavailable (a local devnet without the - * predeploy), after which reads fall back to one call per nonce. */ +/** Set only after an on-chain code check proves Multicall3 is absent. A + * transient provider error must never permanently change the scaling regime. */ let multicallUnavailable = false; const account = privateKeyToAccount(config.releaserPrivateKey); @@ -65,10 +63,27 @@ const walletClient = createWalletClient({ account, chain: baseChain, transport: const pending = new Map(); let fromBlock = config.startBlock; +/** nonce -> when its "blocked" state was last alerted. A blocked release needs + * a timelocked governance action (>= 48h) to clear, so re-paging it every + * poll would emit thousands of identical alerts and train on-call to ignore + * them; re-alert at a bounded interval instead. */ +const blockedAlertedAt = new Map(); + function log(message: string): void { console.log(`${new Date().toISOString()} ${message}`); } +/** Strip URLs before anything leaves the process: RPC endpoints commonly embed + * API keys, and viem error texts quote the endpoint verbatim. */ +function redact(text: string): string { + return text.replace(/https?:\/\/[^\s"')]+/gi, ""); +} + +/** A release transaction that was mined and reverted. The receipt carries no + * revert data, so the reason is unknowable here; the next cycle's simulation + * classifies it properly (and pruneConsumed drops it if a peer won). */ +class MinedRevert extends Error {} + async function alert(subject: string, detail: string): Promise { console.error(`${new Date().toISOString()} ALERT: ${subject} — ${detail}`); if (!config.alertWebhookUrl) return; @@ -76,7 +91,8 @@ async function alert(subject: string, detail: string): Promise { await fetch(config.alertWebhookUrl, { method: "POST", headers: { "content-type": "application/json" }, - body: JSON.stringify({ service: "pen-releaser", releaser: account.address, subject, detail }), + body: JSON.stringify({ service: "pen-releaser", releaser: account.address, subject, detail: redact(detail) }), + signal: AbortSignal.timeout(10_000), }); } catch (webhookError) { console.error("alert webhook failed", webhookError); @@ -84,23 +100,24 @@ async function alert(subject: string, detail: string): Promise { } function loadState(): void { - try { - const state = JSON.parse(readFileSync(config.stateFile, "utf8")) as PersistedState; - fromBlock = BigInt(state.fromBlock); - for (const p of state.pending) { - pending.set(BigInt(p.nonce), { - nonce: BigInt(p.nonce), - recipient: p.recipient as `0x${string}`, - palletAmount: BigInt(p.palletAmount), - }); - } - } catch { - // First run: start from the configured block with an empty set. + const state = readPersistedState(config.stateFile); + if (!state) return; + assertStateIdentity(state, config); + fromBlock = BigInt(state.fromBlock); + for (const p of state.pending) { + pending.set(BigInt(p.nonce), { + nonce: BigInt(p.nonce), + recipient: p.recipient as `0x${string}`, + palletAmount: BigInt(p.palletAmount), + }); } } function saveState(): void { const state: PersistedState = { + version: 1, + baseChainId: config.baseChainId, + vaultAddress: config.vaultAddress, fromBlock: fromBlock.toString(), pending: [...pending.values()].map((p) => ({ nonce: p.nonce.toString(), @@ -108,12 +125,19 @@ function saveState(): void { palletAmount: p.palletAmount.toString(), })), }; - writeFileSync(config.stateFile, JSON.stringify(state)); + writePersistedState(config.stateFile, state); } /** Scan for newly deferred releases and add them to the pending set. */ async function ingestNewPending(toBlock: bigint, conversionFactor: bigint): Promise { for (const [start, end] of blockRanges(fromBlock, toBlock, config.maxBlockRange)) { + // Probe the range's end block first. Against a load-balanced endpoint, + // `eth_getLogs` can be served by a node that has not reached `end` yet + // and some providers then silently truncate rather than error — which + // would advance the cursor past a ReleasePending we never saw, orphaning + // that deferral until a human notices the monitor's liveness alert. A + // lagging node fails this probe loudly instead, and the cycle replays. + await publicClient.getBlock({ blockNumber: end }); const logs = await publicClient.getContractEvents({ address: config.vaultAddress, abi: vaultAbi, @@ -142,50 +166,61 @@ async function ingestNewPending(toBlock: bigint, conversionFactor: bigint): Prom * the cycle. Before this fallback existed a missing predeploy threw on every * cycle that had anything pending -- which is precisely when the releaser * matters -- so it silently never drained a single deferred release. */ -async function readConsumed(nonces: bigint[]): Promise { +async function readConsumed(nonces: bigint[], blockNumber: bigint): Promise { const single = (nonce: bigint) => publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "nonceConsumed", args: [nonce], + blockNumber, }); - if (!multicallUnavailable) { - try { - const results = await publicClient.multicall({ - contracts: nonces.map((nonce) => ({ - address: config.vaultAddress, - abi: vaultAbi, - functionName: "nonceConsumed" as const, - args: [nonce] as const, - })), - allowFailure: true, - }); - return results.map((r) => r.status === "success" && r.result === true); - } catch (error) { - multicallUnavailable = true; - await alert( - "multicall unavailable, using per-nonce reads", - `verify Multicall3 at ${MULTICALL3}: ${error}`, - ); + const flags: boolean[] = []; + for (const batch of chunks(nonces, config.readBatchSize)) { + if (!multicallUnavailable) { + try { + const results = await publicClient.multicall({ + contracts: batch.map((nonce) => ({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "nonceConsumed" as const, + args: [nonce] as const, + })), + allowFailure: false, + blockNumber, + }); + flags.push(...(results as boolean[])); + continue; + } catch (error) { + await alert( + "multicall batch failed, using bounded per-nonce reads for this batch", + `${error}`, + ); + } } + flags.push(...(await Promise.all(batch.map(single)))); } - return Promise.all(nonces.map(single)); + return flags; } -/** Drop entries the vault has already consumed (by us, a peer, or a rival tuple). */ -async function pruneConsumed(): Promise { +/** Drop entries the vault has already consumed (by us, a peer, or a rival + * tuple) — read at the finality boundary, so an entry only ever leaves the + * durable pending set once its consumption cannot be reorged away. */ +async function pruneConsumed(blockNumber: bigint): Promise { const entries = [...pending.values()]; if (entries.length === 0) return; - const consumed = await readConsumed(entries.map((p) => p.nonce)); + const consumed = await readConsumed(entries.map((p) => p.nonce), blockNumber); entries.forEach((entry, i) => { - if (consumed[i]) pending.delete(entry.nonce); + if (consumed[i]) { + pending.delete(entry.nonce); + blockedAlertedAt.delete(entry.nonce); + } }); } /** Try to push each pending release through; classify what comes back. */ -async function drainPending(): Promise { +async function drainPending(conversionFactor: bigint, dailyCap: bigint): Promise { for (const p of [...pending.values()]) { const label = `nonce=${p.nonce} recipient=${p.recipient} amount=${p.palletAmount}`; try { @@ -200,26 +235,74 @@ async function drainPending(): Promise { const txHash = await walletClient.writeContract(request); const receipt = await publicClient.waitForTransactionReceipt({ hash: txHash }); if (receipt.status !== "success") { - throw new Error(`release reverted on-chain: ${txHash}`); + throw new MinedRevert(`release reverted on-chain: ${txHash}`); } - pending.delete(p.nonce); - log(`released: ${label} tx=${txHash}`); + // Deliberately NOT deleted here: the entry leaves the durable pending + // set only when `pruneConsumed` sees the nonce consumed at the + // finality boundary. Until then a re-attempt is a cheap simulation + // that classifies as PendingFinality and stays quiet — and if the + // release were reorged away, the entry is still ours to retry. + log(`released (awaiting ${config.baseFinalityTag}): ${label} tx=${txHash}`); } catch (error) { + if (error instanceof MinedRevert) { + // Usually a benign race (a peer released, or the daily allowance + // was consumed between simulation and inclusion). Stay quiet: the + // next cycle prunes it if consumed and re-simulates otherwise, + // which yields a decodable, classifiable reason. + log(`release reverted on-chain, re-evaluating next cycle: ${label}`); + continue; + } const reason = error instanceof Error ? error.message : String(error); - switch (classifyReleaseFailure(reason)) { + // A peer may have consumed the nonce after our simulation. Settle + // that race from state before relying on decoded custom errors. + const confirmed = await publicClient.getBlock({ blockTag: config.baseFinalityTag }); + if (confirmed.number === null) throw new Error(`${config.baseFinalityTag} Base block has no number`); + const consumedConfirmed = await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "nonceConsumed", + args: [p.nonce], + blockNumber: confirmed.number, + }); + const consumedLatest = consumedConfirmed ? true : await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "nonceConsumed", + args: [p.nonce], + }); + const decoded = contractErrorName(error); + // Only a consumption read at the finality boundary may drop the entry. + // A NonceAlreadyConsumed decoded from a LATEST-state simulation on a + // node ahead of the ones we read is the same pending-finality case. + const errorName = consumedConfirmed + ? "NonceAlreadyConsumed" + : consumedLatest || decoded === "NonceAlreadyConsumed" + ? "PendingFinality" + : decoded === "ExceedsDailyCap" && p.palletAmount * conversionFactor > dailyCap + ? "ExceedsDailyCapPermanently" + : decoded; + switch (classifyReleaseFailure(errorName)) { case "done": pending.delete(p.nonce); + blockedAlertedAt.delete(p.nonce); log(`skip (already consumed): ${label}`); break; case "retry": // Expected while the daily bucket refills — stay quiet. break; - case "blocked": - await alert( - "release blocked above the per-release cap", - `${label} — needs a governance setCaps to clear; it cannot self-heal`, - ); + case "blocked": { + // Clearing this needs a timelocked governance action, so the + // condition persists for days; page at a bounded interval. + const lastAlerted = blockedAlertedAt.get(p.nonce) ?? 0; + if (Date.now() - lastAlerted >= config.blockedAlertIntervalMs) { + blockedAlertedAt.set(p.nonce, Date.now()); + await alert( + "release blocked and needs operator action", + `${label} — ${errorName}; it cannot self-heal`, + ); + } break; + } case "unexpected": await alert("unexpected release failure", `${label} — ${reason}`); break; @@ -242,17 +325,32 @@ async function main(): Promise { abi: vaultAbi, functionName: "conversionFactor", }); + const multicallCode = await publicClient.getBytecode({ address: MULTICALL3 }); + if (!multicallCode || multicallCode === "0x") { + multicallUnavailable = true; + await alert( + "multicall unavailable, using bounded per-nonce reads", + `no contract code at ${MULTICALL3}`, + ); + } log(`releaser ${account.address} started; scanning from block ${fromBlock}, ${pending.size} pending`); await checkGasBalance(); for (;;) { try { - const latest = await publicClient.getBlockNumber(); - await ingestNewPending(latest, conversionFactor); - await pruneConsumed(); - await drainPending(); + const confirmed = await publicClient.getBlock({ blockTag: config.baseFinalityTag }); + if (confirmed.number === null) throw new Error(`${config.baseFinalityTag} Base block has no number`); + await ingestNewPending(confirmed.number, conversionFactor); + await pruneConsumed(confirmed.number); + const dailyCap = await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "dailyCap", + blockNumber: confirmed.number, + }); + await drainPending(conversionFactor, dailyCap); saveState(); - log(`ok: ${pending.size} pending, scanned through block ${fromBlock - 1n}`); + log(`ok: ${pending.size} pending, scanned ${config.baseFinalityTag} through block ${fromBlock - 1n}`); } catch (cycleError) { await alert("releaser cycle failed", `${cycleError}`); } diff --git a/releaser/src/vaultAbi.ts b/releaser/src/vaultAbi.ts index 60f34e2b0..532413166 100644 --- a/releaser/src/vaultAbi.ts +++ b/releaser/src/vaultAbi.ts @@ -1,5 +1,32 @@ /** Minimal MigrationVault ABI: only what the releaser needs. */ export const vaultAbi = [ + { type: "error", name: "NonceAlreadyConsumed", inputs: [{ name: "nonce", type: "uint64" }] }, + { + type: "error", + name: "NotEnoughApprovals", + inputs: [ + { name: "active", type: "uint256" }, + { name: "required", type: "uint256" }, + ], + }, + { type: "error", name: "EnforcedPause", inputs: [] }, + { + type: "error", + name: "ExceedsPerReleaseCap", + inputs: [ + { name: "amount", type: "uint256" }, + { name: "cap", type: "uint256" }, + ], + }, + { + type: "error", + name: "ExceedsDailyCap", + inputs: [ + { name: "requested", type: "uint256" }, + { name: "available", type: "uint256" }, + ], + }, + { type: "error", name: "InsufficientVaultBalance", inputs: [] }, { type: "event", name: "ReleasePending", @@ -34,4 +61,11 @@ export const vaultAbi = [ inputs: [], outputs: [{ type: "uint256" }], }, + { + type: "function", + name: "dailyCap", + stateMutability: "view", + inputs: [], + outputs: [{ type: "uint256" }], + }, ] as const; From 9da3dd11f03fa6f0b1a2e562640c15c119587387 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:51:04 +0200 Subject: [PATCH 45/61] monitor: Persist the two-chain cursor and pending tuples atomically The Pendulum cursor, the Base cursor, the unresolved canonical tuples with their finalized-block timestamps, the alert throttles and the source-supply anchor are written durably and bound to the Base chain, vault and Pendulum genesis. Malformed or mismatched state is fatal rather than a silent reset of the security history. --- monitor/.gitignore | 2 + monitor/package.json | 2 +- monitor/src/state.test.ts | 71 +++++++++++++++++++ monitor/src/state.ts | 140 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 214 insertions(+), 1 deletion(-) create mode 100644 monitor/src/state.test.ts create mode 100644 monitor/src/state.ts diff --git a/monitor/.gitignore b/monitor/.gitignore index b94707787..cbb90b2e6 100644 --- a/monitor/.gitignore +++ b/monitor/.gitignore @@ -1,2 +1,4 @@ node_modules/ dist/ +monitor-state.json +monitor-state.json.*.tmp diff --git a/monitor/package.json b/monitor/package.json index f1a8b893d..c9575d5be 100644 --- a/monitor/package.json +++ b/monitor/package.json @@ -8,7 +8,7 @@ "build": "tsc", "start": "node dist/main.js", "typecheck": "tsc --noEmit", - "test": "tsc && node --test dist/checks.test.js" + "test": "tsc && node --test dist/*.test.js" }, "dependencies": { "@polkadot/api": "^11.3.1", diff --git a/monitor/src/state.test.ts b/monitor/src/state.test.ts new file mode 100644 index 000000000..ba6f96c5b --- /dev/null +++ b/monitor/src/state.test.ts @@ -0,0 +1,71 @@ +import assert from "node:assert/strict"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; +import { + assertMonitorStateIdentity, + loadMonitorState, + saveMonitorState, + type PersistedMonitorState, +} from "./state.js"; + +const state: PersistedMonitorState = { + version: 1, + baseChainId: 8453, + vaultAddress: "0x1111111111111111111111111111111111111111", + pendulumGenesisHash: `0x${"33".repeat(32)}`, + lastPendulumBlock: 100, + nextExpectedNonce: "3", + baseFromBlock: "200", + migrations: [{ + nonce: "2", + recipient: "0x2222222222222222222222222222222222222222", + palletAmount: "100", + firstSeenMs: 1_700_000_000_000, + sourceBlock: 100, + }], + livenessAlertedAt: [["2", 1_700_000_001_000]], + unmatchedBaseFirstSeenAt: [["Approved:3", 1_700_000_002_000]], +}; + +test("monitor state survives a durable replacement with its security history", () => { + const directory = mkdtempSync(join(tmpdir(), "pen-monitor-state-")); + const path = join(directory, "state.json"); + try { + assert.equal(loadMonitorState(path), undefined); + saveMonitorState(path, state); + const loaded = loadMonitorState(path); + assert.deepEqual(loaded, state); + assert.doesNotThrow(() => assertMonitorStateIdentity(loaded!, state)); + assert.throws( + () => assertMonitorStateIdentity(loaded!, { ...state, pendulumGenesisHash: `0x${"44".repeat(32)}` }), + /Pendulum genesis/, + ); + } finally { + rmSync(directory, { recursive: true, force: true }); + } +}); + +test("corrupt monitor state fails rather than restarting grace timers", () => { + const directory = mkdtempSync(join(tmpdir(), "pen-monitor-state-")); + const path = join(directory, "state.json"); + try { + writeFileSync(path, "{"); + assert.throws(() => loadMonitorState(path), /refusing to reset security history/); + } finally { + rmSync(directory, { recursive: true, force: true }); + } +}); + +test("missing unmatched-event history is rejected rather than resetting its grace period", () => { + const directory = mkdtempSync(join(tmpdir(), "pen-monitor-state-")); + const path = join(directory, "state.json"); + try { + const { unmatchedBaseFirstSeenAt: _, ...incomplete } = state; + writeFileSync(path, JSON.stringify(incomplete)); + assert.throws(() => loadMonitorState(path), /invalid or legacy schema/); + } finally { + rmSync(directory, { recursive: true, force: true }); + } +}); diff --git a/monitor/src/state.ts b/monitor/src/state.ts new file mode 100644 index 000000000..caf936a90 --- /dev/null +++ b/monitor/src/state.ts @@ -0,0 +1,140 @@ +import { + closeSync, + fsyncSync, + openSync, + readFileSync, + renameSync, + unlinkSync, + writeFileSync, +} from "node:fs"; +import { dirname } from "node:path"; + +export interface PersistedMigration { + nonce: string; + recipient: `0x${string}`; + palletAmount: string; + firstSeenMs: number; + sourceBlock: number; +} + +export interface PersistedMonitorState { + version: 1; + baseChainId: number; + vaultAddress: `0x${string}`; + pendulumGenesisHash: `0x${string}`; + lastPendulumBlock: number; + nextExpectedNonce: string; + baseFromBlock: string; + migrations: PersistedMigration[]; + livenessAlertedAt: Array<[string, number]>; + unmatchedBaseFirstSeenAt: Array<[string, number]>; + /** Pendulum totalIssuance + TotalMigrated when first observed (round 9); + * absent in state written before it existed and re-anchored on load. */ + issuanceAnchor?: string; +} + +function isMissing(error: unknown): boolean { + return (error as NodeJS.ErrnoException)?.code === "ENOENT"; +} + +function isUint(value: unknown): value is string { + return typeof value === "string" && /^\d+$/.test(value); +} + +export function loadMonitorState(path: string): PersistedMonitorState | undefined { + let raw: string; + try { + raw = readFileSync(path, "utf8"); + } catch (error) { + if (isMissing(error)) return undefined; + throw new Error(`cannot read monitor state ${path}`, { cause: error }); + } + let value: unknown; + try { + value = JSON.parse(raw); + } catch (error) { + throw new Error(`monitor state ${path} is not valid JSON; refusing to reset security history`, { cause: error }); + } + const state = value as Partial; + const badMigration = !Array.isArray(state.migrations) || state.migrations.some((migration) => + !isUint(migration?.nonce) || + typeof migration?.recipient !== "string" || + !/^0x[0-9a-fA-F]{40}$/.test(migration.recipient) || + !isUint(migration?.palletAmount) || + !Number.isSafeInteger(migration?.firstSeenMs) || + !Number.isSafeInteger(migration?.sourceBlock), + ); + const badAlerts = !Array.isArray(state.livenessAlertedAt) || state.livenessAlertedAt.some( + (entry) => !Array.isArray(entry) || entry.length !== 2 || !isUint(entry[0]) || !Number.isSafeInteger(entry[1]), + ); + const badUnmatchedEvents = !Array.isArray(state.unmatchedBaseFirstSeenAt) || state.unmatchedBaseFirstSeenAt.some( + (entry) => + !Array.isArray(entry) || + entry.length !== 2 || + typeof entry[0] !== "string" || + !/^(Approved|Released):\d+$/.test(entry[0]) || + !Number.isSafeInteger(entry[1]), + ); + if ( + state.version !== 1 || + !Number.isSafeInteger(state.baseChainId) || + typeof state.vaultAddress !== "string" || + !/^0x[0-9a-fA-F]{40}$/.test(state.vaultAddress) || + typeof state.pendulumGenesisHash !== "string" || + !/^0x[0-9a-fA-F]{64}$/.test(state.pendulumGenesisHash) || + !Number.isSafeInteger(state.lastPendulumBlock) || + !isUint(state.nextExpectedNonce) || + !isUint(state.baseFromBlock) || + (state.issuanceAnchor !== undefined && !isUint(state.issuanceAnchor)) || + badMigration || + badAlerts || + badUnmatchedEvents + ) { + throw new Error(`monitor state ${path} has an invalid or legacy schema; refusing to reset security history`); + } + return state as PersistedMonitorState; +} + +export function assertMonitorStateIdentity( + state: { baseChainId: number; vaultAddress: string; pendulumGenesisHash: string }, + expected: { baseChainId: number; vaultAddress: string; pendulumGenesisHash: string }, +): void { + if (state.baseChainId !== expected.baseChainId) { + throw new Error(`state Base chain ${state.baseChainId} does not match configured chain ${expected.baseChainId}`); + } + if (state.vaultAddress.toLowerCase() !== expected.vaultAddress.toLowerCase()) { + throw new Error(`state vault ${state.vaultAddress} does not match configured vault ${expected.vaultAddress}`); + } + if (state.pendulumGenesisHash.toLowerCase() !== expected.pendulumGenesisHash.toLowerCase()) { + throw new Error( + `state Pendulum genesis ${state.pendulumGenesisHash} does not match connected chain ${expected.pendulumGenesisHash}`, + ); + } +} + +export function saveMonitorState(path: string, state: PersistedMonitorState): void { + const temporary = `${path}.${process.pid}.tmp`; + let file: number | undefined; + try { + file = openSync(temporary, "w", 0o600); + writeFileSync(file, JSON.stringify(state)); + fsyncSync(file); + closeSync(file); + file = undefined; + renameSync(temporary, path); + const directory = openSync(dirname(path), "r"); + try { + fsyncSync(directory); + } finally { + closeSync(directory); + } + } catch (error) { + if (file !== undefined) closeSync(file); + try { + unlinkSync(temporary); + } catch (cleanupError) { + if (!isMissing(cleanupError)) console.error(`could not remove temporary state ${temporary}`, cleanupError); + } + throw new Error(`cannot durably write monitor state ${path}`, { cause: error }); + } +} From 95e7e475e0dc5f86d4d5b5ac515b62130425fcd5 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:51:04 +0200 Subject: [PATCH 46/61] monitor: Reconcile every safe Base event against its finalized Pendulum burn Finalized MigrationInitiated events become the canonical records; every safe Base Approved and Released event must match one exactly (nonce, recipient, pallet amount, and the converted token amount). A mismatch against a known nonce pauses immediately. An unknown nonce pages on first sight, then pauses as soon as it is provable (the finalized source view has passed the moment the event was first observed, so the burn would already be ingested) or after a bounded grace if the source view stays behind; a stalling source view pages on its own beforehand. The pause runs before the alert webhook, never behind it. Aggregate conservation stays as independent defence. New guards: Pendulum totalIssuance + TotalMigrated is anchored and any growth pages (minted PEN would be honestly released from the fixed-supply vault); liveness also covers quorum-reached-but-unreleased migrations; unreleasable burns (zero or vault recipient) are paged once and excluded from the pending count; log ranges probe their end block; long catch-ups persist as they go. --- monitor/README.md | 43 +- monitor/src/checks.test.ts | 93 +++- monitor/src/checks.ts | 113 ++++- monitor/src/main.ts | 887 ++++++++++++++++++++++++++++++------- 4 files changed, 925 insertions(+), 211 deletions(-) diff --git a/monitor/README.md b/monitor/README.md index 2cf6c7323..f8a42e83c 100644 --- a/monitor/README.md +++ b/monitor/README.md @@ -8,22 +8,35 @@ Checks every poll: | Check | Meaning | Reaction | |---|---|---| -| M2a: `totalReleased <= TotalMigrated × conversionFactor` | Tokens may never leave the vault without a corresponding finalized burn on Pendulum. A violation is the signature of attestor-quorum compromise. | Alert + auto-pause the vault (if `GUARDIAN_PRIVATE_KEY` is set) | +| M2 tuple match | Every safe Base `Approved` and `Released` event must match one finalized Pendulum `MigrationInitiated` nonce, recipient, and pallet amount; `Released.tokenAmount` must equal `palletAmount × conversionFactor`. | Alert + auto-pause. A mismatch against a *known* nonce pauses immediately. An *unknown* nonce first pages (`UNVERIFIED BASE EVENT`), then pauses as soon as it is provable — the finalized source view passed the event's own timestamp and the nonce still does not exist — or after `UNMATCHED_EVENT_GRACE_SECONDS` if the source view stays behind (a stalled source also pages on its own, see below) | +| M2a aggregate | `totalReleased <= TotalMigrated × conversionFactor`. This remains independent defence-in-depth around the event reconciliation. | Alert + auto-pause | | M2b: `balanceOf(vault) + totalReleased + totalSwept >= totalSupply` | Vault-internal conservation. Only a **deficit** alerts: a surplus is a harmless inbound transfer (donation, or a migration whose recipient is the vault) and is ignored, so it cannot false-trigger a pause. | Alert + auto-pause on a deficit | -| M4: every nonce older than `GRACE_SECONDS` is consumed on Base | Liveness of the attestor fleet (outage, cap deferral, pause). Per-nonce reads are batched via Multicall3 so a large backlog cannot starve the checks above. | Alert | +| M4: every old migration reaches active approval quorum — and then releases | Liveness of the attestor fleet (`below approval quorum`), and of everything after it (`quorum reached but not released`: a release starved of daily allowance, paused, or made releasable by a threshold cut without a `ReleasePending`, which the releaser cannot see). | Alert, throttled per nonce | +| Source supply | `totalIssuance + TotalMigrated` on Pendulum must never grow past its first-seen anchor: minted PEN is burned honestly and drains the fixed-supply vault ahead of late migrators, satisfying every other check. | Alert (`SOURCE SUPPLY GREW`); pausing is a human decision | +| Unreleasable burns | A burn to the zero or vault address can never be approved or released; it is paged once and excluded from the pending count so it cannot page liveness forever or block RB-7. | Alert once | ## Configuration (environment) | Variable | Meaning | |---|---| -| `PENDULUM_WS` | WebSocket of the monitor's own Pendulum node | -| `BASE_RPC_URL` | Base JSON-RPC endpoint (ideally a different provider than the attestors use) | +| `PENDULUM_WS` | WebSocket of the monitor's own Pendulum node, run with `--state-pruning archive` (see below) | +| `BASE_RPC_URL` | Base JSON-RPC endpoint — a **single dedicated node**, and a different provider than the attestors use. Load-balanced pools can serve `eth_getLogs` from a lagging replica; each scan range is probed for its end block first, which turns a behind-node into a loud replay instead of a silent gap, but a dedicated node removes the hazard entirely | | `VAULT_ADDRESS` | MigrationVault address on Base | | `POLL_INTERVAL_MS` | Poll cadence (default 60s) | | `GRACE_SECONDS` | Liveness alert threshold (default 30 min) | +| `UNMATCHED_EVENT_GRACE_SECONDS` | Time a lagging Pendulum source view gets to reveal the burn behind a safe Base event before a cannot-verify pause (default 10 min). The first sighting of an unmatched event pages immediately, so this is the window operators have to repair a stalled node. Size it to your node-ops response time | +| `SOURCE_CLOCK_SKEW_MARGIN_SECONDS` | Clock-skew allowance for the fabrication proof (default 120s): once the finalized Pendulum view has passed the moment the monitor first observed an unmatched Base event by this margin and the nonce still does not exist, the event is provably fabricated and pauses without waiting out the grace. The anchor is the monitor's own clock, not the Base block timestamp (which trails real time after a sequencer outage); a future-dated source view proves nothing | +| `SOURCE_STALE_ALERT_SECONDS` | Age of the finalized Pendulum view (vs. wall clock) past which the monitor pages that it is going blind (default 300s) | +| `ISSUANCE_TOLERANCE` | Growth of Pendulum `totalIssuance + TotalMigrated` over its first-seen anchor (12-decimal pallet units) tolerated before `SOURCE SUPPLY GREW` pages (default 0). That sum cannot grow under any legitimate flow; growth means PEN was minted at the source and can be honestly burned against the fixed-supply vault | | `ALERT_WEBHOOK_URL` | Webhook receiving JSON alerts — wire this to paging | | `GUARDIAN_PRIVATE_KEY` | Optional: a guardian key enabling automatic `pause()` on conservation violations | | `BASE_CHAIN_ID` | Default 8453 | +| `PENDULUM_START_BLOCK` | **Required on first run:** first block to scan, normally the pallet activation block or the next finalized block before unpausing | +| `PENDULUM_START_NONCE` | Nonce expected at that block (default 0; set explicitly when starting after any migrations) | +| `BASE_START_BLOCK` | **Required on first run:** vault deployment block or an earlier block | +| `STATE_FILE` | Durable two-chain cursor, pending tuples, and liveness timestamps (default `./monitor-state.json`) | +| `BASE_FINALITY_TAG` | `safe` (default) or `finalized`; only events inside this boundary are trusted | +| `BASE_MAX_BLOCK_RANGE` | Maximum `eth_getLogs` range (default 10,000 blocks) | ## Run @@ -34,6 +47,22 @@ npm start ``` Run it under a process manager and treat "monitor down" itself as a paging -condition: an unwatched migration is the risk model failing silently. Note the -liveness state (`nonceFirstSeen`) is in-memory — after a restart, grace timers -restart from zero, which can only delay (never lose) a liveness alert. +condition: an unwatched migration is the risk model failing silently. The +two-chain cursors, unresolved canonical tuples, their original finalized block +timestamps, and alert throttles are atomically persisted. A malformed or +Pendulum-genesis/Base-chain/vault-mismatched state file is fatal rather than +silently resetting the security history. + +**Node requirements.** The Pendulum node must run with `--state-pruning +archive` (or `archive-canonical`): the monitor catches up block by block +through historical state, and a default-pruned node (256 blocks ≈ 51 min) +cannot serve that after any monitor outage longer than the pruning horizon — +the monitor then wedges loudly on restart. Recovery is pointing `PENDULUM_WS` +at an archive node, never editing the state file. A monitor whose *running* +source view stalls pages `PENDULUM SOURCE VIEW STALLED` well before the +unmatched-event grace can force a cannot-verify pause; treat that page as +"restore the node now". + +An automatic pause is reported as successful only after the transaction is +canonical inside the configured Base confirmation boundary and `paused()` is +true there. A merely-mined pause is reported as pending finality. diff --git a/monitor/src/checks.test.ts b/monitor/src/checks.test.ts index 91c0e4507..c4803013b 100644 --- a/monitor/src/checks.test.ts +++ b/monitor/src/checks.test.ts @@ -8,7 +8,17 @@ import assert from "node:assert/strict"; import { test } from "node:test"; -import { isStale, newNonces, releasedExceedsMigrated, vaultConservationDeficit } from "./checks.js"; +import { + approvalMismatch, + blockRanges, + chunks, + isStale, + provablyUnsourced, + releaseMismatch, + releasedExceedsMigrated, + shouldAwaitSource, + vaultConservationDeficit, +} from "./checks.js"; const CF = 1_000_000n; // 12 -> 18 decimals const SUPPLY = 150_000_000n * 10n ** 18n; @@ -50,30 +60,67 @@ test("M4: staleness respects the grace period", () => { assert.equal(isStale(now - 1801 * 1000, now, grace), true); // just past grace }); -test("M4: newNonces incorporates each nonce exactly once (no re-add of consumed)", () => { - // Round-7 regression: the old scan re-added every nonce 0..nextNonce each - // poll, so a consumed-and-pruned nonce was re-read via Multicall forever, - // making the scan O(all migrations) instead of O(pending backlog). - const firstSeen = new Map(); - let incorporated = 0n; +test("an unseen future source nonce gets bounded RPC-lag grace, never an old nonce", () => { + const now = 10_000_000; + assert.equal(shouldAwaitSource(8n, 8n, now, now, 600), true); + assert.equal(shouldAwaitSource(9n, 8n, now - 599_000, now, 600), true); + assert.equal(shouldAwaitSource(9n, 8n, now - 601_000, now, 600), false); + assert.equal(shouldAwaitSource(7n, 8n, now, now, 600), false); +}); - // Poll 1: five migrations exist — all freshly stamped. - for (const nonce of newNonces(incorporated, 5n)) firstSeen.set(nonce, 1_000); - incorporated = 5n; - assert.deepEqual([...firstSeen.keys()], [0n, 1n, 2n, 3n, 4n]); +test("a fabricated event is proven unsourced once the source view passes its first observation", () => { + const firstSeen = 10_000_000; + const margin = 120_000; + const now = firstSeen + 600_000; + // Source finalized view has moved past the moment the event was first + // observed (plus skew margin): the burn would already have been ingested — + // fabricated, pause without waiting out the grace. + assert.equal(provablyUnsourced(firstSeen, firstSeen + margin, now, margin), true); + assert.equal(provablyUnsourced(firstSeen, firstSeen + margin + 1, now, margin), true); + // A lagging source view trails the observation: NOT proof, so the RPC-lag + // grace applies and node lag can never fast-path a false pause. + assert.equal(provablyUnsourced(firstSeen, firstSeen + margin - 1, now, margin), false); + assert.equal(provablyUnsourced(firstSeen, firstSeen - 300_000, now, margin), false); +}); - // Nonces 0,1,2 get consumed on Base and are pruned from the pending set. - firstSeen.delete(0n); - firstSeen.delete(1n); - firstSeen.delete(2n); +test("a future-dated source view proves nothing", () => { + // A collator clock ahead of real time must not let the view "pass" an + // observation it has not genuinely caught up with. + const firstSeen = 10_000_000; + const margin = 120_000; + const now = firstSeen + 10_000; + assert.equal(provablyUnsourced(firstSeen, now + margin + 1, now, margin), false); + assert.equal(provablyUnsourced(firstSeen, now + margin, now, margin), true); +}); - // Poll 2: nextNonce unchanged — nothing to incorporate, and the consumed - // nonces must NOT reappear (the bug that this fix closes). - assert.deepEqual(newNonces(incorporated, 5n), []); - assert.deepEqual([...firstSeen.keys()], [3n, 4n]); +test("the proof anchor is the observation time, not a Base block timestamp that may trail real time", () => { + // An event observed now whose Base block timestamp trails real time by a + // sequencer catch-up: the view being past that OLD timestamp is not proof + // (the predicate never sees it), only being past the observation is. + const firstSeen = 10_000_000; + const margin = 120_000; + assert.equal(provablyUnsourced(firstSeen, firstSeen + 60_000, firstSeen + 60_000, margin), false); +}); + +test("M2 tuple matching rejects missing, wrong-recipient and wrong-amount approvals", () => { + const expected = { nonce: 7n, recipient: "0x1111111111111111111111111111111111111111", palletAmount: 100n }; + assert.match(approvalMismatch(undefined, expected)!, /no finalized/); + assert.match( + approvalMismatch(expected, { ...expected, recipient: "0x2222222222222222222222222222222222222222" })!, + /recipient/, + ); + assert.match(approvalMismatch(expected, { ...expected, palletAmount: 99n })!, /pallet amount/); + assert.equal(approvalMismatch(expected, { ...expected, recipient: expected.recipient.toUpperCase() }), undefined); +}); + +test("M2 release matching includes the one-and-only decimal conversion", () => { + const expected = { nonce: 7n, recipient: "0x1111111111111111111111111111111111111111", palletAmount: 100n }; + assert.equal(releaseMismatch(expected, { ...expected, tokenAmount: 100n * CF }, CF), undefined); + assert.match(releaseMismatch(expected, { ...expected, tokenAmount: 100n * CF + 1n }, CF)!, /token amount/); +}); - // Poll 3: two new migrations arrive — only those are freshly incorporated. - for (const nonce of newNonces(incorporated, 7n)) firstSeen.set(nonce, 2_000); - incorporated = 7n; - assert.deepEqual([...firstSeen.keys()], [3n, 4n, 5n, 6n]); +test("RPC work is split into inclusive block ranges and bounded batches", () => { + assert.deepEqual(blockRanges(1n, 10n, 4n), [[1n, 4n], [5n, 8n], [9n, 10n]]); + assert.deepEqual(blockRanges(11n, 10n, 4n), []); + assert.deepEqual(chunks([1, 2, 3, 4, 5], 2), [[1, 2], [3, 4], [5]]); }); diff --git a/monitor/src/checks.ts b/monitor/src/checks.ts index 5e7643de2..92082992b 100644 --- a/monitor/src/checks.ts +++ b/monitor/src/checks.ts @@ -55,21 +55,104 @@ export function isStale(firstSeenMs: number, nowMs: number, graceSeconds: number } /** - * Nonces observed for the first time this poll: the half-open range - * [incorporatedUpTo, nextNonce). The caller advances its high-water mark to - * `nextNonce` after stamping these, so each nonce enters the liveness set - * EXACTLY ONCE. A nonce already seen — and possibly since consumed and pruned - * from the pending set — is never re-added. + * A Base event whose nonce is at or beyond the source node's next nonce may be + * legitimate data observed through a temporarily faster RPC. Give that source + * view a bounded window to catch up. Older missing nonces cannot be explained + * by node lag and must fail immediately. + */ +export function shouldAwaitSource( + eventNonce: bigint, + nextExpectedNonce: bigint, + firstSeenMs: number, + nowMs: number, + graceSeconds: number, +): boolean { + return eventNonce >= nextExpectedNonce && !isStale(firstSeenMs, nowMs, graceSeconds); +} + +/** + * A Base event is PROVABLY without a Pendulum source once the monitor's + * finalized source view has passed the wall-clock moment the monitor FIRST + * OBSERVED the event (plus a clock-skew margin) and the nonce still does not + * exist. * - * This is what keeps the per-poll `nonceConsumed` scan O(pending backlog) - * rather than O(all migrations ever created). The earlier scan re-added every - * nonce `0..nextNonce` each poll (any nonce not currently in the map), so a - * consumed-and-pruned nonce was re-inserted and re-read via Multicall every - * poll forever — silently defeating the round-5 batching optimisation and - * letting the scan grow without bound for the whole migration window (round 7). + * Why this is sound: a legitimate release's burn is relay-FINALIZED strictly + * before any attestor submits an approval, so the burn block's timestamp + * precedes the real time at which the monitor could first observe that + * approval on Base. Substrate timestamps are strictly monotone, so a finalized + * head whose timestamp is past that observation time by more than any collator + * clock drift already contains every block the burn could live in. If the + * nonce is still unknown then, no amount of further waiting can reveal it — + * the event is fabricated, and the auto-pause must not sit out the RPC-lag + * grace period. + * + * The anchor is the monitor's own clock, deliberately NOT the Base block + * timestamp: OP-stack L2 block timestamps trail real time by the length of any + * sequencer outage while it catches up, and an artificially old event + * timestamp would let a merely-lagging source "prove" a legitimate event + * fabricated (review round 9). A lagging source never satisfies this predicate + * (its head timestamp trails the observation), and a future-dated source view + * (a collator clock ahead of real time) proves nothing either and is excluded, + * so neither can turn node lag into a false pause. */ -export function newNonces(incorporatedUpTo: bigint, nextNonce: bigint): bigint[] { - const fresh: bigint[] = []; - for (let nonce = incorporatedUpTo; nonce < nextNonce; nonce++) fresh.push(nonce); - return fresh; +export function provablyUnsourced( + firstSeenMs: number, + sourceFinalizedTsMs: number, + nowMs: number, + skewMarginMs: number, +): boolean { + if (sourceFinalizedTsMs > nowMs + skewMarginMs) return false; + return sourceFinalizedTsMs >= firstSeenMs + skewMarginMs; +} + +export interface MigrationTuple { + nonce: bigint; + recipient: string; + palletAmount: bigint; +} + +function sameAddress(left: string, right: string): boolean { + return left.toLowerCase() === right.toLowerCase(); +} + +export function approvalMismatch(expected: MigrationTuple | undefined, actual: MigrationTuple): string | undefined { + if (!expected) return `nonce ${actual.nonce} has no finalized Pendulum migration`; + if (!sameAddress(expected.recipient, actual.recipient)) { + return `nonce ${actual.nonce} recipient ${actual.recipient} != finalized ${expected.recipient}`; + } + if (expected.palletAmount !== actual.palletAmount) { + return `nonce ${actual.nonce} pallet amount ${actual.palletAmount} != finalized ${expected.palletAmount}`; + } + return undefined; +} + +export function releaseMismatch( + expected: MigrationTuple | undefined, + actual: MigrationTuple & { tokenAmount: bigint }, + conversionFactor: bigint, +): string | undefined { + const tupleMismatch = approvalMismatch(expected, actual); + if (tupleMismatch) return tupleMismatch; + const expectedTokenAmount = actual.palletAmount * conversionFactor; + if (actual.tokenAmount !== expectedTokenAmount) { + return `nonce ${actual.nonce} token amount ${actual.tokenAmount} != converted ${expectedTokenAmount}`; + } + return undefined; +} + +export function blockRanges(fromBlock: bigint, toBlock: bigint, maxRange: bigint): Array<[bigint, bigint]> { + if (maxRange <= 0n) throw new Error(`invalid maxRange ${maxRange}`); + const result: Array<[bigint, bigint]> = []; + for (let start = fromBlock; start <= toBlock; start += maxRange) { + const end = start + maxRange - 1n; + result.push([start, end > toBlock ? toBlock : end]); + } + return result; +} + +export function chunks(items: T[], size: number): T[][] { + if (!Number.isSafeInteger(size) || size <= 0) throw new Error(`invalid chunk size ${size}`); + const result: T[][] = []; + for (let start = 0; start < items.length; start += size) result.push(items.slice(start, start + size)); + return result; } diff --git a/monitor/src/main.ts b/monitor/src/main.ts index fdbcb921a..533620efb 100644 --- a/monitor/src/main.ts +++ b/monitor/src/main.ts @@ -1,29 +1,41 @@ /** - * PEN migration invariant monitor (PRD §6.5). + * Independent PEN migration invariant monitor (PRD §6.5). * - * Runs on infrastructure SEPARATE from every attestor and reads both chains - * independently. Each poll it checks, at the finalized head of Pendulum and - * the latest Base block: - * - * (M2a) totalReleased on Base <= TotalMigrated on Pendulum * conversionFactor - * (a violation means tokens were released that were never burned — - * the strongest possible signal of attestor compromise) - * (M2b) balanceOf(vault) + totalReleased == totalSupply - * (conservation inside the vault itself) - * (M4) liveness: every migration nonce older than GRACE_SECONDS is consumed - * on Base (detects a stalled attestor fleet) - * - * On an M2a violation the monitor alerts AND — when GUARDIAN_PRIVATE_KEY is - * configured (design option in PRD M3) — pauses the vault immediately. + * The monitor owns durable cursors on both chains. Finalized Pendulum + * MigrationInitiated events are the canonical nonce/recipient/amount records; + * every safe Base Approved and Released event must match one of those records + * exactly. Aggregate conservation remains a second, independent defence. */ import { ApiPromise, WsProvider } from "@polkadot/api"; -import { createPublicClient, createWalletClient, defineChain, http } from "viem"; +import { + createPublicClient, + createWalletClient, + defineChain, + encodeAbiParameters, + http, + keccak256, +} from "viem"; import { privateKeyToAccount } from "viem/accounts"; -import { isStale, newNonces, releasedExceedsMigrated, vaultConservationDeficit } from "./checks.js"; +import { + approvalMismatch, + blockRanges, + chunks, + isStale, + provablyUnsourced, + releaseMismatch, + releasedExceedsMigrated, + shouldAwaitSource, + type MigrationTuple, + vaultConservationDeficit, +} from "./checks.js"; +import { + assertMonitorStateIdentity, + loadMonitorState, + saveMonitorState, + type PersistedMonitorState, +} from "./state.js"; -// Canonical Multicall3 deployment (same address on Base and every major chain), -// used to batch the per-nonce liveness reads into a handful of RPC round-trips. const MULTICALL3_ADDRESS = "0xcA11bde05977b3631167028862bE2a173976CA11" as const; const vaultAbi = [ @@ -32,14 +44,35 @@ const vaultAbi = [ { type: "function", name: "conversionFactor", stateMutability: "view", inputs: [], outputs: [{ type: "uint256" }] }, { type: "function", name: "token", stateMutability: "view", inputs: [], outputs: [{ type: "address" }] }, { type: "function", name: "paused", stateMutability: "view", inputs: [], outputs: [{ type: "bool" }] }, + { type: "function", name: "threshold", stateMutability: "view", inputs: [], outputs: [{ type: "uint256" }] }, { type: "function", - name: "nonceConsumed", + name: "activeApprovals", stateMutability: "view", - inputs: [{ name: "nonce", type: "uint64" }], - outputs: [{ type: "bool" }], + inputs: [{ name: "payload", type: "bytes32" }], + outputs: [{ type: "uint256" }], }, { type: "function", name: "pause", stateMutability: "nonpayable", inputs: [], outputs: [] }, + { + type: "event", + name: "Approved", + inputs: [ + { name: "nonce", type: "uint64", indexed: true }, + { name: "recipient", type: "address", indexed: true }, + { name: "palletAmount", type: "uint256", indexed: false }, + { name: "attestor", type: "address", indexed: true }, + ], + }, + { + type: "event", + name: "Released", + inputs: [ + { name: "nonce", type: "uint64", indexed: true }, + { name: "recipient", type: "address", indexed: true }, + { name: "palletAmount", type: "uint256", indexed: false }, + { name: "tokenAmount", type: "uint256", indexed: false }, + ], + }, ] as const; const erc20Abi = [ @@ -59,19 +92,68 @@ function required(name: string): string { return value; } +/** A numeric environment value, validated at startup: a typo must fail loudly + * here, not surface later as NaN-driven behavior (a NaN grace, for instance, + * would silently disable the very checks this daemon exists for). */ +function envNumber(name: string, fallback: number | undefined): number { + const raw = process.env[name]; + if (raw === undefined || raw === "") { + if (fallback === undefined) throw new Error(`Missing required environment variable ${name}`); + return fallback; + } + const value = Number(raw); + if (!Number.isFinite(value) || value < 0) throw new Error(`invalid ${name}: ${raw}`); + return value; +} + +function finalityTag(): "safe" | "finalized" { + const value = process.env.BASE_FINALITY_TAG ?? "safe"; + if (value !== "safe" && value !== "finalized") throw new Error(`invalid BASE_FINALITY_TAG ${value}`); + return value; +} + +const baseFinalityTag = finalityTag(); + const config = { pendulumWs: required("PENDULUM_WS"), baseRpcUrl: required("BASE_RPC_URL"), vaultAddress: required("VAULT_ADDRESS") as `0x${string}`, - pollIntervalMs: Number(process.env.POLL_INTERVAL_MS ?? "60000"), - /** Seconds a migration may stay unreleased before a liveness alert (M4). */ - graceSeconds: Number(process.env.GRACE_SECONDS ?? "1800"), + pollIntervalMs: envNumber("POLL_INTERVAL_MS", 60_000), + graceSeconds: envNumber("GRACE_SECONDS", 1800), + unmatchedEventGraceSeconds: envNumber("UNMATCHED_EVENT_GRACE_SECONDS", 600), alertWebhookUrl: process.env.ALERT_WEBHOOK_URL, - /** Optional: enables auto-pause on a conservation violation (M3). */ guardianPrivateKey: process.env.GUARDIAN_PRIVATE_KEY as `0x${string}` | undefined, - baseChainId: Number(process.env.BASE_CHAIN_ID ?? "8453"), + baseChainId: envNumber("BASE_CHAIN_ID", 8453), + baseFinalityTag, + baseFinalityPollMs: envNumber("BASE_FINALITY_POLL_MS", 2000), + /** Bound on confirming the auto-pause inside the finality boundary; + * `finalized` trails `safe` by L1 finality, hence the longer default. */ + baseFinalityTimeoutMs: envNumber( + "BASE_FINALITY_TIMEOUT_MS", + baseFinalityTag === "finalized" ? 2_700_000 : 900_000, + ), + pendulumStartBlock: envNumber("PENDULUM_START_BLOCK", undefined), + pendulumStartNonce: BigInt(process.env.PENDULUM_START_NONCE ?? "0"), + baseStartBlock: BigInt(required("BASE_START_BLOCK")), + baseMaxBlockRange: BigInt(process.env.BASE_MAX_BLOCK_RANGE ?? "10000"), + readBatchSize: envNumber("READ_BATCH_SIZE", 100), + stateFile: process.env.STATE_FILE ?? "./monitor-state.json", + /** Cross-chain clock-skew allowance for `provablyUnsourced`: how far the + * finalized Pendulum view must pass a Base event's timestamp before a + * missing nonce counts as proof of fabrication rather than lag. */ + sourceClockSkewMarginMs: envNumber("SOURCE_CLOCK_SKEW_MARGIN_SECONDS", 120) * 1000, + /** Age of the finalized Pendulum view (vs. wall clock) past which the + * monitor pages that it is going blind — BEFORE any unmatched event can + * reach the grace deadline, so operators get the whole grace window to + * restore the node ahead of a cannot-verify auto-pause. */ + sourceStaleAlertMs: envNumber("SOURCE_STALE_ALERT_SECONDS", 300) * 1000, + /** Growth of (Pendulum totalIssuance + TotalMigrated) over its first-seen + * anchor, in 12-decimal pallet units, tolerated before paging. */ + issuanceToleranceRaw: BigInt(process.env.ISSUANCE_TOLERANCE ?? "0"), }; +const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000"; + const baseChain = defineChain({ id: config.baseChainId, name: "base", @@ -81,10 +163,122 @@ const baseChain = defineChain({ }); const publicClient = createPublicClient({ chain: baseChain, transport: http(config.baseRpcUrl) }); +interface MigrationRecord extends MigrationTuple { + firstSeenMs: number; + sourceBlock: number; +} + +interface RuntimeState { + baseChainId: number; + vaultAddress: `0x${string}`; + pendulumGenesisHash: `0x${string}` | undefined; + lastPendulumBlock: number; + nextExpectedNonce: bigint; + baseFromBlock: bigint; + migrations: Map; + livenessAlertedAt: Map; + unmatchedBaseFirstSeenAt: Map; + /** Pendulum totalIssuance + TotalMigrated at first observation. Under every + * legitimate flow this sum can only fall (teleport-out) and later recover + * to at most its original value (teleport-in); growth means PEN was minted + * at the source, which honest attestation would then release from the + * fixed-supply vault. Alert-only: the response is a human decision. */ + issuanceAnchor: bigint | undefined; +} + +function hydrateState(): RuntimeState { + const persisted = loadMonitorState(config.stateFile); + if (!persisted) { + return { + baseChainId: config.baseChainId, + vaultAddress: config.vaultAddress, + pendulumGenesisHash: undefined, + lastPendulumBlock: config.pendulumStartBlock - 1, + nextExpectedNonce: config.pendulumStartNonce, + baseFromBlock: config.baseStartBlock, + migrations: new Map(), + livenessAlertedAt: new Map(), + unmatchedBaseFirstSeenAt: new Map(), + issuanceAnchor: undefined, + }; + } + return { + baseChainId: persisted.baseChainId, + vaultAddress: persisted.vaultAddress, + pendulumGenesisHash: persisted.pendulumGenesisHash, + lastPendulumBlock: persisted.lastPendulumBlock, + nextExpectedNonce: BigInt(persisted.nextExpectedNonce), + baseFromBlock: BigInt(persisted.baseFromBlock), + migrations: new Map(persisted.migrations.map((migration) => [ + BigInt(migration.nonce), + { + nonce: BigInt(migration.nonce), + recipient: migration.recipient, + palletAmount: BigInt(migration.palletAmount), + firstSeenMs: migration.firstSeenMs, + sourceBlock: migration.sourceBlock, + }, + ])), + livenessAlertedAt: new Map(persisted.livenessAlertedAt.map(([nonce, at]) => [BigInt(nonce), at])), + unmatchedBaseFirstSeenAt: new Map(persisted.unmatchedBaseFirstSeenAt), + issuanceAnchor: persisted.issuanceAnchor === undefined ? undefined : BigInt(persisted.issuanceAnchor), + }; +} + +function cloneState(source: RuntimeState): RuntimeState { + return { + baseChainId: source.baseChainId, + vaultAddress: source.vaultAddress, + pendulumGenesisHash: source.pendulumGenesisHash, + lastPendulumBlock: source.lastPendulumBlock, + nextExpectedNonce: source.nextExpectedNonce, + baseFromBlock: source.baseFromBlock, + migrations: new Map(source.migrations), + livenessAlertedAt: new Map(source.livenessAlertedAt), + unmatchedBaseFirstSeenAt: new Map(source.unmatchedBaseFirstSeenAt), + issuanceAnchor: source.issuanceAnchor, + }; +} + +function persistState(state: RuntimeState): void { + if (!state.pendulumGenesisHash) throw new Error("cannot persist monitor state before Pendulum identity is known"); + const persisted: PersistedMonitorState = { + version: 1, + baseChainId: state.baseChainId, + vaultAddress: state.vaultAddress, + pendulumGenesisHash: state.pendulumGenesisHash, + lastPendulumBlock: state.lastPendulumBlock, + nextExpectedNonce: state.nextExpectedNonce.toString(), + baseFromBlock: state.baseFromBlock.toString(), + migrations: [...state.migrations.values()].map((migration) => ({ + nonce: migration.nonce.toString(), + recipient: migration.recipient as `0x${string}`, + palletAmount: migration.palletAmount.toString(), + firstSeenMs: migration.firstSeenMs, + sourceBlock: migration.sourceBlock, + })), + livenessAlertedAt: [...state.livenessAlertedAt].map(([nonce, at]) => [nonce.toString(), at]), + unmatchedBaseFirstSeenAt: [...state.unmatchedBaseFirstSeenAt], + issuanceAnchor: state.issuanceAnchor?.toString(), + }; + saveMonitorState(config.stateFile, persisted); +} + +let state: RuntimeState; +let multicallUnavailable = false; +let lastSourceStallAlertMs = 0; +let lastIssuanceAlertMs = 0; + function log(message: string): void { console.log(`${new Date().toISOString()} ${message}`); } +/** Strip URLs before anything leaves the process: RPC endpoints commonly embed + * API keys, and viem error texts quote the endpoint verbatim. */ +function redact(text: string): string { + return text.replace(/https?:\/\/[^\s"')]+/gi, ""); +} + async function alert(subject: string, detail: string): Promise { console.error(`${new Date().toISOString()} ALERT: ${subject} — ${detail}`); if (!config.alertWebhookUrl) return; @@ -92,201 +286,562 @@ async function alert(subject: string, detail: string): Promise { await fetch(config.alertWebhookUrl, { method: "POST", headers: { "content-type": "application/json" }, - body: JSON.stringify({ service: "pen-monitor", subject, detail }), + body: JSON.stringify({ service: "pen-monitor", subject, detail: redact(detail) }), + // The webhook is an external dependency; it must never stall the + // check loop (or the pause that follows a violation) indefinitely. + signal: AbortSignal.timeout(10_000), }); } catch (webhookError) { console.error("alert webhook failed", webhookError); } } -async function pauseVault(): Promise { +async function waitForBaseFinality( + hash: `0x${string}`, + receiptBlock: bigint, + receiptBlockHash: `0x${string}`, +): Promise { + const deadline = Date.now() + config.baseFinalityTimeoutMs; + for (;;) { + const confirmed = await publicClient.getBlock({ blockTag: config.baseFinalityTag }); + if (confirmed.number !== null && confirmed.number >= receiptBlock) { + const canonicalReceipt = await publicClient.getTransactionReceipt({ hash }); + if (canonicalReceipt.status !== "success" || canonicalReceipt.blockHash !== receiptBlockHash) { + throw new Error(`pause transaction ${hash} was reorged before becoming ${config.baseFinalityTag}`); + } + return confirmed.number; + } + if (Date.now() >= deadline) { + throw new Error(`timed out waiting for pause ${hash} to become ${config.baseFinalityTag}`); + } + await new Promise((resolve) => setTimeout(resolve, config.baseFinalityPollMs)); + } +} + +async function pauseVault(): Promise { if (!config.guardianPrivateKey) { await alert("AUTO-PAUSE UNAVAILABLE", "no GUARDIAN_PRIVATE_KEY configured; pause manually NOW"); - return; + return false; } - const guardian = privateKeyToAccount(config.guardianPrivateKey); - const walletClient = createWalletClient({ account: guardian, chain: baseChain, transport: http(config.baseRpcUrl) }); try { - const txHash = await walletClient.writeContract({ + const confirmed = await publicClient.getBlock({ blockTag: config.baseFinalityTag }); + if (confirmed.number === null) throw new Error(`${config.baseFinalityTag} block has no number`); + const pausedConfirmed = await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "paused", + blockNumber: confirmed.number, + }); + if (pausedConfirmed) { + await alert("vault pause confirmed", `already paused in Base ${config.baseFinalityTag} block ${confirmed.number}`); + return true; + } + const pausedLatest = await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "paused", + }); + if (pausedLatest) { + await alert("AUTO-PAUSE PENDING FINALITY", `pause is mined but not yet ${config.baseFinalityTag}`); + return false; + } + + const guardian = privateKeyToAccount(config.guardianPrivateKey); + const walletClient = createWalletClient({ account: guardian, chain: baseChain, transport: http(config.baseRpcUrl) }); + const { request } = await publicClient.simulateContract({ + account: guardian, address: config.vaultAddress, abi: vaultAbi, functionName: "pause", }); - await alert("vault auto-paused", `tx=${txHash}`); + const txHash = await walletClient.writeContract(request); + const receipt = await publicClient.waitForTransactionReceipt({ hash: txHash }); + if (receipt.status !== "success") throw new Error(`pause transaction reverted: ${txHash}`); + const safeBlock = await waitForBaseFinality(txHash, receipt.blockNumber, receipt.blockHash); + const paused = await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "paused", + blockNumber: safeBlock, + }); + if (!paused) throw new Error(`pause transaction ${txHash} is confirmed but paused() is false`); + await alert("vault auto-pause confirmed", `tx=${txHash} ${config.baseFinalityTag}Block=${safeBlock}`); + return true; } catch (pauseError) { await alert("AUTO-PAUSE FAILED", `${pauseError}; pause manually NOW`); + return false; } } -/** Timestamps (ms) at which the monitor first saw each pallet nonce count. */ -const nonceFirstSeen = new Map(); - -/** Timestamp (ms) of the last liveness alert per nonce. Re-alerts are throttled - * to at most once per grace period, so a pause backlog — or a burn to a - * structurally-unreleasable address (zero / the vault, which never consumes) — - * does not re-fire the highest-severity page on every single poll and train - * on-call to ignore it. Cleared when the nonce is finally consumed. */ -const livenessAlertedAt = new Map(); - -/** High-water mark: exclusive upper bound of nonces already incorporated into - * `nonceFirstSeen`. Only nonces at or beyond this are stamped each poll, so a - * nonce that was consumed and pruned from the pending set is never re-added. - * Without it the scan re-inserted every `0..nextNonce` nonce every poll (any - * not currently in the map), re-reading consumed ones via Multicall forever - * and making the scan O(all migrations ever) instead of O(pending) — silently - * defeating the round-5 batching (round 7). */ -let nextNonceIncorporated = 0n; - -let multicallUnavailable = false; +async function securityViolation(subject: string, detail: string): Promise { + // Pause FIRST. The alert webhook is an external dependency with its own + // latency; nothing may sit between detecting a violation and the pause. + const notified = alert(subject, detail); + const paused = await pauseVault(); + await notified; + return paused; +} -/** Read `nonceConsumed` for many nonces, batched through Multicall3. +/** + * Decide what to do with a safe Base event whose nonce the finalized Pendulum + * view does not (yet) know. * - * Falls back to plain concurrent reads if the batch call fails — e.g. on a - * chain where Multicall3 is not deployed at the canonical address, or a - * provider that rejects the batch. The fallback still works (just chattier), - * so a misconfigured multicall degrades liveness detection rather than - * crashing the whole check cycle and blinding the conservation alerts. */ -async function readNonceConsumed(nonces: bigint[]): Promise { - if (!multicallUnavailable) { - try { - return (await publicClient.multicall({ - allowFailure: false, - contracts: nonces.map((nonce) => ({ - address: config.vaultAddress, - abi: vaultAbi, - functionName: "nonceConsumed", - args: [nonce], - })), - })) as boolean[]; - } catch (multicallError) { - // Latch so we do not re-attempt (and re-log) the batch every poll. - multicallUnavailable = true; - await alert( - "multicall unavailable, using per-nonce reads", - `liveness reads fall back to individual calls; verify Multicall3 at ${MULTICALL3_ADDRESS}: ${multicallError}`, - ); + * Three outcomes: + * - "proven": the source view has passed the moment this event was first + * observed and the nonce still does not exist — fabricated, + * pause immediately (the RPC-lag grace must not extend an + * attacker's dwell time); + * - "await": the source view still trails the event and the grace has not + * expired — legitimate node lag looks exactly like this; + * - "expired": the grace ran out with the source still behind — the monitor + * has been unable to verify releases for the whole window. + * + * The first observation of an unmatched event pages immediately (deduplicated + * via the persisted first-seen map), so operators get the entire grace window + * to repair a lagging source node before "expired" forces the pause. + */ +async function unmatchedDisposition( + kind: "Approved" | "Released", + key: string, + nonce: bigint, + eventBlockNumber: bigint, + draft: RuntimeState, + observedAt: number, + sourceFinalizedTsMs: number, +): Promise<"proven" | "await" | "expired"> { + const alreadyKnown = draft.unmatchedBaseFirstSeenAt.has(key); + const firstSeen = draft.unmatchedBaseFirstSeenAt.get(key) ?? observedAt; + draft.unmatchedBaseFirstSeenAt.set(key, firstSeen); + if (!alreadyKnown) { + await alert( + "UNVERIFIED BASE EVENT", + `${kind} nonce ${nonce} at Base block ${eventBlockNumber} has no finalized Pendulum source yet; ` + + `waiting up to ${config.unmatchedEventGraceSeconds}s for the source view to reveal its burn`, + ); + } + if (provablyUnsourced(firstSeen, sourceFinalizedTsMs, observedAt, config.sourceClockSkewMarginMs)) return "proven"; + return shouldAwaitSource(nonce, draft.nextExpectedNonce, firstSeen, observedAt, config.unmatchedEventGraceSeconds) + ? "await" + : "expired"; +} + +async function ingestPendulumEvents(api: ApiPromise, draft: RuntimeState, finalizedBlock: number): Promise { + for (let block = draft.lastPendulumBlock + 1; block <= finalizedBlock; block++) { + const blockHash = await api.rpc.chain.getBlockHash(block); + const apiAt = await api.at(blockHash); + const firstSeenMs = Number((await apiAt.query.timestamp.now()).toString().replaceAll(",", "")); + const records = (await apiAt.query.system.events()) as unknown as { + event: { section: string; method: string; data: unknown[] }; + }[]; + for (const record of records) { + const { section, method, data } = record.event; + if (section !== "tokenMigration" || method !== "MigrationInitiated") continue; + if (data.length !== 4) throw new Error(`MigrationInitiated in block ${block} has ${data.length} fields, expected 4`); + const [nonceValue, , recipientValue, amountValue] = data as [ + { toBigInt(): bigint }, + unknown, + { toHex(): string }, + { toBigInt(): bigint }, + ]; + const nonce = nonceValue.toBigInt(); + const recipient = recipientValue.toHex() as `0x${string}`; + if (!/^0x[0-9a-fA-F]{40}$/.test(recipient)) { + throw new Error(`cannot decode base_address in block ${block}: ${recipient}`); + } + if (nonce !== draft.nextExpectedNonce) { + throw new Error(`migration nonce gap: finalized event ${nonce}, expected ${draft.nextExpectedNonce}`); + } + draft.nextExpectedNonce++; + if (recipient === ZERO_ADDRESS || recipient.toLowerCase() === config.vaultAddress.toLowerCase()) { + // The vault deterministically rejects this tuple (ZeroAddress / + // RecipientIsVault) and the attestors skip it, so it can never be + // approved or released. Tracking it would keep the pending count + // non-zero forever and page liveness every grace period (and make + // RB-7's "zero outstanding nonces" unsatisfiable). Record it once + // and leave it out of reconciliation. + await alert( + "CRITICAL: unreleasable migration burned on Pendulum", + `nonce ${nonce} in block ${block} burned ${amountValue.toBigInt()} to ${recipient}; the vault can never release it`, + ); + continue; + } + draft.migrations.set(nonce, { + nonce, + recipient, + palletAmount: amountValue.toBigInt(), + firstSeenMs, + sourceBlock: block, + }); + } + draft.lastPendulumBlock = block; + // Checkpoint a long catch-up as it goes: the Pendulum-side cursor is + // self-consistent on its own (the Base cursor only ever trails it), so a + // transient RPC error hours into a replay must not discard the progress + // and restart the whole replay — against a flaky endpoint that never + // converges, leaving the monitor blind indefinitely. + if (block % 200 === 0) { + state = cloneState(draft); + persistState(state); } } +} - // Fallback: read in bounded concurrent batches to avoid a request storm. - const CHUNK = 100; - const flags: boolean[] = []; - for (let start = 0; start < nonces.length; start += CHUNK) { - const chunk = nonces.slice(start, start + CHUNK); - const chunkFlags = await Promise.all( - chunk.map((nonce) => - publicClient.readContract({ - address: config.vaultAddress, - abi: vaultAbi, - functionName: "nonceConsumed", - args: [nonce], - }), - ), - ); - flags.push(...chunkFlags); +async function reconcileBaseEvents( + draft: RuntimeState, + toBlock: bigint, + conversionFactor: bigint, + sourceFinalizedTsMs: number, +): Promise<{ violation?: string; awaitingSource?: string }> { + const releasedInScan = new Set(); + let firstViolation: string | undefined; + for (const [start, end] of blockRanges(draft.baseFromBlock, toBlock, config.baseMaxBlockRange)) { + // Probe the range's end block first. Against a load-balanced endpoint, + // `eth_getLogs` can be served by a node that has not reached `end` yet + // and some providers then silently truncate rather than error — which + // would advance the cursor past events this monitor never reconciled. A + // lagging node fails this probe loudly instead, and the cycle replays. + await publicClient.getBlock({ blockNumber: end }); + const approvals = await publicClient.getContractEvents({ + address: config.vaultAddress, + abi: vaultAbi, + eventName: "Approved", + fromBlock: start, + toBlock: end, + }); + const releases = await publicClient.getContractEvents({ + address: config.vaultAddress, + abi: vaultAbi, + eventName: "Released", + fromBlock: start, + toBlock: end, + }); + const observedAt = Date.now(); + let awaitingSource: string | undefined; + + // Preflight the complete range before deleting released migrations. If a + // source RPC is merely behind, the range must be replayed intact later. + for (const event of approvals) { + const actual = event.args as { nonce: bigint; recipient: `0x${string}`; palletAmount: bigint }; + const expected = draft.migrations.get(actual.nonce); + const mismatch = approvalMismatch(expected, actual); + const key = `Approved:${actual.nonce}`; + if (!mismatch) { + draft.unmatchedBaseFirstSeenAt.delete(key); + continue; + } + if (!expected) { + const disposition = await unmatchedDisposition( + "Approved", + key, + actual.nonce, + event.blockNumber, + draft, + observedAt, + sourceFinalizedTsMs, + ); + if (disposition === "await") { + awaitingSource ??= `Approved nonce ${actual.nonce} at Base block ${event.blockNumber}`; + continue; + } + firstViolation ??= `Approved at Base block ${event.blockNumber}: ${mismatch} (` + + (disposition === "proven" + ? "source view has passed the event's timestamp — fabricated" + : "source view still behind after the full grace period") + ")"; + continue; + } + firstViolation ??= `Approved at Base block ${event.blockNumber}: ${mismatch}`; + } + for (const event of releases) { + const actual = event.args as { + nonce: bigint; + recipient: `0x${string}`; + palletAmount: bigint; + tokenAmount: bigint; + }; + const expected = draft.migrations.get(actual.nonce); + const mismatch = releaseMismatch(expected, actual, conversionFactor); + const key = `Released:${actual.nonce}`; + if (!mismatch) { + draft.unmatchedBaseFirstSeenAt.delete(key); + continue; + } + if (!expected) { + const disposition = await unmatchedDisposition( + "Released", + key, + actual.nonce, + event.blockNumber, + draft, + observedAt, + sourceFinalizedTsMs, + ); + if (disposition === "await") { + awaitingSource ??= `Released nonce ${actual.nonce} at Base block ${event.blockNumber}`; + continue; + } + firstViolation ??= `Released at Base block ${event.blockNumber}: ${mismatch} (` + + (disposition === "proven" + ? "source view has passed the event's timestamp — fabricated" + : "source view still behind after the full grace period") + ")"; + continue; + } + firstViolation ??= `Released at Base block ${event.blockNumber}: ${mismatch}`; + } + if (awaitingSource && !firstViolation) return { awaitingSource }; + + for (const event of releases) { + const actual = event.args as { + nonce: bigint; + recipient: `0x${string}`; + palletAmount: bigint; + tokenAmount: bigint; + }; + if (releasedInScan.has(actual.nonce)) { + firstViolation ??= `duplicate Released event for nonce ${actual.nonce}`; + continue; + } + const mismatch = releaseMismatch(draft.migrations.get(actual.nonce), actual, conversionFactor); + if (mismatch) { + firstViolation ??= `Released at Base block ${event.blockNumber}: ${mismatch}`; + continue; + } + releasedInScan.add(actual.nonce); + draft.migrations.delete(actual.nonce); + draft.livenessAlertedAt.delete(actual.nonce); + } + draft.baseFromBlock = end + 1n; + } + return { violation: firstViolation }; +} + +function payloadHash(migration: MigrationRecord): `0x${string}` { + return keccak256(encodeAbiParameters( + [{ type: "uint64" }, { type: "address" }, { type: "uint256" }], + [migration.nonce, migration.recipient as `0x${string}`, migration.palletAmount], + )); +} + +async function activeApprovalCounts(migrations: MigrationRecord[], blockNumber: bigint): Promise { + const counts: bigint[] = []; + for (const batch of chunks(migrations, config.readBatchSize)) { + const single = (migration: MigrationRecord) => publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "activeApprovals", + args: [payloadHash(migration)], + blockNumber, + }); + if (!multicallUnavailable) { + try { + const result = await publicClient.multicall({ + allowFailure: false, + blockNumber, + contracts: batch.map((migration) => ({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "activeApprovals" as const, + args: [payloadHash(migration)] as const, + })), + }); + counts.push(...(result as bigint[])); + continue; + } catch (error) { + await alert("multicall batch failed, using bounded reads for this batch", `${error}`); + } + } + counts.push(...(await Promise.all(batch.map(single)))); } - return flags; + return counts; } async function check(api: ApiPromise): Promise { - // --- Base side, pinned to one block --- - // Read Base FIRST, then Pendulum's monotonically-growing totalMigrated at a - // strictly-later snapshot: this guarantees totalMigrated >= what any Base - // release could have been attested against, so the M2a check can never - // false-positive on a burn that finalized between the two reads. Pinning - // every Base read to one block keeps totalReleased and vaultBalance from - // skewing against each other (a release landing mid-cycle). - const blockNumber = await publicClient.getBlockNumber(); + const confirmedBase = await publicClient.getBlock({ blockTag: config.baseFinalityTag }); + if (confirmedBase.number === null) throw new Error(`${config.baseFinalityTag} Base block has no number`); + const baseBlock = confirmedBase.number; + + // Read Pendulum after choosing the Base snapshot. TotalMigrated can only + // grow, so this ordering cannot false-positive if a burn finalizes mid-poll. + const finalizedHash = await api.rpc.chain.getFinalizedHead(); + const finalizedHeader = await api.rpc.chain.getHeader(finalizedHash); + const finalizedBlock = finalizedHeader.number.toNumber(); + const apiAt = await api.at(finalizedHash); + const sourceFinalizedTsMs = Number((await apiAt.query.timestamp.now()).toString().replaceAll(",", "")); + // A stalling source view is this daemon going blind: it cannot verify any + // new Base event, and once the unmatched-event grace expires that forces a + // cannot-verify pause. Page as soon as the view goes stale, so operators + // get the entire grace window to restore the node before that happens. + if ( + Date.now() - sourceFinalizedTsMs > config.sourceStaleAlertMs && + Date.now() - lastSourceStallAlertMs > config.unmatchedEventGraceSeconds * 1000 + ) { + lastSourceStallAlertMs = Date.now(); + await alert( + "PENDULUM SOURCE VIEW STALLED", + `finalized head ${finalizedBlock} is ${Math.round((Date.now() - sourceFinalizedTsMs) / 1000)}s old; ` + + `the monitor cannot verify new Base events against a stalled source — restore the node before ` + + `the ${config.unmatchedEventGraceSeconds}s unmatched-event grace forces a pause`, + ); + } + const draft = cloneState(state); + await ingestPendulumEvents(api, draft, finalizedBlock); + const [totalMigratedValue, chainNextNonceValue, totalIssuanceValue] = await Promise.all([ + apiAt.query.tokenMigration.totalMigrated(), + apiAt.query.tokenMigration.nextNonce(), + apiAt.query.balances.totalIssuance(), + ]); + const totalMigrated = BigInt(totalMigratedValue.toString().replaceAll(",", "")); + const chainNextNonce = BigInt(chainNextNonceValue.toString().replaceAll(",", "")); + if (chainNextNonce !== draft.nextExpectedNonce) { + throw new Error(`finalized nextNonce ${chainNextNonce} != monitor event cursor ${draft.nextExpectedNonce}`); + } + // Source-chain supply guard. Every burn of freshly MINTED Pendulum PEN (a + // passed setBalance referendum, an unexpected teleport-in) is a genuine + // finalized burn: attestors approve it honestly, every tuple matches, and + // M2a/M2b stay satisfied — while the fixed-supply vault drains ahead of + // honest late migrators. issuance + migrated cannot grow under any + // legitimate flow, so growth over the first-seen anchor is the one signal + // for this. Alert only: pausing is a human decision here. + const totalIssuance = BigInt(totalIssuanceValue.toString().replaceAll(",", "")); + const sourceSupply = totalIssuance + totalMigrated; + if (draft.issuanceAnchor === undefined) { + draft.issuanceAnchor = sourceSupply; + log(`anchored source supply: issuance ${totalIssuance} + migrated ${totalMigrated} = ${sourceSupply}`); + } else if ( + sourceSupply > draft.issuanceAnchor + config.issuanceToleranceRaw && + Date.now() - lastIssuanceAlertMs > config.graceSeconds * 1000 + ) { + lastIssuanceAlertMs = Date.now(); + await alert( + "SOURCE SUPPLY GREW", + `Pendulum issuance ${totalIssuance} + migrated ${totalMigrated} = ${sourceSupply} exceeds the anchor ` + + `${draft.issuanceAnchor} by ${sourceSupply - draft.issuanceAnchor} pallet units: PEN was minted at the ` + + `source and can be burned against the fixed-supply vault — investigate before it migrates (RB-3)`, + ); + } + const [totalReleased, totalSwept, conversionFactor, tokenAddress] = await Promise.all([ - publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "totalReleased", blockNumber }), - publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "totalSwept", blockNumber }), - publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "conversionFactor", blockNumber }), - publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "token", blockNumber }), + publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "totalReleased", blockNumber: baseBlock }), + publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "totalSwept", blockNumber: baseBlock }), + publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "conversionFactor", blockNumber: baseBlock }), + publicClient.readContract({ address: config.vaultAddress, abi: vaultAbi, functionName: "token", blockNumber: baseBlock }), ]); + + const reconciliation = await reconcileBaseEvents(draft, baseBlock, conversionFactor, sourceFinalizedTsMs); + if (reconciliation.violation) { + if (await securityViolation("MIGRATION TUPLE VIOLATION", reconciliation.violation)) { + state = draft; + persistState(state); + } + return; + } + const [totalSupply, vaultBalance] = await Promise.all([ - publicClient.readContract({ address: tokenAddress, abi: erc20Abi, functionName: "totalSupply", blockNumber }), + publicClient.readContract({ address: tokenAddress, abi: erc20Abi, functionName: "totalSupply", blockNumber: baseBlock }), publicClient.readContract({ address: tokenAddress, abi: erc20Abi, functionName: "balanceOf", args: [config.vaultAddress], - blockNumber, + blockNumber: baseBlock, }), ]); - - // --- Pendulum side, at the finalized head (read after Base, see above) --- - const finalizedHash = await api.rpc.chain.getFinalizedHead(); - const apiAt = await api.at(finalizedHash); - const totalMigrated = BigInt((await apiAt.query.tokenMigration.totalMigrated()).toString()); - const nextNonce = BigInt((await apiAt.query.tokenMigration.nextNonce()).toString()); - - // (M2a) Nothing may leave the vault that was not burned on Pendulum. - if (releasedExceedsMigrated(totalReleased, totalMigrated, conversionFactor)) { - await alert( - "CONSERVATION VIOLATION", - `released ${totalReleased} > migrated ${totalMigrated * conversionFactor} (token units)`, - ); - await pauseVault(); - return; - } - - // (M2b) Vault-internal conservation. Only a DEFICIT signals real loss; a - // surplus is a harmless inbound transfer (donation, or a migration whose - // recipient is the vault) and must not trip the check — otherwise a dust - // transfer would pause the vault every poll until the window-close sweep. - // totalSwept accounts for the intended end-of-window sweep. if (vaultConservationDeficit(vaultBalance, totalReleased, totalSwept, totalSupply)) { - await alert( + if (await securityViolation( "VAULT BALANCE DEFICIT", `balance ${vaultBalance} + released ${totalReleased} + swept ${totalSwept} < supply ${totalSupply}`, + )) { + state = draft; + persistState(state); + } + return; + } + if (reconciliation.awaitingSource) { + state = draft; + persistState(state); + log( + `waiting up to ${config.unmatchedEventGraceSeconds}s for Pendulum RPC to catch up with ` + + reconciliation.awaitingSource, ); - await pauseVault(); + return; + } + if (releasedExceedsMigrated(totalReleased, totalMigrated, conversionFactor)) { + if (await securityViolation( + "CONSERVATION VIOLATION", + `released ${totalReleased} > migrated ${totalMigrated * conversionFactor} (token units)`, + )) { + state = draft; + persistState(state); + } return; } - // (M4) Liveness: nonces the monitor has known about for longer than the - // grace period must be consumed on Base. Batch the per-nonce reads through - // Multicall3 so a large release backlog (e.g. during a pause) cannot make a - // cycle outrun the poll interval and starve the conservation checks above. + const pending = [...draft.migrations.values()]; + const threshold = await publicClient.readContract({ + address: config.vaultAddress, + abi: vaultAbi, + functionName: "threshold", + blockNumber: baseBlock, + }); + const approvals = await activeApprovalCounts(pending, baseBlock); const now = Date.now(); - for (const nonce of newNonces(nextNonceIncorporated, nextNonce)) { - nonceFirstSeen.set(nonce, now); - } - nextNonceIncorporated = nextNonce; - const pendingNonces = [...nonceFirstSeen.keys()]; - if (pendingNonces.length > 0) { - const consumedFlags = await readNonceConsumed(pendingNonces); - for (let i = 0; i < pendingNonces.length; i++) { - const nonce = pendingNonces[i]; - if (consumedFlags[i]) { - nonceFirstSeen.delete(nonce); - livenessAlertedAt.delete(nonce); - } else if (isStale(nonceFirstSeen.get(nonce) ?? now, now, config.graceSeconds)) { - // Throttle: page immediately on first staleness, then at most once - // per grace period, so an ongoing outage stays visible without - // storming (a permanently-unreleasable nonce would otherwise page - // every poll forever). - const lastAlerted = livenessAlertedAt.get(nonce); - if (lastAlerted === undefined || now - lastAlerted >= config.graceSeconds * 1000) { - livenessAlertedAt.set(nonce, now); - await alert( - "LIVENESS: migration not released", - `nonce ${nonce} unreleased for over ${config.graceSeconds}s — attestor outage, cap deferral, pause, or a burn to an unreleasable address?`, - ); - } - } + for (let index = 0; index < pending.length; index++) { + const migration = pending[index]; + if (!isStale(migration.firstSeenMs, now, config.graceSeconds)) continue; + const lastAlerted = draft.livenessAlertedAt.get(migration.nonce); + if (lastAlerted !== undefined && now - lastAlerted < config.graceSeconds * 1000) continue; + draft.livenessAlertedAt.set(migration.nonce, now); + if (approvals[index] >= threshold) { + // Quorum is present but nothing released it: cap-deferred and + // starved of daily allowance, paused, or made releasable by a + // threshold cut without ever emitting ReleasePending (so the + // releaser cannot see it). Silent indefinitely before round 9. + await alert( + "LIVENESS: quorum reached but not released", + `nonce ${migration.nonce} has ${approvals[index]}/${threshold} active approvals and is still unreleased ` + + `after ${config.graceSeconds}s — check the daily allowance / pause state, or call release() directly ` + + `if it qualified through a threshold cut (RB-6)`, + ); + continue; } + await alert( + "LIVENESS: migration below approval quorum", + `nonce ${migration.nonce} has ${approvals[index]}/${threshold} active approvals after ${config.graceSeconds}s`, + ); } + state = draft; + persistState(state); log( `ok: migrated=${totalMigrated} released=${totalReleased} swept=${totalSwept} ` + - `pending=${nonceFirstSeen.size} vaultBalance=${vaultBalance}`, + `pending=${state.migrations.size} vaultBalance=${vaultBalance} base=${baseBlock} pendulum=${finalizedBlock}`, ); } async function main(): Promise { + state = hydrateState(); + const multicallCode = await publicClient.getBytecode({ address: MULTICALL3_ADDRESS }); + if (!multicallCode || multicallCode === "0x") { + multicallUnavailable = true; + await alert("multicall unavailable, using bounded individual reads", `no contract code at ${MULTICALL3_ADDRESS}`); + } const api = await ApiPromise.create({ provider: new WsProvider(config.pendulumWs) }); - log(`monitor started, polling every ${config.pollIntervalMs}ms`); + const pendulumGenesisHash = api.genesisHash.toHex(); + if (state.pendulumGenesisHash) { + assertMonitorStateIdentity( + { + baseChainId: state.baseChainId, + vaultAddress: state.vaultAddress, + pendulumGenesisHash: state.pendulumGenesisHash, + }, + { ...config, pendulumGenesisHash }, + ); + } else { + state.pendulumGenesisHash = pendulumGenesisHash; + } + log( + `monitor started; Pendulum after ${state.lastPendulumBlock}, Base from ${state.baseFromBlock}, ` + + `polling every ${config.pollIntervalMs}ms`, + ); for (;;) { try { await check(api); From 7ed88c3513e98a5ae7b2ba8f3f151899fc6fe39b Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:52:27 +0200 Subject: [PATCH 47/61] contracts: Base quorum on circulating supply with an absolute floor Quorum was a fraction of the full past total supply, which includes the vault's unmigrated balance, while only migrated-and-delegated PEN can vote. Post-handover that could deadlock permanently: a pause (whose unpause is an admin action behind this governor) freezes releases, so circulating voting supply can never grow to quorum. MigrationVault.setToken now one-time-delegates the vault's balance to a constant dead-address vote sink, which checkpoints the unmigrated supply in the token's vote history; PENGovernor.quorum() subtracts the sink's past votes from the denominator and applies an absolute quorumFloor (new constructor and QUORUM_FLOOR deploy parameter) so early proposals are not trivially cheap. The two sink constants are asserted equal in the tests; vault-held PEN provably never votes. --- contracts/.env.example | 15 ++++++-- contracts/script/DeployGovernance.s.sol | 11 ++++-- contracts/src/MigrationVault.sol | 16 ++++++++ contracts/src/PENGovernor.sol | 50 ++++++++++++++++++++++--- contracts/test/MigrationVault.t.sol | 15 ++++++++ contracts/test/PENGovernor.t.sol | 47 ++++++++++++++++++++++- testing/.env.rehearsal.example | 3 ++ testing/src/rehearsal-deploy.mjs | 4 +- 8 files changed, 146 insertions(+), 15 deletions(-) diff --git a/contracts/.env.example b/contracts/.env.example index 70a067ec0..f5a5b9ff6 100644 --- a/contracts/.env.example +++ b/contracts/.env.example @@ -58,8 +58,15 @@ TIMELOCK_DELAY=172800 # 48h (PRD V5) VOTING_DELAY=86400 # 1 day, seconds (timestamp clock) VOTING_PERIOD=432000 # 5 days PROPOSAL_THRESHOLD= # token units required to propose -# Percent of TOTAL supply (fixed at 150M), counting For+Abstain votes actually -# cast. Set this against the delegated supply observed at handover, not a guess -# now: the vault's unmigrated balance is in the denominator but never votes, so -# too high a figure makes governance unusable early (PRD G1). +# Percent of CIRCULATING supply (total minus the vault's unmigrated balance, +# which setToken parks at the vote sink — review round 8), counting For+Abstain +# votes actually cast. Bounded below by QUORUM_FLOOR. QUORUM_FRACTION=2 +# Absolute quorum floor, 18-decimal token units. DECIDE THIS BEFORE PHASE 5 — +# there is no safe default and the script refuses to run without it. Size it +# so quorum is honestly reachable soon after launch (it must be, or an unpause +# proposal can never pass), and understand what it does NOT do: quorum stake +# is bought, not burned, so no floor prices out capture of a ~150M-PEN vault. +# Capture resistance comes from TIMELOCK_CANCELLER below, the guardian pause, +# and the vault caps — not from this number (review round 9). +QUORUM_FLOOR= diff --git a/contracts/script/DeployGovernance.s.sol b/contracts/script/DeployGovernance.s.sol index f84fdc3c7..4839967b2 100644 --- a/contracts/script/DeployGovernance.s.sol +++ b/contracts/script/DeployGovernance.s.sol @@ -22,8 +22,12 @@ import {IVotes} from "@openzeppelin/contracts/governance/utils/IVotes.sol"; /// VOTING_DELAY seconds before voting starts (timestamp clock) /// VOTING_PERIOD seconds of voting /// PROPOSAL_THRESHOLD token units needed to propose -/// QUORUM_FRACTION percent of total supply (start low; vault balance -/// counts toward total supply, see PRD G1) +/// QUORUM_FRACTION percent of CIRCULATING supply (total minus the +/// vault's unmigrated balance parked at the vote +/// sink, see PRD G1 as revised in review round 8) +/// QUORUM_FLOOR absolute quorum lower bound, 18-decimal token +/// units; keeps early proposals from being trivially +/// cheap while circulating supply is still small contract DeployGovernance is Script { function run() external { address token = vm.envAddress("PEN_TOKEN"); @@ -32,6 +36,7 @@ contract DeployGovernance is Script { uint32 votingPeriod = uint32(vm.envUint("VOTING_PERIOD")); uint256 proposalThreshold = vm.envUint("PROPOSAL_THRESHOLD"); uint256 quorumFraction = vm.envUint("QUORUM_FRACTION"); + uint256 quorumFloor = vm.envUint("QUORUM_FLOOR"); vm.startBroadcast(); @@ -41,7 +46,7 @@ contract DeployGovernance is Script { new TimelockController(timelockDelay, empty, empty, msg.sender); PENGovernor governor = new PENGovernor( - IVotes(token), timelock, votingDelay, votingPeriod, proposalThreshold, quorumFraction + IVotes(token), timelock, votingDelay, votingPeriod, proposalThreshold, quorumFraction, quorumFloor ); // Only the Governor proposes/cancels; anyone may execute after the delay. diff --git a/contracts/src/MigrationVault.sol b/contracts/src/MigrationVault.sol index acabd7bee..769b83ac2 100644 --- a/contracts/src/MigrationVault.sol +++ b/contracts/src/MigrationVault.sol @@ -3,6 +3,7 @@ pragma solidity 0.8.26; import {IERC20} from "@openzeppelin/contracts/token/ERC20/IERC20.sol"; import {SafeERC20} from "@openzeppelin/contracts/token/ERC20/utils/SafeERC20.sol"; +import {IVotes} from "@openzeppelin/contracts/governance/utils/IVotes.sol"; /// @title MigrationVault — releases pre-minted PEN as holders migrate from Pendulum /// @notice Holds the entire unmigrated PEN supply. Each attestor independently @@ -73,6 +74,17 @@ contract MigrationVault { /// hold the token's entire supply (pre-mint model, ADR-001). IERC20 public token; + /// @notice Where the vault's own (unmigrated) voting power is parked. + /// `setToken` delegates the vault's balance here once, which + /// checkpoints the unmigrated supply in the token's vote history: + /// PENGovernor subtracts it from its quorum denominator so quorum + /// tracks CIRCULATING supply, and vault-held tokens can never + /// vote. Must stay in exact lockstep with + /// `PENGovernor.QUORUM_SINK` (asserted in the tests). A constant — + /// an admin-chosen delegatee would hand whoever controls admin the + /// full unmigrated voting power. + address public constant VOTE_SINK = 0x000000000000000000000000000000000000dEaD; + /// @notice Admin of all parameters; a TimelockController post-bootstrap. address public admin; address public pendingAdmin; @@ -205,12 +217,16 @@ contract MigrationVault { /// @notice One-time wiring of the token, required because vault and token /// reference each other: the vault is deployed first, then PEN /// mints its full supply here, then the admin calls this. + /// The token must be ERC20Votes (PEN is): the vault parks its + /// voting power at `VOTE_SINK` so governance quorum can track the + /// circulating supply. function setToken(IERC20 token_) external onlyAdmin { if (address(token) != address(0)) revert TokenAlreadySet(); if (address(token_) == address(0)) revert ZeroAddress(); uint256 supply = token_.totalSupply(); if (supply == 0 || token_.balanceOf(address(this)) != supply) revert VaultMustHoldFullSupply(); token = token_; + IVotes(address(token_)).delegate(VOTE_SINK); emit TokenSet(address(token_)); } diff --git a/contracts/src/PENGovernor.sol b/contracts/src/PENGovernor.sol index 4c4c43dec..c0c5b4fe7 100644 --- a/contracts/src/PENGovernor.sol +++ b/contracts/src/PENGovernor.sol @@ -18,10 +18,15 @@ import {IVotes} from "@openzeppelin/contracts/governance/utils/IVotes.sol"; /// bootstrap phase. PEN uses the EIP-6372 timestamp clock, so all /// Governor periods below are in seconds. /// -/// Quorum caveat (PRD G1): quorum is a fraction of *total* supply, -/// which includes the unmigrated balance held by the vault. Start -/// with a low fraction while migration is in progress and raise it -/// via governance as circulating supply grows. +/// Quorum (PRD G1, revised in review round 8): quorum is a fraction of +/// the CIRCULATING supply — total supply minus the unmigrated balance +/// parked at `QUORUM_SINK` by the MigrationVault — bounded below by an +/// absolute `quorumFloor`. A full-supply denominator could deadlock +/// governance permanently: with the vault holding most of the supply +/// early on, a pause (whose unpause is admin-only, i.e. behind this +/// governor) could freeze releases while the votable supply can never +/// grow to quorum. The sink-based denominator tracks what can actually +/// vote; the floor keeps day-one capture from being trivial. contract PENGovernor is Governor, GovernorSettings, @@ -30,20 +35,53 @@ contract PENGovernor is GovernorVotesQuorumFraction, GovernorTimelockControl { + /// @notice Where the MigrationVault parks the voting power of the + /// unmigrated supply (`MigrationVault.VOTE_SINK` — the two + /// constants must stay in exact lockstep, asserted in the tests). + /// Delegating checkpoints the vault's balance in the token's vote + /// history, which lets `quorum()` subtract it per timepoint. + address public constant QUORUM_SINK = 0x000000000000000000000000000000000000dEaD; + + /// @notice Absolute lower bound on quorum, in token units. Guards the + /// early window in which circulating supply is small enough that a + /// purely fractional quorum would make proposals trivially cheap. + uint256 public immutable quorumFloor; + constructor( IVotes token, TimelockController timelock, uint48 votingDelay_, // seconds (timestamp clock) uint32 votingPeriod_, // seconds uint256 proposalThreshold_, // token units - uint256 quorumFraction // percent of total supply + uint256 quorumFraction, // percent of circulating supply + uint256 quorumFloor_ // token units ) Governor("PENGovernor") GovernorSettings(votingDelay_, votingPeriod_, proposalThreshold_) GovernorVotes(token) GovernorVotesQuorumFraction(quorumFraction) GovernorTimelockControl(timelock) - {} + { + quorumFloor = quorumFloor_; + } + + /// @notice Quorum as a fraction of the circulating supply at `timepoint`, + /// never below `quorumFloor`. Circulating = past total supply + /// minus the votes parked at `QUORUM_SINK` (the vault's unmigrated + /// balance, plus anything a holder knowingly burns there — which + /// only ever costs the delegator its own voting power, so lowering + /// quorum this way is strictly dominated by just voting). + function quorum(uint256 timepoint) + public + view + override(Governor, GovernorVotesQuorumFraction) + returns (uint256) + { + uint256 parked = token().getPastVotes(QUORUM_SINK, timepoint); + uint256 circulating = token().getPastTotalSupply(timepoint) - parked; + uint256 fractional = (circulating * quorumNumerator(timepoint)) / quorumDenominator(); + return fractional > quorumFloor ? fractional : quorumFloor; + } // ----- required overrides for the Governor composition ----- diff --git a/contracts/test/MigrationVault.t.sol b/contracts/test/MigrationVault.t.sol index 9c678d614..d3cfdf88d 100644 --- a/contracts/test/MigrationVault.t.sol +++ b/contracts/test/MigrationVault.t.sol @@ -60,6 +60,21 @@ contract MigrationVaultTest is Test { fresh.setToken(IERC20(address(pen))); } + function test_SetTokenParksVaultVotesInSink() public { + // setToken delegates the vault's balance to the vote sink: the + // unmigrated supply is checkpointed there (so PENGovernor's quorum can + // subtract it) and vault-held tokens can never vote. + assertEq(pen.delegates(address(vault)), vault.VOTE_SINK()); + assertEq(pen.getVotes(vault.VOTE_SINK()), MAX_ISSUANCE); + + uint256 palletAmount = 5e12; + approveAs(0, 0, recipient, palletAmount); + approveAs(1, 0, recipient, palletAmount); + approveAs(2, 0, recipient, palletAmount); + // The released amount left the vault, and with it the parked votes. + assertEq(pen.getVotes(vault.VOTE_SINK()), MAX_ISSUANCE - 5e18); + } + function test_ConstructorRejectsThresholdBelowTwo() public { vm.expectRevert(MigrationVault.InvalidThreshold.selector); new MigrationVault(admin, guardian, attestors, 1, CONVERSION_FACTOR, PER_RELEASE_CAP, DAILY_CAP, earliestSweep); diff --git a/contracts/test/PENGovernor.t.sol b/contracts/test/PENGovernor.t.sol index 9597ee0a0..7cb3de62b 100644 --- a/contracts/test/PENGovernor.t.sol +++ b/contracts/test/PENGovernor.t.sol @@ -5,6 +5,7 @@ import {Test} from "forge-std/Test.sol"; import {TimelockController} from "@openzeppelin/contracts/governance/TimelockController.sol"; import {IGovernor} from "@openzeppelin/contracts/governance/IGovernor.sol"; import {IVotes} from "@openzeppelin/contracts/governance/utils/IVotes.sol"; +import {IERC20} from "@openzeppelin/contracts/token/ERC20/IERC20.sol"; import {PEN} from "../src/PEN.sol"; import {PENGovernor} from "../src/PENGovernor.sol"; import {MigrationVault} from "../src/MigrationVault.sol"; @@ -14,6 +15,7 @@ contract PENGovernorTest is Test { uint256 internal constant TIMELOCK_DELAY = 2 days; uint48 internal constant VOTING_DELAY = 1 days; uint32 internal constant VOTING_PERIOD = 5 days; + uint256 internal constant QUORUM_FLOOR = 1_000e18; PEN internal pen; PENGovernor internal governor; @@ -32,7 +34,7 @@ contract PENGovernorTest is Test { timelock = new TimelockController(TIMELOCK_DELAY, empty, empty, address(this)); governor = new PENGovernor( - IVotes(address(pen)), timelock, VOTING_DELAY, VOTING_PERIOD, 1_000e18, 4 + IVotes(address(pen)), timelock, VOTING_DELAY, VOTING_PERIOD, 1_000e18, 4, QUORUM_FLOOR ); timelock.grantRole(timelock.PROPOSER_ROLE(), address(governor)); @@ -92,6 +94,49 @@ contract PENGovernorTest is Test { assertEq(vault.dailyCap(), 9e24); } + function test_QuorumSinkMatchesVaultVoteSink() public view { + // The vault parks unmigrated voting power exactly where the governor + // subtracts it. If these ever diverge, quorum silently reverts to a + // full-supply denominator and the deadlock the sink exists to prevent + // (an unpause proposal that can never reach quorum) comes back. + assertEq(governor.QUORUM_SINK(), vault.VOTE_SINK()); + } + + function test_QuorumTracksCirculatingSupplyWithFloor() public { + // A real vault+token pair: the full supply starts unmigrated. + address[] memory quorumAttestors = new address[](3); + quorumAttestors[0] = makeAddr("qa0"); + quorumAttestors[1] = makeAddr("qa1"); + quorumAttestors[2] = makeAddr("qa2"); + MigrationVault migrationVault = new MigrationVault( + address(this), guardian, quorumAttestors, 2, 1e6, 1e30, 1e30, block.timestamp + 365 days + ); + PEN token = new PEN(address(migrationVault), MAX_ISSUANCE); + migrationVault.setToken(IERC20(address(token))); + PENGovernor gov = new PENGovernor( + IVotes(address(token)), timelock, VOTING_DELAY, VOTING_PERIOD, 1_000e18, 4, QUORUM_FLOOR + ); + + vm.warp(block.timestamp + 1); + uint256 beforeMigration = block.timestamp - 1; + // Nothing migrated: the fractional quorum is zero and the floor holds, + // instead of 4% of the 150M total supply (6M) that nobody could reach. + assertEq(gov.quorum(beforeMigration), QUORUM_FLOOR); + + // Release 50M PEN to a holder through the real approval path. + address bob = makeAddr("bob"); + uint256 palletAmount = 50_000_000e12; + vm.prank(quorumAttestors[0]); + migrationVault.approve(0, bob, palletAmount); + vm.prank(quorumAttestors[1]); + migrationVault.approve(0, bob, palletAmount); + + vm.warp(block.timestamp + 1); + uint256 afterMigration = block.timestamp - 1; + // Quorum is now 4% of the 50M circulating, not 4% of the 150M total. + assertEq(gov.quorum(afterMigration), (50_000_000e18 * 4) / 100); + } + function test_ProposalBelowThresholdReverts() public { address pleb = makeAddr("pleb"); address[] memory targets = new address[](1); diff --git a/testing/.env.rehearsal.example b/testing/.env.rehearsal.example index b389c1c06..5b8104227 100644 --- a/testing/.env.rehearsal.example +++ b/testing/.env.rehearsal.example @@ -50,3 +50,6 @@ GOV_VOTING_DELAY=60 GOV_VOTING_PERIOD=240 GOV_PROPOSAL_THRESHOLD_PEN=1000 GOV_QUORUM_FRACTION=0 +# Absolute quorum floor in whole PEN (round 8: quorum is a fraction of +# CIRCULATING supply bounded below by this floor; 0 keeps the drill trivial). +GOV_QUORUM_FLOOR_PEN=0 diff --git a/testing/src/rehearsal-deploy.mjs b/testing/src/rehearsal-deploy.mjs index 1904520f3..5f0a61634 100644 --- a/testing/src/rehearsal-deploy.mjs +++ b/testing/src/rehearsal-deploy.mjs @@ -84,6 +84,7 @@ export function governanceDrillParams(env) { votingPeriod: Number(env.GOV_VOTING_PERIOD ?? "240"), proposalThreshold: BigInt(env.GOV_PROPOSAL_THRESHOLD_PEN ?? "1000") * PEN_18, quorumFraction: Number(env.GOV_QUORUM_FRACTION ?? "0"), + quorumFloor: BigInt(env.GOV_QUORUM_FLOOR_PEN ?? "0") * PEN_18, }; } @@ -98,8 +99,9 @@ export function deployGovernanceToSepolia({ env, pen, log }) { VOTING_PERIOD: String(params.votingPeriod), PROPOSAL_THRESHOLD: params.proposalThreshold.toString(), QUORUM_FRACTION: String(params.quorumFraction), + QUORUM_FLOOR: params.quorumFloor.toString(), }; - log(`deploying governance (timelock ${params.timelockDelay}s, voting ${params.votingDelay}s+${params.votingPeriod}s, quorum ${params.quorumFraction}%) ...`); + log(`deploying governance (timelock ${params.timelockDelay}s, voting ${params.votingDelay}s+${params.votingPeriod}s, quorum ${params.quorumFraction}% of circulating, floor ${params.quorumFloor / PEN_18} PEN) ...`); execFileSync( "forge", [ From 9d9e81fd3cba12d405a4f7fd1053f09dac70ddc9 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:52:27 +0200 Subject: [PATCH 48/61] contracts: Let a human canceller veto a queued proposal during the delay CANCELLER_ROLE was granted only to the Governor, and OZ Governor lets only the proposer cancel, and only before voting starts. Once a proposal with quorum-clearing stake had passed, nothing could stop it during the 48h delay: one executor transaction could batch unpause, attacker attestors, unbounded caps and fabricated approvals, undoing the guardian's pause inside the same batch. DeployGovernance takes an optional TIMELOCK_CANCELLER (intended: the guardian Safe) and grants it the role; the test proves the veto against a queued proposal. --- contracts/.env.example | 6 ++++ contracts/script/DeployGovernance.s.sol | 14 +++++++++- contracts/test/PENGovernor.t.sol | 37 +++++++++++++++++++++++++ 3 files changed, 56 insertions(+), 1 deletion(-) diff --git a/contracts/.env.example b/contracts/.env.example index f5a5b9ff6..da43b480d 100644 --- a/contracts/.env.example +++ b/contracts/.env.example @@ -70,3 +70,9 @@ QUORUM_FRACTION=2 # Capture resistance comes from TIMELOCK_CANCELLER below, the guardian pause, # and the vault caps — not from this number (review round 9). QUORUM_FLOOR= +# Optional but STRONGLY recommended: the guardian (or council) Safe, granted +# CANCELLER_ROLE on the timelock so a hostile proposal that passed can still be +# vetoed during the 48h delay. Without it nothing can stop a queued proposal +# (OZ Governor lets only the proposer cancel, only before voting starts), so +# the delay is not a reaction window at all (review round 9). +TIMELOCK_CANCELLER= diff --git a/contracts/script/DeployGovernance.s.sol b/contracts/script/DeployGovernance.s.sol index 4839967b2..6e88e23bd 100644 --- a/contracts/script/DeployGovernance.s.sol +++ b/contracts/script/DeployGovernance.s.sol @@ -28,6 +28,13 @@ import {IVotes} from "@openzeppelin/contracts/governance/utils/IVotes.sol"; /// QUORUM_FLOOR absolute quorum lower bound, 18-decimal token /// units; keeps early proposals from being trivially /// cheap while circulating supply is still small +/// TIMELOCK_CANCELLER optional: an address (the guardian/council Safe) +/// that can cancel a QUEUED operation during the +/// timelock delay. Without it only the Governor holds +/// CANCELLER_ROLE, and OZ Governor lets only the +/// proposer cancel, only before voting starts — so a +/// hostile proposal that passed could not be stopped +/// during the 48h delay by anyone (review round 9) contract DeployGovernance is Script { function run() external { address token = vm.envAddress("PEN_TOKEN"); @@ -37,6 +44,7 @@ contract DeployGovernance is Script { uint256 proposalThreshold = vm.envUint("PROPOSAL_THRESHOLD"); uint256 quorumFraction = vm.envUint("QUORUM_FRACTION"); uint256 quorumFloor = vm.envUint("QUORUM_FLOOR"); + address canceller = vm.envOr("TIMELOCK_CANCELLER", address(0)); vm.startBroadcast(); @@ -49,9 +57,13 @@ contract DeployGovernance is Script { IVotes(token), timelock, votingDelay, votingPeriod, proposalThreshold, quorumFraction, quorumFloor ); - // Only the Governor proposes/cancels; anyone may execute after the delay. + // Only the Governor proposes; anyone may execute after the delay. The + // Governor cancels its own operations, and an optional human canceller + // can veto a queued operation during the delay — the reaction window + // the timelock exists to provide. timelock.grantRole(timelock.PROPOSER_ROLE(), address(governor)); timelock.grantRole(timelock.CANCELLER_ROLE(), address(governor)); + if (canceller != address(0)) timelock.grantRole(timelock.CANCELLER_ROLE(), canceller); timelock.grantRole(timelock.EXECUTOR_ROLE(), address(0)); // Leave the timelock self-administered: changes require a proposal. timelock.renounceRole(timelock.DEFAULT_ADMIN_ROLE(), msg.sender); diff --git a/contracts/test/PENGovernor.t.sol b/contracts/test/PENGovernor.t.sol index 7cb3de62b..f939c9ccc 100644 --- a/contracts/test/PENGovernor.t.sol +++ b/contracts/test/PENGovernor.t.sol @@ -39,6 +39,9 @@ contract PENGovernorTest is Test { timelock.grantRole(timelock.PROPOSER_ROLE(), address(governor)); timelock.grantRole(timelock.CANCELLER_ROLE(), address(governor)); + // The human veto during the timelock delay (DeployGovernance's + // optional TIMELOCK_CANCELLER): the guardian Safe. + timelock.grantRole(timelock.CANCELLER_ROLE(), guardian); timelock.grantRole(timelock.EXECUTOR_ROLE(), address(0)); timelock.renounceRole(timelock.DEFAULT_ADMIN_ROLE(), address(this)); @@ -94,6 +97,40 @@ contract PENGovernorTest is Test { assertEq(vault.dailyCap(), 9e24); } + function test_CancellerCanVetoAQueuedProposalDuringTheDelay() public { + // A passed proposal is the one thing OZ Governor cannot cancel itself + // (only the proposer, only before voting). The timelock delay is a + // reaction window only if someone holds CANCELLER_ROLE on the timelock. + address[] memory targets = new address[](1); + targets[0] = address(vault); + uint256[] memory values = new uint256[](1); + bytes[] memory calldatas = new bytes[](1); + calldatas[0] = abi.encodeCall(MigrationVault.setCaps, (5e24, 9e24)); + string memory description = "Hostile: raise caps before a drain"; + bytes32 descriptionHash = keccak256(bytes(description)); + + vm.prank(alice); + uint256 proposalId = governor.propose(targets, values, calldatas, description); + vm.warp(block.timestamp + VOTING_DELAY + 1); + vm.prank(alice); + governor.castVote(proposalId, 1); + vm.warp(block.timestamp + VOTING_PERIOD + 1); + governor.queue(targets, values, calldatas, descriptionHash); + assertEq(uint256(governor.state(proposalId)), uint256(IGovernor.ProposalState.Queued)); + + // GovernorTimelockControl salts the operation with the governor address. + bytes32 salt = bytes20(address(governor)) ^ descriptionHash; + bytes32 operationId = timelock.hashOperationBatch(targets, values, calldatas, 0, salt); + vm.prank(guardian); + timelock.cancel(operationId); + + assertEq(uint256(governor.state(proposalId)), uint256(IGovernor.ProposalState.Canceled)); + vm.warp(block.timestamp + TIMELOCK_DELAY + 1); + vm.expectRevert(); + governor.execute(targets, values, calldatas, descriptionHash); + assertEq(vault.perReleaseCap(), 1e24, "caps unchanged after veto"); + } + function test_QuorumSinkMatchesVaultVoteSink() public view { // The vault parks unmigrated voting power exactly where the governor // subtracts it. If these ever diverge, quorum silently reverts to a From e3a14b9402cecf6a7103d20d85f3bf49b8b3e946 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:52:28 +0200 Subject: [PATCH 49/61] contracts: Enforce cap invariants and reject a past sweep floor perReleaseCap > dailyCap opened a band of amounts that pass the per-release check but can never fit the daily allowance: burned on Pendulum, deferred forever, clearable only by a timelocked setCaps. The rehearsal config had exactly that shape. The constructor and setCaps now reject it (CapsInverted). dailyCap is bounded by MAX_DAILY_CAP: an uncap of type(uint256).max overflowed _decayedConsumed under checked arithmetic and would have reverted every threshold-crossing approve() and release() until a second timelocked setCaps. An earliestSweepTimestamp in the past is rejected. The rehearsal per-release cap default now equals the daily cap. --- contracts/.env.example | 3 +- contracts/src/MigrationVault.sol | 26 +++++++++++++++++ contracts/test/MigrationVault.t.sol | 43 +++++++++++++++++++++++++++++ testing/.env.rehearsal.example | 4 ++- testing/src/rehearsal-deploy.mjs | 4 ++- 5 files changed, 77 insertions(+), 3 deletions(-) diff --git a/contracts/.env.example b/contracts/.env.example index da43b480d..66d5acb7d 100644 --- a/contracts/.env.example +++ b/contracts/.env.example @@ -28,7 +28,8 @@ MAX_ISSUANCE=150000000000000000000000000 # blocked by the DAILY cap heals by itself as the rolling bucket refills and # the releaser retries it, whereas one above the PER-RELEASE cap can only be # cleared by a governance setCaps behind the 48h timelock. Keeping them equal -# removes that permanently-stuck band entirely. +# removes that permanently-stuck band entirely — and since review round 9 the +# vault enforces PER_RELEASE_CAP <= DAILY_CAP on-chain (CapsInverted). # # Soft launch: 1,000,000 PEN (below, active). Raise both to 3,000,000 by # governance once the soft-launch checks pass — 3M/day is ~2% of supply per diff --git a/contracts/src/MigrationVault.sol b/contracts/src/MigrationVault.sol index 769b83ac2..9d055ff00 100644 --- a/contracts/src/MigrationVault.sol +++ b/contracts/src/MigrationVault.sol @@ -49,6 +49,9 @@ contract MigrationVault { error InsufficientVaultBalance(); error ExceedsSweepable(uint256 requested, uint256 sweepable); error SweepSettlingAfterThresholdCut(uint256 allowedFrom); + error CapsInverted(uint256 perReleaseCap, uint256 dailyCap); + error CapTooLarge(uint256 dailyCap); + error SweepTimestampInPast(); // ---------------------------------------------------------------- events @@ -167,6 +170,14 @@ contract MigrationVault { uint256 public thresholdReducedAt; uint256 public constant SWEEP_SETTLING_PERIOD = 7 days; + /// @notice Upper bound on `dailyCap`. `_decayedConsumed` multiplies the + /// elapsed seconds by `dailyCap` under checked arithmetic; an + /// "uncap" of type(uint256).max would make that product overflow + /// and revert every threshold-crossing approve() and release() + /// until a second timelocked setCaps lands. 2^128 token units is + /// ~3e20 PEN — no real cap comes anywhere near it. + uint256 public constant MAX_DAILY_CAP = type(uint128).max; + // ---------------------------------------------------------------- modifiers modifier onlyAdmin() { @@ -194,6 +205,10 @@ contract MigrationVault { if (admin_ == address(0) || guardian_ == address(0)) revert ZeroAddress(); if (threshold_ < 2 || threshold_ > attestors_.length) revert InvalidThreshold(); if (conversionFactor_ == 0) revert ZeroAmount(); + _validateCaps(perReleaseCap_, dailyCap_); + // A floor in the past would make the immutable "cannot sweep before" + // guarantee to holders void from block one. + if (earliestSweepTimestamp_ <= block.timestamp) revert SweepTimestampInPast(); admin = admin_; guardian = guardian_; @@ -426,11 +441,22 @@ contract MigrationVault { } function setCaps(uint256 perReleaseCap_, uint256 dailyCap_) external onlyAdmin { + _validateCaps(perReleaseCap_, dailyCap_); perReleaseCap = perReleaseCap_; dailyCap = dailyCap_; emit CapsUpdated(perReleaseCap_, dailyCap_); } + /// @dev A release in the band (dailyCap, perReleaseCap] passes the + /// per-release check but can never fit the daily allowance (which is + /// capped at dailyCap), so it defers forever — burned on Pendulum, + /// releasable only after a timelocked setCaps. Refuse such a pair at + /// the source instead of relying on a deployment convention. + function _validateCaps(uint256 perReleaseCap_, uint256 dailyCap_) internal pure { + if (perReleaseCap_ > dailyCap_) revert CapsInverted(perReleaseCap_, dailyCap_); + if (dailyCap_ > MAX_DAILY_CAP) revert CapTooLarge(dailyCap_); + } + function setGuardian(address guardian_) external onlyAdmin { if (guardian_ == address(0)) revert ZeroAddress(); guardian = guardian_; diff --git a/contracts/test/MigrationVault.t.sol b/contracts/test/MigrationVault.t.sol index d3cfdf88d..9c9fae3c6 100644 --- a/contracts/test/MigrationVault.t.sol +++ b/contracts/test/MigrationVault.t.sol @@ -75,6 +75,49 @@ contract MigrationVaultTest is Test { assertEq(pen.getVotes(vault.VOTE_SINK()), MAX_ISSUANCE - 5e18); } + function test_CapsMustNotBeInverted() public { + // perReleaseCap > dailyCap opens a band of amounts that pass the + // per-release check but can never fit the daily allowance: burned on + // Pendulum, deferred forever, clearable only by a timelocked setCaps. + vm.expectRevert(abi.encodeWithSelector(MigrationVault.CapsInverted.selector, DAILY_CAP + 1, DAILY_CAP)); + new MigrationVault(admin, guardian, attestors, 3, CONVERSION_FACTOR, DAILY_CAP + 1, DAILY_CAP, earliestSweep); + + vm.prank(admin); + vm.expectRevert(abi.encodeWithSelector(MigrationVault.CapsInverted.selector, DAILY_CAP + 1, DAILY_CAP)); + vault.setCaps(DAILY_CAP + 1, DAILY_CAP); + + // Equal caps are the intended shape and remain allowed. + vm.prank(admin); + vault.setCaps(DAILY_CAP, DAILY_CAP); + assertEq(vault.perReleaseCap(), DAILY_CAP); + } + + function test_DailyCapIsBoundedSoDecayCannotOverflow() public { + // An "uncap" of type(uint256).max would overflow _decayedConsumed's + // elapsed * dailyCap product and revert every threshold-crossing + // approve() and release() until a second timelocked setCaps. + uint256 maxCap = vault.MAX_DAILY_CAP(); + uint256 huge = maxCap + 1; + vm.prank(admin); + vm.expectRevert(abi.encodeWithSelector(MigrationVault.CapTooLarge.selector, huge)); + vault.setCaps(huge, huge); + + // The largest allowed cap must still leave approve()/release() working. + vm.prank(admin); + vault.setCaps(maxCap, maxCap); + approveAs(0, 0, recipient, 5e12); + approveAs(1, 0, recipient, 5e12); + approveAs(2, 0, recipient, 5e12); + assertEq(pen.balanceOf(recipient), 5e18); + } + + function test_ConstructorRejectsSweepTimestampInPast() public { + vm.expectRevert(MigrationVault.SweepTimestampInPast.selector); + new MigrationVault( + admin, guardian, attestors, 3, CONVERSION_FACTOR, PER_RELEASE_CAP, DAILY_CAP, block.timestamp + ); + } + function test_ConstructorRejectsThresholdBelowTwo() public { vm.expectRevert(MigrationVault.InvalidThreshold.selector); new MigrationVault(admin, guardian, attestors, 1, CONVERSION_FACTOR, PER_RELEASE_CAP, DAILY_CAP, earliestSweep); diff --git a/testing/.env.rehearsal.example b/testing/.env.rehearsal.example index 5b8104227..5cb78e569 100644 --- a/testing/.env.rehearsal.example +++ b/testing/.env.rehearsal.example @@ -34,7 +34,9 @@ RELEASER_PRIVATE_KEY= # release drain by itself inside one session. Production values are validated # separately on Anvil in phase 1, where time can be warped. REHEARSAL_DAILY_CAP_PEN=28800 -REHEARSAL_PER_RELEASE_CAP_PEN=50000 +# Must not exceed the daily cap: the vault enforces perReleaseCap <= dailyCap +# (round 9), since anything in between could never fit the daily allowance. +REHEARSAL_PER_RELEASE_CAP_PEN=28800 # How far ahead the immutable sweep floor sits. Short so the sweep path is # reachable in a long session; production is 2027-03-01. diff --git a/testing/src/rehearsal-deploy.mjs b/testing/src/rehearsal-deploy.mjs index 5f0a61634..1fc0bd822 100644 --- a/testing/src/rehearsal-deploy.mjs +++ b/testing/src/rehearsal-deploy.mjs @@ -19,7 +19,9 @@ const CONTRACTS = path.join(ROOT, "contracts"); export function rehearsalParams(env) { const dailyCapPen = BigInt(env.REHEARSAL_DAILY_CAP_PEN ?? "28800"); - const perReleaseCapPen = BigInt(env.REHEARSAL_PER_RELEASE_CAP_PEN ?? "50000"); + // Equal to the daily cap: the vault rejects perReleaseCap > dailyCap since + // round 9 (that band could never release). + const perReleaseCapPen = BigInt(env.REHEARSAL_PER_RELEASE_CAP_PEN ?? "28800"); return { maxIssuance: 150_000_000n * PEN_18, dailyCap: dailyCapPen * PEN_18, From 8f89a6f1d830eba107d38b80dd662f9ee51945db Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:52:28 +0200 Subject: [PATCH 50/61] testing: Drive the monitor's durable cursors and drill the new alarms Phases 3, 5 and 6 start the monitor with its required start block, start nonce, Base start block and state file, and clear that state between runs. Phase 3 gains two drills: a real vault deficit (via Anvil impersonation) must alert and reach a confirmed auto-pause while a later surplus must not re-trigger it, and a fabricated approval tuple must be caught before quorum. The RB-6 drill rewinds the attestor checkpoint by editing lastProcessedBlock in place, keeping the identity fields the loader requires. Phase 4 exits non-zero on failure. --- testing/src/anvil.mjs | 15 +++++++ testing/src/drills.mjs | 23 ++++++++--- testing/src/phase3-e2e.mjs | 69 ++++++++++++++++++++++++++++++-- testing/src/phase4-zombienet.mjs | 2 +- testing/src/rehearsal.mjs | 14 ++++++- 5 files changed, 112 insertions(+), 11 deletions(-) diff --git a/testing/src/anvil.mjs b/testing/src/anvil.mjs index ee5975160..162820601 100644 --- a/testing/src/anvil.mjs +++ b/testing/src/anvil.mjs @@ -44,6 +44,21 @@ export async function send(account, params) { return pub.waitForTransactionReceipt({ hash }); } +/** Send as an arbitrary address on Anvil. This is intentionally isolated to + * local failure drills; it lets the harness model an impossible token outflow + * from the vault and prove the independent monitor catches it. */ +export async function sendImpersonated(address, params) { + await pub.request({ method: "anvil_impersonateAccount", params: [address] }); + await pub.request({ method: "anvil_setBalance", params: [address, "0x8ac7230489e80000"] }); // 10 ETH + const impersonated = createWalletClient({ account: address, chain: anvilChain, transport: http(RPC) }); + try { + const hash = await impersonated.writeContract({ ...params, account: address }); + return await pub.waitForTransactionReceipt({ hash }); + } finally { + await pub.request({ method: "anvil_stopImpersonatingAccount", params: [address] }); + } +} + /** Attempt a call and return the revert reason instead of throwing. */ export async function expectRevert(account, params) { try { diff --git a/testing/src/drills.mjs b/testing/src/drills.mjs index 21e94de22..c2c91e66b 100644 --- a/testing/src/drills.mjs +++ b/testing/src/drills.mjs @@ -22,7 +22,7 @@ */ import { execSync } from "node:child_process"; -import { mkdirSync, writeFileSync } from "node:fs"; +import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; import path from "node:path"; import { ApiPromise, WsProvider } from "@polkadot/api"; import { Keyring } from "@polkadot/keyring"; @@ -131,7 +131,7 @@ async function main() { section("Attestor fleet"); killMatching(["dist/main.js"]); clearState(["attestor/cp1.json", "attestor/cp2.json", "attestor/cp3.json", "attestor/cp4.json", - "releaser/releaser-state.json"]); + "monitor/monitor-state.json", "releaser/releaser-state.json"]); const pendulumWs = api._options?.provider?.endpoint ?? process.env.PENDULUM_WS; const baseEnv = { BASE_RPC_URL: env.BASE_SEPOLIA_RPC_URL, VAULT_ADDRESS: vault, @@ -139,6 +139,7 @@ async function main() { }; const pendulumHead = (await api.query.system.number()).toString(); const baseHead = String(await ctx.pub.getBlockNumber()); + const monitorStartNonce = (await api.query.tokenMigration.nextNonce()).toString(); const attestorEnv = (i) => ({ ...baseEnv, PENDULUM_WS: pendulumWs, ATTESTOR_PRIVATE_KEY: env[`ATTESTOR_${i}_PRIVATE_KEY`], @@ -148,7 +149,16 @@ async function main() { start(`attestor${i}`, "attestor", attestorEnv(i)); await sleep(3000); } - start("monitor", "monitor", { ...baseEnv, PENDULUM_WS: pendulumWs, GRACE_SECONDS: "300" }); + start("monitor", "monitor", { + ...baseEnv, + PENDULUM_WS: pendulumWs, + GRACE_SECONDS: "300", + GUARDIAN_PRIVATE_KEY: env.GUARDIAN_PRIVATE_KEY, + PENDULUM_START_BLOCK: (BigInt(pendulumHead) + 1n).toString(), + PENDULUM_START_NONCE: monitorStartNonce, + BASE_START_BLOCK: baseHead, + STATE_FILE: "./monitor-state.json", + }); start("releaser", "releaser", { ...baseEnv, RELEASER_PRIVATE_KEY: env.RELEASER_PRIVATE_KEY, START_BLOCK: baseHead }); log(`fleet up (Pendulum from #${pendulumHead}, Base from #${baseHead})`); @@ -239,8 +249,11 @@ async function main() { // RB-6's operator step: restart the re-added attestor with its checkpoint // rewound to before the affected migrations, so it re-scans and re-signs. await stopAndWait("attestor4"); - writeFileSync(path.join(ROOT, "attestor/cp4.json"), - JSON.stringify({ lastProcessedBlock: held.atBlock - 1 })); + // Rewinding is the one sanctioned checkpoint edit (RB-6 step 3): lower + // lastProcessedBlock and keep the identity fields the loader requires — + // a bare { lastProcessedBlock } is rejected as a legacy schema. + const cp4 = path.join(ROOT, "attestor/cp4.json"); + writeFileSync(cp4, JSON.stringify({ ...JSON.parse(readFileSync(cp4, "utf8")), lastProcessedBlock: held.atBlock - 1 })); start("attestor4", "attestor", attestorEnv(4)); await waitReleased(held, "the held migration to release after re-approval"); assertEq(await read(V, "activeApprovals", [heldPayload]), 3n, "final active approvals"); diff --git a/testing/src/phase3-e2e.mjs b/testing/src/phase3-e2e.mjs index 70b397ce2..7c374fee0 100644 --- a/testing/src/phase3-e2e.mjs +++ b/testing/src/phase3-e2e.mjs @@ -21,7 +21,7 @@ import { ApiPromise, WsProvider } from "@polkadot/api"; import { Keyring } from "@polkadot/keyring"; import { cryptoWaitReady } from "@polkadot/util-crypto"; import { assert, assertEq, check, section, summarise } from "./harness.mjs"; -import { accounts, attestors, keys, pub, releaser as releaserAcct, RPC, send } from "./anvil.mjs"; +import { accounts, attestors, keys, pub, releaser as releaserAcct, RPC, send, sendImpersonated } from "./anvil.mjs"; import { erc20Abi, vaultAbi } from "./abi.mjs"; import { deployStack, PARAMS } from "./deploy.mjs"; import { alive, clearState, killStrays, logs, start, stopAll, stopAndWait, waitFor } from "./daemons.mjs"; @@ -70,13 +70,15 @@ async function migrate(amount, baseAddress) { // --- start the fleet ------------------------------------------------------- killStrays(); clearState(["attestor/cp1.json", "attestor/cp2.json", "attestor/cp3.json", "attestor/cp4.json", - "releaser/releaser-state.json"]); + "monitor/monitor-state.json", "releaser/releaser-state.json"]); const baseEnv = { BASE_RPC_URL: RPC, VAULT_ADDRESS: vault, BASE_CHAIN_ID: "31337", POLL_INTERVAL_MS: "2000" }; const startBlock = String(await pub.getBlockNumber()); // Where the attestors begin scanning Pendulum. Must be the current head: a // fork sits at ~7.4M blocks, and starting from 0 would have each daemon walk // every historical block before reaching anything under test. const pendulumHead = (await api.query.system.number()).toString(); +const monitorPendulumStart = (BigInt(pendulumHead) + 1n).toString(); +const monitorStartNonce = (await api.query.tokenMigration.nextNonce()).toString(); console.log(` attestors scan Pendulum from block ${pendulumHead}`); for (let i = 0; i < 4; i++) { @@ -85,7 +87,17 @@ for (let i = 0; i < 4; i++) { ATTESTOR_PRIVATE_KEY: keys[i + 1], CHECKPOINT_FILE: `./cp${i + 1}.json`, START_BLOCK: pendulumHead, }); } -start("monitor", "monitor", { ...baseEnv, PENDULUM_WS: CHOPSTICKS, GRACE_SECONDS: "30" }); +start("monitor", "monitor", { + ...baseEnv, + PENDULUM_WS: CHOPSTICKS, + GRACE_SECONDS: "30", + UNMATCHED_EVENT_GRACE_SECONDS: "0", + GUARDIAN_PRIVATE_KEY: keys[5], + PENDULUM_START_BLOCK: monitorPendulumStart, + PENDULUM_START_NONCE: monitorStartNonce, + BASE_START_BLOCK: startBlock, + STATE_FILE: "./monitor-state.json", +}); start("releaser", "releaser", { ...baseEnv, RELEASER_PRIVATE_KEY: keys[7], START_BLOCK: startBlock }); const recipient = "0x000000000000000000000000000000000000beef"; @@ -190,6 +202,57 @@ await check("conservation holds across the whole run", async () => { assertEq(bal + released + swept, supply, "conservation identity"); }); +await check("a real vault deficit alerts and reaches a confirmed automatic pause", async () => { + const deficit = 1n * 10n ** 18n; + const before = logs("monitor").length; + await sendImpersonated(vault, { ...P, functionName: "transfer", args: [admin.address, deficit] }); + await waitFor(() => logs("monitor").slice(before).includes("VAULT BALANCE DEFICIT"), { + timeoutMs: 30_000, + label: "the monitor deficit alert", + }); + await waitFor(async () => (await read(V, "paused")) === true, { + timeoutMs: 30_000, + label: "the vault to be paused", + }); + await waitFor(() => logs("monitor").slice(before).includes("vault auto-pause confirmed"), { + timeoutMs: 30_000, + label: "confirmed auto-pause reporting", + }); + + // Restore the missing token, then add a harmless one-wei surplus from a + // migrated holder. The monitor must recover without treating the donation + // as another conservation failure. + await send(admin, { ...P, functionName: "transfer", args: [vault, deficit] }); + await sendImpersonated(recipient, { ...P, functionName: "transfer", args: [vault, 1n] }); + const restoredAt = logs("monitor").length; + await waitFor(() => logs("monitor").slice(restoredAt).includes("ok:"), { + timeoutMs: 30_000, + label: "the monitor to accept restored conservation plus a surplus", + }); + await send(admin, { ...V, functionName: "unpause", args: [] }); + const stableAt = logs("monitor").length; + await new Promise((resolve) => setTimeout(resolve, 5_000)); + assert(!logs("monitor").slice(stableAt).includes("VAULT BALANCE DEFICIT"), "surplus retriggered deficit alarm"); + assertEq(await read(V, "paused"), false, "vault remains unpaused after healthy checks"); +}); + +await check("a mismatched Base approval is caught before it can reach quorum", async () => { + const before = logs("monitor").length; + await send(attestors[0], { + ...V, + functionName: "approve", + args: [9_999_999n, admin.address, MIN], + }); + await waitFor(() => logs("monitor").slice(before).includes("MIGRATION TUPLE VIOLATION"), { + timeoutMs: 30_000, + label: "the monitor to reject a fabricated approval tuple", + }); + await waitFor(async () => (await read(V, "paused")) === true, { + timeoutMs: 30_000, + label: "the mismatched approval to trigger an automatic pause", + }); +}); + const ok = summarise(); stopAll(); await api.disconnect(); diff --git a/testing/src/phase4-zombienet.mjs b/testing/src/phase4-zombienet.mjs index 1e1fb1c69..f5f8fde26 100644 --- a/testing/src/phase4-zombienet.mjs +++ b/testing/src/phase4-zombienet.mjs @@ -110,4 +110,4 @@ await check("subscribeFinalizedHeads delivers monotonically increasing heads", a }); await api.disconnect(); -summarise(); +process.exit(summarise() ? 0 : 1); diff --git a/testing/src/rehearsal.mjs b/testing/src/rehearsal.mjs index d8b62ccbc..692f32311 100644 --- a/testing/src/rehearsal.mjs +++ b/testing/src/rehearsal.mjs @@ -292,7 +292,7 @@ async function main() { section("Attestor fleet"); killMatching(["dist/main.js"]); clearState(["attestor/cp1.json", "attestor/cp2.json", "attestor/cp3.json", "attestor/cp4.json", - "releaser/releaser-state.json"]); + "monitor/monitor-state.json", "releaser/releaser-state.json"]); const pendulumWs = api._options?.provider?.endpoint ?? process.env.PENDULUM_WS; const baseEnv = { BASE_RPC_URL: env.BASE_SEPOLIA_RPC_URL, @@ -307,6 +307,7 @@ async function main() { }; const pendulumHead = (await api.query.system.number()).toString(); const baseHead = String(await ctx.pub.getBlockNumber()); + const monitorStartNonce = (await api.query.tokenMigration.nextNonce()).toString(); for (let i = 0; i < 4; i++) { start(`attestor${i + 1}`, "attestor", { ...baseEnv, PENDULUM_WS: pendulumWs, @@ -315,7 +316,16 @@ async function main() { }); await sleep(3000); } - start("monitor", "monitor", { ...baseEnv, PENDULUM_WS: pendulumWs, GRACE_SECONDS: "300" }); + start("monitor", "monitor", { + ...baseEnv, + PENDULUM_WS: pendulumWs, + GRACE_SECONDS: "300", + GUARDIAN_PRIVATE_KEY: env.GUARDIAN_PRIVATE_KEY, + PENDULUM_START_BLOCK: (BigInt(pendulumHead) + 1n).toString(), + PENDULUM_START_NONCE: monitorStartNonce, + BASE_START_BLOCK: baseHead, + STATE_FILE: "./monitor-state.json", + }); start("releaser", "releaser", { ...baseEnv, RELEASER_PRIVATE_KEY: env.RELEASER_PRIVATE_KEY, START_BLOCK: baseHead }); log(`fleet started (Pendulum from #${pendulumHead}, Base from #${baseHead})`); From 12dc4469f0186056824049590afe5dc3d758ea91 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:52:28 +0200 Subject: [PATCH 51/61] docs: Record review rounds 8 and 9 and realign the runbooks Round 8 (durable state, finality gating, governance quorum) and round 9 (multi-agent audit: governance capture, source-chain inflation, cap semantics, daemon regressions) are logged with findings, resolutions, refuted claims and the decisions still open. The runbooks gain an alert vocabulary table and are corrected where they described pre-round-8 behaviour: RB-2 recovery (archive node, never edit the checkpoint), RB-3 source-supply response, RB-6 re-add rewind and the threshold-cut alert, RB-7 accounting for unreleasable burns. --- docs/pen-migration-implementation-overview.md | 18 +- docs/pen-migration-internal-review.md | 338 ++++++++++++++++++ docs/pen-migration-local-test-plan.md | 16 +- docs/pen-migration-review-handover.md | 26 +- docs/pen-migration-runbooks.md | 80 ++++- 5 files changed, 439 insertions(+), 39 deletions(-) diff --git a/docs/pen-migration-implementation-overview.md b/docs/pen-migration-implementation-overview.md index 3ca1393b4..82f050ff1 100644 --- a/docs/pen-migration-implementation-overview.md +++ b/docs/pen-migration-implementation-overview.md @@ -26,19 +26,19 @@ watched by an independent monitor that can auto-pause. One-way by design. | Component | Location | Status | |---|---|---| -| `token-migration` pallet | `pallets/token-migration/` | Burn-and-emit `migrate` (user) + `migrate_treasury`/`set_treasury_destination` (governance, fixed Base destination) extrinsics sharing one nonce space and event; unique nonces, dust/ED + lock handling, KeepAlive treasury withdraw, ships paused, pause origin; 21 unit tests + benchmark test suite (frame-benchmarking v2) | +| `token-migration` pallet | `pallets/token-migration/` | Burn-and-emit `migrate` (user) + one-time-destination `migrate_treasury` path sharing one nonce space and event; unique nonces, dust/ED + lock handling, KeepAlive treasury withdraw, ships paused, pause origin; 22 unit tests + benchmark test suite (frame-benchmarking v2) | | Runtime wiring | `runtime/pendulum/src/lib.rs` | Pallet index 102, minimum migration amount 100 PEN (sized to dominate the attestor fleet's per-migration Base gas, so dust spam cannot grief it), pause = root/half-council or 2/3 technical committee, added to `BaseFilter` whitelist and `define_benchmarks`; compiles with and without `runtime-benchmarks` (Foucoco intentionally skipped — that chain is no longer live; validation is local plus Base Sepolia) | | `PEN.sol` | `contracts/src/` | Fixed-supply `ERC20 + ERC20Permit + ERC20Votes`, EIP-6372 timestamp clock, full supply minted to vault, no owner/mint/proxy | | `MigrationVault.sol` | `contracts/src/` | 3-of-4 on-chain approvals per exact tuple, permanent nonce consumption, 12→18 decimal conversion in one place, per-release + daily caps (defer, not kill), guardian pause (approvals recorded while paused), rotation retroactively invalidates removed attestors, two-step admin, pending-release accounting protecting the timelocked remainder sweep | -| `PENGovernor.sol` | `contracts/src/` | OZ Governor composition through a TimelockController (hybrid governance, timestamp clock) | +| `PENGovernor.sol` | `contracts/src/` | OZ Governor composition through a TimelockController (hybrid governance, timestamp clock); quorum is a fraction of **circulating** supply (total minus the vault's unmigrated balance, parked at a vote sink by `setToken`) bounded below by an absolute `quorumFloor` — a full-supply denominator could make quorum unreachable early and permanently deadlock an unpause (round 8) | | Deploy scripts | `contracts/script/` | `Deploy.s.sol` (vault→token→setToken dance, admin handover to bootstrap Safe), `DeployGovernance.s.sol` (timelock+governor role wiring, deployer admin renounced); parameters documented in `contracts/.env.example` | -| Contract tests | `contracts/test/` | 37 Foundry tests incl. fuzz (supply invariant), full Governor proposal lifecycle, replay/race/rotation/caps/pause/sweep-pending scenarios | -| Attestor daemon | `attestor/` | TypeScript; finalized-heads-only, strictly ordered blocks, crash-safe checkpoint, idempotent + race-tolerant approvals, fail-fast on decode errors (4-field shape asserted), startup set-membership check, low-gas/webhook alerts; ops guide in its README | -| Invariant monitor | `monitor/` | Independent watchdog: conservation checks (block-pinned reads) + per-nonce liveness batched via Multicall3; webhook alerts; optional guardian auto-pause | +| Contract tests | `contracts/test/` | 44 Foundry tests incl. fuzz (supply invariant), full Governor proposal lifecycle and a canceller veto, circulating-supply quorum, caps invariants, replay/race/rotation/caps/pause/sweep-pending scenarios | +| Attestor daemon | `attestor/` | TypeScript; finalized-heads-only, strictly ordered blocks processed at latest Base state with the durable checkpoint trailing at the `safe`/`finalized` boundary (round 8: processing and durability decoupled, so lost races never alert and throughput is submission latency), idempotent + race-tolerant approvals, structural transient-error classification, fail-fast on decode errors (4-field shape asserted), startup set-membership check, low-gas/webhook alerts; ops guide in its README | +| Invariant monitor | `monitor/` | Independent watchdog: durable finalized-Pendulum ↔ safe-Base tuple reconciliation, aggregate conservation, active-approval liveness batched via Multicall3, and receipt/state-confirmed guardian auto-pause. Unmatched Base events page on first sight; a fabricated one is proven (source view past the event's timestamp) and pauses immediately, while a lagging source gets a bounded grace and its own stall paging (round 8) | | Releaser | `releaser/` | Drains cap-deferred releases via the permissionless `release()`; unprivileged gas-only key; classifies self-healing vs governance-blocked failures | | Test harness | `testing/` | Automates all four phases of the local test plan: contracts on Anvil, the pallet against a Chopsticks fork of live mainnet state, the full attestor/monitor/releaser pipeline end to end, and relay-chain finality under Zombienet | | Runbooks | `docs/pen-migration-runbooks.md` | RB-1…RB-7: key compromise, outage, invariant breach, pause/unpause, runtime upgrade, attestor rotation, window close | -| Internal security review | `docs/pen-migration-internal-review.md` | The project's security-assurance record across seven adversarial rounds | +| Internal security review | `docs/pen-migration-internal-review.md` | The project's security-assurance record across nine adversarial rounds (round 9: multi-agent, fresh angles — governance capture, source-chain inflation, economics, supply chain, runbook drift) | ### Portal repo (`feat/pen-base-migration`, PR #655) @@ -66,14 +66,14 @@ restricted to consumed nonces. Details and verified-not-vulnerable list in | Suite | Result | |---|---| -| `cargo test -p token-migration` | 21 (22 with `runtime-benchmarks`) | +| `cargo test -p token-migration` | 22 (23 with `runtime-benchmarks`) | | `cargo check -p pendulum-runtime` | clean, both feature sets | | `forge test` | 37, incl. 512-run fuzz and a full Governor lifecycle | -| `attestor` / `monitor` / `releaser` | 6 / 7 / 7 | +| `attestor` / `monitor` / `releaser` | 8 / 13 / 11 | | Portal `yarn build` (tsc + vite) | clean against `main` | | `testing/src/phase1-base.mjs` | 11/11 against the real deploy script on Anvil | | `testing/src/phase2-pendulum.mjs` | 14/14 against a Chopsticks fork of live mainnet state | -| `testing/src/phase3-e2e.mjs` | 7/7 end to end, four attestors + monitor + releaser | +| `testing/src/phase3-e2e.mjs` | 9 checks defined, including injected deficit + confirmed auto-pause and fabricated-approval tuple rejection; expanded suite requires a running Chopsticks instance and is pending a full rerun | | `testing/src/phase4-zombienet.mjs` | 7/7 against a real relay (~2-block parachain finality lag) | | `testing/src/rehearsal.mjs` | 15/15 full stack: local Zombienet Pendulum + Base Sepolia | | `testing/src/drills.mjs` | 13/13 failure drills (RB-1/3/4/6/7) on the Sepolia stack | diff --git a/docs/pen-migration-internal-review.md b/docs/pen-migration-internal-review.md index 034a75af3..084467697 100644 --- a/docs/pen-migration-internal-review.md +++ b/docs/pen-migration-internal-review.md @@ -465,6 +465,344 @@ post-upgrade extrinsic cannot be submitted through Chopsticks at all, because it serves pre-fork metadata even across `--resume` while executing the new runtime, which rejects the stale-metadata signature as `badProof`. +## Round 8 (2026-08-31, independent reviewer over the round-7+ commits and the uncommitted durable-state/finality work) + +Scope per the round-8 handover: the six attestor/releaser commits since round +7, the uncommitted working-tree changes (durable state in all three daemons, +the monitor's per-tuple reconciliation, Base-finality gating, the pallet's +one-time treasury anchor), governance wiring, runtime wiring, and the portal. +No fund-loss bug was found; the on-chain invariants of rounds 1–7 held under +re-derivation. The significant findings were all in the newest, uncommitted +daemon code — and notably in code that had **never run under real Base +safe-lag**: Anvil serves `safe == latest`, so the phase-3 harness exercises +the finality gating as a no-op, and the 2026-08-28 Sepolia rehearsal predates +it. + +### H1(r8). HIGH — Finality gating made every lost race an alert storm and serialized fleet throughput +The uncommitted change pointed the benign-race recheck (`alreadyHandled`/ +`alreadyHandledSettled`) at the Base `safe` block and put a +`waitForBaseFinality` inline in `approve()`. Consequences: (a) the guaranteed +per-migration race loser saw "not handled" at `safe` for the whole safe-lag +(minutes), fell through to a synthetic "pending finality" transient error, and +re-alerted on every finalized head until `safe` caught up — an alert storm on +the *normal* k-of-n race, the fifth instance of the round-1 class; (b) each +winner blocked serially on its own transaction reaching `safe` before touching +the next event, collapsing per-attestor throughput to roughly one migration +per Base batch interval — hours of backlog under launch-day load, with the +monitor's liveness grace then paging for everything queued. + +**Resolution (fixed, restructured):** processing and durability are decoupled. +Blocks are processed strictly in order against **latest** Base state (lost +races conclude benign from latest, as the rehearsal fix intended), while the +durable checkpoint trails separately: it advances past a block only once every +releasable event in it reads as handled at the `safe`/`finalized` boundary +(`advanceCheckpoint` in `attestor/src/main.ts`). Event-less blocks checkpoint +with zero Base reads. A block whose approvals refuse to settle (reorged away +with nothing replacing them) is re-approved idempotently after +`BASE_FINALITY_TIMEOUT_MS` with a single alert — which also makes reorg +recovery automatic where the previous design needed a restart. Crash recovery +re-processes exactly the non-durable suffix. Transient alerts are throttled to +one per minute (everything still logged). + +### M1(r8). MEDIUM — Unmatched-event grace could false-pause on a stalled source node, and hid a real attack for its duration +Two failure modes of one mechanism: (a) a monitor whose own Pendulum node +served a live-but-stalled finalized view for longer than +`UNMATCHED_EVENT_GRACE_SECONDS` (default 600s) — a stuck peer set or a slow +restart-sync — turned every fresh legitimate approval into `MIGRATION TUPLE +VIOLATION` and auto-paused the vault (post-handover unpause: quorum + ≥48h +timelock); (b) during the grace the monitor only *logged*, and `awaitingSource` +returned before the M2a aggregate check, so a genuinely fabricated tuple +(compromised quorum, high nonce) got up to 600s of cap-bounded releases with +zero paging — a ~10× dwell regression versus the pre-rewrite monitor. + +**Resolution (fixed):** three changes in `monitor/src/`. (1) The first sighting +of an unmatched event **pages immediately** (`UNVERIFIED BASE EVENT`, +deduplicated through the persisted first-seen map), so operators get the whole +grace window. (2) A **fabrication proof** short-circuits the grace: a burn is +relay-finalized strictly before any attestor approves, and Substrate +timestamps are monotone — so once the finalized source view's timestamp passes +the Base event's own block timestamp (plus `SOURCE_CLOCK_SKEW_MARGIN_SECONDS`, +default 120s) and the nonce still does not exist, the event is provably +fabricated and pauses at once (`provablyUnsourced` in `checks.ts`, unit +tested). A merely-lagging source can never satisfy the proof, so node lag +cannot fast-path a false pause. (3) A finalized source view older than +`SOURCE_STALE_ALERT_SECONDS` (default 300s) pages `PENDULUM SOURCE VIEW +STALLED` on its own — before any grace deadline can force a cannot-verify +pause. The grace-expiry pause itself is retained deliberately: a monitor blind +past its grace with unverified releases flowing is a pause-worthy state, and it +now arrives announced. + +### M2(r8). MEDIUM — Full-supply quorum denominator could deadlock governance into a permanent pause +`PENGovernor` quorum was a fraction of *total* past supply — including the +vault's unmigrated ~150M — while only migrated-and-delegated PEN can vote. +Post-handover, a pause (guardian key compromise, monitor false positive, or a +legitimate incident) with quorum unreachable would be **permanent**: unpause, +`setCaps`, `clearStalePending` and `sweepRemainder` are all admin actions +behind the timelock, the paused vault freezes releases, so circulating voting +supply can never grow to quorum — a self-locking deadlock with no alternate +admin path. (Mitigating: the handover-acceptance proposal itself proves quorum +once; the deadlock needed participation to decay afterwards.) + +**Resolution (fixed, design change — needs explicit sign-off):** quorum now +tracks **circulating** supply. `MigrationVault.setToken` one-time-delegates the +vault's balance to a constant dead-address vote sink +(`MigrationVault.VOTE_SINK` == `PENGovernor.QUORUM_SINK`, lockstep asserted in +tests), which checkpoints the unmigrated supply in the token's vote history; +`PENGovernor.quorum()` subtracts the sink's past votes from the denominator and +applies an absolute `quorumFloor` (new constructor/deploy parameter, +`QUORUM_FLOOR`) so early proposals are not trivially cheap. Third parties +delegating to the sink only forfeit their own voting power (strictly dominated +by voting), so the mechanism is not abusable to raise or unfairly drop quorum +below the floor. Release gas grows ~10k for the sink checkpoint update. +Covered by `test_QuorumTracksCirculatingSupplyWithFloor`, +`test_QuorumSinkMatchesVaultVoteSink`, `test_SetTokenParksVaultVotesInSink`. +Handover should still be gated on measured delegated voting power +comfortably exceeding `max(floor, fraction × circulating)`. + +### M3(r8). MEDIUM — Daemon catch-up requires archive state; a >51-minute outage wedged them on default-pruned nodes +Monitor and attestor catch up block by block via `api.at()` + +`system.events()`, which needs per-block state; a default Substrate node prunes +state to 256 blocks (~51 min). Any daemon outage past the horizon wedged the +restart loudly but indefinitely, and the (correct) "never edit the state file" +rule left no sanctioned recovery. **Resolution (documented):** both READMEs now +require `--state-pruning archive` (or `archive-canonical`) and name the +recovery path — point `PENDULUM_WS` at an archive node, never edit state. + +### L1(r8). LOW — Load-balanced `eth_getLogs` could silently truncate a scan range +A pool node behind the `safe` head can serve a truncated log range without +erroring on some providers, letting the monitor/releaser cursor advance past +events never seen (stale pending entries, lost per-tuple evidence; aggregates +unaffected). **Resolution (fixed):** both daemons probe each range's end block +(`getBlock({blockNumber: end})`) before scanning — a lagging node now fails the +cycle loudly and the range replays — and both READMEs recommend a single +dedicated Base endpoint. + +### L2(r8). LOW — Releaser re-paged governance-blocked releases every poll +`ExceedsPerReleaseCap`/`InsufficientVaultBalance` need a ≥48h timelocked action +to clear but alerted every 60s (~2,900 identical pages per timelock period). +**Resolution (fixed):** per-nonce throttling (`BLOCKED_ALERT_INTERVAL_MS`, +default 6h), cleared when the nonce resolves. + +### L3(r8). LOW — Bare numeric status codes in the transient classifier collided with nonce labels +The drills-round word-boundary fix did not cover `nonce=429` (and 502/503/504) +embedded in error labels: a genuine failure at those nonces classified as +transient — an infinite noisy retry instead of the PRD-A5 exit. **Resolution +(fixed):** classification is structural first — `HttpRequestError.status`, +`TimeoutError`, socket error codes, walked through the error `cause` chain — +with a word-only text fallback (bare digit patterns removed entirely, WebSocket +disconnect phrasing added). Extracted to `attestor/src/checks.ts` and unit +tested, including the `nonce=429` regression. The releaser's synthetic +"PendingFinality" message channel was removed along with the inline waits. + +### L4(r8). LOW — Config footguns +`BASE_FINALITY_TAG=finalized` with the 15-minute default timeout guaranteed at +least one spurious timeout per approval; unvalidated numeric envs became NaN +silently (turning backoffs into busy-loops or disabling graces). **Resolution +(fixed):** all numeric envs are validated at startup across the three daemons, +and finality-related defaults scale with the chosen tag (45 min for +`finalized`). The releaser's inline finality wait was removed outright: a +successful `release()` now leaves the durable pending set only when +`pruneConsumed` sees the nonce consumed at the boundary, which is both durable +against reorgs and stall-free. + +### Noted, not fixed (INFO) +Portal: a daily-cap-deferred release shows "waiting for approvals (3/3) … +normally a few minutes" indefinitely (detect `approvals ≥ threshold && +!released` and say it is queued behind the cap); migrate-entire-balance can +fail post-fee (`InsufficientBalance`/`WouldLeaveDust`) because the max button +uses the display float and fees are withdrawn first. Repo hygiene: ~420MB of +untracked local artifacts at the repo root are one `git add .` from landing in +the PR. `spec_version` is still 25 at HEAD; the bump to 26 stays on the +release checklist. + +### Round 8 explicitly verified as not vulnerable +The one-time treasury-destination anchor (origin-gated, zero-address-checked, +tested; root can still reset it via `killStorage`, an accepted bar-raise from +council-majority to root); treasury migrations share the `MigrationInitiated` +event and nonce sequence, so the monitor's contiguous-nonce cursor handles +them; pallet burn atomicity, lock/vesting enforcement, dust/ED, KeepAlive and +the paused-by-default storage default (the Executive migration tuple touches +only xcmp-queue/identity); the `isUnreleasable` ⇄ `approve()` input-revert +lockstep (unchanged, three conditions); generation accounting cannot +double-add `pendingApprovedAmount` across remove/re-add/re-approve; the leaky +bucket under `setCaps` changes; ABI/event parity across vault, attestor, +monitor, releaser and portal (indexing verified field by field); payload-hash +parity incl. the portal's manual `abi.encode` mirror; monitor draft/persist +crash consistency and M2a read ordering; deploy-script role wiring; the vault's +PEN cannot vote (and after r8 is provably parked at the sink). + +### Round 8 fix verification +All local suites pass post-fix: pallet 22, Foundry 40 (3 new), attestor 14 +(6 new classifier tests), monitor 14 (1 new), releaser 11. The attestor/ +monitor/releaser changes are on the fund-release path: per the standing +practice they need a fresh adversarial pass, and phases 3/5/6 must be re-run — +phase 3 cannot exercise the finality trailing (Anvil's `safe == latest`), so +the Sepolia rehearsal is the first environment where the new checkpoint +behavior actually runs under real safe-lag. The M2(r8) governance change +(circulating-supply quorum + floor) alters deployed-parameter semantics and +needs an explicit team decision on `QUORUM_FRACTION`/`QUORUM_FLOOR` values +before phase 5b re-runs. + +## Round 9 (2026-09-05, multi-agent audit over the round-8 tree; fresh angles) + +**Method.** Ten independent finder agents, one lens each — an adversarial pass +over the round-8 fixes, daemon concurrency, fresh-eyes Solidity, and seven +angles no earlier round had taken (economic/MEV, substrate-runtime +interactions, supply chain, secrets/ops hygiene, portal web security, +cross-component drift, test gaps) — followed by dedup against this log, two +adversarial refuters per finding (code-truth and exploitability), a +completeness critic that added four lenses (source-chain inflation, release +artifact drift, governance-capture economics, runbook drift), and a second +targeted round. 37 agents, 45 raw findings; the funnel's own triage was then +re-judged by hand, because it had dropped several material items. Three +findings are defects in round-8 fixes, which is exactly what the standing +practice predicted. + +### H1(r9). HIGH — Nobody could stop a passed hostile proposal during the timelock delay +`DeployGovernance.s.sol` granted `CANCELLER_ROLE` only to the Governor, and +OZ Governor lets only the proposer cancel, only before voting starts. So once +a proposal with quorum-clearing stake passed, the 48h delay was not a +reaction window: one executor transaction could batch `unpause`, +`addAttestor`×3, an unbounded `setCaps`, and three fabricated approvals — the +guardian's pause (the PRD's stated mitigation for this threat) is undone +inside the same batch. **Resolution (fixed, decision pending):** the script +takes an optional `TIMELOCK_CANCELLER` (intended: the guardian or council +Safe) and grants it `CANCELLER_ROLE`; `test_CancellerCanVetoAQueuedProposalDuringTheDelay` +proves the veto against a queued proposal. **The team must set that address +at phase 5** — the env example now says so. + +### H2(r9). HIGH — Governance capture is cheap by construction; no quorum floor prices it out +Quorum stake is bought, not burned, and the prize is the unmigrated vault. +Round 8's circulating-supply quorum was necessary (a full-supply denominator +deadlocks unpause) but makes early capture cheaper still; no floor value +reconciles "honestly reachable soon after launch" with "unprofitable to +capture". No production `QUORUM_FLOOR` was decided anywhere, the env example +still described a total-supply quorum, and the only in-repo value was the +rehearsal's 0. **Resolution:** capture resistance is re-assigned explicitly to +H1's canceller, the guardian pause and the vault caps — not to the floor — +and `contracts/.env.example` now records both parameters as decisions to make +before phase 5. Also noted: the round-8 handover gate ("measured delegated +voting power exceeds quorum") is satisfiable by an attacker's own stake — +measure *diverse* participation. + +### H3(r9). HIGH — Minted-then-burned PEN drains the vault with every check satisfied +A passed Pendulum referendum (1 PEN deposit, `EnsureSigned` submission, root +enactment — the exact vector of the real July 2026 proposal #2) or an +unexpected teleport-in mints PEN; migrating it is a genuine finalized burn, +honestly attested, tuple-matched, and within M2a/M2b — while the immutable +150M vault drains ahead of late honest migrators. No component read Pendulum +issuance; round 7 had cleared only staking inflation. **Resolution (fixed, +alert-only):** the monitor anchors `totalIssuance + TotalMigrated` on first +observation (persisted; that sum cannot grow under any legitimate flow, since +teleport-in can only restore what teleport-out burned) and pages +`SOURCE SUPPLY GREW` on growth beyond `ISSUANCE_TOLERANCE`; RB-3 gained the +response (pause the *pallet*). Pausing is deliberately a human decision. The +governance-side mitigation — council/technical-committee vigilance on opaque +preimage proposals for the window's duration — is a Pendulum governance +matter recorded here as an open decision. + +### M1(r9). MEDIUM — Quorum-approved-but-unreleased migrations were silent everywhere +Three causes, one symptom. (a) Nothing enforced `perReleaseCap ≤ dailyCap`, so +an amount in between passes the per-release check but can never fit the daily +allowance — the rehearsal config itself had that shape (50,000 > 28,800). +(b) The leaky bucket is first-come-first-served with no reservation, so +sustained small self-migrations can starve a large deferred release at +near-zero net cost. (c) A threshold cut can make a payload releasable without +a `ReleasePending`, invisible to the releaser. In all three the releaser +retried quietly, the monitor's liveness loop skipped anything at quorum, and +the portal said "a few minutes". **Resolution (fixed):** the vault enforces +`perReleaseCap ≤ dailyCap` (`CapsInverted`) in the constructor and `setCaps`; +the monitor pages `LIVENESS: quorum reached but not released` after the grace; +the releaser classifies an amount above `dailyCap` itself as blocked +(`ExceedsDailyCapPermanently`) rather than a refill wait; RB-6 step 4 no +longer claims a listing that did not exist. Residual: (b) is now *visible*, +not prevented — a reservation scheme is a vault design change, recorded as a +decision. + +### M2(r9). MEDIUM — Round 8's fabrication proof was unsound under Base timestamp lag +The proof anchored on the Base block timestamp of the unmatched event. OP-stack +L2 timestamps trail real time by the length of a sequencer outage while it +catches up, so after such an outage a merely-lagging monitor node could +"prove" a legitimate event fabricated and pause without grace. **Resolution +(fixed):** `provablyUnsourced` anchors on the monitor's own first-observation +time (the burn is finalized before the approval can be observed at all), and a +future-dated source view (collator clock ahead) proves nothing. Regression +tests cover both; the Base-timestamp fetch is gone. + +### M3(r9). MEDIUM — Monitor catch-up was all-or-nothing per cycle +After a long outage the whole replay ran inside one `check()` and persisted +only at the end; one transient RPC error discarded hours of progress, and a +flaky endpoint could keep it blind indefinitely. **Resolution (fixed):** the +Pendulum-side cursor (self-consistent on its own; the Base cursor only ever +trails it) is checkpointed every 200 blocks during ingest. + +### M4(r9). MEDIUM — `setCaps(type(uint256).max, …)` would brick approve() and release() +`_decayedConsumed` multiplies elapsed seconds by `dailyCap` under checked +arithmetic; an "uncap" overflowed it, reverting every threshold-crossing +`approve()` (a deterministic revert outside `isUnreleasable`, i.e. the +fleet-halt class) until a second timelocked `setCaps`. **Resolution (fixed):** +`MAX_DAILY_CAP = 2^128` enforced with the caps invariant; tested at the bound. + +### Lower-severity, all fixed +- Releaser: a `NonceAlreadyConsumed` decoded from a *latest*-state simulation + could delete a pending entry before consumption was confirmed at the + boundary (round-8 regression) — now classified as pending finality; a mined + revert (no revert data) no longer pages as "unexpected" but is re-evaluated + next cycle. +- Monitor: `securityViolation` awaited the alert webhook *before* pausing, with + no timeout — now pauses first; every daemon's webhook call has a 10s + timeout and redacts URLs (viem error texts quote the RPC endpoint, which + commonly embeds an API key). +- Attestor: a push-driven loop with no watchdog idled silently when the node + stopped finalizing — `HEAD_STALL_ALERT_MS` watchdog added; state reads at + the `safe` block outside a node's retained window (`missing trie node` + while the batcher lags) are classified transient instead of fatal. +- Monitor: a burn to the zero/vault address kept the pending count non-zero + forever (paging liveness every grace period and making RB-7's precondition + unsatisfiable) — paged once as `CRITICAL: unreleasable migration burned` + and excluded from reconciliation; RB-7 step 3 accounts for them. +- Vault constructor rejects an `earliestSweepTimestamp` in the past. +- The RB-6 drill rewound a checkpoint by writing the legacy shape the round-8 + loader rejects — fixed; the sanctioned rewind procedure is now in RB-6. +- Runbooks re-aligned with post-round-8 behaviour: alert vocabulary table, + RB-1 trigger, RB-2 recovery (archive node, never edit the checkpoint, + `START_BLOCK` applies only without a checkpoint), RB-3 step 6, RB-6 steps 3–4. + +### Noted, not fixed — decisions and other repos +- **Spam economics (MEDIUM):** the 100 PEN minimum is retained capital, not a + cost — a self-migration returns it on Base — so spam is bounded only by the + spammer's Pendulum holdings and the daily cap, which the spam then consumes + (feeding M1(b)). Options: a burned-only fee on `migrate`, a per-account rate + limit, or accepting it with the monitor's new paging. Decision. +- **Portal (separate repo):** `NumericInput` paste strips commas as thousands + separators, so a pasted `500,5` migrates 5005 PEN (to the user's own Base + address — not lost, but 10× and irreversible; the same paste path predates + the numora switch); the smart-contract and above-cap warnings fail open + while their RPC reads are loading or failed; confirmation checkboxes do not + reset when the address or amount changes; the headline copy says "3-of-5" + where the deployed set is 3-of-4. +- **Process:** no CI job runs the Foundry or daemon suites; testing scripts + pass the deployer key on the forge argv (throwaway keys — use `--account`/a + keystore for production deploys); phase 4's header claims to prove attestor + finality gating but starts no attestor; the phase-3 fabricated-approval + drill passes through the grace-expiry path (grace 0), never the proof path. + +### Round 9 explicitly refuted +The live PEN teleport channel does not invalidate `MAX_ISSUANCE` (teleports are +supply-conserving; 150M was rounded up from ~149.93M live issuance, and +AssetHub's PEN can only originate from Pendulum burns). A compromised quorum +cannot permanently brick `sweepRemainder`: the fabricated nonce is consumable +by the rotated honest quorum, after which `clearStalePending` clears it (its +documented purpose, tested). + +### Round 9 fix verification +Local suites after the fixes: pallet 22, Foundry 44 (4 new), attestor 14, +monitor 16 (2 new), releaser 12 (1 new). The same standing practice applies +as after round 8 — these changes touch the vault (caps invariants, sweep +guard), the governance deployment, and all three daemons, so they need a +fresh adversarial pass and the Sepolia re-run, with `REHEARSAL_PER_RELEASE_CAP_PEN` +now equal to the daily cap and `TIMELOCK_CANCELLER`/`QUORUM_FLOOR` set. + ## Residual risks and standing practices (no external audit — risk accepted) - Every change to the fund-release path (vault release/approve/sweep logic, pallet burn path) gets a fresh independent adversarial review round before diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 66930de12..d3cd761b4 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -175,6 +175,17 @@ BASE_RPC_URL=http://localhost:8545 VAULT_ADDRESS=0x… \ RELEASER_PRIVATE_KEY=0x… START_BLOCK=0 npm start ``` +The monitor's first-run boundaries are mandatory: use the first Pendulum block +that can contain a migration (plus the nonce expected there) and the vault's +Base deployment block. Afterwards its atomic state file owns both cursors: + +```bash +PENDULUM_WS=ws://127.0.0.1:9944 BASE_RPC_URL=http://localhost:8545 \ +VAULT_ADDRESS=0x… GUARDIAN_PRIVATE_KEY=0x… \ +PENDULUM_START_BLOCK=123 PENDULUM_START_NONCE=0 BASE_START_BLOCK=45 \ +STATE_FILE=./monitor-state.json npm start +``` + Checks: 1. **Full path:** unpause the pallet, `migrate` from a funded account → within @@ -204,7 +215,10 @@ Checks: `VITE_MIGRATION_VAULT_ADDRESS` set to the Anvil vault; migrate through the UI and watch the status card go 0/3 → 3/3 → released. -**Pass:** 7/7 from the script. +**Pass criterion:** 9/9 from the script. The final two checks impersonate the vault on +Anvil to inject a real deficit and confirm the guardian pause through Base +finality/state verification, then submit a fabricated one-attestor approval to +prove the monitor rejects a tuple mismatch before it can reach quorum. Three traps cost real debugging time and are worth knowing before you run it: a wasm built with `--features runtime-benchmarks` cannot be used as a diff --git a/docs/pen-migration-review-handover.md b/docs/pen-migration-review-handover.md index 68b1d0b49..18a73979b 100644 --- a/docs/pen-migration-review-handover.md +++ b/docs/pen-migration-review-handover.md @@ -15,7 +15,8 @@ daemons watch relay-**finalized** blocks and submit `approve(nonce, recipient, amount)` to a vault on Base; the 3rd matching approval releases pre-minted fixed-supply ERC-20 PEN (12→18 decimals, ×1e6, converted in exactly one place). Rate caps bound worst-case loss; an -independent monitor holds a conservation invariant and can auto-pause; a +independent monitor reconciles every safe Base tuple to its finalized Pendulum +source, holds aggregate conservation invariants, and can auto-pause; a permissionless releaser drains cap-deferred releases. Governance: OZ Governor + Timelock on Base, with the vault admin handed to the timelock by proposal. @@ -72,8 +73,8 @@ Every real bug found post-unit-tests belongs to one of these. The most likely Try to falsify these directly — each is load-bearing: -- `vault.balanceOf + totalReleased + totalSwept == PEN.totalSupply`, with - surplus tolerated and only deficit alarming. +- `vault.balanceOf + totalReleased + totalSwept >= PEN.totalSupply`, with + equality under vault-controlled flows, surplus tolerated, and only a deficit alarming. - A nonce releases at most once, ever, across user AND treasury migrations (one shared sequence); no tuple `(nonce, recipient, amount)` can be released with different arguments than were approved. @@ -102,19 +103,20 @@ Try to falsify these directly — each is load-bearing: classifier): newest fund-release-path code, reviewed once, by the author. Specifically: can `isTransientRpcError` misclassify anything fatal as transient (silent-stall) or vice versa (fleet death)? Can the - `alreadyHandledSettled` backoff interact badly with checkpointing or the - serialized block-processing promise chain? + safe/finalized `alreadyHandledSettled` recheck and latest-state pending + detection interact badly with durable checkpointing or the serialized + block-processing promise chain? 2. **The portal UI** — one full-diff pass (r4) only. Amount parsing and decimal display, EIP-55 handling in `src/helpers/ethereum.ts`, the payload-hash mirror of the vault's `abi.encode`, and what the status card does on RPC lag or a deferred release. 3. **`migrate_treasury` / `set_treasury_destination`** — the governance-only burn path; less exercised than user `migrate`. Check origin gating, the - fixed-destination logic, KeepAlive semantics against the real treasury + one-time destination anchor, KeepAlive semantics against the real treasury account. -4. **Monitor liveness (M4)** — `nonceFirstSeen` map growth, alert - deduplication, GRACE_SECONDS interaction with finality lag; r5/r7 touched - it twice, which historically predicts a third issue. +4. **Monitor reconciliation/liveness** — durable two-chain cursor and pending + tuple growth, active-approval alert deduplication, and the bounded + `UNMATCHED_EVENT_GRACE_SECONDS` interaction with independently lagging RPCs. 5. **Governance wiring** — `PENGovernor.sol` composition and `DeployGovernance.s.sol` ran on-chain for the first time on 2026-08-31. Quorum counts For+Abstain against **full** `totalSupply` (the unreleased @@ -138,9 +140,9 @@ Try to falsify these directly — each is load-bearing: ## Running things ```bash -cargo test -p token-migration # 21 (22 with --features runtime-benchmarks) -cd contracts && forge test # 37 -cd attestor && npm test # 6 (monitor: 7, releaser: 7) +cargo test -p token-migration # 22 (23 with --features runtime-benchmarks) +cd contracts && forge test # 44 +cd attestor && npm test # 14 (monitor: 16, releaser: 12) node testing/src/phase1-base.mjs # needs: anvil --port 8545 node testing/src/phase2-pendulum.mjs # needs: chopsticks per docs/pen-migration-local-test-plan.md ``` diff --git a/docs/pen-migration-runbooks.md b/docs/pen-migration-runbooks.md index d87187054..34ffec5e4 100644 --- a/docs/pen-migration-runbooks.md +++ b/docs/pen-migration-runbooks.md @@ -11,6 +11,22 @@ tech lead → guardian Safe signers → admin Safe signers. --- +## Alert vocabulary (post round 8/9) → runbook + +| Alert | Source | Meaning | Go to | +|---|---|---|---| +| `MIGRATION TUPLE VIOLATION` | monitor | A safe Base `Approved`/`Released` contradicts (or provably lacks) a finalized Pendulum burn. Auto-pause fires with it. | RB-1 / RB-3 | +| `UNVERIFIED BASE EVENT` | monitor | A safe Base event whose nonce the monitor's Pendulum view does not know *yet*. Either the monitor's node lags (see `PENDULUM SOURCE VIEW STALLED`) or the event is fabricated; escalates to a violation once provable or after `UNMATCHED_EVENT_GRACE_SECONDS`. | check the monitor's node first; RB-1 if it escalates | +| `PENDULUM SOURCE VIEW STALLED` | monitor | The monitor's finalized Pendulum view is stale — it is going blind. Restore the node before the grace forces a cannot-verify pause. | RB-2 (monitor's node) | +| `CONSERVATION VIOLATION` / `VAULT BALANCE DEFICIT` | monitor | Aggregate loss on Base. Auto-pause fires with it. | RB-3 | +| `SOURCE SUPPLY GREW` | monitor | Pendulum `totalIssuance + TotalMigrated` grew: PEN was minted at the source (a referendum, an unexpected teleport-in) and can be burned against the fixed-supply vault. Not a vault fault; a human decision. | RB-3 step 6 | +| `LIVENESS: migration below approval quorum` | monitor | Attestors are not approving. | RB-2 | +| `LIVENESS: quorum reached but not released` | monitor | Approved but unreleased: cap-deferred and starved, paused, or made releasable by a threshold cut with no `ReleasePending` (the releaser cannot see those). | RB-6 step 4 / RB-4 | +| `CRITICAL: unreleasable migration burned` | monitor / attestor | A burn to the zero or vault address; can never release; excluded from the monitor's pending count. | note for RB-7; nothing to do | +| `release blocked and needs operator action` | releaser | Above the per-release cap, above the daily cap itself, or under-funded vault: needs a governance action. | RB-4 / RB-7 | +| `approvals not durable at the Base finality boundary` | attestor | A block's approvals were reorged away or Base finality is stalled; re-submitted automatically. | watch; RB-2 if persistent | +| `no finalized Pendulum head received` | attestor | The attestor's node stopped finalizing or its subscription died. | RB-2 | + > **Confirming state against a public RPC.** Public endpoints are load-balanced > and give no read-after-write guarantee: a read issued straight after a > confirmed transaction can land on a node that has not imported that block yet, @@ -22,8 +38,12 @@ tech lead → guardian Safe signers → admin Safe signers. ## RB-1: Suspected attestor key compromise **Trigger:** an `Approved` event from an attestor for a tuple that does not -match any finalized Pendulum `MigrationInitiated` event (monitor M2a alert, or -manual observation), or an operator reports infrastructure compromise. +match any finalized Pendulum `MigrationInitiated` event (monitor +`MIGRATION TUPLE VIOLATION`, or manual observation), or an operator reports +infrastructure compromise. An `UNVERIFIED BASE EVENT` page on its own is not +yet evidence: it also fires when the monitor's Pendulum node merely lags — +check for `PENDULUM SOURCE VIEW STALLED` first. If the node is healthy the +monitor escalates to the violation (and pauses) within minutes. 1. **Pause first, investigate second.** Any guardian signer (or the monitor's auto-pause) calls `vault.pause()`. Releases stop; approvals keep recording. @@ -53,10 +73,15 @@ period) or an attestor's own restart-loop/low-gas alerts. harmlessly (approvals missing, nothing to roll back) — escalate to the affected operators. 2. Common causes, in order of frequency: Base gas wallet empty (fund it; the - daemon logs the address), Pendulum node not synced/finalizing, checkpoint - file pointing at a pruned block (re-point `START_BLOCK` at a block the node - still has, never past unprocessed migrations), daemon restart-loop after a - runtime upgrade (see RB-5). + daemon logs the address), Pendulum node not synced/finalizing, the node no + longer holding the historical state the daemon needs to catch up (an + outage longer than the node's pruning horizon — the daemon wedges loudly + on restart: point `PENDULUM_WS` at an archive node; **never edit the + checkpoint**, and note `START_BLOCK` only applies when no checkpoint file + exists), Base RPC unable to serve state at the `safe` block while the + batcher lags (`missing trie node` — the daemon waits it out; if it + persists, use an endpoint with deeper state history), daemon restart-loop + after a runtime upgrade (see RB-5). 3. After recovery the daemon catches up from its checkpoint automatically; duplicate approvals are impossible (pre-checks + contract dedup). 4. Verify recovery: the queued nonces release as approvals arrive; monitor @@ -87,6 +112,15 @@ vault address as a release recipient at `approve`. policy. Consider whether caps need lowering before resumption. 5. Unpause only with sign-off from the admin Safe quorum and a written incident report. +6. **`SOURCE SUPPLY GREW`** is the source-chain variant: no vault invariant is + broken, but Pendulum now holds more PEN than the vault was sized for, and + every burn of it is a genuine burn the attestors will honestly release. + Identify the mint (a referendum enacting `setBalance`/`update_balance`, a + teleport-in) from the Pendulum explorer. If it is not a legitimate + teleport round-trip: pause the **pallet** (`setPaused(true)`, RB-4) to stop + further burns while governance decides, and treat any migration of the + minted PEN as loss for disclosure purposes. The monitor cannot pause the + pallet; only governance can. ## RB-4: Pause / unpause (routine procedure) @@ -145,9 +179,13 @@ first. plus a generous attestor-processing margin; hours, not minutes) so every already-finalized migration reaches the vault and is released. 3. **Reconcile via the monitor:** confirm `TotalMigrated × conversionFactor == - totalReleased + pendingApprovedAmount` and that the monitor reports **zero** - outstanding/unreleased nonces for a sustained window. Resolve any pending - or deferred releases (raise caps / unpause / `release`) before proceeding. + totalReleased + pendingApprovedAmount + unreleasable` and that the monitor + reports **zero** outstanding/unreleased nonces for a sustained window. + `unreleasable` is the sum of burns to the zero or vault address (each paged + once as `CRITICAL: unreleasable migration burned`); the monitor excludes + them from its pending count because they can never release. Resolve any + pending or deferred releases (raise caps / unpause / `release`) before + proceeding. 4. **Compute the sweep amount** off-chain: `balance − pendingApprovedAmount`, and sanity-check it against expected unmigrated supply. Do not sweep more. 5. **Sweep:** admin (timelock) calls `sweepRemainder(destination, amount)`. @@ -173,14 +211,22 @@ calls `release(nonce, recipient, palletAmount)`. their daemon to be live and approving, then `removeAttestor(old)`. 3. Removed attestors' recorded approvals stop counting immediately; pending migrations that relied on them simply need approvals from the remaining - set (the new attestor's daemon backfills from its `START_BLOCK` — set it - to a block before the oldest unreleased migration). + set. A **new** key with no checkpoint file backfills from its + `START_BLOCK` — set it to a block before the oldest unreleased migration. + A **re-added** existing daemon must re-sign under its new generation: + stop it, lower `lastProcessedBlock` in its checkpoint file to a block + before the oldest unreleased migration **keeping every other field** (the + identity fields are required; a bare `{lastProcessedBlock}` is rejected), + and restart. Rewinding backwards is the one sanctioned checkpoint edit — + re-processing is idempotent. 4. **Lowering the threshold** (`setThreshold` to a smaller value) can make a payload that was one approval short suddenly releasable, *without* routing through `approve()` — so its owed amount is not registered in - `pendingApprovedAmount`. The contract guards this: `sweepRemainder` is - blocked for `SWEEP_SETTLING_PERIOD` (7 days) after any threshold decrease. - During that window, call `release(...)` on every migration that the new, - lower threshold now satisfies (the M4 liveness monitor lists them), so each - is properly released or re-registered before the next sweep. Never sweep - right after cutting the threshold. + `pendingApprovedAmount` and no `ReleasePending` was ever emitted, which + means the **releaser cannot see it**. The contract guards the sweep: + `sweepRemainder` is blocked for `SWEEP_SETTLING_PERIOD` (7 days) after any + threshold decrease. During that window, call `release(...)` on every + migration that the new, lower threshold now satisfies — the monitor pages + `LIVENESS: quorum reached but not released` for each once it is older than + `GRACE_SECONDS` — so each is properly released or re-registered before the + next sweep. Never sweep right after cutting the threshold. From 6ac055e1631d0d918fd8dd80f515c5f41dfc8a20 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 15:52:28 +0200 Subject: [PATCH 52/61] ci: Run the Foundry and daemon suites The migration stack lives outside the Rust workspace, so test-code.yml never ran it: the contracts and the three daemons were only ever tested by hand. A green check now means the suites the review log cites ran. --- .github/workflows/migration-tests.yml | 65 +++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 .github/workflows/migration-tests.yml diff --git a/.github/workflows/migration-tests.yml b/.github/workflows/migration-tests.yml new file mode 100644 index 000000000..2d2c5a5eb --- /dev/null +++ b/.github/workflows/migration-tests.yml @@ -0,0 +1,65 @@ +name: Migration Tests + +# The PEN -> Base migration stack lives outside the Rust workspace: the +# Foundry contracts and the three TypeScript daemons. `test-code.yml` covers +# the pallet through `cargo test`; this job covers everything else, so a +# green check on the PR means the suites the review log cites actually ran. + +on: + pull_request: + paths: + - "contracts/**" + - "attestor/**" + - "monitor/**" + - "releaser/**" + - ".github/workflows/migration-tests.yml" + push: + branches: + - main + +jobs: + contracts: + name: Foundry (contracts) + runs-on: ubuntu-latest + defaults: + run: + working-directory: contracts + steps: + - uses: actions/checkout@v5 + with: + submodules: recursive + + - uses: foundry-rs/foundry-toolchain@v1 + with: + version: stable + + - name: Build + run: forge build --sizes + + - name: Test + run: forge test -vvv + + daemons: + name: ${{ matrix.package }} (node) + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + package: [attestor, monitor, releaser] + defaults: + run: + working-directory: ${{ matrix.package }} + steps: + - uses: actions/checkout@v5 + + - uses: actions/setup-node@v4 + with: + node-version: 20 + cache: npm + cache-dependency-path: ${{ matrix.package }}/package-lock.json + + - name: Install + run: npm ci + + - name: Typecheck and test + run: npm test From 6622d94fa10e3120e1bda0a91e4d66a95332f6fa Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 16:04:32 +0200 Subject: [PATCH 53/61] attestor: Treat a pre-vault finality boundary as not yet durable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The durability check reads the vault at the safe/finalized block. On a chain whose boundary still trails the vault's deployment (a fresh deployment on a testnet, or Anvil, whose safe tag is genesis until the chain is an epoch deep) that read returns no data, and the attestor treated it as fatal: every attestor exited on its first checkpoint, with nothing to read afterwards. A zero-data read at the boundary now means "nothing can be durable there yet" — the checkpoint simply waits. --- attestor/src/main.ts | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/attestor/src/main.ts b/attestor/src/main.ts index 20f41e6f1..d68166aa5 100644 --- a/attestor/src/main.ts +++ b/attestor/src/main.ts @@ -23,6 +23,8 @@ import { ApiPromise, WsProvider } from "@polkadot/api"; import { + BaseError, + ContractFunctionZeroDataError, createPublicClient, createWalletClient, defineChain, @@ -301,12 +303,23 @@ async function main(): Promise { let lastTransientAlertMs = 0; /** All releasable events of `entry` are resolved at `blockNumber` — either - * the nonce is consumed (released) or our approval is recorded there. */ + * the nonce is consumed (released) or our approval is recorded there. A + * boundary block that predates the vault (a fresh deployment on a chain + * whose safe head still trails it) cannot have anything durable in it: + * the read returns no data, which means "not yet", never "fatal". */ async function eventsDurableAt(entry: UnconfirmedBlock, blockNumber: bigint): Promise { - for (const event of entry.events) { - if (!(await alreadyHandledAt(event, blockNumber))) return false; + try { + for (const event of entry.events) { + if (!(await alreadyHandledAt(event, blockNumber))) return false; + } + return true; + } catch (error) { + if (error instanceof BaseError && error.walk((e) => e instanceof ContractFunctionZeroDataError)) { + log(`vault has no state at ${config.baseFinalityTag} block ${blockNumber} yet; checkpoint waits`); + return false; + } + throw error; } - return true; } /** Advance the durable checkpoint through every leading block whose events From 1622aea4320cc44260250761f47460a91a06496b Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 16:04:33 +0200 Subject: [PATCH 54/61] testing: Tee daemon logs; require an Anvil whose safe tag tracks latest Daemon output was kept only in the harness's in-memory ring buffer, so a silent attestor exit mid-run left no stderr to read. Every daemon's output is now also written to testing/.logs/.log. Anvil resolves safe/finalized to genesis until the chain is 32 blocks deep, which handed the daemons' finality-boundary reads a pre-vault block. The harness now refuses to run unless Anvil's safe tag tracks latest, and the documented command is anvil --port 8545 --slots-in-an-epoch 0. --- docs/pen-migration-local-test-plan.md | 2 +- docs/pen-migration-review-handover.md | 2 +- testing/.gitignore | 1 + testing/src/anvil.mjs | 15 +++++++++++++++ testing/src/daemons.mjs | 12 ++++++++++-- testing/src/phase1-base.mjs | 2 +- 6 files changed, 29 insertions(+), 5 deletions(-) diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index d3cd761b4..8f1ea9ac1 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -56,7 +56,7 @@ Goal: the contracts behave as specified against a real EVM, and the deploy script works with realistic parameters. ```bash -anvil --port 8545 # terminal 1 +anvil --port 8545 --slots-in-an-epoch 0 # terminal 1 cd contracts cp .env.example .env # fill in: 4 attestor addrs, Safes, caps, MAX_ISSUANCE, EARLIEST_SWEEP_TS forge script script/Deploy.s.sol --rpc-url http://localhost:8545 --broadcast diff --git a/docs/pen-migration-review-handover.md b/docs/pen-migration-review-handover.md index 18a73979b..ab3496b56 100644 --- a/docs/pen-migration-review-handover.md +++ b/docs/pen-migration-review-handover.md @@ -143,7 +143,7 @@ Try to falsify these directly — each is load-bearing: cargo test -p token-migration # 22 (23 with --features runtime-benchmarks) cd contracts && forge test # 44 cd attestor && npm test # 14 (monitor: 16, releaser: 12) -node testing/src/phase1-base.mjs # needs: anvil --port 8545 +node testing/src/phase1-base.mjs # needs: anvil --port 8545 --slots-in-an-epoch 0 node testing/src/phase2-pendulum.mjs # needs: chopsticks per docs/pen-migration-local-test-plan.md ``` diff --git a/testing/.gitignore b/testing/.gitignore index 50f41b552..b44d1f6fd 100644 --- a/testing/.gitignore +++ b/testing/.gitignore @@ -1,3 +1,4 @@ node_modules/ .chopsticks-db.sqlite* .chopsticks-e2e.sqlite* +.logs/ diff --git a/testing/src/anvil.mjs b/testing/src/anvil.mjs index 162820601..f71c1cf0d 100644 --- a/testing/src/anvil.mjs +++ b/testing/src/anvil.mjs @@ -35,6 +35,21 @@ export const anvilChain = defineChain({ }); export const pub = createPublicClient({ chain: anvilChain, transport: http(RPC) }); + +// The daemons pin their durability reads to Base `safe`/`finalized`. Anvil +// resolves those tags to GENESIS until the chain is an epoch (32 blocks) deep, +// so a default Anvil hands every attestor a pre-vault block on its first +// checkpoint and the fleet dies silently (found the hard way). Refuse to run +// against an Anvil started without `--slots-in-an-epoch 0`. +{ + const [latest, safe] = await Promise.all([pub.getBlock({ blockTag: "latest" }), pub.getBlock({ blockTag: "safe" })]); + if (safe.number !== latest.number) { + throw new Error( + `Anvil at ${RPC} resolves "safe" to block ${safe.number} while latest is ${latest.number}; ` + + "start it with `anvil --port 8545 --slots-in-an-epoch 0` so the daemons' finality-boundary reads see the vault", + ); + } +} export const wallet = (account) => createWalletClient({ account, chain: anvilChain, transport: http(RPC) }); /** Send a contract call and wait for it to land, returning the receipt. */ diff --git a/testing/src/daemons.mjs b/testing/src/daemons.mjs index 33465afb3..a0e3c3138 100644 --- a/testing/src/daemons.mjs +++ b/testing/src/daemons.mjs @@ -4,7 +4,7 @@ */ import { execSync, spawn } from "node:child_process"; -import { rmSync } from "node:fs"; +import { createWriteStream, mkdirSync, rmSync } from "node:fs"; import path from "node:path"; import { fileURLToPath } from "node:url"; @@ -14,20 +14,28 @@ const running = new Map(); /** Start a built service. Output is captured so a crash is diagnosable, and * `exited` records whether the process died -- the attestor race regression * is precisely "a daemon that should have kept running did not". */ +const LOG_DIR = path.join(ROOT, "testing", ".logs"); + export function start(name, dir, env) { const proc = spawn("node", ["dist/main.js"], { cwd: path.join(ROOT, dir), env: { ...process.env, ...env }, stdio: ["ignore", "pipe", "pipe"], }); + // Tee every daemon's output to testing/.logs/.log as well: the + // in-memory ring buffer dies with the harness, which left a silent attestor + // exit during a phase-3 run with no stderr to read afterwards. + mkdirSync(LOG_DIR, { recursive: true }); + const file = createWriteStream(path.join(LOG_DIR, `${name}.log`), { flags: "a" }); const rec = { proc, name, out: [], exited: null }; const capture = (chunk) => { + file.write(chunk); for (const line of String(chunk).split("\n")) if (line.trim()) rec.out.push(line); if (rec.out.length > 500) rec.out.splice(0, rec.out.length - 500); }; proc.stdout.on("data", capture); proc.stderr.on("data", capture); - proc.on("exit", (code) => { rec.exited = code ?? -1; }); + proc.on("exit", (code) => { rec.exited = code ?? -1; file.end(`\n[harness] ${name} exited with code ${code}\n`); }); running.set(name, rec); return rec; } diff --git a/testing/src/phase1-base.mjs b/testing/src/phase1-base.mjs index 25f42221f..e0abf769f 100644 --- a/testing/src/phase1-base.mjs +++ b/testing/src/phase1-base.mjs @@ -6,7 +6,7 @@ * handover. That is the point: unit tests exercise the contract, this * exercises the thing we will actually ship and the script that ships it. * - * anvil --port 8545 + * anvil --port 8545 --slots-in-an-epoch 0 * node testing/src/phase1-base.mjs */ From 224ff6d10a63d4f721130583b9745438b4aed1a9 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 16:10:35 +0200 Subject: [PATCH 55/61] releaser: Buffer the release() gas limit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A release writes an ERC20Votes checkpoint for the vault's vote sink, and whether that is an overwrite or a new entry depends on the block timestamp the transaction lands in — a state the estimate cannot know. Phase 3 saw an exact estimate run out of gas in that write. Double it, as the attestor already does for approve(). --- releaser/src/main.ts | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/releaser/src/main.ts b/releaser/src/main.ts index 53a6be957..f54b1228e 100644 --- a/releaser/src/main.ts +++ b/releaser/src/main.ts @@ -232,7 +232,19 @@ async function drainPending(conversionFactor: bigint, dailyCap: bigint): Promise functionName: "release", args: [p.nonce, p.recipient, p.palletAmount], }); - const txHash = await walletClient.writeContract(request); + // Pad the gas limit: a release writes an ERC20Votes checkpoint for the + // vault's vote sink, and whether that is an overwrite or a new entry + // depends on the block timestamp the transaction lands in — a state + // the estimate cannot know. An exact estimate can run out of gas in + // that write (seen in phase 3); doubling it is cheap insurance. + const gasLimit = (await publicClient.estimateContractGas({ + account, + address: config.vaultAddress, + abi: vaultAbi, + functionName: "release", + args: [p.nonce, p.recipient, p.palletAmount], + })) * 2n; + const txHash = await walletClient.writeContract({ ...request, gas: gasLimit }); const receipt = await publicClient.waitForTransactionReceipt({ hash: txHash }); if (receipt.status !== "success") { throw new MinedRevert(`release reverted on-chain: ${txHash}`); From 67702bd6764b01c5287d576e34fc6fcceec46af4 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 16:10:35 +0200 Subject: [PATCH 56/61] testing: Pass explicit gas in the harness; keep the drill's caps equal Anvil fills a missing gas limit from an estimate taken at the current wall-clock second; a PEN transfer touching the vault writes a vote-sink checkpoint keyed by timestamp, so the estimate (overwrite) undershoots the execution one second later (new entry) and the deficit drill's impersonated transfer ran out of gas. Every harness transaction now carries an explicit limit. The cap-deferral drill sets equal caps: the vault rejects perReleaseCap > dailyCap since round 9. --- testing/src/anvil.mjs | 12 ++++++++++-- testing/src/phase3-e2e.mjs | 5 ++++- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/testing/src/anvil.mjs b/testing/src/anvil.mjs index f71c1cf0d..1305dca6b 100644 --- a/testing/src/anvil.mjs +++ b/testing/src/anvil.mjs @@ -11,6 +11,14 @@ import { privateKeyToAccount } from "viem/accounts"; export const RPC = process.env.BASE_RPC_URL ?? "http://127.0.0.1:8545"; +/** Explicit gas for every harness transaction. Anvil fills a missing limit + * from eth_estimateGas, which runs at the current wall-clock second; the + * vault delegates its votes to the sink, so a PEN transfer touching the vault + * writes an ERC20Votes checkpoint keyed by block timestamp — an OVERWRITE at + * estimation time becomes a NEW entry when the block lands one second later, + * and the exact estimate runs out of gas in the checkpoint write. */ +const HARNESS_GAS = 1_000_000n; + const KEYS = [ "0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80", // 0 deployer "0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d", // 1 attestor A @@ -55,7 +63,7 @@ export const wallet = (account) => createWalletClient({ account, chain: anvilCha /** Send a contract call and wait for it to land, returning the receipt. */ export async function send(account, params) { const { request } = await pub.simulateContract({ account, ...params }); - const hash = await wallet(account).writeContract(request); + const hash = await wallet(account).writeContract({ ...request, gas: HARNESS_GAS }); return pub.waitForTransactionReceipt({ hash }); } @@ -67,7 +75,7 @@ export async function sendImpersonated(address, params) { await pub.request({ method: "anvil_setBalance", params: [address, "0x8ac7230489e80000"] }); // 10 ETH const impersonated = createWalletClient({ account: address, chain: anvilChain, transport: http(RPC) }); try { - const hash = await impersonated.writeContract({ ...params, account: address }); + const hash = await impersonated.writeContract({ gas: HARNESS_GAS, ...params, account: address }); return await pub.waitForTransactionReceipt({ hash }); } finally { await pub.request({ method: "anvil_stopImpersonatingAccount", params: [address] }); diff --git a/testing/src/phase3-e2e.mjs b/testing/src/phase3-e2e.mjs index 7c374fee0..ed25769c0 100644 --- a/testing/src/phase3-e2e.mjs +++ b/testing/src/phase3-e2e.mjs @@ -155,8 +155,11 @@ section("Cap deferral and the releaser"); await check("a cap-deferred release is drained by the releaser, unattended", async () => { // Squeeze the daily budget so the next migration cannot release immediately. + // Both caps equal one minimum-sized migration: the vault rejects + // perReleaseCap > dailyCap (round 9), and a MIN migration still fits the + // per-release cap exactly while the second one must wait for the refill. const tiny = MIN * CF; // one minimum-sized migration's worth - await send(admin, { ...V, functionName: "setCaps", args: [tiny * 10n, tiny] }); + await send(admin, { ...V, functionName: "setCaps", args: [tiny, tiny] }); // Earlier checks in this run consumed the (much larger) original budget. // The bucket refills proportionally to the CURRENT dailyCap, so after // lowering it the old consumption decays slowly -- warp past it, or the From e0a36b981ecae02ee07dffc8d3c7a1ec9f72fd01 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Sun, 6 Sep 2026 16:10:35 +0200 Subject: [PATCH 57/61] docs: Record the phase 1 and phase 3 results on the round-9 revision --- docs/pen-migration-internal-review.md | 31 +++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/pen-migration-internal-review.md b/docs/pen-migration-internal-review.md index 084467697..1f0702fe5 100644 --- a/docs/pen-migration-internal-review.md +++ b/docs/pen-migration-internal-review.md @@ -803,6 +803,37 @@ guard), the governance deployment, and all three daemons, so they need a fresh adversarial pass and the Sepolia re-run, with `REHEARSAL_PER_RELEASE_CAP_PEN` now equal to the daily cap and `TIMELOCK_CANCELLER`/`QUORUM_FLOOR` set. +### Phase 3 re-run on the round-9 revision (2026-09-06) + +Phase 1 passed 11/11 on the first attempt. Phase 3 (Chopsticks mainnet fork +with the runtime wasm override + Anvil; four attestors, monitor, releaser) +reached 9/9 after three findings that only a live fleet could produce: + +- **Every attestor exited on its first checkpoint** — silently, because the + harness kept daemon output only in memory. Cause: the durability read at + the `safe` block hit a block that predates the vault. Anvil resolves + `safe`/`finalized` to genesis until the chain is 32 blocks deep (the round-8 + claim "Anvil's safe equals latest" was wrong — it holds only with + `--slots-in-an-epoch 0`, which the harness now requires and documents). The + attestor now treats a zero-data read at the boundary as "not yet durable" + rather than fatal, which is also the right behaviour for a fresh testnet + deployment whose safe head still trails the vault. Daemon output is teed to + `testing/.logs/`. +- **`setCaps(perRelease > daily)` in the cap-deferral drill** now reverts + `CapsInverted`, as intended; the drill uses equal caps. +- **An exact gas estimate ran out of gas inside the ERC20Votes checkpoint + write.** The vault's vote-sink delegation (round 8) makes every transfer + touching the vault write a checkpoint keyed by block timestamp; an estimate + computed in one second (overwrite) undershoots execution in the next (new + entry). The harness passes an explicit gas limit, and the releaser now + doubles its `release()` estimate like the attestor does for `approve()`. + On Base the estimate runs at the pending block's timestamp and is + conservative, but the buffer costs nothing. + +Note for later drills: `--slots-in-an-epoch 1` makes Anvil trail `safe` by one +block, which would exercise the checkpoint trailing locally; the drills would +then need to mine an extra block after each action. + ## Residual risks and standing practices (no external audit — risk accepted) - Every change to the fund-release path (vault release/approve/sweep logic, pallet burn path) gets a fresh independent adversarial review round before From c1a2fdd122139148a9a1c63e3e2af323418f076f Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Tue, 8 Sep 2026 11:07:41 +0200 Subject: [PATCH 58/61] runtime: Bump the Pendulum spec version to 26 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The upgrade adds the token-migration pallet at index 102 and its BaseFilter arm — no existing call index, argument or signed extension changes — so transaction_version stays at 11: signed transactions built against spec 25 remain valid across enactment. --- runtime/pendulum/src/lib.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime/pendulum/src/lib.rs b/runtime/pendulum/src/lib.rs index 725577577..e98cd0841 100644 --- a/runtime/pendulum/src/lib.rs +++ b/runtime/pendulum/src/lib.rs @@ -234,7 +234,7 @@ pub const VERSION: RuntimeVersion = RuntimeVersion { spec_name: create_runtime_str!("pendulum"), impl_name: create_runtime_str!("pendulum"), authoring_version: 1, - spec_version: 25, + spec_version: 26, impl_version: 0, apis: RUNTIME_API_VERSIONS, transaction_version: 11, From 7ee6fa55112e142740e7e2c0d6a02db2437ec445 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Tue, 8 Sep 2026 11:13:03 +0200 Subject: [PATCH 59/61] pallet: Accept the dev-machine benchmark weights as production weights The exit criteria asked for a re-run on reference hardware; the project has never benchmarked on such hardware and the weights are measured, not hand estimates. Record the decision where both the checklist and the file header made the opposite promise. --- docs/pen-migration-local-test-plan.md | 7 +++++-- pallets/token-migration/src/default_weights.rs | 8 ++++---- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/pen-migration-local-test-plan.md b/docs/pen-migration-local-test-plan.md index 8f1ea9ac1..7e24bdfa3 100644 --- a/docs/pen-migration-local-test-plan.md +++ b/docs/pen-migration-local-test-plan.md @@ -450,8 +450,11 @@ UPGRADE_WASM=/path/to/spec+1.wasm node testing/src/drill-rb5-upgrade.mjs - [ ] Finality gating confirmed against a real relay: finalized lags best, and the attestors act only on the finalized stream. - [ ] The monitor alerts on a real injected deficit and tolerates a surplus. -- [ ] Benchmarks re-run on reference hardware and the generated weights - replace the manual estimates. +- [x] Benchmark weights: generated on a development machine (2026-08-27, + steps 50 / repeat 20) and consciously accepted as the production + weights (decision 2026-09-06) — the project has never benchmarked on + collator-grade reference hardware. They are measured, not hand + estimates; regenerate only if the pallet's extrinsics change. - [ ] A dry run of the deploy script with the **final** production parameters, reviewed by someone other than whoever wrote the `.env`. - [ ] The phase 5 rehearsal green on Base Sepolia against the shipped revision. diff --git a/pallets/token-migration/src/default_weights.rs b/pallets/token-migration/src/default_weights.rs index 4e4d7a0a2..d1b314017 100644 --- a/pallets/token-migration/src/default_weights.rs +++ b/pallets/token-migration/src/default_weights.rs @@ -5,10 +5,10 @@ //! WASM-EXECUTION: `Compiled`, DB CACHE: 1024 //! //! Generated on a development machine, not on collator-grade reference -//! hardware, so treat these as measured-but-provisional: they are strictly -//! better than the hand-written estimates they replace, but should be -//! regenerated on production hardware before the runtime upgrade if the -//! timeline allows. +//! hardware, and accepted as the production weights (decision 2026-09-06): +//! the project has never benchmarked on reference hardware, and these are +//! measured rather than hand estimated. Regenerate if the pallet's +//! extrinsics change. //! //! Regenerate with: //! From 49f690bdf06be05bf1836aa7cdd70f9ca9e82294 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Tue, 8 Sep 2026 11:16:58 +0200 Subject: [PATCH 60/61] docs: Record phase 2 and the RB-5 drill on the spec-26 runtime --- docs/pen-migration-internal-review.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/pen-migration-internal-review.md b/docs/pen-migration-internal-review.md index 1f0702fe5..8d5e2b15d 100644 --- a/docs/pen-migration-internal-review.md +++ b/docs/pen-migration-internal-review.md @@ -834,6 +834,20 @@ Note for later drills: `--slots-in-an-epoch 1` makes Anvil trail `safe` by one block, which would exercise the checkpoint trailing locally; the drills would then need to mine an extra block after each action. +### Spec-26 runtime: phase 2 and RB-5 (2026-09-08) + +The runtime was built from the committed tree with `spec_version` 26 +(`transaction_version` unchanged at 11: no existing extrinsic encoding +changes). Local artifact `pendulum_runtime.compact.compressed.wasm`, 2,206,797 +bytes, sha256 `8e82e2e8f0e17eb2174aa114cef2b999e52b905bb6e87b51a0aae8e72100c782` +— the reproducible build from the release pipeline is the artifact to submit; +compare hashes. Against a fresh Chopsticks fork of live mainnet with that wasm +as override, the fork reports spec 26 and **phase 2 passed 14/14**, including +ships-paused with no storage written. **RB-5 passed 5/5**: the wasm written to +`:code` under a running four-attestor fleet on a pristine spec-25 fork; every +attestor decoded post-upgrade blocks without a restart and rode a node +restart. Dev-machine benchmark weights were accepted as production weights. + ## Residual risks and standing practices (no external audit — risk accepted) - Every change to the fund-release path (vault release/approve/sweep logic, pallet burn path) gets a fresh independent adversarial review round before From dce3028066f4b875a99c5c0bf9c392a8008ce343 Mon Sep 17 00:00:00 2001 From: Marcel Ebert Date: Tue, 8 Sep 2026 15:34:50 +0200 Subject: [PATCH 61/61] runtime: Keep the Pendulum spec version at 25 on this branch The bump to 26 belongs to the release PR that follows this one, so that "spec 26" is pinned to the exact commit the reproducible build runs on rather than to whatever is on main between merge and release. This reverts c1a2fdd; the same change is re-applied on release/pendulum-26. --- runtime/pendulum/src/lib.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime/pendulum/src/lib.rs b/runtime/pendulum/src/lib.rs index e98cd0841..725577577 100644 --- a/runtime/pendulum/src/lib.rs +++ b/runtime/pendulum/src/lib.rs @@ -234,7 +234,7 @@ pub const VERSION: RuntimeVersion = RuntimeVersion { spec_name: create_runtime_str!("pendulum"), impl_name: create_runtime_str!("pendulum"), authoring_version: 1, - spec_version: 26, + spec_version: 25, impl_version: 0, apis: RUNTIME_API_VERSIONS, transaction_version: 11,