Repository navigation
feat(create-spec): enforce the hanging_api and stateful rules in the method-schema gate - #137
Conversation
8a44f0c to
2c87d74
Compare
…AG-3644 item 4) (#153) * feat(create-spec): gate node-operator RPCs out of base collections (MAG-3644) MAG-3644 item 4: should create-spec flag node-admin RPCs during synthesis? Yes — and the first catalogue sweep proves why. `bch.json` served BCHN's parkblock/unparkblock/finalizeblock from BCH's base collection, enabled, at 10 CU. It reached main, two Phase 11 reviews passed over it, and it was only caught once a reviewer was bumped to opus. Review is not a reliable gate for this; a script is. ## The predicate enabled == true AND add_on == "" AND the method acts on the NODE rather than the chain. `add_on` is part of the predicate, not a detail. An add-on is an opt-in boundary a provider can decline; a base collection is not — every provider serving the chain over that interface offers the method, and importing specs inherit it through CombineCollections. MAG-3644's severity argument turns on exactly this, so `debug_setHead` under a `debug` add-on is reported as INFO, not failed. The classes are grouped by what an attacker gets — consensus view, peering, lifecycle, mining, keys, regtest mining — because someone asking "is my new method one of these?" needs the reason, not the list. `personal_*` is matched by prefix; enumerating it would fall behind a client release. Out of scope on purpose: submitblock/submitblocklight. Block submission changes CHAIN state, not the node's view of it, and is arguably legitimate for a mining consumer. Its stateful/deterministic typing belongs to the method-schema gate (and to PR #137, which is already in that territory). ## Why a baseline file exists The sweep found the exposure is not bch-only: etc.json ETC 28 methods at add_on "" — admin_*, miner_*, and the entire personal_* namespace: unlockAccount, importRawKey, newAccount, sendTransaction, signTransaction. Key material and signing on the PROVIDER'S node, as priced relays. Filed as MAG-3808 (High) rather than buried here. zcash.json generate, setgenerate — regtest block fabrication on a mainnet spec. A warn-only gate would reproduce the exact failure this is meant to stop, so the gate hard-fails and pre-existing exposure goes in node_admin_baseline.txt with its ticket. A row there is a debt record, not an exemption — and test_check_node_admin_rpcs.sh asserts every shipped spec is clean or baselined, so the ledger cannot drift. bch.json's three rows are marked TEMPORARY: this branch is cut from main, where they are still enabled. PR #152 disables them; delete the rows on merge. ## Item 1's measurement is in the docs, not just the ticket Reproduced on BCHN 29.1.0 with two peered regtest nodes: parkblock does nothing. The node logs "Unpark chain up to block … as it has accumulated enough PoW" and carries on. Self-healing on the most-work chain. finalizeblock strands the node permanently. The provider held at height 11 while the network reached 20, rejecting honest headers with bad-header-finalization (code 259) and banning the peers offering them. So the ticket's headline claim was wrong about which method is the weapon. phase3.2 carries that as a worked example, with the lesson: check the mechanism, because the obvious-looking candidate was the harmless one. Also documented: a probe cannot decide this. Gateways whitelist methods and geth deprecated personal_*, so probes look clean while a self-hosted node serves them. The free-tier rule's converse — a -32601 is not evidence of safety either. ## Mechanics - spec_guards.yml gets a job scoped to the PR's changed root specs, matching how disabled-count already resolves them. Catalogue-wide assertion would make every spec PR answer for files it never touched. - bash-3.2-clean (grep -Fxq against a temp file, no `declare -A`), like check_disabled_count.sh, because this is a guard reviewers run by hand. Verified under both /bin/bash 3.2.57 and bash 5.3.15. - Emits add_on last in the jq row: tab is IFS whitespace, so an empty field in the middle silently shifts everything after it. Skill-only changeset, so spec_pipeline.yml does not trigger (paths: "*.json"). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(create-spec): make the node-admin baseline self-cleaning A baseline row whose method is no longer exposed is not harmless. It sits there describing a fixed problem, and if the fix is ever reverted it silently re-permits the method — the guard sees a known row and says nothing. The ledger has to clean itself or it becomes the hiding place it was written not to be. So a stale row now FAILS, and the fix is to delete it. The exception is a row tagged `TEMPORARY`, which exists precisely because it is expected to go stale: a fix already in flight on another branch. Those are reported loudly under "STALE, TEMPORARY (the in-flight fix landed — delete these rows now)" but do not fail, because the merge that makes them stale is usually someone else's and should not turn their CI red. bch.json's three rows are tagged accordingly. The token is machine-readable rather than a comment, which is the point — the previous version stated the same intent in prose with nothing to enforce it, so the rows would have outlived PR #152 silently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(create-spec): drop bch.json's TEMPORARY baseline rows — #152 landed The three rows existed because this branch was cut from main while parkblock / finalizeblock / unparkblock were still enabled there. #152 disabled them, so the rows now describe an exposure that no longer exists. The mechanism worked as designed on its first real test. After merging main the guard reported: === STALE, TEMPORARY (the in-flight fix landed — delete these rows now) === bch.json BCH jsonrpc finalizeblock bch.json BCH jsonrpc parkblock bch.json BCH jsonrpc unparkblock Reported, not failed — which is the whole point of the TEMPORARY tag: #152's merge did not turn CI red on this branch or anyone else's, but the rows could not quietly outlive their reason either. An untagged row going stale still fails, because a stale row would silently re-permit the method if the fix were reverted. Suite still green, including the catalogue sweep over all shipped specs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
avitenzer
left a comment
There was a problem hiding this comment.
I tested this at 550776ff. Both self-test suites pass (5/5 each), it still merges cleanly into main (21 commits behind), and the rule it encodes still matches the router: timeout_ms replaces the CU term, hanging_api adds 2× block time, with a 1 s floor. I also ran both checkers over all 155 specs on origin/main, plus qtum.json (#187) and dcr.json (#186).
1. Update mode will hit false FAILs, and Phase 10a calls them fixer regressions (blocking)
check_hanging_api.sh checks the whole candidate file. The scope note says the 152 pre-existing rows across ~30 established specs are "a separate cleanup, not this gate's job". The full-catalog run matches that count exactly: 28 files, 152 FAIL rows.
The problem is that in update mode (SKILL.md:91) the candidate is one of those established specs, and Phase 6's static gates still run there. Three open update PRs are already affected. On origin/main, before any change:
| Spec | Open update PR | FAIL rows |
|---|---|---|
zcash.json |
#150 | 1 |
solana.json |
#164 | 1 |
starknet.json |
#167 | 6 |
Each of these fails the gate on rows the updater never touched. SKILL.md Phase 10a then states "Any FAIL row here is a fixer-introduced regression, not an original defect. Feed it back as a CRITICAL item", so the fixer gets sent to edit methods outside the PR's scope. That is the drift that update mode exists to prevent (#80).
Possible fixes: fail only on rows that are new relative to the same file on origin/main (report the rest as INFO), or skip checks 2 and 3 in update mode for apis the updater did not add or change.
2. Stale line citations (non-blocking)
These have moved on smart-router main:
protocol/chainlib/common.go:575is now:600.GetRelayTimeoutnow also bounds a caller override withBoundCallerRelayTimeout, and the CU /timeout_ms/ hanging arithmetic moved intorouterRelayTimeout. The rule is unchanged, but the quoted body no longer matches.consumer_websocket_manager.go:298is now:525.SKILL.md:263, cited in the FAIL message and the header: this PR's own SKILL.md edit puts the rule at:331.
implied < 1000 hard-codes the default floor. That floor is now MinimumTimePerRelayDelay, configurable via --min-relay-timeout, so a deployment that raises it also raises the real budget. This doesn't break the check, but a comment saying it assumes the default would help.
3. Real catalog findings from check_stateful.sh (FYI, outside this PR)
monad.json and optimism.json declare eth_sendTransaction with stateful: 0. Those two are the only check_stateful failures in the catalog, and they look like genuine misclassifications worth a ticket.
qtum.json and dcr.json pass both checks.
… (MAG-3389)
Three rules for category.hanging_api, all derived from the one function
that reads it -- protocol/chainlib/common.go:575:
func GetRelayTimeout(chainMessage, averageBlockTime) time.Duration {
if chainMessage.TimeoutOverride() != 0 { return chainMessage.TimeoutOverride() }
extraRelayTimeout := 0
if IsHangingApi(chainMessage) { extraRelayTimeout = averageBlockTime * 2 }
relayTimeAddition := common.GetTimePerCu(GetComputeUnits(chainMessage))
if chainMessage.GetApi().TimeoutMs > 0 {
relayTimeAddition = time.Millisecond * time.Duration(chainMessage.GetApi().TimeoutMs)
}
return extraRelayTimeout + relayTimeAddition
}
1. hanging_api must not appear on a SUBSCRIBE-tagged API.
consumer_websocket_manager.go:298 branches on the SUBSCRIBE function
tag straight into StartSubscription. All four GetRelayTimeout call
sites are on the unary path, and neither subscription manager
references it -- so hanging_api, compute_units and timeout_ms are all
dead inputs on a subscription. Its lifetime is its socket's; the WS
manager's only timeouts are two hardcoded 10s constants for
unsubscribe teardown. 3 of 288 SUBSCRIBE-registered APIs in the
catalogue set the flag; PR #136 clears 2 of them.
SUBSCRIBE names are collected inheritance-aware, walking the imports
graph the way check_directive_presence.sh does, because an L2 that
imports ETH1 carries an empty parse_directives array of its own.
2. hanging_api: true requires an explicit timeout_ms. This automates the
rule at SKILL.md:263 and spec-builder.md:59, which both flagged it as
having no validator coverage.
3. timeout_ms must be at least max(1s, CU*100ms). It REPLACES the CU
term rather than adding to it, so a smaller value shortens the relay
budget relative to setting nothing -- the opposite of the intent.
Caught live on Acala: CU 1000 implies a 100 000ms base, so a
reflexive timeout_ms: 30000 would have cut the budget from 124s to
54s. Zero specs in the catalogue currently violate this, so it is a
pure regression guard.
Rule 1 short-circuits: a subscription is never judged on its timeout,
since neither field is read.
Wired into gate 9 (method-schema) rather than added as a tenth gate --
both are per-method structural field checks, and gate 7 (pruning)
already sets the precedent of one gate running two scripts.
Scope: candidate file only, which is how the pipeline uses it. 152 APIs
across ~30 established specs (ethereum, cosmossdk, tendermint, solana,
kusama) predate rule 2 and would fail if it ran repo-wide. Separate
cleanup, tracked in MAG-3389.
Also corrects the test-suite table, stale since check_disabled_count.sh
(PR #133) and check_internal_paths.sh landed without being listed: 12 ->
15 rows.
test_check_hanging_api.sh covers all three rules, rule-1 precedence, the
1s CU floor at 999/1000, inherited SUBSCRIBE directives, and non-hanging
APIs being out of scope. All 15 suites pass under bash 5.3.
…AG-3389)
category.stateful is a routing instruction, not a label. It maps to
CONSISTENCY_SELECT_ALL_PROVIDERS and the router acts on it in four places
(rpcsmartrouter_server.go:3336, :3585, :3835, :4735): the call is fanned
out to every provider and excluded from cross-validation
(cross_validation_policy.go:422) and the recovery probe
(recovery_probe.go:112-133). Both directions fail silently --
stateful: 0 on a write sends the broadcast to a single provider, and
stateful: 1 on a read fans every call out to all of them and drops
cross-validation.
Unlike hanging_api this cannot be settled from the router source: "does
this method change chain state" is chain knowledge. Nor by vote --
author_submitAndWatchExtrinsic splits 10 specs at 0 against 11 at 1,
with kusama.json and polkadot.json on opposite sides, so a majority rule
produces no answer exactly where one is needed.
Three layers of decreasing confidence:
1. Curated write list -> FAIL. Substrate submit/submitAndWatch,
eth_sendRawTransaction/sendTransaction/sendRawTransactionSync,
BroadcastTx, broadcast_tx_*, Solana sendTransaction, the three
starknet add*Transaction. Curated rather than voted, so it fires
even when the whole catalogue agrees and is wrong.
2. Curated read list -> FAIL. Simulation and codec helpers.
eth_fillTransaction is on it by name: SPEC_GUIDE.md and phase3.2
both call it the classic trap.
3. Cross-spec consensus -> INFO, never FAIL. Fires when >=90% of the
OTHER specs declaring a name disagree and at least 4 declare it.
Advisory because 24 of 5,539 distinct method names genuinely
disagree across the catalogue.
REST entries are keyed "TYPE /path" where the verb disambiguates: GET
/cosmos/tx/v1beta1/txs is a query, POST to the same path is BroadcastTx.
The verb resolves a name collision, it does not classify -- treating
POST as "is a write" is the bug that put four read endpoints on the
write path in cosmossdk.json.
Calibration over the 140-spec catalogue: 29 FAIL rows in 16 specs and 11
INFO rows. Every FAIL is a real defect -- the 10 specs carrying the watch
pair at 0, babylon/kava/sei on decode/amino, monad/optimism on
eth_sendTransaction, and cosmossdk's four REST reads. The INFO rows are
lit.json's seven node-admin methods at 0 against 16 specs at 1, and
bittensor/etc on eth_submitWork, eth_submitHashrate and
transaction_v1_stop.
Two defects found while building it, both caught by the tests:
- the consensus dedupe key dropped the filename, so `sort -u`
collapsed the whole catalogue to one row per distinct value and the
threshold could never be reached;
- the candidate was counted in its own consensus denominator, which
contradicted the "N/M other specs" wording and pushed 4-of-4
agreement below the 90% bar.
Wired into gate 9 alongside check_method_schema.sh and
check_hanging_api.sh; gate 9's INFO rows are added to the advisory
routing table, which now lists four gates.
test_check_stateful.sh covers both curated lists, the fillTransaction
trap, REST verb disambiguation in both directions, consensus being
advisory rather than fatal, the sibling threshold, and both of the
defects above. All 16 suites pass under bash 5.3.
…-run Three corrections found by re-checking claims I had asserted rather than verified. 1. stateful is NOT read on the subscription path either. check_stateful.sh told the reader a stateful: 0 write "goes to a single provider instead of all of them". For a SUBSCRIBE-tagged method that is wrong, and wrong the same way the hanging_api claim was: GetStateful is unreachable from the subscription path. It appears in rpcsmartrouter_server.go, recovery_probe.go, unified_relay_state_machine.go and common.go -- and in neither subscription manager, exactly like GetRelayTimeout. But it does not follow that the flag is inert there, which is where this differs from hanging_api. ApiHasStatefulCategory (base_chain_parser.go:549, reached via rpcsmartrouter_server.go:456) looks the method up by NAME, independent of transport, and the cross-validation policy guard uses it to reject an enabled per-method CV policy on a write. So the write list still FAILs on subscription methods rather than short-circuiting them the way check_hanging_api.sh rule 1 does -- hanging_api has no equivalent name-keyed reader, stateful does. The message and header now say precisely which effects are unary-only and which are not. 2. Phase 10a: re-run the offline gates after the fixer. Gate 9 runs in Phase 6, before Phase 10. The only post-fix validation was `jq .`, so any defect the FIXER introduces shipped unguarded -- and the fixer is the step most likely to introduce these, because it applies field-level instructions without surrounding context. check_hanging_api.sh rule 3 is the clearest case: told only "a hanging API needs a timeout_ms", the natural response is a flat 30000, which on a CU-1000 method shortens the relay budget instead of lengthening it. That is the mistake made by hand while writing MAG-3389, and the guard against it could not fire where it mattered. Phase 10a re-runs check_method_schema, check_hanging_api, check_stateful and check_unused_fields, all offline and sub-second. It also finally wires check_disabled_count.sh (PR #133), which was left unwired for precisely this timing reason. 3. "Every FAIL is a genuine defect" was overstated. monad and optimism carry eth_sendTransaction at enabled: false, so those 2 of the 29 rows are latent, not live. Corrected to 27 live and 2 latent -- the same distinction that applies to a disabled method anywhere. Also: a rest collection with no collection_data.type could not match a verb-keyed curated entry and skipped both lists in silence. Every rest collection in the catalogue currently has one; if that ever stops being true it now emits an INFO row instead of passing quietly. All 16 suites pass.
…in update mode Review on 550776f: on an established spec, Phase 6 and Phase 10a failed rows the run never touched (zcash 1, solana 1, starknet 6 hanging_api rows; monad and optimism 1 stateful row each), and Phase 10a called them fixer regressions. - check_hanging_api.sh and check_stateful.sh take --base <file>. A FAIL row the base fails with identical text moves to INFO as pre-existing; new or changed rows still FAIL. No base, no change. An unreadable base exits 2. - The base is judged in the candidate's directory by its own imports, so an inherited SUBSCRIBE directive resolves the same way for both. - Phase 6 passes origin/main's copy in update, add-testnet and add-collection mode; Phase 10a passes the pre-fix snapshot. - Citations: GetRelayTimeout is common.go:600, and the spec inputs are read by routerRelayTimeout (:613). The SUBSCRIBE branch is consumer_websocket_manager.go:525. The SKILL.md rule is cited by section, and the 1s floor is noted as MinimumTimePerRelayDelay's default. All five specs above pass with themselves as base and still fail without one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…sh 5.2 An unquoted $* in a heredoc joins with spaces, not IFS, before bash 5.3, so the two-api base-mode fixtures were invalid JSON on ubuntu-24.04 and test_check_stateful.sh failed in CI while passing on bash 5.3. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
944a8b0 to
f819e75
Compare
…collection Two holes in the base mode added in 7fb2e84, both found in review: - Phase 10a passed the pre-fix snapshot as --base. That snapshot is taken after the Phase-6 fixer, so on a new chain a Phase-6 FAIL the fixer left in place (the Acala timeout_ms case this gate exists for) was demoted to INFO as pre-existing and passed. Phase 10a now uses the same base as Phase 6: the file on origin/main, or none for a new chain. - Base rows were matched as a set of "index/interface/name" lines, so a method already failing in one collection excused the same defect in an add-on collection the run added. Rows now name the collection (interface, REST verb, internal_path, add_on) and are demoted by count. Also corrects what the docs and messages claim about the router: hanging_api has three readers, not one; rule 3 shortens the relay window, not the attempt budget; the subscription path that ignores hanging_api is the WebSocket one. Refreshes check_stateful.sh's smart-router citations and the calibration figures, names MAG-3401 as the cleanup ticket, routes the method-schema ADVISORY block in Phase 6, and drops the circular --expect <N> from Phase 10a. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds
check_hanging_api.shandcheck_stateful.shto themethod-schemagate — the twocategory.*flags that steer routing and had no validator coverage between them. Ticket: MAG-3389. Companion to #136, which fixes the specs this gate would flag.Why
category.hanging_apisets the relay window.GetRelayTimeout(protocol/chainlib/common.go:600in smart-routerd13051cf) bounds a caller's timeout override withBoundCallerRelayTimeout; the spec's own inputs are read byrouterRelayTimeout(common.go:613):The flag has two other readers: the attempt budget (
GetTimeoutInfo,common.go:803→GetTimeoutForProcessing, which raises it to 6×DefaultTimeoutfor a hanging call,protocol/common/timeout.go:198) and the recovery probe (recovery_probe.go:136). The rules below concern the window. Two properties ofrouterRelayTimeoutgenerate all three.The three rules
1.
hanging_apimust not appear on aSUBSCRIBE-tagged API → FAILconsumer_websocket_manager.go:525branches on the SUBSCRIBE function tag straight intoStartSubscription. EveryGetRelayTimeoutcall site (five today:rpcsmartrouter_server.go:2305,:5334,:5507,:6069andcommon.go:117) is on the unary path, and no subscription manager references it. A subscription's lifetime is its socket's — the WS manager's only timeouts are two hardcoded 10s constants for unsubscribe teardown.So over WebSocket, where a subscription is served,
hanging_api,compute_unitsandtimeout_msare dead inputs on a SUBSCRIBE-tagged API, and setting the flag asserts something the subscription path never reads. (A SUBSCRIBE call sent over HTTP takes the unary path like any other, but cannot deliver a subscription there.) 3 of 288 SUBSCRIBE-registered APIs in the catalogue did it when this was written. #136 cleared two and #141 the last (dydx.json); onmaintoday (155 specs) none is left.SUBSCRIBE names are collected inheritance-aware, walking the
importsgraph the waycheck_directive_presence.shdoes — an L2 importing ETH1 has an emptyparse_directivesarray of its own.2.
hanging_api: truerequires an explicittimeout_ms→ FAILAutomates the rule already stated in SKILL.md's Phase 6 pre-flight checklist and
agents/spec-builder.md:59, both of which flagged it as "no validator covers this." Now one does.3.
timeout_msmust be at leastmax(1s, CU × 100ms)→ FAILtimeout_msreplaces the CU term rather than adding to it, so a smaller value shortens the relay window relative to setting nothing at all — the opposite of why anyone sets it. (The window decides when the router tries another provider; the attempt budget above is separate.)Caught live while writing #136: Acala's watch pair at CU 1000 carries a 100 000 ms implied base, so a reflexive
timeout_ms: 30000would have cut the window from 124s to 54s. Zero specs currently violate this, so it is a pure regression guard.The 1s is the default
MinimumTimePerRelayDelay. A deployment that raises it with--min-relay-timeoutraises the real window too, so the check stays conservative there.Rule 1 short-circuits — a subscription is never judged on its timeout, since neither field is read.
Part 2 —
check_stateful.shcategory.statefulis a routing instruction, not a label. It maps toCONSISTENCY_SELECT_ALL_PROVIDERS, and the router acts on it in four places (rpcsmartrouter_server.go:1384,:4112,:4921,:6008): the call is fanned out to every provider and excluded from cross-validation (theValidateNoStatefulPoliciesguard,cross_validation_policy.go:572, wired atcross_validation_boot.go:107-119) and the recovery probe (recovery_probe.go:112-133).Both directions fail silently:
stateful: 0on a writestateful: 1on a readScope caveat, verified rather than assumed. Those per-relay effects are unary-only. Like
GetRelayTimeout,GetStatefulis unreachable from the subscription path — it appears inrpcsmartrouter_server.go,recovery_probe.go,unified_relay_state_machine.goandcommon.go, and in neither subscription manager. So on a SUBSCRIBE-tagged method such asauthor_submitAndWatchExtrinsic, provider fan-out does not depend on this flag.It does not follow that the flag is inert there, and this is where
statefuldiffers fromhanging_api.ApiHasStatefulCategory(base_chain_parser.go:608, viacross_validation_boot.go:107) looks the method up by name, independent of transport, and the cross-validation policy guard uses it to reject an enabled per-method CV policy on a write. That is why the write list still FAILs on subscription methods rather than short-circuiting them the waycheck_hanging_api.shrule 1 does:hanging_apihas no name-keyed reader,statefuldoes.Why this one can't be done the same way
hanging_api's rules fall out of the router source. "Does this method change chain state" is chain knowledge instead.And it can't be settled by vote either: before #136
author_submitAndWatchExtrinsicsplit 10 specs at0against 11 at1, withkusama.jsonandpolkadot.jsonon opposite sides. A majority rule produced no answer exactly where one was needed. (After #136 and #141, all 21 declare1.)So: three layers of decreasing confidence.
1. Curated write list → FAIL. The Substrate submit/submitAndWatch family,
eth_sendRawTransaction/eth_sendTransaction/eth_sendRawTransactionSync,BroadcastTx,broadcast_tx_*, SolanasendTransaction, the three Starknetadd*Transaction. Curated rather than voted, so it fires even when the whole catalogue agrees and is wrong.2. Curated read list → FAIL. Simulation and codec helpers.
eth_fillTransactionis on it by name — SPEC_GUIDE.md and phase3.2 both call it the classic trap: a*_fill*/*_prepare*helper whose argument shape looks like a transaction.3. Cross-spec consensus → INFO, never FAIL. Fires when ≥90% of the other specs declaring a name disagree with the candidate, and at least 4 declare it. Advisory, because some names genuinely disagree across chains (24 of 5,539 distinct method names at the 2026-09-06 calibration).
REST entries are keyed
"TYPE /path"where the verb disambiguates —GET /cosmos/tx/v1beta1/txsis a query,POSTto the same path is BroadcastTx. The verb resolves a name collision; it does not classify. Treating POST as "is a write" is exactly the bug that put four read endpoints on the write path incosmossdk.json(#138).Calibration over all 155 specs (re-run 2026-10-07 on
main)2 FAIL rows in 2 specs, 11 INFO rows. Both FAILs are latent:
monadandoptimismcarryeth_sendTransactionatstateful: 0on a method that isenabled: false. They are not fixed here. They belong to the catalogue cleanup in MAG-3401.Fixed since the first calibration (2026-09-06: 29 FAIL rows in 16 of 140 specs):
avtandbsxby #136;babylon,kava,sei(decode/aminoat1) pluscosmossdk's four REST reads by #138; and the watch pair atstateful: 0onastar,enjin,hydration,kusama,kusamaassethub,moonriver,polkadot_asset_hubandsdnby #141. All three have merged.The 11 INFO rows are unchanged:
lit.json's seven node-admin methods (author_insertKey,author_rotateKeys,offchain_localStorageSet,system_add/removeReservedPeer,system_add/resetLogFilter) at0against 16 specs at1, plusbittensor/etconeth_submitWork,eth_submitHashrateandtransaction_v1_stop.Two defects the tests caught
Worth naming because both would have made the consensus layer quietly useless rather than wrong:
sort -ucollapsed the whole catalogue to one row per distinct value and the threshold could never be reached.Phase 10a — so the gates fire where it matters
Gate 9 runs in Phase 6, before the Phase 10 fixer, and the only post-fix validation was
jq .. Any defect the fixer introduces therefore shipped unguarded — and the fixer is the likeliest source, since it applies field-level instructions without surrounding context.check_hanging_api.shrule 3 is the sharpest example: told only "a hanging API needs atimeout_ms", the natural response is a flat30000, which on a CU-1000 method shortens the window. That is exactly the mistake made by hand on Acala while writing #136 — so without this the guard could not fire on the case that motivated it.This PR adds Phase 10a, re-running
check_method_schema,check_hanging_api,check_statefulandcheck_unused_fieldsafter the fixer — all offline, sub-second, no network. It also finally wirescheck_disabled_count.sh(#133) after the fixer, in its report-only form: the count it prints is the value the PR body'sdisabled-countmarker must carry. It was left unwired for precisely this timing reason.Established specs:
--baseAn update, add-testnet or add-collection run edits one of the established specs, and 28 of them still carry rule-2 rows (152 APIs;
btc.jsonhas 3), as domonadandoptimismforcheck_stateful.sh. Judged on the whole file, both gates would fail such a run on rows it never touched, and Phase 10a would send them back to the fixer as regressions. That widens a surgical PR, and on an add-collection PR it would trip #163's guard.So both scripts take
--base <file>. A FAIL row the base already fails with identical text moves to=== INFO ===as[pre-existing …]; a new or changed row still FAILs.origin/main, and nothing for a new chain. Phase 10a deliberately does not use the pre-fix snapshot: that is taken after the Phase-6 fixer, so a Phase-6 defect that survived would be excused as pre-existing. Run on a new chain with the Acala mistake, Phase 10a fails it; judged against the snapshot instead, it passed.internal_path,add_on, e.g.jsonrpc@debug) and are matched by count. So a method that already fails in one collection does not excuse the same defect in a collection the run adds, which is the add-collection case.imports. An inherited SUBSCRIBE directive resolves the same way for both, and a candidate that newly imports a SUBSCRIBE parent still FAILs.On
main,zcash.jsonandsolana.jsoncarry onecheck_hanging_api.shFAIL row each,starknet.jsonsix, andmonad.jsonandoptimism.jsononecheck_stateful.shrow each. All five pass against themselves as base, and across all 155 specs every row moves to INFO.Wiring
Both folded into gate 9 (
method-schema) rather than added as new gates — all three scripts are per-method field checks, and gate 7 (pruning) already sets the precedent of one gate running multiple scripts.Updated: the gate table, TESTING.md §9,
method-schema-validator.md, and two SKILL.md pre-flight bullets. Both of those bullets previously said the rule had no validator coverage; both now do, so the pre-flight section no longer carries any unchecked item.The
method-schemagate's ADVISORY block (check_stateful.sh's consensus rows, and[pre-existing …]rows) is added to Phase 6's severity routing, which now lists four gates. Those rows are printed to the user, never sent to the fixer, and never fail the gate.Scope
Both scripts read the candidate file only, which is how the pipeline uses it. 152 APIs across 28 established specs (
ethereum,cosmossdk,tendermint,solana,kusama…; re-counted 2026-10-05) predate rule 2 and would fail if this ran repo-wide — a separate cleanup, tracked in MAG-3401, deliberately not attempted here. An update of one of those specs is judged with--base, above.Tests
test_check_hanging_api.sh— 20 cases: each of the three rules, rule-1 precedence over rules 2–3, the 1s CU floor at exactly 999 and 1000, inherited SUBSCRIBE directives resolved throughimports, non-hanging APIs ignored, a spec with no hanging APIs at all, and base mode (an unchanged failing row is INFO; a changed, edited or new one FAILs; no base judges every row; imports resolved in context and by the base's own; a failing row in a new add-on collection FAILs; rows matched by count; an unreadable base exits 2).test_check_stateful.sh— 17 cases: both curated lists in both directions, theeth_fillTransactiontrap, REST verb disambiguation (POST /txsmust be1,GET /txsmust not be forced), consensus being advisory rather than fatal, the 4-sibling threshold, regression cases for both defects above, and the same base-mode cases.All 19 suites pass (2026-10-07, rebased on
main) under bash 5.3 on macOS and bash 5.2 on Ubuntu 24.04, the CI runner. Under bash 3.2, 7 fail, as TESTING.md documents.Also corrects the test-suite table, which had fallen behind the test files: it now has 19 rows, one per test file, including the node-admin, collection-addition and update-diff tests that came in from
main.