test: harden the block-improvement wait against empty payloads - #12859
Draft
Marchhill wants to merge 1 commit into
Draft
test: harden the block-improvement wait against empty payloads#12859Marchhill wants to merge 1 commit into
Marchhill wants to merge 1 commit into
Conversation
WaitForImprovedBlock matched only on the parent hash. The payload service publishes an initial empty block for a parent and improves it afterwards, so the first BlockImproved event satisfied the wait regardless of whether the tx pool had yet made the submitted transactions selectable — TxPool head processing runs on a background channel, so its account/nonce view lags the forkchoiceUpdated that advanced the head. engine_getPayload* could then return the empty block, and Witness_includes_bytecode_and_storage_proof_for_a_called_contract failed intermittently because the called contract had no code in the witness. Add an additive minTransactions parameter (default 0 keeps every existing call site unchanged) and thread the submitted transaction count through BuildAmsterdamPayload. Improvements are monotonic — each context is seeded with the previous best block — so observing an improvement with N transactions guarantees the retrieved payload keeps at least N. The two V6 call sites that submit transactions and then assert on block hash, receipts root, or block contents carried the same latent race, so they now require their transaction counts too.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
MergeTestBlockchain.WaitForImprovedBlockmatched only on the parent hash.PayloadPreparationServicepublishes an initial empty block for a parent and then improves it, so the firstBlockImprovedevent satisfied the wait regardless of whether the tx pool had yet made the submitted transactions selectable.TxPoolhead processing runs on a background channel (_headBlocksChannel/ProcessNewHeads), so its account/nonce view lags theforkchoiceUpdatedthat advanced the head —engine_getPayload*could therefore hand back the empty block.That is what made
EngineModuleTests.Witness_includes_bytecode_and_storage_proof_for_a_called_contractflaky onRun Nethermind.Merge.Plugin.Test [no-intrinsics]: when the transaction missed its block, the called contract had no code andwitness.Codeslacked the runtime bytecode.DOTNET_EnableHWIntrinsic=0slows keccak/ecrecover on the sender-resolution and pool-insert path, widening the window — which is why only the[no-intrinsics]variant surfaced it.minTransactionsparameter toWaitForImprovedBlock. The default of0leaves all ~20 existing call sites behaviourally unchanged.BuildAmsterdamPayload, and assert the retrieved payload actually carries them so a regression fails at the cause rather than in a downstream witness assertion.V6call sites that submit transactions and then assert on block hash, receipts root, or block contents — they carried the identical latent race.minTransactions.Improvements are monotonic —
ImproveBlockseeds each new context with the previous best block — so observing an improvement with N transactions guarantees the retrieved payload keeps at least N.Types of changes
Testing
Requires testing
If yes, did you write tests?
Notes on testing
The flake would not reproduce directly on a fast dev box, so the race was demonstrated with a throwaway probe rather than assumed:
engine_getPayloadV6returned a 0-transaction payload — the exact CI symptom.minTransactions: 1the wait provably did not complete on the empty improvement (improvementWait.IsCompletedwasfalse) and completed only once an improvement carried the transaction; the payload then had 1 transaction.an empty improvement must not satisfy minTransactions / Expected: False / But was: True, confirming it is not vacuous.DOTNET_EnableHWIntrinsic=0: 1683 passed, 0 failed, 2 skipped — matching the counts from the green CI re-run.DOTNET_EnableHWIntrinsic=0plus 10 busy loops on 12 cores (load average ~65) to widen the window: 12/12 consecutive green runs of the witness and V6 suites.Documentation
Requires documentation update
Requires explanation in Release Notes
Remarks
Test-infrastructure only; no production code is touched. Unrelated to the separate
ExpiryQueue_ReleasesStorageAfterCumulativeChurnflake inNethermind.TxPool.Test.