Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion .agents/skills/write-test/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,16 @@ Conventions and patterns for writing consensus tests. Run this skill before writ
- Exception: use `blockchain_test` when the test needs more than one transaction (a `state_test` holds exactly one) or more than one block (e.g. transaction-ordering or fork-transition tests).
- Anti-pattern: wrapping one transaction in a `Block` to reach `blockchain_test`. A `state_test` can assert the transaction's gas used and receipt logs (the tx's `expected_receipt=TransactionReceipt(cumulative_gas_used=...)`), reserve state gas (the tx's `state_gas_reservoir=`), and other block-header fields (`blockchain_test_header_verify=Header(...)`) without it.

## Fail Loudly

Wire a test's premises so that a change fails it instead of silently retargeting what it exercises.

- **Assert what helpers assume.** A helper that derives a boundary from gas costs asserts its preconditions, so a repricing fails the test instead of moving the measurement.
- **Derive parameters from what the test asserts.** If a boundary is `len(slots) * COST`, compute it from the same `slots` the expectation checks.
- **Assert `post` alongside a BAL expectation.** The BAL checks access, `post` checks state; neither implies the other.
- **Witness that the code path ran.** Where success and never-executed look identical, leave a trace the `post` can assert: an `SSTORE`d sentinel, a balance, or `Account.NONEXISTENT`.
- **Pin block premises.** A block that must be exactly full asserts its `gas_used` with `header_verify=Header(...)`.

## Pre-State Setup

- `pre.fund_eoa()` — create funded EOA, returns Address. Accepts `amount=`, `nonce=`
Expand All @@ -34,6 +44,8 @@ Conventions and patterns for writing consensus tests. Run this skill before writ
- `storage = Storage()` then `storage.store_next(expected_value)` — auto-increments slot
- `Op.SSTORE(storage.store_next(sender), Op.ORIGIN)` — build bytecode + expected storage in one step
- Post-state: `post = {contract: Account(storage=storage)}`
- `Account(storage=...)` compares storage **exhaustively** — any slot you omit must be zero. Opt an omitted key out with `Storage.set_expect_any(key)`.
- BAL expectation fields behave differently: a field left unset is not checked, `[]` asserts empty, and a non-empty list matches as an **ordered subsequence** (extra actual entries are skipped, yours must appear in order). `BalAccountExpectation()` with no field set raises — use `.empty()` for an account with no changes, and `{address: None}` to assert an address is absent.

## Markers

Expand All @@ -43,7 +55,7 @@ Conventions and patterns for writing consensus tests. Run this skill before writ
- `@pytest.mark.with_all_call_opcodes` — parametrize CALL/CALLCODE/DELEGATECALL/STATICCALL
- `@pytest.mark.with_all_evm_code_types` — parametrize across EVM code types
- `@pytest.mark.slow` — excluded by default in fill
- `@pytest.mark.exception_test` — marks tests expecting exceptions
- `@pytest.mark.exception_test` — marks tests expecting exceptions. When only some parametrized cases raise, put it on the `pytest.param(..., marks=...)`, not the function, or the passing cases fail.

## Fork-Aware Logic

Expand All @@ -68,6 +80,7 @@ Never hand-reconstruct a gas amount by summing `fork.gas_costs()` constants (`NE
- Rule: omit `gas_limit`. It auto-fills so the transaction executes in full without running out of gas.
- Exception: set `gas_limit` explicitly for gas-sensitive tests (intrinsic-gas boundaries, OOG, code-deposit limits, or gas metering).
- Anti-pattern: the `gas_limit=fork.transaction_gas_limit_cap()` boilerplate is now redundant.
- A transaction that runs out of gas consumes exactly its `gas_limit`, so calling an `Om.OOG` contract pins a transaction's gas used to a chosen value without any cost arithmetic. Useful when a block's `gas_used` must land on an exact number.

## Exception Testing

Expand Down
2 changes: 2 additions & 0 deletions docs/writing_tests/fork_methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,8 @@ fork.tx_types() # Returns list of supported transaction types
fork.contract_creating_tx_types() # Returns list of tx types that can create contracts
fork.precompiles() # Returns list of precompile addresses
fork.system_contracts() # Returns list of system contract addresses
fork.system_contract_request_types() # Request classes triggered through a system contract (Prague+)
fork.system_contract_call_phases() # When the block calls each system contract: before/after transactions, or never
```

### EVM Features
Expand Down
29 changes: 20 additions & 9 deletions packages/testing/src/execution_testing/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,22 @@
TransactionException,
)
from .fixtures import BaseFixture, FixtureCollector
from .forks import Fork, GasCosts, RefundTypes, TransitionFork
from .forks import (
BuilderDepositRequest,
BuilderExitRequest,
ConsolidationRequest,
DepositRequest,
FeeSystemContractRequest,
Fork,
GasCosts,
RefundTypes,
Requests,
SystemCallPhase,
SystemContractRequest,
TransitionFork,
WithdrawalRequest,
create_deposit_log_bytes,
)
from .recipient_type import RecipientType
from .specs import (
BaseTest,
Expand Down Expand Up @@ -64,34 +79,27 @@
Blob,
BlockAccessList,
BlockAccessListExpectation,
BuilderDepositRequest,
BuilderExitRequest,
ChainConfig,
ConsolidationRequest,
DepositRequest,
Environment,
FeeSystemContractRequest,
NetworkWrappedTransaction,
Removable,
Requests,
SystemContractInteractionBase,
SystemContractInteractionContract,
SystemContractInteractionMeasuredOutOfGasContract,
SystemContractInteractionTransaction,
SystemContractRequest,
TestParameterGroup,
TestPhaseManager,
Transaction,
TransactionLog,
TransactionReceipt,
TransactionType,
Withdrawal,
WithdrawalRequest,
add_kzg_version,
ceiling_division,
compute_create2_address,
compute_create_address,
compute_deterministic_create2_address,
fee_increment_blocks,
keccak256,
relay_contract_code,
)
Expand Down Expand Up @@ -178,6 +186,8 @@
"Environment",
"EOA",
"FeeSystemContractRequest",
"fee_increment_blocks",
"create_deposit_log_bytes",
"FixedIterationsBytecode",
"FixtureCollector",
"Fork",
Expand Down Expand Up @@ -208,6 +218,7 @@
"StateTestFiller",
"Storage",
"Switch",
"SystemCallPhase",
"SystemContractInteractionBase",
"SystemContractInteractionContract",
"SystemContractInteractionMeasuredOutOfGasContract",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
to_json,
)
from execution_testing.fixtures.blockchain import FixtureHeader
from execution_testing.forks import Fork, TransitionFork
from execution_testing.forks import Fork, Requests, TransitionFork
from execution_testing.rpc import EngineRPC, EthRPC
from execution_testing.test_types import (
DETERMINISTIC_FACTORY_ADDRESS,
Expand All @@ -31,7 +31,6 @@
BlockAccessList,
ChainConfig,
Environment,
Requests,
Withdrawal,
)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
ruleset,
)
from execution_testing.fixtures.blockchain import FixtureHeader
from execution_testing.forks import Osaka
from execution_testing.forks import Osaka, Requests
from execution_testing.rpc import EngineRPC, EthRPC
from execution_testing.test_types import (
DETERMINISTIC_FACTORY_ADDRESS,
Expand All @@ -48,7 +48,6 @@
Alloc,
ChainConfig,
Environment,
Requests,
Transaction,
Withdrawal,
compute_deterministic_create2_address,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

from execution_testing.base_types import Bytes, Hash
from execution_testing.exceptions import ExceptionBase, ExceptionMapper
from execution_testing.forks import Fork, TransitionFork
from execution_testing.forks import Fork, Requests, TransitionFork
from execution_testing.logging import get_logger
from execution_testing.rpc import (
BlockNumberType,
Expand All @@ -37,7 +37,6 @@
from execution_testing.test_types import (
Alloc,
Environment,
Requests,
Transaction,
Withdrawal,
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@
Cancun,
Fork,
Paris,
Requests,
Shanghai,
TransitionFork,
ssz_schema_fork_key,
Expand All @@ -76,7 +77,6 @@
BlockAccessList,
Environment,
Removable,
Requests,
TestPhase,
Transaction,
Withdrawal,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,19 @@
EngineAPIError,
TransactionException,
)
from execution_testing.forks import Amsterdam, Prague
from execution_testing.test_types import (
EOA,
AuthorizationTuple,
from execution_testing.forks import (
Amsterdam,
ConsolidationRequest,
DepositRequest,
Prague,
Requests,
WithdrawalRequest,
)
from execution_testing.test_types import (
EOA,
AuthorizationTuple,
Transaction,
Withdrawal,
WithdrawalRequest,
)

from ..blockchain import (
Expand Down
31 changes: 30 additions & 1 deletion packages/testing/src/execution_testing/forks/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
"""Ethereum test fork definitions."""

from .base_fork import RefundTypes
from .base_fork import RefundTypes, SystemCallPhase
from .forks.eips.amsterdam.eip_8282 import (
BuilderDepositRequest,
BuilderExitRequest,
)
from .forks.eips.prague.eip_6110 import (
DepositRequest,
create_deposit_log_bytes,
)
from .forks.eips.prague.eip_7002 import WithdrawalRequest
from .forks.eips.prague.eip_7251 import ConsolidationRequest
from .forks.forks import (
BPO1,
BPO2,
Expand Down Expand Up @@ -75,8 +85,27 @@
transition_fork_from_to,
transition_fork_to,
)
from .requests import (
FeeSystemContractRequest,
RequestBase,
Requests,
SystemContractRequest,
requests_list_to_bytes,
)

__all__ = [
"BuilderDepositRequest",
"BuilderExitRequest",
"ConsolidationRequest",
"DepositRequest",
"FeeSystemContractRequest",
"RequestBase",
"Requests",
"SystemCallPhase",
"SystemContractRequest",
"WithdrawalRequest",
"create_deposit_log_bytes",
"requests_list_to_bytes",
"ALL_FORKS_WITH_TRANSITIONS",
"ALL_FORKS",
"ALL_TRANSITION_FORKS",
Expand Down
23 changes: 23 additions & 0 deletions packages/testing/src/execution_testing/forks/base_fork.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@

from ..recipient_type import RecipientType
from .gas_costs import GasCosts
from .requests import SystemContractRequest


class MemoryExpansionGasCalculator(Protocol):
Expand Down Expand Up @@ -262,6 +263,14 @@ class RefundTypes(Enum):
AUTHORIZATION_EXISTING_AUTHORITY = auto()


class SystemCallPhase(Enum):
"""When a block calls a system contract, if at all."""

NONE = "none"
BEFORE_TRANSACTIONS = "before_transactions"
AFTER_TRANSACTIONS = "after_transactions"


class BaseForkMeta(ABCMeta):
"""Metaclass for BaseFork."""

Expand Down Expand Up @@ -1102,6 +1111,20 @@ def system_contracts(cls) -> List[Address]:
"""Return list of system contracts supported by the fork."""
pass

@classmethod
@abstractmethod
def system_contract_call_phases(cls) -> Mapping[Address, SystemCallPhase]:
"""Return when the block calls each of its system contracts."""
pass

@classmethod
@abstractmethod
def system_contract_request_types(
cls,
) -> List[Type[SystemContractRequest]]:
"""Return the request classes triggered through a system contract."""
pass

@classmethod
@abstractmethod
def deterministic_factory_predeploy_address(cls) -> Address | None:
Expand Down
Loading
Loading