Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions consensus/bor/bor.go
Original file line number Diff line number Diff line change
Expand Up @@ -1622,6 +1622,34 @@ func (c *Bor) runMilestoneFetcher() {
}
}

// stateTracingHooks is implemented by state wrappers (state.NewHookedState)
// that emit tracing hooks for the state they wrap.
type stateTracingHooks interface {
Hooks() *tracing.Hooks
}

// systemTxVMConfig returns the vm.Config to use when applying bor system
// transactions (span commits and state-sync events) over the given state.
//
// The tracer is derived from the state itself rather than taken from
// c.vmConfig: canonical block import passes a hooked state when a live tracer
// is configured, so system transactions keep being traced there. Every other
// caller — the miner and eth_simulateV1 via FinalizeAndAssemble, or historical
// state regeneration via Finalize — passes a plain state and must not fire the
// node-wide live tracer: those run outside the import goroutine, and invoking
// the singleton live tracer concurrently corrupts its state and can crash the
// node.
func (c *Bor) systemTxVMConfig(state vm.StateDB) vm.Config {
cfg := c.vmConfig
if hooked, ok := state.(stateTracingHooks); ok {
cfg.Tracer = hooked.Hooks()
} else {
cfg.Tracer = nil
}

return cfg
}

func (c *Bor) checkAndCommitSpan(
state vm.StateDB,
header *types.Header,
Expand Down Expand Up @@ -1744,7 +1772,7 @@ func (c *Bor) FetchAndCommitSpan(
)
}

return c.spanner.CommitSpan(ctx, minSpan, validators, producers, state, header, chain, c.vmConfig)
return c.spanner.CommitSpan(ctx, minSpan, validators, producers, state, header, chain, c.systemTxVMConfig(state))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

systemTxVMConfig sets cfg.Tracer = nil whenever the passed-in state does not implement stateTracingHooks, and the parallel state processor passes an unwrapped *state.StateDB to Finalize. With --vmtrace enabled, bor span-commit and state-sync system transactions therefore execute with no tracer on the parallel execution path.

Impact: Observability only — no state, receipt, or consensus output changes — but live-tracer streams lose bor system transactions on the parallel-processor path with no error or warning, making the gap data-dependent and hard to detect downstream by consumers building external indexes/state feeds.

Evidence — the helper drops the tracer rather than falling back to the engine config:

// consensus/bor/bor.go:1651-1660
func (c *Bor) systemTxVMConfig(state vm.StateDB) vm.Config {
	cfg := c.vmConfig
	if hooked, ok := state.(stateTracingHooks); ok {
 cfg.Tracer = hooked.Hooks()
	} else {
 cfg.Tracer = nil
	}

	return cfg
}

Only *hookedStateDB satisfies stateTracingHooks (core/state/statedb_hooked.go:314), and the two import paths differ:

// core/state_processor.go — serial path wraps the state, tracing preserved
var tracingStateDB = vm.StateDB(statedb)
if hooks := cfg.Tracer; hooks != nil {
	tracingStateDB = state.NewHookedState(statedb, hooks)
}
receipts, err = p.chain.Engine().Finalize(p.chain, header, tracingStateDB, block.Body(), receipts)
// core/parallel_state_processor.go:431 — plain statedb, else-branch taken, tracer nil
receipts, err = p.chain.Engine().Finalize(p.bc.hc, header, statedb, block.Body(), receipts)

The PR and commit bodies declare an intentional change away from c.vmConfig for system transactions, so the contract change itself is deliberate; what needs confirmation is whether dropping tracing on the parallel import path is also intended, since canonical import was meant to keep tracer coverage.

Suggested fix: either wrap the state in core/parallel_state_processor.go before calling Finalize, mirroring the serial path's state.NewHookedState(statedb, hooks), or fall back to the engine-configured tracer instead of nil:

cfg := c.vmConfig
if hooked, ok := state.(stateTracingHooks); ok {
	cfg.Tracer = hooked.Hooks()
}
// otherwise keep cfg.Tracer from c.vmConfig
return cfg

Suggested test: import a block through the parallel state processor with a live tracer configured and assert that OnTxStart/OnEnter fire for the bor span-commit and state-sync system transactions.

Relatedly, the added consensus/bor/vmconfig_test.go (TestSystemTxVMConfig) only covers the helper in isolation; consider adding coverage asserting that system-transaction tracing is preserved on canonical import and suppressed for FinalizeAndAssemble/eth_simulateV1.

}

// CommitStates commit states
Expand Down Expand Up @@ -1849,6 +1877,8 @@ func (c *Bor) CommitStates(

var gasUsed uint64

vmConfig := c.systemTxVMConfig(state)

for _, eventRecord := range eventRecords {
if eventRecord.ID <= lastStateID {
continue
Expand Down Expand Up @@ -1894,7 +1924,7 @@ func (c *Bor) CommitStates(

// Receipt construction expects the receiver call to emit at least one log.
// A receiver without code can complete without producing one.
gasUsed, err = c.GenesisContractsClient.CommitState(eventRecord, state, header, chain, c.vmConfig)
gasUsed, err = c.GenesisContractsClient.CommitState(eventRecord, state, header, chain, vmConfig)
if err != nil {
return nil, err
}
Expand Down
117 changes: 117 additions & 0 deletions consensus/bor/vmconfig_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
package bor

import (
"math/big"
"testing"
"time"

"github.com/stretchr/testify/require"

"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/consensus/bor/clerk"
"github.com/ethereum/go-ethereum/consensus/bor/statefull"
"github.com/ethereum/go-ethereum/consensus/bor/valset"
"github.com/ethereum/go-ethereum/core/state"
"github.com/ethereum/go-ethereum/core/tracing"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/core/vm"
)

// TestSystemTxVMConfig verifies that system transactions (span commits and
// state-sync events) are traced only when the caller traces the state they
// are applied to. The node-wide live tracer stored in c.vmConfig must never
// leak into contexts that pass a plain state (miner and eth_simulateV1 via
// FinalizeAndAssemble, historical state regeneration via Finalize): those run
// outside the import goroutine, and invoking the singleton live tracer
// concurrently corrupts it.
func TestSystemTxVMConfig(t *testing.T) {
t.Parallel()

liveTracer := &tracing.Hooks{
OnEnter: func(depth int, typ byte, from, to common.Address, input []byte, gas uint64, value *big.Int) {
t.Error("live tracer must not be invoked for untraced states")
},
}
c := &Bor{vmConfig: vm.Config{Tracer: liveTracer, NoBaseFee: true}}

plainState, err := state.New(types.EmptyRootHash, state.NewDatabaseForTesting())
require.NoError(t, err)

// A plain state (regeneration, miner, eth_simulateV1) must not get the
// live tracer, but keeps the rest of the config.
cfg := c.systemTxVMConfig(plainState)
require.Nil(t, cfg.Tracer)
require.True(t, cfg.NoBaseFee)

// A hooked state (canonical import with a live tracer) keeps being traced
// with the hooks that trace the state itself.
importHooks := &tracing.Hooks{}
cfg = c.systemTxVMConfig(state.NewHookedState(plainState, importHooks))
require.Same(t, importHooks, cfg.Tracer)
}

// capturingGenesisContract records the vm.Config each CommitState call receives.
type capturingGenesisContract struct {
lastStateID uint64
captured []vm.Config
}

func (m *capturingGenesisContract) CommitState(event *clerk.EventRecordWithTime, state vm.StateDB, header *types.Header, chCtx statefull.ChainContext, vmCfg vm.Config) (uint64, error) {
m.captured = append(m.captured, vmCfg)
return 0, nil
}

func (m *capturingGenesisContract) LastStateId(st *state.StateDB, number uint64, hash common.Hash) (*big.Int, error) {
return big.NewInt(int64(m.lastStateID)), nil
}

// TestCommitStates_SystemTxTracer asserts end to end through CommitStates that
// state-sync system transactions receive the tracer of the state they mutate:
// the hooks tracing the state during canonical import, and no tracer at all
// for plain states (miner and eth_simulateV1 via FinalizeAndAssemble,
// historical state regeneration via Finalize) — never the node-wide live
// tracer stored in c.vmConfig.
func TestCommitStates_SystemTxTracer(t *testing.T) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The span-commit sink of the new state-derived vm.Config has no test assertion. TestCommitStates_SystemTxTracer only captures and asserts the state-sync (CommitState) sink; the only spanner double discards the config it receives:

// consensus/bor/bor_test.go:72
func (s *fakeSpanner) CommitSpan(ctx context.Context, _ borTypes.Span, _ []stakeTypes.MinimalVal, _ []stakeTypes.MinimalVal, _ vm.StateDB, _ *types.Header, _ core.ChainContext, _ vm.Config) error {
	if s.shouldFailCommit {
 return errors.New("span commit failed")

The changed call site is behavior-bearing — base consensus/bor/bor.go:1747 passed c.vmConfig, head consensus/bor/bor.go:1775 passes the state-derived config:

return c.spanner.CommitSpan(ctx, minSpan, validators, producers, state, header, chain, c.systemTxVMConfig(state))

A repo-wide search over consensus/bor found no assertion on the vm.Config given to CommitSpan.

Impact: No incorrect behavior today — production wiring is correct and systemTxVMConfig is unit-tested directly. But a future edit or bad merge restoring c.vmConfig at consensus/bor/bor.go:1775, or a state wrapper that stops implementing stateTracingHooks on the span path, would keep the whole consensus/bor suite green while re-introducing exactly the defect this PR fixes: the node-wide live tracer being invoked for span-commit system txs on plain states from the miner, eth_simulateV1 (via FinalizeAndAssemble) and historical regeneration (via Finalize). This PR establishes that both system-tx sinks derive their tracer from the state they mutate; only one half is pinned.

Suggested fix: Give fakeSpanner a capturedVMConfigs []vm.Config field, append the argument in CommitSpan, and add a sibling test:

func TestCommitSpan_SystemTxTracer(t *testing.T) {
	b.vmConfig = vm.Config{Tracer: &tracing.Hooks{}, NoBaseFee: true}
	// (a) plain state: tracer dropped, other fields preserved
	_ = b.FetchAndCommitSpan(/* pre-Rio header */)
	require.Nil(t, sp.capturedVMConfigs[0].Tracer)
	require.True(t, sp.capturedVMConfigs[0].NoBaseFee)
	// (b) hooked state: tracer taken from the state
	_ = b.FetchAndCommitSpan(/* with state.NewHookedState(plain, importHooks) */)
	require.Same(t, importHooks, sp.capturedVMConfigs[0].Tracer)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining for now to keep the PR focused: both system-tx sinks route through the single systemTxVMConfig helper, whose plain-state and hooked-state behavior is pinned directly by TestSystemTxVMConfig, and the state-sync sink is additionally covered end-to-end by TestCommitStates_SystemTxTracer. A call-site regression at CommitSpan would have to bypass the helper entirely, which review would catch on a one-line diff. If maintainers would like the span-side capture as an extra pin, I'm happy to add it here or in a follow-up.

t.Parallel()

addr1 := common.HexToAddress("0x1")
sp := &fakeSpanner{vals: []*valset.Validator{{Address: addr1, VotingPower: 1}}}

now := time.Now()
chain, b := newChainAndBorForTest(t, sp, indoreBorConfig(), true, addr1, uint64(now.Unix()))

// The node-wide live tracer configured at startup: must never reach system txs.
b.vmConfig = vm.Config{Tracer: &tracing.Hooks{}, NoBaseFee: true}

gc := &capturingGenesisContract{lastStateID: 0}
b.GenesisContractsClient = gc
b.SetHeimdallClient(&mockHeimdallClient{
events: []*clerk.EventRecordWithTime{
{
EventRecord: clerk.EventRecord{ID: 1, Contract: common.HexToAddress("0x1001"), Data: []byte{0x01}, ChainID: "1"},
Time: now.Add(-10 * time.Second),
},
},
})

genesis := chain.HeaderChain().GetHeaderByNumber(0)
header := &types.Header{Number: big.NewInt(16), ParentHash: genesis.Hash(), Time: uint64(now.Unix())}
cx := statefull.ChainContext{Chain: chain.HeaderChain(), Bor: b}

// Plain state (regeneration, miner, eth_simulateV1): no tracer, rest of the
// engine config preserved.
_, err := b.CommitStates(newStateDBForTest(t, genesis.Root), header, cx)
require.NoError(t, err)
require.Len(t, gc.captured, 1)
require.Nil(t, gc.captured[0].Tracer)
require.True(t, gc.captured[0].NoBaseFee)

// Hooked state (canonical import with a live tracer): traced with the hooks
// tracing the state itself.
gc.captured = nil
importHooks := &tracing.Hooks{}
_, err = b.CommitStates(state.NewHookedState(newStateDBForTest(t, genesis.Root), importHooks), header, cx)
require.NoError(t, err)
require.Len(t, gc.captured, 1)
require.Same(t, importHooks, gc.captured[0].Tracer)
}
34 changes: 31 additions & 3 deletions core/parallel_state_processor.go
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,17 @@ func (p *ParallelStateProcessor) Process(block *types.Block, statedb *state.Stat

context := NewEVMBlockContext(header, p.bc.hc, author)

vmenv := vm.NewEVM(context, statedb, config, cfg)
// Mirror the serial processor: when the caller configured a tracer, run the
// pre-execution system calls and bor system transactions (span commits and
// state-sync events, via Finalize below) over a hooked state so they are
// traced with the caller's hooks and the tracer's VMContext reports the
// same state instance those calls mutate.
var tracingStateDB = vm.StateDB(statedb)
if hooks := cfg.Tracer; hooks != nil {
tracingStateDB = state.NewHookedState(statedb, hooks)
}

vmenv := vm.NewEVM(context, tracingStateDB, config, cfg)

if beaconRoot := block.BeaconRoot(); beaconRoot != nil {
ProcessBeaconBlockRoot(*beaconRoot, vmenv)
Expand Down Expand Up @@ -404,11 +414,27 @@ func (p *ParallelStateProcessor) Process(block *types.Block, statedb *state.Stat
// Polygon/bor: EIP-6110, EIP-7002, and EIP-7251 are not supported
var requests [][]byte

// State-sync transactions (post-Madhugiri) are the last tx in body. In order to produce
// accurate traces for state-sync transactions, we fire the OnTxStart and OnTxEnd hooks
// here before calling Finalize, mirroring the serial processor.
var (
hasStateSyncTx = len(txs) > 0 && txs[len(txs)-1].Type() == types.StateSyncTxType
stateSyncReceipt *types.Receipt
stateSyncEndErr error
)
if hooks := cfg.Tracer; hooks != nil && hasStateSyncTx && hooks.OnTxStart != nil && hooks.OnTxEnd != nil {
hooks.OnTxStart(vmenv.GetVMContext(), txs[len(txs)-1], params.BorSystemAddress)
Comment thread
claude[bot] marked this conversation as resolved.
defer func() {
hooks.OnTxEnd(stateSyncReceipt, stateSyncEndErr)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the parallel path, a recovered panic fires the newly added OnTxEnd(nil, nil), reporting the synthetic state-sync tx as successfully ended while Process returns an error.

The new hook scope in core/parallel_state_processor.go:

if hooks := cfg.Tracer; hooks != nil && hasStateSyncTx && hooks.OnTxStart != nil && hooks.OnTxEnd != nil {
	hooks.OnTxStart(vmenv.GetVMContext(), txs[len(txs)-1], params.BorSystemAddress)
	defer func() {
 hooks.OnTxEnd(stateSyncReceipt, stateSyncEndErr)
	}()
}

is registered at head lines 425-427, after the function-level recover defer at lines 265-272:

func (p *ParallelStateProcessor) Process(...) (processResult *ProcessResult, err error) {
	defer func() {
 if r := recover(); r != nil {
 log.Error("recovered from panic during parallel execution", "err", r)
 processResult = nil
 err = fmt.Errorf("panic during parallel execution: %v", r)
 }
	}()

Defers run LIFO, so on a panic inside p.chain.Engine().Finalize (bor CommitSpan/CommitStates) or the receipt/log handling that follows, OnTxEnd runs first with the named locals stateSyncReceipt and stateSyncEndErr still nil, and only then does recover set err. The explicit error returns in the same hunk (lines 434-437 and 443-445) do set stateSyncEndErr, so the recovered-panic window is the only path that emits OnTxEnd(nil, nil). The serial processor being mirrored in core/state_processor.go has no recover, so it does not have this window.

Impact: Bounded to tracer output — no state, receipt, or consensus value is affected. Any consumer reconstructing tx outcomes from OnTxStart/OnTxEnd pairs records a successful state-sync transaction for a block that never completed, and live/streaming tracers emit that trace. This is a new-behavior hunk (the parallel path now hands Finalize a hooked state so bor system transactions are traced); please confirm the intended hook semantics on the recovered-panic path.

Suggested fix: Make the panic case observable to the hook:

completed := false
if hooks := cfg.Tracer; hooks != nil && hasStateSyncTx && hooks.OnTxStart != nil && hooks.OnTxEnd != nil {
	hooks.OnTxStart(vmenv.GetVMContext(), txs[len(txs)-1], params.BorSystemAddress)
	defer func() {
 if !completed && stateSyncEndErr == nil {
 stateSyncEndErr = errors.New("panic during parallel execution")
 }
 hooks.OnTxEnd(stateSyncReceipt, stateSyncEndErr)
	}()
}

with completed = true set immediately before the successful return. Alternatively, register the OnTxEnd defer before the recover defer so recover can populate stateSyncEndErr first.

Suggested test: Run ParallelStateProcessor.Process on a post-Madhugiri block whose last tx is a state-sync tx, with a stub tracing.Hooks recording OnTxStart/OnTxEnd and an engine whose Finalize panics; assert Process returns an error and the recorded OnTxEnd carries a non-nil error rather than (nil, nil).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining this one deliberately: the deferred OnTxEnd firing with a nil receipt during panic unwinding is the established hook contract in this codebase — ApplyTransactionWithEVM's deferred hooks.OnTxEnd(receipt, err) behaves identically on the serial path, so tracers must already tolerate nil receipts. Special-casing the parallel copy with a completed flag would make it diverge from the serial block it mirrors (the drift concern raised in the other thread), for a path that is unreachable today (eth/backend.go refuses --vmtrace with ParallelEVM enabled) and where the process already surfaces the failure as an error. If the hook semantics on panic paths should change, that's better done consistently for both processors in a follow-up.

}()
}

// Finalize the block, applying any consensus engine specific extras (e.g. block rewards), apply
// state sync event (if any), and append the receipt.
receiptsCountBeforeFinalize := len(receipts)
receipts, err = p.chain.Engine().Finalize(p.bc.hc, header, statedb, block.Body(), receipts)
receipts, err = p.chain.Engine().Finalize(p.bc.hc, header, tracingStateDB, block.Body(), receipts)
if err != nil {
stateSyncEndErr = err
return nil, err
}

Expand All @@ -417,12 +443,14 @@ func (p *ParallelStateProcessor) Process(block *types.Block, statedb *state.Stat
// Defense-in-depth: if insertStateSyncTransactionAndCalculateReceipt silently failed
// to add the receipt, the count will be off.
if len(block.Transactions()) != len(receipts) {
return nil, fmt.Errorf("%w: receipt count mismatch, txs=%d receipts=%d", ErrStateSyncMismatch, len(block.Transactions()), len(receipts))
stateSyncEndErr = fmt.Errorf("%w: receipt count mismatch, txs=%d receipts=%d", ErrStateSyncMismatch, len(block.Transactions()), len(receipts))
return nil, stateSyncEndErr
}
appliedNewStateSyncReceipt := receiptsCountBeforeFinalize+1 == len(receipts)

if appliedNewStateSyncReceipt {
allLogs = append(allLogs, receipts[len(receipts)-1].Logs...)
stateSyncReceipt = receipts[len(receipts)-1]
}
}

Expand Down
5 changes: 5 additions & 0 deletions core/state/statedb_hooked.go
Original file line number Diff line number Diff line change
Expand Up @@ -309,3 +309,8 @@ func (s *hookedStateDB) Logs() []*types.Log {
func (s *hookedStateDB) Inner() *StateDB {
return s.inner
}

// Hooks returns the tracing hooks this state emits to.
func (s *hookedStateDB) Hooks() *tracing.Hooks {
return s.hooks
}
Loading