Context
PR #819 adds batched forward epoch sync. In try_prepare_forward_epoch_sync, the prepare probe uses a hardcoded timeout:
Duration::from_millis(1_000), // with 2 retries
See aptos-core/consensus/src/block_storage/sync_manager/forward_epoch_sync.rs.
By contrast, fetch uses RPC_TIMEOUT_MSEC (5s from block_retrieval).
Problem
Cold build_forward_epoch_sync_index does O(epoch) work:
get_all(EpochByBlockNumber)get_qc_range- parent walk
- full
LedgerInfoSchema scan
RCA for an epoch-50 zero-hash stuck case measured ~1.3s for a similar QC scan alone. A 1s prepare probe therefore often times out → Ok(None) → legacy reverse sync (HashValue::zero()), re-entering the all-or-nothing zero-hash path that forward epoch sync is meant to avoid — especially on large epochs or loaded serving peers.
The short timeout was intentional for rolling-upgrade capability probing (old peers that cannot decode the new enum variant), but it also treats slow-but-capable peers as unavailable and silently falls back.
Recommendation
- Raise the default prepare timeout substantially (e.g. align with or exceed fetch /
RPC_TIMEOUT_MSEC, or use a dedicated larger budget suitable for cold index build). Concrete suggestion: at least 5s, preferably 10s+, pending measurement on cold vs warm index paths.
- Expose it as a node-configurable parameter (e.g. under consensus / local config alongside existing sync knobs such as
max_blocks_to_request), not a magic literal in the prepare call site.
- Optionally:
- name the constant clearly if a code default remains
- add metrics for prepare timeout vs success vs legacy fallback
Related
Non-goals
This issue is scoped to timeout default + configurability. Full redesign of prepare/index build is out of scope. Busy/semaphore behavior and index warm-path improvements are related but should stay separate unless briefly noted.
Context
PR #819 adds batched forward epoch sync. In
try_prepare_forward_epoch_sync, the prepare probe uses a hardcoded timeout:See
aptos-core/consensus/src/block_storage/sync_manager/forward_epoch_sync.rs.By contrast, fetch uses
RPC_TIMEOUT_MSEC(5s fromblock_retrieval).Problem
Cold
build_forward_epoch_sync_indexdoes O(epoch) work:get_all(EpochByBlockNumber)get_qc_rangeLedgerInfoSchemascanRCA for an epoch-50 zero-hash stuck case measured ~1.3s for a similar QC scan alone. A 1s prepare probe therefore often times out →
Ok(None)→ legacy reverse sync (HashValue::zero()), re-entering the all-or-nothing zero-hash path that forward epoch sync is meant to avoid — especially on large epochs or loaded serving peers.The short timeout was intentional for rolling-upgrade capability probing (old peers that cannot decode the new enum variant), but it also treats slow-but-capable peers as unavailable and silently falls back.
Recommendation
RPC_TIMEOUT_MSEC, or use a dedicated larger budget suitable for cold index build). Concrete suggestion: at least 5s, preferably 10s+, pending measurement on cold vs warm index paths.max_blocks_to_request), not a magic literal in the prepare call site.Related
Non-goals
This issue is scoped to timeout default + configurability. Full redesign of prepare/index build is out of scope. Busy/semaphore behavior and index warm-path improvements are related but should stay separate unless briefly noted.