Repository navigation
fix(endpointstate): report every answered poll on the url's series, not only one the tip store kept (MAG-4347) - #592
Merged
Conversation
…ot only one the tip store kept (MAG-4347) The poll-block hook fired only when the per-url tip store accepted the poll's block, and that hook is the write behind rpc_endpoint_url_latest_block and the answered-until time the per-url stuck alerts read. A block below a still-fresh stored tip is a straggler the store drops, so a url that answered every poll wrote nothing for itself and went silent on its own series. The fleet gate makes that a steady state rather than an occasional event: each peer skip adopts the peer's block again, which keeps the stored tip fresh, so every forced poll that answers lower than the peer is a straggler. Solana reaches it by design, because the tracker polls getLatestBlockhash at finalized commitment while relays harvest the slot at the client's, which is newer. So does the non-relayed provider of a url two providers share, on a single pod. The hook now fires on every ANSWERED poll, with the block the url answered rather than the stored tip. A url stuck on an old block therefore still reads as stuck: its series stands still while its answered-until time keeps moving, which is the shape the alerts look for. A failed poll still reports nothing, so a url that stops answering stops both and reads as down within one window of its last answer. This must land before smart-router-infrastructure PR 840 applies the per-url stuck alerts, or Solana pods false-alert on urls that are answering. Tests: TestAnsweredPollBelowAFreshPeerTipStillReportsTheAnswer, from the repro Anna Rayer attached to the ticket, and TestStuckUrlKeepsAnsweringWithItsOldBlock both fail on main. TestAnsweredUntilStopsWhenTheUrlStopsAnswering is the other half of the acceptance: the fix must not make a dead url look alive. TestOnPollBlock_ReportsEveryAnsweredPoll pinned the defect — its straggler row asserted that a block below a fresh tip is not reported — and now asserts the answer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AnnaR-prog
reviewed
Nov 3, 2026
AnnaR-prog
left a comment
Contributor
There was a problem hiding this comment.
Reviewed at 13c2437. I tested it merged with current main 6d585d4.
The change is correct.
answeredBlockis set only for an answered poll with a block, after the generation and attempt-order guards, so a stale attempt, a replaced tracker and a stopped monitor still report nothing, and a failed poll still stops both series.- I read it against the per-url stuck rules on smart-router-infrastructure PR 840. A url frozen at an old block now writes that block on every answered poll, so its series stands still while its answered-until time moves, and
sr-provider-stuckfires for it. On main, the same url's polls were stragglers below the fresh tip, so it read as down instead of stuck. The rules only usechanges()andoffset, so the change cannot make them misfire. - Solana's GET_BLOCKNUM in lava-specs polls
getLatestBlockhashat finalized commitment, so the straggler case is the normal case there, as the description says. One side effect: a url whose relays harvest a newer slot than its polls now has a non-monotonic per-url series (it dips to the finalized slot after each poll). No rule or dashboard depends on it being monotonic. - With main's
observation.gooverlaid, the repro reports 1 of 4 answered polls and three tests fail. They pass here, also with-race, and with #600 merged on top.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AnnaR-prog
approved these changes
Nov 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A url that answers every poll could read as down on its own per-url series, because the write behind that series only happened when the router's tip store kept the poll's block.
Jira ticket: MAG-4347
This must land before smart-router-infrastructure PR 840 applies the per-url stuck alerts, or Solana pods will false-alert on urls that are answering. It is independent of the three WebSocket PRs (#589, #590, #591) and can merge in any order relative to them.
Summary
EndpointMonitor.recordPollObservationcalledonPollBlockonly when the per-url tip store accepted the poll's block. That hook is the write behindrpc_endpoint_url_latest_blockand the answered-until time the per-url stuck alerts read. A block below a still-fresh stored tip is a straggler the store drops, so an answered poll wrote nothing for the url.The fleet gate makes that a steady state rather than an occasional event: each peer skip adopts the peer's block again, which keeps the stored tip fresh, so every forced poll that answers lower than the peer is a straggler. Two shapes reach it without anything being wrong:
getLatestBlockhashat finalized commitment; relays harvest the slot at the client's commitment, which is newer, and relay observations are published to the fleet store. A pod that serves no relays for a url adopts a peer's newer slot on each skip, and its own finalized polls are stragglers inside the stored tip's freshness horizon.The hook now fires on every answered poll, with the block the url answered rather than the stored tip. Keeping the answered block is what preserves the alert's signal: a url stuck on an old block still reads as stuck, because its series stands still while its answered-until time keeps moving. A failed poll still reports nothing, so a url that stops answering stops both and reads as down within one window of its last answer.
The ticket suggested recording the answered-until time separately from the block value. One write does both here:
SetEndpointURLLatestBlocksets the value and the answered-until stamp together, so reporting the answered block is the smaller change and keeps one source for the series.Test plan
TestAnsweredPollBelowAFreshPeerTipStillReportsTheAnswer— from the repro Anna Rayer attached to the ticket. A peer reports one block above what this pod's polls answer; four polls are answered and all four must be reported, each with the answered block and not the peer's.TestStuckUrlKeepsAnsweringWithItsOldBlock— a url frozen far below the fleet reports its own frozen block every poll.TestAnsweredUntilStopsWhenTheUrlStopsAnswering— the other half of the acceptance: the fix must not make a dead url look alive.TestOnPollBlock_ReportsEveryAnsweredPollpinned the defect. Its straggler row asserted that a block below a fresh tip is not reported, which is the behaviour this ticket calls wrong; it now asserts the answer. Worth a look, since it is the one place a reviewer can see the contract change.main, verified withgo test -overlayagainstorigin/main's copy ofobservation.go. On main the repro reports 1 write for 4 answered polls.go test -count=1onprotocol/endpointstate(24s) andprotocol/rpcsmartrouter(243s) on the committed tree: ok.protocol/endpointstatealso passes under-raceon the committed tree.go build -buildvcs=false ./...andgo veton the committed tree.CI runs no unit tests on this repo, so the local runs above are the only test evidence.
🤖 Generated with Claude Code