Skip to content

fix: eleven behaviour drifts between duplicated code paths (MAG-4278) - #523

Merged
AnnaR-prog merged 23 commits into
mainfrom
fix/MAG-4278-drift-fixes
Oct 18, 2026
Merged

AnnaR-prog merged 23 commits into
mainfrom
fix/MAG-4278-drift-fixes

Conversation

@avitenzer

Copy link
Copy Markdown
Contributor

Jira ticket: MAG-4278

What this fixes

MAG-4278 lists twenty places where two copies of the same logic drifted apart and the drift changes behaviour. This PR takes the eleven that are self-contained: no open PR edits the same code and none touches admission or failover semantics. Each item is its own commit, with a test that fails on the old copy and pins the copy that was right.

Ticket item Fix Test
2 The GraphQL listener reads request headers through detachedReqHeaders like the other three listeners (it was created from the pre-MAG-3881 shape; fiber's header strings alias the per-connection buffer and the metrics goroutine reads them after the reply) GraphQL case in TestListeners_KeepNoHeaderPastTheRequest
6 A secondary-cache hit runs the same filterCachedReplyMetadata a primary hit does, so the spec's pass_reply headers (the Aptos ledger-state set, for one) are replayed from both tiers instead of only the transport pair TestSecondaryCacheHitReplaysTheSameFilteredHeadersAsAPrimaryHit
7 Tendermint batch elements are classified with the single-reply Tendermint rule (envelope, then result.response.{code,log}); the batch walker takes the element rule as a parameter and the JSON-RPC batch keeps its envelope-only rule TestCheckResponseErrorForTendermintBatch_SeesTheAppErrorASingleReplySees
8 The pool-empty bulk release clears BlockedProviderSessionUsedStatus like the per-provider release and the operator reset, so a provider it returns to rotation can be re-blocked TestResetValidAddresses_ClearsTheRedemptionFlagLikeTheOtherReleases
9 ValidateCollect fails a url whose head probe failed (a fetch-latest-block row, no vacuous latest-distance passes against 2^64-1) like validate does, so smartrouter health stops reporting such a node green TestValidateCollect_FailedHeadProbeFailsTheURL
10 The WebSocket rate-limit reply echoes the request id verbatim through MarshalJsonRPCErrorWithRequestID instead of coercing it to an int (string ids, and numeric strings through Atoi on the quoted token, came back as "id":0) TestHandleRateLimitReached_EchoesTheCallersId
12 The REST POST handler answers an undeclared verb with 501 like the catch-all handler, instead of a masked 500 TestRestListener_UndeclaredVerbIs501OnEveryHandler
14 test rpcsmartrouter reads --use-tls and --allow-insecure-connection from its own flag set (only the root command binds its flags to viper, so both read false before); test and health accept and pass --github-token/--gitlab-token for private spec repositories TestTestAndHealthCommandsReadTheirOwnFlags
15 /debug/reset-all resets the chain trackers (which also drops the process-wide endpoint-tip entries, the "tip re-seeds after reset" behaviour) and the probe back-off like reset-scores; reset-scores reports the probe back-off count it already reset TestDebugResetAll_SmartRouter_ResetsTrackersAndProbeBackoffLikeResetScores
16 The local-directory spec loader warns about files it could not decode when other files loaded, as the remote loaders do; before, a corrupt --use-static-spec file made its chain vanish with no log line TestGetAllSpecsFromLocalDir_ReportsACorruptSibling
19 Tendermint SendURI runs a transport failure through HandleNodeError like the REST and JSON-RPC proxies, so the router gets a coded LavaError (LavaErrorConnectionRefused for a refused dial) instead of the raw *url.Error TestTendermintSendURI_ClassifiesTransportErrorsLikeTheOtherProxies

Scope notes:

  • Item 14: the health command registers --use-tls/--allow-insecure-connection but never reads them, before and after this PR. That is the "never-read CLI flags" follow-up on MAG-4243, not a drift between copies, so it stays there.
  • Item 15: reset-all now lists chain-trackers and probe-backoff in its cleared array; reset-scores gains a probe_backoff_reset count in its reply.

Not in this PR (the ticket's remaining items)

Verification

  • go build ./..., go vet ./... clean on the committed tree.
  • go test -count=1 ./... on the committed tree: 37 packages ok. protocol/lavasession failed once on TestBlockReason_SurvivesTheEpochTransition: the epoch health probe released the provider before the assertion ran, a known timing race in that test; the probe never reads the flag this PR writes and blockProvider never sets it, so the drain-loop change is a no-op on that path. 13/13 in isolation and the whole package green on rerun.
  • golangci-lint run with the repo config: exactly the 34 findings origin/main reports with this linter version, none new.

🤖 Generated with Claude Code

avitenzer and others added 11 commits October 13, 2026 17:23
…G-4278)

The GraphQL listener still read fiberCtx.GetReqHeaders(), the pre-MAG-3881
shape: fiber's header strings alias the per-connection buffer, and the
metrics goroutine reads them after the reply. Every other listener uses
detachedReqHeaders. The shared keep-no-header-past-the-request test now
covers the GraphQL listener too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Took 14 minutes
… rule (MAG-4278)

A single Tendermint reply is checked by TendermintrpcMessage.CheckResponseError,
which reads the JSON-RPC envelope and then result.response.{code,log}; a batch
went through the JSON-RPC walker, which reads only the envelope. An
application error inside a batched abci_query therefore counted as a success.
The batch walker now takes the per-element rule as a parameter; the JSON-RPC
batch keeps the envelope-only rule, the Tendermint batch uses the single-reply
one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ndler too (MAG-4278)

The catch-all REST handler maps APINotSupportedError to 501; the POST handler
masked the same refusal as a GUID 500. Both handlers now give the same answer,
pinned by a listener test that posts and gets an undeclared path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r proxies (MAG-4278)

The REST and JSON-RPC proxies run an httpClient.Do failure through
HandleNodeError, so the router sees a coded, retryable LavaError and the
node's address is handled by the classifier. The Tendermint URI proxy returned
the raw *url.Error. A refused dial on the URI path now resolves to
LavaErrorConnectionRefused like the REST path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ils (MAG-4278)

validate aborts a url whose three head probes fail; ValidateCollect carried on
with NOT_APPLICABLE, which casts to 2^64-1, so every latest-distance check
passed vacuously and smartrouter health reported the url green. The collect
path now records a failed fetch-latest-block verification and skips the rest,
the verdict validate already gives.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… (MAG-4278)

Every other locally generated JSON-RPC reply echoes the request id verbatim
through MarshalJsonRPCErrorWithRequestID; the rate-limit reply coerced it to an
int, answering string ids (and numeric strings, through Atoi on the quoted
token) with id 0. It now uses the shared marshaller.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… (MAG-4278)

The per-provider release and the operator reset both write
BlockedProviderSessionUnusedStatus when a blocked provider returns to rotation.
The pool-empty bulk release (setValidAddressesToDefaultValue) did not, so a
provider that came back through it kept BlockedProviderSessionUsedStatus and
every later failure read as a second-chance attempt that never re-blocked it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… (MAG-4278)

The remote loaders warn about every file they could not decode; the local
directory loader recorded the error and dropped it as long as one spec loaded,
so a corrupt file vanished silently. The walk is now its own helper that
returns what it recorded, and the loader warns when specs loaded with errors.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ache hit (MAG-4278)

A primary cache hit passes the spec's pass_reply headers through
filterCachedReplyMetadata; a secondary hit of the same entry went through the
foreign-reply sanitiser's own, narrower allow-list and kept only the transport
pair. The secondary path now runs the same filter over the foreign entry's
headers, so both tiers answer with the same header set.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…g/reset-all (MAG-4278)

/debug/reset-all claims to be the superset of /debug/reset-scores, but it
never reset the per-endpoint chain trackers (so the tip re-seeded from the
pre-reset trackers and the process-wide endpoint-tip store) nor the probe
back-off. Both endpoints now reset the same stores, and reset-scores reports
the probe back-off count it already reset.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…G-4278)

The test command read --use-tls and --allow-insecure-connection through viper,
but only the root command binds its flag set to viper, so both were always
false there. Neither the test nor the health command accepted the
--github-token/--gitlab-token flags the root command uses for private spec
repositories, and both loaded specs with empty tokens. The TLS flags are read
from the command's own flag set, and both subcommands register and pass the
spec tokens.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@AnnaR-prog AnnaR-prog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at c276cc0. I tested it as it will land: this branch merged with current main (8ad0ff6). The merge is clean apart from one test call, because #511 made resetValidAddresses return a single value.

All eleven fixes do what they claim on current main.

  • For items 2, 6, 8, 9, 10, 12 and 19, reverting the fix hunk makes its test fail.
  • Item 7's walker composes with #514's batch fold: TestTendermintRpcBatchCall still sees requested 100 and earliest 99. JSON-RPC batches keep the envelope-only rule.
  • The TestBlockReason_SurvivesTheEpochTransition failure is the pre-existing race you describe. It failed 3 of 200 runs here and 6 of 200 on main, with the same message.

523-1 (Medium, item 14): smartrouter test rpcsmartrouter now dials gRPC with TLS only.

  • Before: the command read --use-tls through viper, which has no binding on that command, so UseTLS was false. The connector dialled plaintext first and fell back to TLS, so both kinds of listener worked.
  • Now: the flag's registered default, true, takes effect for the first time. The connector sets up TLS up front and never tries plaintext.
  • Impact: a router's gRPC listener is plaintext unless the listen endpoint enables TLS. So the command's own example, 127.0.0.1:3334 LAV1 grpc, no longer connects without --use-tls=false.
  • Probe: against a plaintext gRPC server, AuthConfig{} connects, and the command's default AuthConfig fails with context deadline exceeded.
  • Not stated: nothing in the PR marks this as a behaviour change.
  • Proposed fix: register --use-tls with a default of false on the test command. The flag then works, the default behaviour stays the same, and --allow-insecure-connection still reaches the TLS retry.

523-2 (Low, item 10): a rate-limited batch or notification gets "id":1.

  • MarshalJsonRPCErrorWithRequestID echoes a top-level id. A batch frame and a notification have none, so the reply keeps the template's hardcoded 1. Before this PR it was 0.
  • A client whose ids start at 1 can settle the wrong request with that reply, while every batch member waits for its timeout.
  • I'm pushing a fix that goes through buildLocalReply, as the unsubscribe replies do: one error per batch member that has an id, and nothing for a notification.

523-3 (Low, item 8): the redemption-flag reset also runs on the epoch rebuild.

  • The write sits in setValidAddressesToDefaultValue, which also serves ReleaseEpochRebuild.
  • On a real epoch tick the sessions are fresh objects, so the write does nothing.
  • The failed-provider readmit and /debug/reset-pairing rebuild within the same epoch and reuse the same session objects. There, the write clears the flag of a redemption that is still in flight, so when that redemption succeeds, the provider is no longer returned to rotation.
  • I'm pushing a fix that skips the write for ReleaseEpochRebuild, with a test.

523-4 (Low, item 15): the reset test checks only strings, and the change to reset-scores isn't stated.

  • TestDebugResetAll_SmartRouter_ResetsTrackersAndProbeBackoffLikeResetScores builds the mux without a router, so it still passes with the two new reset calls removed.
  • On main, neither reset-scores nor reset-all reset the probe back-off; only /debug/reset-probe-backoff did. So "the probe back-off count it already reset" isn't right: this PR adds the reset to both.
  • Two comments still say no other reset reaches the back-off: the one on resetAllProbeBackoff and the one on /debug/reset-probe-backoff.
  • I'm pushing a test with a real tracker, which checks that reset-all drops the tip, plus the comment fixes.

523-5 (Low, item 6): docs/SECONDARY-CACHE.md is out of date.

  • It still says a secondary hit replays only Content-Type and Content-Encoding "by design" (:59-67, :568-576, :706).
  • Since this item, a secondary hit also replays Retry-After, Date and the spec's reply headers, like a primary hit.
  • I'm pushing the doc update.

523-6 (Low, item 9): ValidateCollect stops reporting a URL once its head probe fails.

  • Its doc and README line 83 promise "every verification for every node URL".
  • With this change, a URL whose head probe fails gets one fetch-latest-block row and nothing else.
  • So the checks that need no head (chain-id, trustless-rpc, the archive pruning value) aren't reported. A rate-limited eth_blockNumber alone now turns a URL red with no other diagnostics.
  • The verdict itself is right. The options:
    • keep reporting the head-independent checks after the failed row; or
    • update the doc and the README.
  • Either way, the test's loop over row.Verifications[1:] runs zero times, so it asserts nothing.

523-7 (Low, item 7): the test doesn't pin the parser wiring.

  • It calls CheckResponseErrorForTendermintBatch directly. If newBatchChainMessage is pointed back at the JSON-RPC checker, the test stays green.
  • I'm adding a parser-level test that parses a batched abci_query with the LAVA spec.

523-8 (Nit, item 19): a comment overstates the change.

  • The new comment says the URI path now masks the transport error.
  • The coded error still carries the node URL (NewLavaError(classified, err.Error())), as it does on the REST and JSON-RPC paths.
  • I'm rewording it.

Description:

  • The lint numbers are stale. Main is at 0 findings, and this PR adds none.
  • Item 12 reaches fewer specs than described. The bundled cosmoshub and lava specs inherit a POST collection, so only specs with no POST collection answer 501.

AnnaR-prog and others added 9 commits October 15, 2026 18:04
The merge is clean apart from one test call: #511 (MAG-4244) made
resetValidAddresses return only didReset, so the item-8 test reads the
single result.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… notification not at all (MAG-4278)

MarshalJsonRPCErrorWithRequestID echoes a top-level id and otherwise keeps
the template's own id, which for the rate-limit error is 1. A batch frame and
a notification have no top-level id, so a rate-limited batch was answered
with one object carrying "id":1 (before the id fix it was 0). Clients often
start their ids at 1, so that reply can settle a different request, and every
other member of the batch waits for its timeout.

The rate-limit reply now goes through buildLocalReply like the other local
replies in this file: a batch gets an array with one error per member that
has an id, and a notification gets no reply (JSON-RPC 2.0 sections 4.1 and 6).
The read loop sends nothing when the reply is empty.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…build re-blocks (MAG-4278)

The redemption-flag reset added to setValidAddressesToDefaultValue runs for
all three of its callers, not only the pool-empty release. On
ReleaseEpochRebuild, UpdateAllProviders re-blocks every carried-over provider
right after the drain, so none of them is back in rotation. The
failed-provider readmit and /debug/reset-pairing call UpdateAllProviders in
the same epoch with the same session objects, so the reset erased the flag
of a redemption session still in flight. OnSessionDone reads that flag to
return a provider whose redemption succeeded to rotation, so that success
no longer did.

The reset now skips ReleaseEpochRebuild, as the release log line in the same
function already does. Every path that later returns such a provider to
rotation clears the flag itself.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…te the back-off comments (MAG-4278)

TestDebugResetAll_SmartRouter_ResetsTrackersAndProbeBackoffLikeResetScores
builds the debug mux with no router, so it checks only the strings in the
reply: it stays green with the two new reset calls removed. The new test
seeds a real per-endpoint tip through an EndpointMonitor and checks that
reset-all drops it.

Before this change neither reset-scores nor reset-all touched the probe
back-off; only /debug/reset-probe-backoff did. Now both reset it, but the
comments on resetAllProbeBackoff and /debug/reset-probe-backoff still said
no other reset reaches it, and the reset-scores comment did not list it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…r (MAG-4278)

TestCheckResponseErrorForTendermintBatch_SeesTheAppErrorASingleReplySees
calls CheckResponseErrorForTendermintBatch directly. The fix it covers is
the one-line wiring in TendermintChainParser.newBatchChainMessage, and the
test stays green when that line points back at the JSON-RPC walker. The new
test parses a batched abci_query with the Tendermint parser and classifies
the reply through the chain message, so it fails if the wiring is reverted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ary hit does (MAG-4278)

The secondary path now runs the primary tier's reply-header filter over the
foreign entry's headers, so a secondary hit replays Content-Type,
Content-Encoding, Retry-After, Date and the spec's reply-direction headers,
minus any router-owned name. docs/SECONDARY-CACHE.md still said the secondary
keeps only Content-Type and Content-Encoding "by design", in the "No identity
leakage" bullet, in the demo walkthrough and in the requirements table.
Neither PR stacked on this one (#524, #525) touches the document.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…it does not mask them (MAG-4278)

The comment on SendURI's transport-error branch said the error is classified
"and masked" and that the raw *url.Error "used to leave here with the node's
address in it", which reads as if the address no longer does. handleAndClassify
builds the coded error with NewLavaError(classified, err.Error()), so the text
still carries the full node URL, as on the REST and JSON-RPC paths. A node URL
that embeds an API key in its path is no more hidden here than before, and the
comment should not suggest otherwise.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…default (MAG-4278)

Reading --use-tls from the test command's own flag set made its registered
default (true) take effect for the first time. Before, viper had no binding
on that command and read false, and the gRPC connector dialled plaintext
first and TLS only if that failed. With true it dials TLS only, so a
router's gRPC listener, which is plaintext unless its endpoint enables TLS,
became unreachable without --use-tls=false. That includes the command's own
Example (127.0.0.1:3334 LAV1 grpc).

--use-tls now defaults to false, so the command dials as it did before;
--use-tls forces TLS, and --allow-insecure-connection accepts a self-signed
certificate on either TLS path. The flag help says what the default does,
and a test dials a plaintext gRPC listener with the default flags.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ateCollect's head probe fails (MAG-4278)

Failing the url when its head probe fails is right: verifying against
NOT_APPLICABLE made every latest-distance check pass vacuously. But ending
the url there dropped every other check from the smartrouter health report,
against ValidateCollect's documented contract to run every verification. A
node that answers chain-id and refuses or rate-limits only eth_blockNumber
(a 429 ends the head probe's retries at once) then read the same as a dead
one.

After the failed fetch-latest-block row, ValidateCollect now goes through
every verification. One that needs the head (the predicate needsLatestBlock
applies, now verificationNeedsHead) is reported failed with "latest block
unknown" and the head error, without being sent. Every other one runs, with
0 as the latest block, which Verify reads as unknown. LatestBlock stays
NOT_APPLICABLE, so the row keeps no height and the health rollup still
fails it.

The PR test's loop over the remaining rows ran zero times. It now checks
each row against the spec's verifications. A new test uses a router that
refuses only eth_blockNumber: chain-id is reported ok, pruning fails as
head-unknown without being sent, and the url does not pass. A health_cmd
test pins the rollup: the row is not ok, it is not backfilled, and the
top-level ok is false. The README describes the row.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AnnaR-prog

Copy link
Copy Markdown
Contributor

Review fixes pushed: b9a1443

The branch now merges main (c258f7f). #511 made resetValidAddresses return one value, so the item-8 test reads that single result.

Finding Status
523-1 (Medium): the test command's TLS-only default Fixed in 604361f, as the reviewer decided. --use-tls now defaults to false on test rpcsmartrouter. The flag works, and the default is still plaintext first, then TLS. subcommand_flags_test.go pins the default and the parsed value. A new test reaches a plaintext gRPC listener with the default flags.
523-2 (Low): a rate-limited batch or notification gets "id":1 Fixed in 302932b. The reply goes through buildLocalReply: a batch gets one error per member that has an id, and a notification gets nothing. Test: TestHandleRateLimitReached_AnswersABatchPerMemberAndANotificationNotAtAll.
523-3 (Low): the redemption flag is cleared on the epoch rebuild Fixed in 9ef1fc5. The reset now skips ReleaseEpochRebuild. Test: TestSameEpochRebuild_KeepsTheRedemptionFlagOfAReblockedProvider.
523-4 (Low): the reset test checks only strings Fixed in 5fb8bc8. TestDebugResetAll_SmartRouter_DropsTheEndpointTip uses a real tracker and fails with the two reset calls removed. The comments now say which resets clear the probe back-off. The description's "already reset" wording is still to correct.
523-5 (Low): docs/SECONDARY-CACHE.md Fixed in a1de465.
523-6 (Low): ValidateCollect after a failed head probe Fixed in b9a1443, as the reviewer decided. After the failed fetch-latest-block row, every check that needs no head still runs and is reported. A check that needs the head fails with "latest block unknown" and isn't sent. LatestBlock stays unset. The old loop now asserts each row. New tests cover a router that refuses only eth_blockNumber, plus the health rollup (row not ok, no backfill, top-level ok false).
523-7 (Low): the parser wiring isn't pinned Fixed in 7c54ab7. TestTendermintBatchParsedMessage_SeesTheAppErrorASingleReplySees parses a batched abci_query with the LAVA spec.
523-8 (Nit): the URI-path comment Fixed in 2e7a0c7.

Verified on b9a1443:

  • go build ./... and go vet are clean.
  • go test passes on ./protocol/chainlib/..., ./protocol/rpcsmartrouter/ and ./utils/keeper/.
  • golangci-lint reports 0 issues on the touched packages and against main.
  • On the commit before the last two, -race also passes on chainlib and lavasession.

AnnaR-prog added a commit that referenced this pull request Oct 17, 2026
… into refactor/MAG-4279-batch1a

One conflict, protocol/chainlib/chainproxy/connector.go: #511 (MAG-4244)
folded the connector's three dial sites into a grpcDialOptions helper on
main; this branch replaces the same three sites with dialOptions over the
shared common.NodeUrl.GRPCDialOptions, which carries the same MAG-2218
auth-headers and receive-size cap. Keep this branch's side.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AnnaR-prog added a commit that referenced this pull request Oct 17, 2026
…MAG-4279-batch1a

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AnnaR-prog added a commit that referenced this pull request Oct 17, 2026
…ew fixes) into refactor/MAG-4279-descriptor-resolver

Clean merge.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AnnaR-prog added a commit that referenced this pull request Oct 17, 2026
…s) into refactor/MAG-4279-descriptor-resolver

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AnnaR-prog and others added 2 commits October 17, 2026 20:42
…mption-flag test (MAG-4278)

TestSameEpochRebuild_KeepsTheRedemptionFlagOfAReblockedProvider went
through UpdateAllProviders twice, and so raced its deferred
checkAndUnblockHealthyReBlockedProviders: that runs on a goroutine after a
random sleep of up to 500 ms and may release the re-blocked provider,
clearing its flag as every release does, before the assertions run. It
failed that way in a full-suite run (the same race the older
TestBlockReason_SurvivesTheEpochTransition is known for).

The test now runs setValidAddressesToDefaultValue with
ReleaseEpochRebuild under the lock, the way UpdateAllProviders does, and
checks the flag the in-flight redemption reads. 30/30 green; with the
drain's route guard removed it fails.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…er starts (MAG-4278)

start() reset latestChangeTime with a plain write, outside blockQueueMu,
although ResetLatestBlock writes the same field under that lock and the
file's own rule is that every access outside the lock goes through the
locked helpers. A reset that lands while a tracker is still starting is a
data race: /debug/reset-scores could already hit it, and with this branch
/debug/reset-all resets the trackers too. Under -race,
TestDebugResetAll_SmartRouter_DropsTheEndpointTip, which resets a tracker
it has just created, reported it (ResetLatestBlock against start).

start() now goes through setLatestChangeTime.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AnnaR-prog

Copy link
Copy Markdown
Contributor

Two more fixes pushed: a40be3f

Both were found in an integration run of this PR with #519, #520, #522, #524 and #525 on main.

  • 59bd166 (test): one of my 523-3 tests was flaky.
    • TestSameEpochRebuild_KeepsTheRedemptionFlagOfAReblockedProvider failed once in a full-suite run.
    • Cause: it went through UpdateAllProviders, which races that call's deferred checkAndUnblockHealthyReBlockedProviders. That runs on a goroutine after a random sleep of up to 500 ms, and it can release the provider and clear its flag before the test's assertions run. TestBlockReason_SurvivesTheEpochTransition is known for the same race.
    • The test now calls setValidAddressesToDefaultValue(…, ReleaseEpochRebuild) directly, under the lock. It passes 30/30 and still fails with the route guard removed.
  • a40be3f (fix): a data race when a reset lands while a tracker is starting.
    • ChainTracker.start() reset latestChangeTime with a plain write outside blockQueueMu. ResetLatestBlock writes the same field under that lock.
    • /debug/reset-scores could already hit this. With this PR, /debug/reset-all resets trackers too.
    • Under -race, TestDebugResetAll_SmartRouter_DropsTheEndpointTip reported it.
    • start() now uses setLatestChangeTime, and the test passes 20/20 under -race.

Already failing on main: go test -race ./protocol/chaintracker/ fails with 14 test-level races, in TestChainTrackerCallbacks and TestChainTrackerFetchSpreadAcrossPollingTime. The race sites are the same with this PR.

On the integration tree (main plus all six PRs):

  • go test ./...: all 38 packages pass.
  • -race passes on rpcsmartrouter, chainlib and lavasession.
  • golangci-lint: 0 issues.

@AnnaR-prog AnnaR-prog left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 6b298fe: main 027cc1c (after #519 and #520) plus this PR and the review fixes, including the two follow-up fixes from the integration run (the redemption-flag test and the latestChangeTime race).

  • On this exact tree, go build ./..., go vet ./protocol/... ./utils/... and the tests of ./protocol/lavasession/, ./protocol/rpcsmartrouter/, ./protocol/chainlib/..., ./protocol/common/..., ./protocol/chaintracker/ and ./utils/... pass.
  • I also validated this PR together with #519, #520, #522, #524 and #525 on one integration tree: golangci-lint reports 0 issues, go test ./... passes, and -race is clean on the touched packages.
  • go test -race ./protocol/chaintracker/ fails on main too (test-level races that predate this PR); it is not counted above.

AnnaR-prog added a commit that referenced this pull request Oct 18, 2026
…into refactor/MAG-4279-batch1a

Brings in #523's last two fixes and main after #519 and #520, so this
branch carries the exact #523 that is about to be squash-merged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AnnaR-prog
AnnaR-prog merged commit f4ebf95 into main Oct 18, 2026
@AnnaR-prog
AnnaR-prog deleted the fix/MAG-4278-drift-fixes branch October 18, 2026 09:12
AnnaR-prog added a commit that referenced this pull request Oct 18, 2026
Main now holds #523 as squash f4ebf95. This branch already carries that
exact #523 (its final head, 6b298fe), so the tree does not change; the
merge only makes main an ancestor for the retargeted PR.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AnnaR-prog added a commit that referenced this pull request Oct 18, 2026
… into refactor/MAG-4279-descriptor-resolver

Brings in #523's last two fixes and main after #519, #520 and #523, so
this branch carries the exact #524 that is about to be squash-merged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants