Repository navigation
fix(rpcsmartrouter): a client that joins during the last client's unsubscribe gets its own subscription (MAG-4345) - #589
Conversation
…ubscribe gets its own subscription (MAG-4345) Unsubscribe decided "last client" under the lock, called the node off the lock, and only then tore the subscription down. The entry stayed in the active map while the node call was in flight, so a client arriving for the same params joined it, a local step with no node call. The local cleanup then saw the joiner's mapping and tore nothing down, but the node had already dropped the subscription: the joiner got no pushes, and its channel was never closed, so it waited until it disconnected. The subscription now leaves the active map in the first critical section, before the node is called. A subscription the router has told the node to drop takes no new clients, and a client arriving meanwhile creates a subscription of its own on the node. A restore that reaches its commit point while the call is in flight finds nothing to commit onto and ends the subscription it re-opened (the MAG-2547 discard), so the restored-since-read branch goes. A node call that fails no longer returns early with the client still attached: the client asked to leave and the subscription is already retired, so it ends locally and the error goes back with the reply. A client that was not the last when its call started but is by the time the lock is retaken now tells the node, which that path never did. teardownSubscriptionLocked splits into retireSubscriptionLocked (the map entry, identity-guarded, and the counter) and endUpstreamSubscriptionLocked (the upstream half), so Unsubscribe can run the two halves apart. Tests: TestWS_JoinerDuringLastClientUnsubscribeGetsItsOwnSubscription, from Anna Rayer's attached repro (the arriving client subscribes in the background, because it now needs the node the test holds), and TestWS_LastClientUnsubscribeFailingUpstreamStillEndsTheSubscription (the node applies the unsubscribe but never answers). Both fail on main. TestUnsubscribeDuringRestore_TellsTheNodeTheRestoredID now asserts the outcome rather than the path: neither id is left on the node. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Took 20 minutes
… that ended under it (MAG-4345) A client that is not the last one releases the lock between Unsubscribe's two critical sections. If the subscription ends there by another path — the node ends it with an error, or a restore gives it up — the listener's cleanup releases every client, stops counting the subscription and tells the node. Unsubscribe then retook the lock, found no clients left, and tore the subscription down again: totalSubscriptions and the pooled connection's count each went to -1 and the node was told twice. On main the mapper's last-client answer, false once the cleanup had removed every mapping, kept it from doing so; the retire/end split dropped that guard. The teardown now runs only while the subscription is still the live entry under its params. beforeUnsubscribeRelease is a test hook, nil in production, in the shape of beforeRestoreCommit. TestWS_UnsubscribeDoesNotTearDownASubscriptionThatEndedUnderIt fails without the guard (totalSubscriptions -1, pool count -1, two eth_unsubscribe) and passes on main's manager with only the hook added. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…when the client attaches (MAG-4345)
The watcher a subscribe starts looked its subscription up when the
goroutine first ran, to read the client's detached signal. Unsubscribe now
takes the subscription out of the map in its first step, so a client that
unsubscribed before its watcher was scheduled left the watcher with a nil
channel: it stayed parked until the socket closed, one goroutine per
subscribe and unsubscribe on a long-lived socket. On main the entry stayed
mapped for the node's round trip, which hid the race.
The signal is now read where the client is attached — by the creator while
it still owns the subscription, by a joiner under the manager lock — and
handed to the watcher.
TestUnsubscribeThenDisconnect_LeavesNothingBehind ("the disconnect watcher
must not stay parked once the client unsubscribed") failed 6 of 20 runs
without this and passes 30 of 30 with it; it passes 20 of 20 on main.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AnnaR-prog
left a comment
There was a problem hiding this comment.
Reviewed at bbe3923. I tested it merged with current main b875f86, and against main's copy of the manager with go test -overlay.
589-1 (High): the disconnect watcher misses a client that unsubscribes before the watcher runs.
StartSubscriptionand the join path starthandleClientDisconnect. It looks the subscription up when the goroutine first runs, to read the client'sdetachedsignal.Unsubscribenow takes the subscription out of the map in its first step. A client that unsubscribes before its watcher is scheduled leaves the watcher with a nil channel, and the watcher stays parked until the socket closes: one goroutine per subscribe and unsubscribe on a long-lived socket.- On main the entry stayed in the map for the node's round trip, which hid the race.
TestUnsubscribeThenDisconnect_LeavesNothingBehind, already on main, catches it with "the disconnect watcher must not stay parked once the client unsubscribed". It fails in 2 to 12 of 20 runs on this head and passes 20 of 20 on main.- The required unit-tests job reruns a failing test only twice, so this would make the check fail on main and on every PR stacked here.
- I pushed a fix: the signal is read where the client is attached (by the creator while it still owns the subscription, by a joiner under the manager lock) and handed to the watcher. With it the test passes 30 of 30 here, and on #590 and #591.
589-2 (Low): a non-last client's unsubscribe can tear down, a second time, a subscription that ended under it.
- A client that is not the last one releases the lock between
Unsubscribe's two critical sections. Suppose the subscription ends in that gap by another path: the node ends it with an error, or a restore gives up. cleanupSubscriptionthen releases every client, decrements the counters and tells the node.Unsubscribefinds zero clients and runsteardownSubscriptionLockedagain.- With a hook between the two sections:
totalSubscriptionsis -1, the pooled connection's count is -1, and the node gets twoeth_unsubscribecalls. A count below the truth letsmaybeScaleDownclose a connection that still carries a subscription. - On main,
RemoveMappinganswered "not the last client" once the cleanup had removed every mapping. - I pushed three things: a guard (tear down only while the subscription is still the live entry), a test hook in the shape of
beforeRestoreCommit, andTestWS_UnsubscribeDoesNotTearDownASubscriptionThatEndedUnderIt, which fails without the guard.
Verified:
-
Both new tests fail with main's manager and pass here, also under
-race. -
Once the entry is retired, no other teardown path can find it.
Unsubscribeends its upstream half exactly once. A restore in flight reaches the MAG-2547 discard and ends the id it re-opened on the node. -
It merges cleanly with main and builds; vet and lint are clean.
-
protocol/rpcsmartrouter/direct_ws_subscription_manager.go:1880: The watcher readsdetachedhere, when it first runs. Since this PR takes the entry out of the map at the start ofUnsubscribe, a client that unsubscribes before the watcher is scheduled leaves it with a nil channel, parked until the socket closes (589-1). -
protocol/rpcsmartrouter/direct_ws_subscription_manager.go:1131: If the subscription ended by another path while the lock was released (node error, failed restore),cleanupSubscriptionalready released everyone and decremented the counters, and this tears it down again (589-2). Guarding ondwsm.activeSubscriptions[hashedParams] == activeSubrestores whatRemoveMapping'slastClientdid on main.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Description
A client that subscribes to the same params while the last client's
eth_unsubscribeis waiting on the node was left on a subscription the node had already dropped: no pushes, and a channel nobody closed, so it waited until it disconnected.Jira ticket: MAG-4345
Summary
Unsubscribeworked in three steps: decide "last client" under the lock, call the node off the lock, retake the lock and tear down only if nobody else is mapped now. The entry stayed in the active map for the whole node call, so an arriving client joined it — a local step with no node call — and the cleanup afterwards saw the joiner's mapping and tore nothing down.The subscription now leaves the active map in the first critical section, before the node is called. One rule: a subscription the router has told the node to drop takes no new clients. An arriving client creates a subscription of its own on the node instead.
Three consequences, all in this PR:
restoredSinceReadbranch goes. A restore that reaches its commit point while the call is in flight now finds nothing to commit onto and ends the subscription it re-opened, which is the MAG-2547 discard path.TestUnsubscribeDuringRestore_TellsTheNodeTheRestoredIDasserts the outcome rather than the path: neither the old id nor the restored one is left on the node.restoredSinceReadwas false.teardownSubscriptionLockedsplits intoretireSubscriptionLocked(the map entry, identity-guarded, and the counter) andendUpstreamSubscriptionLocked(the upstream half), soUnsubscribecan run the two halves apart.teardownSubscriptionLockeditself is unchanged for its other five callers.The mock node gained
unsubscribeUnanswered: it applies an unsubscribe and never answers it, as some gateways do.Test plan
TestWS_JoinerDuringLastClientUnsubscribeGetsItsOwnSubscription— from the repro Anna Rayer attached to the ticket. The arriving client subscribes on a background goroutine, because it now needs the node that the test is holding. Asserts it creates its own upstream subscription, receives pushes, and that nothing is left on the node, in the pool or in the counters once both clients go.TestWS_LastClientUnsubscribeFailingUpstreamStillEndsTheSubscription— the node applies the unsubscribe and never answers; the client leaves, the subscription ends, the error reaches the client, and a second unsubscribe finds nothing.main(verified withgo test -overlayagainstorigin/main's copy of the manager).go test -count=1 ./protocol/rpcsmartrouter/on the committed tree: ok, 249s.-raceon the committed tree: ok.go build -buildvcs=false ./...andgo vet ./protocol/rpcsmartrouter/on the committed tree.CI runs no unit tests on this repo, so the local runs above are the only test evidence.
🤖 Generated with Claude Code