Skip to content

fix(rpcsmartrouter): a client that joins during the last client's unsubscribe gets its own subscription (MAG-4345) - #589

Merged
AnnaR-prog merged 4 commits into
mainfrom
fix/MAG-4345-joiner-during-unsubscribe
Nov 3, 2026
Merged

AnnaR-prog merged 4 commits into
mainfrom
fix/MAG-4345-joiner-during-unsubscribe

Conversation

@avitenzer

Copy link
Copy Markdown
Contributor

Description

A client that subscribes to the same params while the last client's eth_unsubscribe is waiting on the node was left on a subscription the node had already dropped: no pushes, and a channel nobody closed, so it waited until it disconnected.

Jira ticket: MAG-4345

Summary

Unsubscribe worked in three steps: decide "last client" under the lock, call the node off the lock, retake the lock and tear down only if nobody else is mapped now. The entry stayed in the active map for the whole node call, so an arriving client joined it — a local step with no node call — and the cleanup afterwards saw the joiner's mapping and tore nothing down.

The subscription now leaves the active map in the first critical section, before the node is called. One rule: a subscription the router has told the node to drop takes no new clients. An arriving client creates a subscription of its own on the node instead.

Three consequences, all in this PR:

  • The restoredSinceRead branch goes. A restore that reaches its commit point while the call is in flight now finds nothing to commit onto and ends the subscription it re-opened, which is the MAG-2547 discard path. TestUnsubscribeDuringRestore_TellsTheNodeTheRestoredID asserts the outcome rather than the path: neither the old id nor the restored one is left on the node.
  • A failed node call no longer returns early with the client still attached. The client asked to leave and the subscription is already retired, so it ends locally and the error still goes back with the reply. Keeping the client on it would park it on exactly the dead subscription this fix is about.
  • A client that was not the last when its call started, but is by the time the lock is retaken, now tells the node. That path previously skipped the wire call because restoredSinceRead was false.

teardownSubscriptionLocked splits into retireSubscriptionLocked (the map entry, identity-guarded, and the counter) and endUpstreamSubscriptionLocked (the upstream half), so Unsubscribe can run the two halves apart. teardownSubscriptionLocked itself is unchanged for its other five callers.

The mock node gained unsubscribeUnanswered: it applies an unsubscribe and never answers it, as some gateways do.

Test plan

  • TestWS_JoinerDuringLastClientUnsubscribeGetsItsOwnSubscription — from the repro Anna Rayer attached to the ticket. The arriving client subscribes on a background goroutine, because it now needs the node that the test is holding. Asserts it creates its own upstream subscription, receives pushes, and that nothing is left on the node, in the pool or in the counters once both clients go.
  • TestWS_LastClientUnsubscribeFailingUpstreamStillEndsTheSubscription — the node applies the unsubscribe and never answers; the client leaves, the subscription ends, the error reaches the client, and a second unsubscribe finds nothing.
  • Both fail on main (verified with go test -overlay against origin/main's copy of the manager).
  • go test -count=1 ./protocol/rpcsmartrouter/ on the committed tree: ok, 249s.
  • The WS subscription tests under -race on the committed tree: ok.
  • go build -buildvcs=false ./... and go vet ./protocol/rpcsmartrouter/ on the committed tree.

CI runs no unit tests on this repo, so the local runs above are the only test evidence.

🤖 Generated with Claude Code

…ubscribe gets its own subscription (MAG-4345)

Unsubscribe decided "last client" under the lock, called the node off the
lock, and only then tore the subscription down. The entry stayed in the
active map while the node call was in flight, so a client arriving for the
same params joined it, a local step with no node call. The local cleanup
then saw the joiner's mapping and tore nothing down, but the node had
already dropped the subscription: the joiner got no pushes, and its
channel was never closed, so it waited until it disconnected.

The subscription now leaves the active map in the first critical section,
before the node is called. A subscription the router has told the node to
drop takes no new clients, and a client arriving meanwhile creates a
subscription of its own on the node. A restore that reaches its commit
point while the call is in flight finds nothing to commit onto and ends
the subscription it re-opened (the MAG-2547 discard), so the
restored-since-read branch goes. A node call that fails no longer returns
early with the client still attached: the client asked to leave and the
subscription is already retired, so it ends locally and the error goes
back with the reply. A client that was not the last when its call started
but is by the time the lock is retaken now tells the node, which that
path never did.

teardownSubscriptionLocked splits into retireSubscriptionLocked (the map
entry, identity-guarded, and the counter) and
endUpstreamSubscriptionLocked (the upstream half), so Unsubscribe can run
the two halves apart.

Tests: TestWS_JoinerDuringLastClientUnsubscribeGetsItsOwnSubscription,
from Anna Rayer's attached repro (the arriving client subscribes in the
background, because it now needs the node the test holds), and
TestWS_LastClientUnsubscribeFailingUpstreamStillEndsTheSubscription (the
node applies the unsubscribe but never answers). Both fail on main.
TestUnsubscribeDuringRestore_TellsTheNodeTheRestoredID now asserts the
outcome rather than the path: neither id is left on the node.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Took 20 minutes
AnnaR-prog and others added 2 commits November 3, 2026 13:08
… that ended under it (MAG-4345)

A client that is not the last one releases the lock between Unsubscribe's
two critical sections. If the subscription ends there by another path —
the node ends it with an error, or a restore gives it up — the listener's
cleanup releases every client, stops counting the subscription and tells
the node. Unsubscribe then retook the lock, found no clients left, and tore
the subscription down again: totalSubscriptions and the pooled
connection's count each went to -1 and the node was told twice. On main
the mapper's last-client answer, false once the cleanup had removed every
mapping, kept it from doing so; the retire/end split dropped that guard.

The teardown now runs only while the subscription is still the live entry
under its params. beforeUnsubscribeRelease is a test hook, nil in
production, in the shape of beforeRestoreCommit.

TestWS_UnsubscribeDoesNotTearDownASubscriptionThatEndedUnderIt fails
without the guard (totalSubscriptions -1, pool count -1, two
eth_unsubscribe) and passes on main's manager with only the hook added.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…when the client attaches (MAG-4345)

The watcher a subscribe starts looked its subscription up when the
goroutine first ran, to read the client's detached signal. Unsubscribe now
takes the subscription out of the map in its first step, so a client that
unsubscribed before its watcher was scheduled left the watcher with a nil
channel: it stayed parked until the socket closed, one goroutine per
subscribe and unsubscribe on a long-lived socket. On main the entry stayed
mapped for the node's round trip, which hid the race.

The signal is now read where the client is attached — by the creator while
it still owns the subscription, by a joiner under the manager lock — and
handed to the watcher.

TestUnsubscribeThenDisconnect_LeavesNothingBehind ("the disconnect watcher
must not stay parked once the client unsubscribed") failed 6 of 20 runs
without this and passes 30 of 30 with it; it passes 20 of 20 on main.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@AnnaR-prog AnnaR-prog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at bbe3923. I tested it merged with current main b875f86, and against main's copy of the manager with go test -overlay.

589-1 (High): the disconnect watcher misses a client that unsubscribes before the watcher runs.

  • StartSubscription and the join path start handleClientDisconnect. It looks the subscription up when the goroutine first runs, to read the client's detached signal.
  • Unsubscribe now takes the subscription out of the map in its first step. A client that unsubscribes before its watcher is scheduled leaves the watcher with a nil channel, and the watcher stays parked until the socket closes: one goroutine per subscribe and unsubscribe on a long-lived socket.
  • On main the entry stayed in the map for the node's round trip, which hid the race.
  • TestUnsubscribeThenDisconnect_LeavesNothingBehind, already on main, catches it with "the disconnect watcher must not stay parked once the client unsubscribed". It fails in 2 to 12 of 20 runs on this head and passes 20 of 20 on main.
  • The required unit-tests job reruns a failing test only twice, so this would make the check fail on main and on every PR stacked here.
  • I pushed a fix: the signal is read where the client is attached (by the creator while it still owns the subscription, by a joiner under the manager lock) and handed to the watcher. With it the test passes 30 of 30 here, and on #590 and #591.

589-2 (Low): a non-last client's unsubscribe can tear down, a second time, a subscription that ended under it.

  • A client that is not the last one releases the lock between Unsubscribe's two critical sections. Suppose the subscription ends in that gap by another path: the node ends it with an error, or a restore gives up.
  • cleanupSubscription then releases every client, decrements the counters and tells the node. Unsubscribe finds zero clients and runs teardownSubscriptionLocked again.
  • With a hook between the two sections: totalSubscriptions is -1, the pooled connection's count is -1, and the node gets two eth_unsubscribe calls. A count below the truth lets maybeScaleDown close a connection that still carries a subscription.
  • On main, RemoveMapping answered "not the last client" once the cleanup had removed every mapping.
  • I pushed three things: a guard (tear down only while the subscription is still the live entry), a test hook in the shape of beforeRestoreCommit, and TestWS_UnsubscribeDoesNotTearDownASubscriptionThatEndedUnderIt, which fails without the guard.

Verified:

  • Both new tests fail with main's manager and pass here, also under -race.

  • Once the entry is retired, no other teardown path can find it. Unsubscribe ends its upstream half exactly once. A restore in flight reaches the MAG-2547 discard and ends the id it re-opened on the node.

  • It merges cleanly with main and builds; vet and lint are clean.

  • protocol/rpcsmartrouter/direct_ws_subscription_manager.go:1880: The watcher reads detached here, when it first runs. Since this PR takes the entry out of the map at the start of Unsubscribe, a client that unsubscribes before the watcher is scheduled leaves it with a nil channel, parked until the socket closes (589-1).

  • protocol/rpcsmartrouter/direct_ws_subscription_manager.go:1131: If the subscription ended by another path while the lock was released (node error, failed restore), cleanupSubscription already released everyone and decremented the counters, and this tears it down again (589-2). Guarding on dwsm.activeSubscriptions[hashedParams] == activeSub restores what RemoveMapping's lastClient did on main.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@AnnaR-prog AnnaR-prog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 5e50281: main 12d3215 plus this PR, with both review fixes. The rpcsmartrouter tests pass on this exact tree.

@AnnaR-prog
AnnaR-prog merged commit 322a967 into main Nov 3, 2026
@AnnaR-prog
AnnaR-prog deleted the fix/MAG-4345-joiner-during-unsubscribe branch November 3, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants