websocket: Make unsubscribe helpers accept the request ID returned by subscribe - #701
Conversation
… subscribe The RPC server assigns subscription IDs independently of the request IDs that the subscribe helpers return (devnet confirmed request ID 1 as subscription ID 9710270), so passing a subscribe() return value to the matching unsubscribe method sent an invalid subscription ID: the server replied "Invalid subscription id" and kept streaming notifications, or local bookkeeping raised KeyError. Record the request-ID-to-subscription-ID mapping when the server confirms a subscription, and resolve it in every unsubscribe method. Unconfirmed or already-server-assigned IDs pass through unchanged, preserving the behavior for callers that track server IDs themselves.
|
LGTM |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #701 +/- ##
==========================================
+ Coverage 94.82% 94.99% +0.17%
==========================================
Files 25 25
Lines 1739 1739
==========================================
+ Hits 1649 1652 +3
+ Misses 90 87 -3 🚀 New features to boost your workflow:
|
michaelhly
left a comment
There was a problem hiding this comment.
Thanks for the detailed report — the bug is real and the direction of the fix (record the request ID → server subscription ID mapping on the SubscriptionResult confirmation) is right.
My concern is how the two ID spaces are reconciled: both are plain int and the request-ID map is consulted first, so a caller passing a server ID that numerically collides with a live request ID gets silently mistranslated and cancels the wrong subscription. That's the path README and every integration fixture use, so the "behaves exactly as before" claim doesn't hold in general. Details inline.
I'd like the first comment resolved before merge; the next two are cheap to fix alongside it.
| Returns: | ||
| The server-assigned subscription ID, if known, else the input unchanged. | ||
| """ | ||
| server_subscription = self.request_ids_to_subscriptions.pop(subscription, subscription) |
There was a problem hiding this comment.
Request IDs and server subscription IDs collide here, cancelling the wrong subscription.
Both spaces are plain int and the request-ID map is consulted first, so the "pass through unchanged if unconfirmed" rule can't distinguish an unconfirmed request ID from a server-assigned ID that happens to equal a live request ID. Reproduced on a fresh protocol object (small server IDs, as a local validator assigns):
slot_subscribe() -> req 1, server sub 1; slot_unsubscribe(1) consumes req id 2
root_subscribe() -> req 3, server sub 2 # map {3: 2}
vote_subscribe() -> req 4, server sub 3 # map {3: 2, 4: 3}
vote_unsubscribe(3) # 3 = server id of the vote sub, the README form
wire: {"method":"voteUnsubscribe","id":5,"params":[2]} # cancels the ROOT subscription
remaining: {3: VoteSubscribe...} # vote sub leaks
The PR description says callers that track server-assigned IDs "behave exactly as before"; that only holds while no live request ID equals the server ID being passed. README.md:94/:105 and every integration fixture use the server-ID form, so this is a real backward-compat break rather than a theoretical one.
Suggestion: disambiguate instead of guessing. Checking self.subscriptions first keeps the older contract authoritative:
if subscription in self.subscriptions: # already a server-assigned ID
self.request_ids_to_subscriptions = {k: v for k, v in self.request_ids_to_subscriptions.items() if v != subscription}
self.subscriptions.pop(subscription, None)
return subscriptionOr, better long term, expose the mapping through a distinct return type/API so the two ID spaces are never interchangeable ints.
| The server-assigned subscription ID, if known, else the input unchanged. | ||
| """ | ||
| server_subscription = self.request_ids_to_subscriptions.pop(subscription, subscription) | ||
| self.subscriptions.pop(server_subscription, None) |
There was a problem hiding this comment.
Minor: swapping del for pop(..., None) removes the only local signal that the unsubscribe target isn't a live subscription. Unsubscribing before confirmation (or twice) now silently sends an invalid subscription ID; the server's -32602 comes back as a SoldersSubscriptionError and is raised as SubscriptionError out of an unrelated later recv(), while the real subscription keeps streaming. Dropping the KeyError is the right call, but a returned bool or a logging.warning would let the caller learn the unsubscribe was a no-op at the call site.
| """ | ||
| req_id = self.increment_counter_and_get_id() | ||
| req = AccountUnsubscribe(subscription, req_id) | ||
| req = AccountUnsubscribe(self._pop_server_subscription(subscription), req_id) |
There was a problem hiding this comment.
Local bookkeeping is now torn down before the request is sent.
_pop_server_subscription(...) is evaluated while building the request object, so the mapping and the self.subscriptions entry are gone before await self.send_request(req) runs. Previously the del ran only after a successful send.
If send_request raises (e.g. ConnectionClosed), a reconnect-and-resubscribe routine reading self.subscriptions sees no record of the subscription, and a retried slot_unsubscribe(req_id) now falls through to the pass-through path and sends the raw request ID as the subscription ID. Please resolve the ID, send, then clean up on success.
Same pattern at lines 192, 238, 294, 325, 345, 365, 385 and 405.
|
|
||
| Args: | ||
| subscription: ID of subscription to cancel. | ||
| subscription: The request ID returned by the corresponding ``subscribe`` method. |
There was a problem hiding this comment.
These docstrings (all nine) now document the request-ID form, but README.md:94 and the docs page mirroring it still pass first_resp[0].result — the server-assigned ID — and so do all the integration fixtures. Since the two forms are silently interchangeable and can collide, the public contract ends up documented one way and exercised another. Worth updating README/docs to whichever form is authoritative.
|
Thanks — the collision is real, and I overstated the pass-through compatibility. I'll treat
Pushing a follow-up shortly. |
Treat self.subscriptions as authoritative so a server-assigned ID that collides with a live request ID is not rewritten. Drop the request-ID map after a successful send (including raw send_request), on signature expiry, and only after send succeeds.
|
Pushed in b8a3e79:
The original unit tests plus the collision / leak / failed-send cases are green ( |
|
I am attempting to propose a plan here. The request_id is only needed to increment at local. Once the response is received, it can be discarded. Active subscriptions can be stored as a new strong type in a simple The general idea is something like this: @dataclass
class Subscription:
subscription_id: int
unsubscribe_method: str
subscriptions: list[Subscription] = []
# Subscribe
request_id = send_subscribe_request(...)
response = await wait_for_response(request_id)
sub = Subscription(
subscription_id=response.result,
unsubscribe_method="logsUnsubscribe",
)
subscriptions.append(sub)
# Unsubscribe
subscriptions.remove(sub)
send_request(sub.unsubscribe_method, [sub.subscription_id])Key points
|
|
@kingsznhone agreed that a distinct I'd keep that out of this PR though. Today This PR is the compatibility shim: keep accepting both ints, treat |
|
Thanks! |
|
CI is green across the matrix on a3dabda, including the full integration suite against a live validator on ubuntu — that exercises the server-ID contract ( Recording the residual caveats so they're findable later. None of these block this PR:
Doc nit for whoever picks up the follow-up: the README now says both forms are accepted, but given (1) the server-assigned ID is the canonical one and the request ID is the convenience. Worth stating that explicitly. Thanks @kartsan03 for the repro and the quick turnarounds. |
|
Thanks for the careful review on the first pass — the collision catch was the right one, and the notes made the follow-up straightforward. Glad this landed. |
Problem
Every
*_unsubscribe()helper is unusable with the value returned by its matching*_subscribe()on any real RPC endpoint.Since #647, subscribe helpers return the client request ID used for the subscription request. But the server assigns subscription IDs independently of request IDs, and the unsubscribe helpers forward the given value verbatim as the subscription ID. On devnet today:
So
await ws.slot_unsubscribe(await ws.slot_subscribe())sendsslotUnsubscribe(1), which the node rejects — verified againstwss://api.devnet.solana.comwith raw websockets frames:Through the client itself, the wrong ID also crashes local bookkeeping (
del self.subscriptions[subscription]), sincesubscriptionsis keyed by server-assigned IDs:A fresh local validator happens to assign small subscription IDs that coincide with request IDs, which is why this only surfaces on public endpoints.
Fix
Record the mapping from request ID to server-assigned subscription ID when a
SubscriptionResultconfirmation is processed, and resolve it in all nine unsubscribe helpers via_pop_server_subscription(). Resolution passes the value through unchanged when no confirmation has been seen, so callers that track server-assigned IDs themselves (as the integration tests do) behave exactly as before. Bookkeeping cleanup now usespop(..., None)instead ofdel, so unsubscribing an unconfirmed or already-cancelled subscription no longer raisesKeyError.Verification
Against
https://api.devnet.solana.com/wss://api.devnet.solana.com(Python 3.13):Before this change:
After this change:
Local checks:
ruff format --check src tests— cleanruff check src tests— cleanmypy src— Success: no issues found in 31 source filespytest -m "not integration" --doctest-modules src tests/unit— 155 passed (includes two new unit tests covering the translation and the unconfirmed pass-through)Notes
SubscriptionResult.resultand pass them to the unsubscribe helpers continue to work unchanged (pass-through path).KeyError.