Skip to content

Do not raise deferred connection error as uncaught exception after GOAWAY - #1769

Open
immomo808 wants to merge 2 commits into
connectrpc:mainfrom
immomo808:fix-goaway-idle-uncaught-exception
Open

Do not raise deferred connection error as uncaught exception after GOAWAY#1769
immomo808 wants to merge 2 commits into
connectrpc:mainfrom
immomo808:fix-goaway-idle-uncaught-exception

Conversation

@immomo808

Copy link
Copy Markdown

Fixes #1768.

When a GOAWAY frame with NO_ERROR arrives on a connection without open streams, the session manager destroys the connection with a ConnectError. Node.js defers this error until the socket has closed, and raises it as an "error" event on the session.

If the manager exits the "ready" state before the socket has closed — for example because a new request sees the destroyed connection and reconnects — onExitState() removes the manager's error listeners, and the deferred error is raised as an uncaught exception, crashing the process:

ConnectError: [canceled] received GOAWAY without any open streams
    at ClientHttp2Session.onGoaway (.../http2-session-manager.js:473:26)
Emitted 'error' event on ClientHttp2Session instance at:
    at emitClose (node:internal/http2/core:1164:10)
    ...

We hit this in production behind a GCP internal Application Load Balancer, which sends GOAWAY on idle connections after ~10 minutes (AWS ALBs behave the same way, see #1545).

The fix attaches a listener to swallow the deferred error, following the same pattern already used in abort().

The regression test reproduces the crash deterministically: without the fix, it fails with failureType: 'uncaughtException' and the error above; with the fix, it passes. The full connect-node test suite passes.

🤖 Generated with Claude Code

…AWAY

When a GOAWAY frame with NO_ERROR arrives on a connection without open
streams, the session manager destroys the connection with a ConnectError.
Node.js defers this error until the socket has closed, and raises it as
an "error" event on the session.

If the manager exits the "ready" state before the socket has closed -
for example because a new request sees the destroyed connection and
reconnects - onExitState() removes the manager's error listeners, and
the deferred error is raised as an uncaught exception, crashing the
process.

This can be reproduced with a load balancer that recycles idle
connections (observed with GCP and AWS application load balancers,
which send GOAWAY after their idle timeout), when a request arrives
while the destroyed connection's socket is still closing.

Attach a listener to swallow the deferred error, following the same
pattern already used in abort().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: immomo808 <git@immomo808.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Uncaught exception after GOAWAY on idle connection crashes the process

1 participant