Skip to content

Relax timeouts on MOVING rebind without buffered commands - #3894

Merged
a-TODO-rov merged 1 commit into
mainfrom
topic/ggivo/fix-moving-rebind-relaxed-timeouts
Aug 20, 2026
Merged

Relax timeouts on MOVING rebind without buffered commands#3894
a-TODO-rov merged 1 commit into
mainfrom
topic/ggivo/fix-moving-rebind-relaxed-timeouts

Conversation

@ggivo

@ggivo ggivo commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Relax timeouts on MOVING rebind without buffered commands

Summary

MaintenanceAwareConnectionWatchdog only notified onRebindStarted when a command was in flight at the moment the MOVING push arrived. On an idle connection the relaxed-timeout window was never opened, so commands issued during the ~rebind window timed out with the normal (unrelaxed) timeout. This surfaced as the recurring RelaxedTimeoutConfigurationTest.timeoutUnrelaxedOnMovingTest scenario failure. Rebind-started is now reported for every MOVING notification, regardless of the command stack.

Description

  • Commands issued after the MOVING notification are buffered until the new connection is up, so timeouts must be relaxed for the whole rebind window even when nothing was in flight when the push arrived.
  • In the empty-stack path the channel is closed immediately and channelReadComplete() never fires again, so onRebindCompleted is reported inline there; the non-empty path keeps reporting completion from channelReadComplete() once the stack drains.

Changes

  • onRebindStarted now fires on every MOVING rebind, not only when the command stack is non-empty.
  • The empty-stack path now also reports onRebindCompleted, so relaxed timeouts are lifted after the grace period instead of never being opened/closed.

Testing

MaintenanceAwareConnectionWatchdogUnitTests updated: the empty-stack test previously asserted onRebindStarted was not called — that expectation encoded the bug and now asserts both started and completed notifications; the non-empty-stack tests additionally verify completion is deferred until the stack drains. The scenario test RelaxedTimeoutConfigurationTest#timeoutUnrelaxedOnMovingTest reproduces the bug end-to-end.


Note

Medium Risk
Touches maintenance/MOVING rebind lifecycle and command timeout behavior during cluster endpoint moves; scope is narrow but affects timing-sensitive client paths.

Overview
MOVING rebind handling in MaintenanceAwareConnectionWatchdog now always opens the relaxed-timeout window, including when the connection is idle and no commands were in flight when the push arrived.

notifyRebindStarted runs for every rebind (not only when the command stack is non-empty), so commands buffered while the endpoint is disconnected during rebind get the same relaxed timeouts as in-flight work. On the empty-stack path the channel closes immediately and channelReadComplete() never runs again, so onRebindCompleted is invoked there to end the relaxed window after the grace period; the non-empty path still completes via channelReadComplete() after the stack drains.

Unit tests were updated to match: the idle-connection case now expects both started and completed notifications, and non-empty-stack cases assert completion is not reported until later.

Reviewed by Cursor Bugbot for commit 8132ccc. Bugbot is set up for automated code reviews on this repo. Configure here.

@ggivo

ggivo commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

run integration tests

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8132ccc3e9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@ggivo
ggivo requested a review from tishun August 18, 2026 16:08
@uglide

uglide commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

run scenario tests

onRebindStarted was only reported when a command was in flight when the
MOVING push arrived, so idle connections never opened the relaxed-timeout
window and commands issued during the rebind timed out unrelaxed
(recurring RelaxedTimeoutConfigurationTest#timeoutUnrelaxedOnMovingTest
failure). Report rebind-started on every MOVING notification, and report
completion inline in the empty-stack path where the channel is closed
immediately and channelReadComplete() can no longer fire.
@a-TODO-rov
a-TODO-rov force-pushed the topic/ggivo/fix-moving-rebind-relaxed-timeouts branch from 8132ccc to 1222630 Compare August 20, 2026 07:55
@a-TODO-rov
a-TODO-rov merged commit e3dd4ce into main Aug 20, 2026
10 checks passed
@a-TODO-rov
a-TODO-rov deleted the topic/ggivo/fix-moving-rebind-relaxed-timeouts branch August 20, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants