fix(proxy): correct account failover so a healthy account is always used - #55
Open
locchh wants to merge 2 commits into
Open
fix(proxy): correct account failover so a healthy account is always used#55locchh wants to merge 2 commits into
locchh wants to merge 2 commits into
Conversation
Account selection was blind to several "this account can't serve right now" signals, so the session strategy would stick to an unusable account instead of failing over to a healthy one. The symptom looked model-specific (e.g. "Sonnet works but Opus doesn't") but was purely a timing artifact of which account was unavailable at the moment. Fixes: - Overage-aware rate limiting: when Anthropic returns `anthropic-ratelimit-unified-overage-status: allowed`, the account is still serving requests past its primary limit. Previously ccflare benched it until the multi-hour primary reset, discarding a working account. Now it applies a short cooldown instead of a multi-hour lock. - Failover skips token-refresh-backed-off accounts: refresh failures were tracked only in an in-memory map the load balancer could not see, so an account with a dead/failing token kept being selected. Extracted the registry into refresh-backoff.ts and made SessionStrategy treat a backed-off account as unusable. Self-healing: the account returns once the backoff clears. - Rate limit without a reset time still marks the account: a quota hit whose response carried no parseable reset time was logged but never marked, so selection kept choosing it. Now applies a default cooldown. All three are self-healing; no account is permanently disabled. Verified live: with a dead-token account and a healthy account, requests fail over to the healthy account and both Sonnet and Opus return 200. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Clean up now / Compact database failed silently since the mutation hooks never surfaced .error, so a timed-out or failed request just stopped spinning with no feedback. Also extend the client timeout for both calls since a VACUUM on a large database can legitimately exceed the default 30s. Also fix getErrorType's network-error check, which looked for the substring "fetch failed" but the browser's actual TypeError message is "Failed to fetch" (reversed), so network failures never got the friendly message anywhere in the dashboard. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Account selection was blind to several "this account can't serve right now" signals, so the session strategy would stick to an unusable account instead of failing over to a healthy one. The symptom looked model-specific (e.g. "Sonnet works but Opus doesn't") but was purely a timing artifact of which account was unavailable at the moment.
Fixes:
Overage-aware rate limiting: when Anthropic returns
anthropic-ratelimit-unified-overage-status: allowed, the account is still serving requests past its primary limit. Previously ccflare benched it until the multi-hour primary reset, discarding a working account. Now it applies a short cooldown instead of a multi-hour lock.Failover skips token-refresh-backed-off accounts: refresh failures were tracked only in an in-memory map the load balancer could not see, so an account with a dead/failing token kept being selected. Extracted the registry into refresh-backoff.ts and made SessionStrategy treat a backed-off account as unusable. Self-healing: the account returns once the backoff clears.
Rate limit without a reset time still marks the account: a quota hit whose response carried no parseable reset time was logged but never marked, so selection kept choosing it. Now applies a default cooldown.
All three are self-healing; no account is permanently disabled.
Verified live: with a dead-token account and a healthy account, requests fail over to the healthy account and both Sonnet and Opus return 200.