Skip to content

fix(proxy): correct account failover so a healthy account is always used - #55

Open
locchh wants to merge 2 commits into
snipeship:mainfrom
locchh:fix/account-failover
Open

fix(proxy): correct account failover so a healthy account is always used#55
locchh wants to merge 2 commits into
snipeship:mainfrom
locchh:fix/account-failover

Conversation

@locchh

@locchh locchh commented Jun 24, 2026

Copy link
Copy Markdown

Account selection was blind to several "this account can't serve right now" signals, so the session strategy would stick to an unusable account instead of failing over to a healthy one. The symptom looked model-specific (e.g. "Sonnet works but Opus doesn't") but was purely a timing artifact of which account was unavailable at the moment.

Fixes:

  • Overage-aware rate limiting: when Anthropic returns anthropic-ratelimit-unified-overage-status: allowed, the account is still serving requests past its primary limit. Previously ccflare benched it until the multi-hour primary reset, discarding a working account. Now it applies a short cooldown instead of a multi-hour lock.

  • Failover skips token-refresh-backed-off accounts: refresh failures were tracked only in an in-memory map the load balancer could not see, so an account with a dead/failing token kept being selected. Extracted the registry into refresh-backoff.ts and made SessionStrategy treat a backed-off account as unusable. Self-healing: the account returns once the backoff clears.

  • Rate limit without a reset time still marks the account: a quota hit whose response carried no parseable reset time was logged but never marked, so selection kept choosing it. Now applies a default cooldown.

All three are self-healing; no account is permanently disabled.

Verified live: with a dead-token account and a healthy account, requests fail over to the healthy account and both Sonnet and Opus return 200.

locchh and others added 2 commits June 24, 2026 22:42
Account selection was blind to several "this account can't serve right
now" signals, so the session strategy would stick to an unusable account
instead of failing over to a healthy one. The symptom looked
model-specific (e.g. "Sonnet works but Opus doesn't") but was purely a
timing artifact of which account was unavailable at the moment.

Fixes:

- Overage-aware rate limiting: when Anthropic returns
  `anthropic-ratelimit-unified-overage-status: allowed`, the account is
  still serving requests past its primary limit. Previously ccflare
  benched it until the multi-hour primary reset, discarding a working
  account. Now it applies a short cooldown instead of a multi-hour lock.

- Failover skips token-refresh-backed-off accounts: refresh failures
  were tracked only in an in-memory map the load balancer could not see,
  so an account with a dead/failing token kept being selected. Extracted
  the registry into refresh-backoff.ts and made SessionStrategy treat a
  backed-off account as unusable. Self-healing: the account returns once
  the backoff clears.

- Rate limit without a reset time still marks the account: a quota hit
  whose response carried no parseable reset time was logged but never
  marked, so selection kept choosing it. Now applies a default cooldown.

All three are self-healing; no account is permanently disabled.

Verified live: with a dead-token account and a healthy account, requests
fail over to the healthy account and both Sonnet and Opus return 200.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Clean up now / Compact database failed silently since the mutation
hooks never surfaced .error, so a timed-out or failed request just
stopped spinning with no feedback. Also extend the client timeout for
both calls since a VACUUM on a large database can legitimately exceed
the default 30s.

Also fix getErrorType's network-error check, which looked for the
substring "fetch failed" but the browser's actual TypeError message is
"Failed to fetch" (reversed), so network failures never got the
friendly message anywhere in the dashboard.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant