Skip to content

feat(api): CoinGecko retry-on-throttle + fail-loud + 1-year cache for past-block rewards - #40

Merged
nimrod-teich merged 1 commit into
mainfrom
feat/rewards-retry-throttle
Apr 20, 2026
Merged

nimrod-teich merged 1 commit into
mainfrom
feat/rewards-retry-throttle

Conversation

@nimrod-teich

Copy link
Copy Markdown
Contributor

Summary

Three layered fixes so a historical rewards query never silently bakes a wrong USD price into the response cache:

  1. Retry on CoinGecko 429 with exponential backoff + Retry-After
  2. Fail loud when retries exhausted — return 503, don't fall back to current price
  3. Cache for 1 year (CACHE_TTL.IMMUTABLE = 31_536_000) since past-block data is fully deterministic

The bug we're fixing

Today (post-PR #39):

  • buildHistoricalPriceMap fires 22 parallel CoinGecko calls
  • Free tier throttles ~10-30/min → most fail
  • Failed calls return 0 → filtered out of override map
  • processRewardTokens falls back to current LAVA price
  • Wrong USD gets wrapped in a 200 response
  • Response plugin caches it for however long the TTL is
  • Next year of traffic gets the wrong answer

The fix

(1) Retry. New coingeckoFetch wraps each CoinGecko call:

  • Up to 5 attempts
  • Honors Retry-After header when CoinGecko sends it
  • Else exponential backoff (2, 4, 8, 16, 32 seconds, capped at 60)

(2) Fail loud. fetchTokenUsdPriceAt now distinguishes "no data" from "can't reach":

  • Unknown denom / 404 / empty market_data → return 0, cache indefinitely (permanent fact)
  • Transient failure (rate limit after retries, network error) → throw, don't cache

buildHistoricalPriceMap:

  • LAVA goes first (virtually all reward value is LAVA)
  • LAVA failure propagates (required)
  • Other denoms best-effort — logged and skipped on failure

Route handler catches the LAVA exception and returns 503 with a clear "please retry" message. Cache plugin already skips 4xx/5xx (cache.ts:60), so no bad response lands in Redis.

(3) 1-year cache. New CACHE_TTL.IMMUTABLE applied to:

  • /provider-estimated-rewards?block=N — response fully determined by past state + block-time prices
  • /provider-estimated-rewards/blocks — monthly 17th heights are permanent
  • /provider-rewards?from=&to= when to is older than today - 1 day

Kept on 1-day HISTORICAL:

  • /providers/:addr/avatar (Keybase updates)
  • /burn-rate (mixes immutable blocks with a latest field)

Test plan

  • pnpm --filter @info/api typecheck clean
  • pnpm --filter @info/api test — 113 passing (new: 503 when historical price unavailable AND fetchRewardsBySpec is NOT called — proves we refuse to ship wrong numbers)
  • Deploy + re-run the rewards breakdown script against block 4697952. Expected: all 48 providers, byte-exact 1.00× USD ratio against jsinfo, first request slow (throttled), subsequent requests instant from cache.

Known limitation

Retries can make a first-time cold request take several minutes under heavy throttling. Once the first response succeeds, it's cached for a year — so this only matters for the initial cold fetch. If the free-tier throttling becomes a real bottleneck, we should upgrade to CoinGecko's demo/pro API key (via a x-cg-demo-api-key header on the existing fetch path).

Hardens historical rewards pricing so wrong numbers never land in the
block-keyed cache for a year.

Three changes layered together:

1. Retry on CoinGecko 429 with backoff
   ----------------------------------
   Free tier throttles ~10-30/min. New coingeckoFetch helper wraps each
   CoinGecko request with:
     • up to 5 attempts
     • honors Retry-After header when present
     • exponential backoff (2^attempt seconds, capped at 60s) otherwise
   Applies to both current-price (fetchLavaUsdPrice) and historical-price
   (fetchTokenUsdPriceAt) paths.

2. Fail loud on rate-limit exhaustion
   ----------------------------------
   Before: fetchTokenUsdPriceAt caught all errors and returned 0 →
   processRewardTokens fell back to current price cache → USD figure in
   response was WRONG but cached as if historical → wrong for a year.

   Now: fetchTokenUsdPriceAt throws on transient failures (rate limit,
   network error) and only returns 0 for legitimate "no data" (unknown
   denom, 404, empty market_data). Zeros cache indefinitely (permanent
   facts), throws don't cache (next caller retries).

   buildHistoricalPriceMap makes LAVA required (virtually all reward value
   is LAVA), other denoms best-effort. If LAVA throws, the route returns
   503 with a clear "please retry" message. Cache plugin skips 4xx/5xx so
   no bad response gets written to Redis.

3. 1-year cache for immutable past-block snapshots
   -----------------------------------------------
   New CACHE_TTL.IMMUTABLE = 31_536_000. Applied to:
     • /provider-estimated-rewards?block=N (response fully determined by
       past chain state + block-time prices, both unchangeable)
     • /provider-estimated-rewards/blocks (monthly 17th heights permanent)
     • /provider-rewards?from=&to= when `to` is older than today - 1 day
   Kept on HISTORICAL (1 day):
     • /providers/:addr/avatar (Keybase avatars can change)
     • /burn-rate (mixes immutable blocks with a "latest" field)

Tests updated: 113/113. Added a case asserting that when
buildHistoricalPriceMap throws, the route returns 503 AND does not call
fetchRewardsBySpec — i.e. we refuse to ship a response rather than
silently shipping one with the wrong price.
@nimrod-teich
nimrod-teich merged commit 7c37319 into main Apr 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant