Skip to content

feat(api): fetch historical prices only for denoms present in the block - #41

Merged
nimrod-teich merged 1 commit into
mainfrom
feat/rewards-prices-relevant-denoms
Apr 20, 2026
Merged

nimrod-teich merged 1 commit into
mainfrom
feat/rewards-prices-relevant-denoms

Conversation

@nimrod-teich

Copy link
Copy Markdown
Contributor

Summary

Fully accurate historical rewards pricing without blowing the gateway timeout. The previous approach fetched all 22 known CoinGecko denoms for every historical-block query; with sequential-with-retry-backoff that ran into 504s on cold requests.

Key insight: only a handful of denoms actually appear in any given block's rewards — typically just LAVA + 0-2 IBC denoms. Introspect the rewards response first, then fetch prices only for what's there.

New flow

  1. Fetch raw estimated_provider_rewards for every provider (with x-cosmos-block-height)
  2. Scan all raw responses, extract unique base denoms (post-IBC resolution)
  3. Build historical price map for ONLY those denoms
  4. Format each provider's rewards with the resulting price overrides

Implementation notes

  • rpc/rewards.ts: split fetchRewardsBySpec into composable pieces — fetchRawProviderRewards, extractBaseDenoms, processRawProviderRewards. fetchRewardsBySpec kept as a convenience wrapper (APR route still uses it).
  • rpc/pricing.ts:
    • coingeckoFetch helper: honors Retry-After on 429, exponential backoff, throws after 5 attempts
    • fetchTokenUsdPriceAt: throws on transient failures (no more silent fallback to current price). Returns 0 only for legitimate "no data" cases (unknown denom, 404, empty market_data), and those cache indefinitely
    • buildHistoricalPriceMap: sequential, LAVA first, LAVA failure propagates so the route can 503 rather than cache wrong data
    • fetchDenomTrace gains a persistent in-memory cache (IBC hash → base denom is immutable per channel)
  • routes/provider-estimated-rewards.ts: new two-pass flow as above. When LAVA historical price fails, returns 503 — the cache plugin skips 4xx/5xx so a wrong-price response can't poison the 1-year cache.
  • config.ts: new CACHE_TTL.IMMUTABLE = 31_536_000 (365 days) for past-block endpoints. Past-window /provider-rewards, block-scoped /provider-estimated-rewards?block=, and /provider-estimated-rewards/blocks all graduate from HISTORICAL (1 day) to IMMUTABLE. Avatars + /burn-rate stay on HISTORICAL since they can change.

Test plan

  • pnpm --filter @info/api typecheck clean
  • pnpm --filter @info/api test — 114 passing. New assertions:
    • buildHistoricalPriceMap called with only the denoms actually in the block's rewards (not all 22)
    • 503 when LAVA historical price unavailable AND processRawProviderRewards is NOT called (refuses to ship wrong numbers)
    • live-mode (?block= omitted) doesn't invoke any historical code path
  • Deploy + run rewards breakdown comparison against block 4697952. Expected:
    • All 48 providers present (historical provider coverage, from the PR that shipped earlier)
    • Every per-provider USD = jsinfo byte-exact 1.00× ratio (historical pricing now actually applied)
    • First cold request returns in seconds (not 504 timeout) because only ~1-3 denoms are fetched
    • Cached response served instantly for a year

Known behavior

If a block's rewards contain a denom we've never seen before AND CoinGecko rate-limits the first fetch for that denom for longer than 5 retries (~2 min), the request 503s. The next caller retries from scratch — no bad response gets cached. LAVA never hits this because its price is in the live-cache pre-warm.

Previously /provider-estimated-rewards?block=N called buildHistoricalPriceMap
with all 22 known denoms. Under CoinGecko throttling (sequential + retry
backoff), this blew past the 120s gateway timeout and 504'd — so cold
historical queries effectively never succeeded.

Now the route introspects each block's actual rewards first, extracts the
set of base denoms that appear (after IBC resolution), and fetches
historical prices only for those — typically just LAVA + 0-2 IBC denoms.
First cold response returns in seconds; subsequent requests come from the
1-year IMMUTABLE cache.

Implementation:

- rpc/rewards.ts: split fetchRewardsBySpec into three composable pieces:
  fetchRawProviderRewards (one chain RPC per provider), extractBaseDenoms
  (post-IBC-resolution denom set across N raw responses), and
  processRawProviderRewards (raw → spec-grouped with source breakdown).
  fetchRewardsBySpec is kept as a convenience wrapper for APR callers.

- rpc/pricing.ts:
  • coingeckoFetch helper: honors Retry-After on 429, exponential backoff
    up to 5 attempts capped at 60s per try. Used by both current-price
    (fetchLavaUsdPrice) and historical-price paths.
  • fetchTokenUsdPriceAt: throws on transient failures (rate-limit
    exhaustion, network error) instead of silently returning 0. Zero is
    reserved for legitimate "no data" (unknown denom, 404, empty
    market_data) and cached indefinitely. Throws don't cache — next
    caller retries from scratch.
  • buildHistoricalPriceMap: sequential (not parallel), LAVA first, LAVA
    failure propagates. Other denoms best-effort; on failure we log and
    continue, falling back to the live cache for that denom.
  • fetchDenomTrace gains an in-memory cache since IBC-hash → base-denom
    is immutable per-channel.

- routes/provider-estimated-rewards.ts: two-pass flow
  1. Fetch raw rewards for every provider (with blockHeight).
  2. Extract base denoms across all raws, build historical price map for
     only those, then process each raw with overrides.
  If LAVA historical price fetch throws, return 503 (cache plugin skips
  4xx/5xx so a poisoned wrong-price response can never enter the 1-year
  IMMUTABLE cache).

- config.ts: new CACHE_TTL.IMMUTABLE (365 days). Applied to past-block
  endpoints: /provider-estimated-rewards?block=N,
  /provider-estimated-rewards/blocks, and /provider-rewards with past
  date ranges. Avatars + burn-rate stay on HISTORICAL (1 day) since they
  can legitimately change.

114/114 tests passing. New assertions cover:
  • buildHistoricalPriceMap called with only the denoms actually in the block
  • 503 when LAVA historical price unavailable + processRawProviderRewards
    is NOT called (so no wrong cached response)
  • live-mode doesn't invoke any historical code path
@nimrod-teich
nimrod-teich force-pushed the feat/rewards-prices-relevant-denoms branch from fcbeb00 to 6681672 Compare April 20, 2026 11:43
@nimrod-teich
nimrod-teich merged commit 54e4b95 into main Apr 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant