Skip to content

fix(cost): tier Anthropic cached-input reads by the long-context threshold - #5765

Open
eeshsaxena wants to merge 2 commits into
Helicone:mainfrom
eeshsaxena:fix/anthropic-cached-input-longcontext-tier
Open

fix(cost): tier Anthropic cached-input reads by the long-context threshold#5765
eeshsaxena wants to merge 2 commits into
Helicone:mainfrom
eeshsaxena:fix/anthropic-cached-input-longcontext-tier

Conversation

@eeshsaxena

Copy link
Copy Markdown

Problem

getThresholdValueFunction returns, per provider, the token count used to pick which pricing tier applies to a given cost field. The anthropic branch handles inputCost and outputCost but has no cachedInputCost case, so that field falls through to default: return 0. Every other provider that tiers (vertex, xai, google-ai-studio) has a cachedInputCost case.

Because the threshold value is always 0 for Anthropic cached reads, getPricingTier always returns the base tier for them, even on a request past the long-context threshold.

Concretely, claude-4.5-sonnet:anthropic prices:

threshold 0:      input 0.000003, cacheMultipliers.cachedInput 0.1
threshold 200000: input 0.000006  (cacheMultipliers inherited)

A cached read is priced as cachedInput * tier.input * cacheMultipliers.cachedInput. Above 200k prompt tokens the input rate doubles, so the cached-read price should double too (0.1 * 0.000006). Instead it stays at the base tier (0.1 * 0.000003), so cached reads on long-context Anthropic requests are billed at half the intended rate. Input and output already tier correctly, so within one request the cached-read rate is inconsistent with the input rate it is derived from.

Fix

Add the cachedInputCost case to the anthropic threshold function, returning the same total-prompt value as inputCost/outputCost. Cached reads now select the same tier as input and output.

Tests

packages/__tests__/cost/anthropic-cache-tier.test.ts checks that a >200k request prices cached reads at the tier-1 rate (0.06 for 100k cached tokens, where the base tier would give 0.03), and that a sub-threshold request still uses the base tier.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@vercel

vercel Bot commented Aug 9, 2026

Copy link
Copy Markdown

@eeshsaxena is attempting to deploy a commit to the Helicone Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant