fix(cost): reach higher pricing tiers on non-tiered providers

[arthi-arumugam-git]

Fixes #5690

The bug

getThresholdValueFunction in packages/cost/models/calculate-cost.ts only had cases for vertex, google-ai-studio, anthropic and xai. Every other provider fell through to default: return () => 0. Since getPricingTier picks the highest tier whose threshold is <= the value it is given, a constant 0 always selects the cheapest tier, so any higher tier on those providers was unreachable and large-context requests were under-charged.

The example from the issue, gpt-5.4 on openai with input: 300000, output: 50000:

	input	output	total
charged (base tier)	300000 x $2.50/M = $0.75	50000 x $15/M = $0.75	$1.50
correct (>272K tier)	300000 x $5/M = $1.50	50000 x $22.50/M = $1.125	$2.625

43% under.

I scanned every endpoint config in packages/cost/models/authors/ for pricing.length > 1 to size it. 40 endpoints declare more than one tier; 24 of them sit on a provider the switch did not handle:

handled     anthropic 3, google-ai-studio 3, vertex 7, xai 3
unreachable azure 2, bedrock 3, helicone 10, openai 2, openrouter 7

bedrock is in that list too, which the issue did not mention: the three claude-sonnet-4* Bedrock endpoints each declare a 200000 tier that was never selected.

The fix

The default branch now returns the request's prompt length instead of 0.

For openai, azure, openrouter and helicone serving OpenAI-authored models that is usage.input + (usage.cacheDetails?.cachedInput ?? 0), exactly what the issue proposed and what the xai handler already does.

bedrock, openrouter and helicone also resell Anthropic models, and Anthropic bills cache writes as part of the prompt, which is why the existing anthropic case adds write5m and write1h. Keying the threshold basis purely off the serving provider would silently under-count the prompt for Claude on Bedrock, so the default branch checks the model's author and applies the Anthropic rule wherever an Anthropic-authored model is served from. The promptLength / anthropicPromptLength helpers exist so the two rules are written once.

Models with a single pricing tier are unaffected: their only tier has threshold: 0, so any value selects it. The four existing provider cases are untouched, vertex in particular, which tiers cachedInputCost off the cached-token count alone rather than the prompt.

Tests

Nine cases added to the threshold-based pricing block in packages/__tests__/cost/modelCostFromRegistry.test.ts: gpt-5.4 above and below the 272K threshold on openai, cached input counting toward that threshold, the higher tier on azure / helicone / openrouter (the OpenRouter rates carry that provider's 1.055 markup), Claude Sonnet 4 on bedrock above the threshold only once cache writes are counted and below it otherwise, and a guard that a single-tier model (gpt-4o) still prices the same.

Reverting only packages/cost/models/calculate-cost.ts and keeping the tests:

$ npx jest __tests__/cost/modelCostFromRegistry.test.ts

  ● threshold-based pricing › should use higher tier pricing for OpenAI GPT-5.4 over 272K tokens
    expect(received).toBe(expected) // Object.is equality
    Expected: 1.5
    Received: 0.75

  ● threshold-based pricing › should count cached input toward the GPT-5.4 threshold
    expect(received).toBe(expected) // Object.is equality
    Expected: 1
    Received: 0.5

  ● threshold-based pricing › should use higher tier pricing for GPT-5.4 on azure over 272K tokens
    expect(received).toBeCloseTo(expected, precision)
    Expected: 1.5000000000000002
    Received: 0.7500000000000001

  ● threshold-based pricing › should use higher tier pricing for GPT-5.4 on helicone over 272K tokens
    expect(received).toBeCloseTo(expected, precision)
    Expected: 1.5000000000000002
    Received: 0.7500000000000001

  ● threshold-based pricing › should use higher tier pricing for GPT-5.4 on openrouter over 272K tokens
    expect(received).toBeCloseTo(expected, precision)
    Expected: 1.5825
    Received: 0.79125

  ● threshold-based pricing › should apply the Anthropic threshold rule to Claude Sonnet 4 on Bedrock
    expect(received).toBe(expected) // Object.is equality
    Expected: 0.9
    Received: 0.45

Tests:       6 failed, 21 passed, 27 total

With the fix in place, the whole packages suite:

$ npx jest __tests__/
Test Suites: 1 skipped, 41 passed, 41 of 42 total
Tests:       3 skipped, 587 passed, 590 total
Snapshots:   8 passed, 8 total

npx tsc --noEmit reports nothing on either changed file.

One note on how I ran these: packages/package.json and packages/yarn.lock are not in the repo, so the yarn install --frozen-lockfile step in .github/workflows/packages-test.yml has nothing to install from. I ran the suite against a local jest/ts-jest setup with @helicone-package/* mapped to packages/*, which is what the imports resolve to anyway. Nothing from that setup is in this branch.

[vercel]

@arthi-arumugam-git is attempting to deploy a commit to the Helicone Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Your trial has ended. Reactivate Greptile to resume code reviews.

[arthi-arumugam-git]

A note while this waits, since #5690 is still open and this is a fix for it.

The three red checks here are all Vercel reporting "Authorization required to deploy", which is the fork deploy permission rather than a failing build, so there is no real CI signal on this branch yet.

The change: getThresholdValueFunction in packages/cost/models/calculate-cost.ts only handled vertex, google-ai-studio, anthropic and xai. Everything else fell through to default: return () => 0, and since getPricingTier picks the highest tier whose threshold is at or below the value it receives, a constant zero always selects the cheapest tier. Higher tiers on every other provider were unreachable, so large-context requests were under-charged rather than over-charged, which is the direction nobody complains about and nobody notices.

Glad to rebase, split it, or take a different approach to the provider mapping if you would rather it lived somewhere else.