fix(cost): tier cache-write on the same threshold as input cost - #5779
Open
Taranum01 wants to merge 1 commit into
Open
fix(cost): tier cache-write on the same threshold as input cost#5779Taranum01 wants to merge 1 commit into
Taranum01 wants to merge 1 commit into
Conversation
Fixes Helicone#5766 `packages/cost/models/calculate-cost.ts` priced `cacheWrite5mCost` and `cacheWrite1hCost` directly off the base pricing tier (`preprocessedPricing[0]`), while `inputCost` and `outputCost` routed through `getPricingTier(... getThresholdValue(...))` to pick the correct tier. For Anthropic's long-context threshold (`> 200k` tokens), cache writes count toward the same total-prompt that drives the inputCost threshold. Pricing writes at the base tier undercharges them by ~2x on requests past 200k. Select the same tier for cache writes that inputCost picks, keeping the `cacheMultipliers` lookup scoped to the chosen tier so a provider that varies `write5m`/`write1h` per tier still computes the correct multiplier. Update `packages/__tests__/cost/modelCostFromRegistry.test.ts`: - `should use higher tier pricing for Claude Sonnet 4 over 200K tokens`: assert the cache-write cost at the long-context rate (`5000 * 0.000006 * 1.25`) instead of the previous base-tier value (`5000 * 0.000003 * 1.25`). Validated the fix numerically (250k-token input, 5k cache writes): the post-fix long-context cost is exactly 2x the base-tier value, which matches the issue's expectation.
|
Someone is attempting to deploy a commit to the Helicone Team on Vercel. A member of the Team first needs to authorize it. |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5766
packages/cost/models/calculate-cost.tspricedcacheWrite5mCostandcacheWrite1hCostdirectly off the base pricing tier (preprocessedPricing[0]), whileinputCostandoutputCostrouted throughgetPricingTier(... getThresholdValue(...))to pick the correct tier.For Anthropic's long-context threshold (
> 200ktokens), cache writes count toward the same total-prompt that drives the inputCost threshold. Pricing writes at the base tier undercharges them by ~2x on requests past 200k.Select the same tier for cache writes that inputCost picks, keeping the
cacheMultiplierslookup scoped to the chosen tier so a provider that varieswrite5m/write1hper tier still computes the correct multiplier.Tests:
packages/__tests__/cost/modelCostFromRegistry.test.ts:should use higher tier pricing for Claude Sonnet 4 over 200K tokens: assert the cache-write cost at the long-context rate (5000 * 0.000006 * 1.25) instead of the previous base-tier value (5000 * 0.000003 * 1.25).