Skip to content

fix(cost): tier cache-write on the same threshold as input cost #5779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Taranum01 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(cost): tier cache-write on the same threshold as input cost #5779

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion packages/__tests__/cost/modelCostFromRegistry.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,11 @@ describe("modelCostBreakdownFromRegistry", () => {
expect(breakdown.inputCost).toBe(250000 * 0.000006);
expect(breakdown.outputCost).toBe(50000 * 0.0000225);
expect(breakdown.cachedInputCost).toBe(10000 * 0.000003 * 0.1);
expect(breakdown.cacheWrite5mCost).toBe(5000 * 0.000003 * 1.25);
// Regression for #5766: cache writes tier on the same threshold
// as the input cost (here, the long-context tier), not the base
// tier — the original 0.000003 base-tier rate undercharged
// >200k-token Anthropic requests at the long-context rate.
expect(breakdown.cacheWrite5mCost).toBe(5000 * 0.000006 * 1.25);
}
});

Expand Down
15 changes: 11 additions & 4 deletions packages/cost/models/calculate-cost.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,16 +215,23 @@ export function calculateModelCostBreakdown(params: {
modelUsage.cacheDetails.cachedInput * cachedInputPricing.input * cachedMultiplier;
}

// Cache writes tier on the same threshold as inputCost — the
// long-context threshold is "total prompt tokens above N", and
// cache writes count toward that. Using the base tier (preprocessed
// pricing tier 0) undercharges >200k-token requests at the
// long-context rate. Regression for #5766.
if (modelUsage.cacheDetails.write5m) {
const write5mMultiplier = basePricing.cacheMultipliers?.write5m ?? 1.0;
const write5mPricing = getPricingTier(preprocessedPricing, getThresholdValue(modelUsage, "inputCost"));
const write5mMultiplier = write5mPricing.cacheMultipliers?.write5m ?? 1.0;
breakdown.cacheWrite5mCost =
modelUsage.cacheDetails.write5m * basePricing.input * write5mMultiplier;
modelUsage.cacheDetails.write5m * write5mPricing.input * write5mMultiplier;
}

if (modelUsage.cacheDetails.write1h) {
const write1hMultiplier = basePricing.cacheMultipliers?.write1h ?? 1.0;
const write1hPricing = getPricingTier(preprocessedPricing, getThresholdValue(modelUsage, "inputCost"));
const write1hMultiplier = write1hPricing.cacheMultipliers?.write1h ?? 1.0;
breakdown.cacheWrite1hCost =
modelUsage.cacheDetails.write1h * basePricing.input * write1hMultiplier;
modelUsage.cacheDetails.write1h * write1hPricing.input * write1hMultiplier;
}
}

Expand Down