fix(gemini): correct token usage accounting - #3034
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
oss-maintainer
left a comment
There was a problem hiding this comment.
Summary
Corrects Gemini usage accounting by billing toolUsePromptTokenCount as input and thoughtsTokenCount as output, with sensible fallbacks when candidate counts are missing. Well covered by four new parser-level tests. Main ask is on documentation of the behaviour flip: previously output deliberately excluded thinking tokens, so consumers that added them back will now double-count.
Automated review by github-manager-bot
| int inputTokens = | ||
| metadata.promptTokenCount().orElse(0) | ||
| + metadata.toolUsePromptTokenCount().orElse(0); | ||
| int cachedTokens = metadata.cachedContentTokenCount().orElse(0); |
There was a problem hiding this comment.
Classifying toolUsePromptTokenCount as input looks right and matches how the other providers bill server-side tool turns. Please confirm the same assumption holds for cachedContentTokenCount: in Gemini the cached count is a subset of promptTokenCount, so inputTokens stays correct, but a comment here would prevent a future "fix" from subtracting it twice.
| metadata.candidatesTokenCount() | ||
| .map(candidateTokens -> candidateTokens + thinkingTokens) | ||
| .orElseGet( | ||
| () -> |
There was a problem hiding this comment.
This flips the sign of the previous intentional behaviour (thinking tokens used to be excluded from output). Two consequences worth a explicit note: (1) stored historical usage for Gemini sessions is not comparable across this boundary, and (2) any user-side cost/budget math that added thinkingTokens back on top of outputTokens now double-counts. Could you call this out in the changelog/release note, and check the OpenAI/DashScope parsers so all providers use the same convention?
AgentScope-Java Version
2.0.3-SNAPSHOT
Description
Fixes #3033.
GeminiResponseParser previously excluded tool-use prompt tokens from input usage and subtracted thinking tokens from output usage. This caused inaccurate usage, budget, and cost accounting for responses that use server-side tools or thinking.
This change:
No public API, dependency, or user-facing documentation changes are required. Existing cached-token handling is preserved.
Validation:
Checklist