Skip to content

fix(gemini): correct token usage accounting #3034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
liugy789 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

fix(gemini): correct token usage accounting #3034

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cc8be0f
fix(gemini): correct token usage accounting
liugy789 Sep 8, 2026
04f3a72
fix(gemini): preserve thinking token fallback
liugy789 Sep 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 18 additions & 6 deletions ...i/src/main/java/io/agentscope/extensions/model/gemini/formatter/GeminiResponseParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,14 +94,26 @@ public ChatResponse parseResponse(GenerateContentResponse response, Instant star
if (response.usageMetadata().isPresent()) {
GenerateContentResponseUsageMetadata metadata = response.usageMetadata().get();

int inputTokens = metadata.promptTokenCount().orElse(0);
int inputTokens =
metadata.promptTokenCount().orElse(0)
+ metadata.toolUsePromptTokenCount().orElse(0);
int cachedTokens = metadata.cachedContentTokenCount().orElse(0);

@oss-maintainer oss-maintainer Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Classifying toolUsePromptTokenCount as input looks right and matches how the other providers bill server-side tool turns. Please confirm the same assumption holds for cachedContentTokenCount: in Gemini the cached count is a subset of promptTokenCount, so inputTokens stays correct, but a comment here would prevent a future "fix" from subtracting it twice.

int totalOutputTokens = metadata.candidatesTokenCount().orElse(0);
int thinkingTokens = metadata.thoughtsTokenCount().orElse(0);

// Output tokens exclude thinking tokens (following DashScope behavior)
// In Gemini, candidatesTokenCount includes thinking, so we subtract it
int outputTokens = totalOutputTokens - thinkingTokens;
int outputTokens =
metadata.candidatesTokenCount()
.map(candidateTokens -> candidateTokens + thinkingTokens)
.orElseGet(
() ->

@oss-maintainer oss-maintainer Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This flips the sign of the previous intentional behaviour (thinking tokens used to be excluded from output). Two consequences worth a explicit note: (1) stored historical usage for Gemini sessions is not comparable across this boundary, and (2) any user-side cost/budget math that added thinkingTokens back on top of outputTokens now double-counts. Could you call this out in the changelog/release note, and check the OpenAI/DashScope parsers so all providers use the same convention?

metadata.totalTokenCount()
.map(
total ->
Math.max(
0,
total
- inputTokens))
// Without candidate or total counts,
// thoughts are the only reported output.
.orElse(thinkingTokens));

usage =
ChatUsage.builder()
Expand Down
66 changes: 62 additions & 4 deletions ...c/test/java/io/agentscope/extensions/model/gemini/formatter/GeminiResponseParserTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,9 +216,9 @@ void testParseUsageMetadata() {
GenerateContentResponseUsageMetadata usageMetadata =
GenerateContentResponseUsageMetadata.builder()
.promptTokenCount(100)
.candidatesTokenCount(60) // Includes thinking
.candidatesTokenCount(60)
.thoughtsTokenCount(10) // Thinking tokens
.totalTokenCount(160)
.totalTokenCount(170)
.build();

GenerateContentResponse response =
Expand All @@ -238,13 +238,71 @@ void testParseUsageMetadata() {
// Input tokens = promptTokenCount
assertEquals(100, usage.getInputTokens());

// Output tokens = candidatesTokenCount - thoughtsTokenCount
assertEquals(50, usage.getOutputTokens());
// Output tokens include candidate and model-generated thinking tokens.
assertEquals(70, usage.getOutputTokens());

// Time should be > 0
assertTrue(usage.getTime() >= 0);
}

@Test
void testParseUsageMetadataClassifiesToolUseTokensAsInput() {
GenerateContentResponseUsageMetadata usageMetadata =
GenerateContentResponseUsageMetadata.builder()
.promptTokenCount(500)
.candidatesTokenCount(120)
.toolUsePromptTokenCount(300)
.thoughtsTokenCount(10)
.totalTokenCount(930)
.build();

GenerateContentResponse response =
GenerateContentResponse.builder().usageMetadata(usageMetadata).build();

ChatUsage usage = parser.parseResponse(response, startTime).getUsage();

assertNotNull(usage);
assertEquals(800, usage.getInputTokens());
assertEquals(130, usage.getOutputTokens());
}

@Test
void testParseUsageMetadataUsesTotalWhenCandidateCountIsMissing() {
GenerateContentResponseUsageMetadata usageMetadata =
GenerateContentResponseUsageMetadata.builder()
.promptTokenCount(500)
.toolUsePromptTokenCount(300)
.totalTokenCount(930)
.build();

GenerateContentResponse response =
GenerateContentResponse.builder().usageMetadata(usageMetadata).build();

ChatUsage usage = parser.parseResponse(response, startTime).getUsage();

assertNotNull(usage);
assertEquals(800, usage.getInputTokens());
assertEquals(130, usage.getOutputTokens());
}

@Test
void testParseUsageMetadataUsesThinkingWhenCandidateAndTotalCountsAreMissing() {
GenerateContentResponseUsageMetadata usageMetadata =
GenerateContentResponseUsageMetadata.builder()
.promptTokenCount(500)
.thoughtsTokenCount(10)
.build();

GenerateContentResponse response =
GenerateContentResponse.builder().usageMetadata(usageMetadata).build();

ChatUsage usage = parser.parseResponse(response, startTime).getUsage();

assertNotNull(usage);
assertEquals(500, usage.getInputTokens());
assertEquals(10, usage.getOutputTokens());
}

@Test
void testParseUsageMetadataReadsCachedContentTokenCount() {
// Gemini 报告的 cachedContentTokenCount 必须透传到 ChatUsage.cachedTokens,
Expand Down
Loading