Skip to content

fix(gemini): correct token usage accounting - #3034

Open
liugy789 wants to merge 2 commits into
agentscope-ai:mainfrom
liugy789:fix/gemini-tool-use-token-accounting
Open

fix(gemini): correct token usage accounting#3034
liugy789 wants to merge 2 commits into
agentscope-ai:mainfrom
liugy789:fix/gemini-tool-use-token-accounting

Conversation

@liugy789

@liugy789 liugy789 commented Sep 8, 2026

Copy link
Copy Markdown

AgentScope-Java Version

2.0.3-SNAPSHOT

Description

Fixes #3033.

GeminiResponseParser previously excluded tool-use prompt tokens from input usage and subtracted thinking tokens from output usage. This caused inaccurate usage, budget, and cost accounting for responses that use server-side tools or thinking.

This change:

  • classifies promptTokenCount + toolUsePromptTokenCount as input tokens;
  • classifies candidatesTokenCount + thoughtsTokenCount as output tokens;
  • falls back to totalTokenCount - inputTokens when candidate count is absent;
  • preserves reported thinking tokens when both candidate and total counts are absent;
  • adds parser-level regression coverage for thinking, server-side tool use, the missing-candidate fallback, and the thinking-only fallback.

No public API, dependency, or user-facing documentation changes are required. Existing cached-token handling is preserved.

Validation:

  • GeminiResponseParserTest passes with 15 tests.
  • The Gemini module and its dependencies pass.
  • Spotless and git diff --check pass.
  • Repository-wide GitHub CI is running for the latest commit.

Checklist

  • Code has been formatted with Spotless.
  • All affected tests are passing.
  • Javadoc comments are complete and follow project conventions. No public API was added or changed.
  • Related documentation has been updated. No documentation is required for this internal bug fix.
  • Code is ready for review.

@codecov

codecov Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@oss-maintainer oss-maintainer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Corrects Gemini usage accounting by billing toolUsePromptTokenCount as input and thoughtsTokenCount as output, with sensible fallbacks when candidate counts are missing. Well covered by four new parser-level tests. Main ask is on documentation of the behaviour flip: previously output deliberately excluded thinking tokens, so consumers that added them back will now double-count.


Automated review by github-manager-bot

int inputTokens =
metadata.promptTokenCount().orElse(0)
+ metadata.toolUsePromptTokenCount().orElse(0);
int cachedTokens = metadata.cachedContentTokenCount().orElse(0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Classifying toolUsePromptTokenCount as input looks right and matches how the other providers bill server-side tool turns. Please confirm the same assumption holds for cachedContentTokenCount: in Gemini the cached count is a subset of promptTokenCount, so inputTokens stays correct, but a comment here would prevent a future "fix" from subtracting it twice.

metadata.candidatesTokenCount()
.map(candidateTokens -> candidateTokens + thinkingTokens)
.orElseGet(
() ->

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This flips the sign of the previous intentional behaviour (thinking tokens used to be excluded from output). Two consequences worth a explicit note: (1) stored historical usage for Gemini sessions is not comparable across this boundary, and (2) any user-side cost/budget math that added thinkingTokens back on top of outputTokens now double-counts. Could you call this out in the changelog/release note, and check the OpenAI/DashScope parsers so all providers use the same convention?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Gemini usage metadata misclassifies tool-use and thinking tokens

2 participants