fix(copilot): baseline cost tracking and cache token display - #12762
Merged
Conversation
…en display When OpenRouter's x-total-cost header is missing, estimate cost from token counts using a known model pricing table so cost is always logged. Also extract cache token details from streaming usage chunks (prompt_tokens_details.cached_tokens) and pass them through to PlatformCostLog. On the dashboard side, add cache read/write columns to the logs table and user table, and include cache tokens in the UserCostSummary backend model so they surface in the API response.
github-project-automation
Bot
moved this to 🆕 Needs initial review
in AutoGPT development kanban
github-actions
Bot
added
platform/frontend
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds per-model OpenRouter token pricing and token-based fallback cost estimation, tracks cache read/creation tokens in baseline LLM streaming, persists those cache metrics to platform cost dashboard and OpenAPI, updates frontend tables to show cache metrics, and adds unit tests for estimation and token extraction. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant BaselineService as Baseline Service
participant OpenRouter as OpenRouter Stream
participant State as BaselineState
participant DB as Platform DB / Prisma
participant Frontend
Client->>BaselineService: send chat completion request
BaselineService->>OpenRouter: open streaming request (model, input)
OpenRouter-->>BaselineService: stream chunks (usage + optional x-total-cost header)
BaselineService->>State: snapshot turn_prompt/completion tokens at call start
BaselineService->>State: accumulate chunk usage, cached_tokens, cache_creation_input_tokens
alt x-total-cost header present
BaselineService->>State: set cost_usd from header
else header missing
BaselineService->>BaselineService: call _estimate_cost_from_tokens(model, prompt_delta, completion_delta)
BaselineService->>State: set estimated cost_usd
end
BaselineService->>DB: persist tokens, cache metrics, and cost
Frontend->>DB: query dashboard (includes cache fields)
DB-->>Frontend: return dashboard data
Frontend->>Client: render tables with cache metrics
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🔍 PR Overlap DetectionThis check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early. 🔴 Merge Conflicts DetectedThe following PRs have been tested and will have merge conflicts if merged after this PR. Consider coordinating with the authors.
🟢 Low Risk — File Overlap OnlyThese PRs touch the same files but different sections (click to expand)
Summary: 3 conflict(s), 0 medium risk, 2 low risk (out of 5 PRs with file overlap) Auto-generated on push. Ignores: |
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #12762 +/- ##
==========================================
+ Coverage 63.46% 63.59% +0.13%
==========================================
Files 1815 1815
Lines 131479 132034 +555
Branches 14304 14327 +23
==========================================
+ Hits 83439 83970 +531
- Misses 45436 45463 +27
+ Partials 2604 2601 -3
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…ion_tokens extraction Bug 1: Fallback cost estimation was using accumulated turn_prompt_tokens / turn_completion_tokens across all tool-call rounds, causing compounding over-estimation on the 2nd+ turn. Snapshot token counts before each call and pass only the per-call delta to _estimate_cost_from_tokens. Bug 2: turn_cache_creation_tokens was defined but never populated. Extract cache_creation_input_tokens from prompt_tokens_details (available from some providers such as Anthropic via OpenRouter). Add regression tests for both fixes.
majdyz
commented
Apr 13, 2026
majdyz
commented
Apr 13, 2026
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
autogpt_platform/backend/backend/copilot/baseline/service.py (1)
1373-1392:⚠️ Potential issue | 🟠 MajorBackfilled tokens still never produce a fallback
cost_usd.When a provider omits both
x-total-costand streamingusage, Lines 1355-1371 backfill token counts withtiktoken, but the pricing fallback only runs inside_baseline_llm_caller.persist_and_record_usage(...)still receivescost_usd=Nonefor the exact provider behavior this PR is trying to cover.💡 Proposed fix
if ( state.turn_prompt_tokens == 0 and state.turn_completion_tokens == 0 and not (_stream_error and not state.assistant_text) ): @@ logger.info( "[Baseline] No streaming usage reported; estimated tokens: " "prompt=%d, completion=%d", state.turn_prompt_tokens, state.turn_completion_tokens, ) + if state.cost_usd is None and ( + state.turn_prompt_tokens > 0 or state.turn_completion_tokens > 0 + ): + estimated = _estimate_cost_from_tokens( + active_model, + state.turn_prompt_tokens, + state.turn_completion_tokens, + ) + if estimated is not None: + state.cost_usd = estimated + # Persist token usage to session and record for rate limiting.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/copilot/baseline/service.py` around lines 1373 - 1392, persist_and_record_usage is still called with cost_usd=None when tokens were backfilled; detect when state.cost_usd is None after you compute uncached_prompt (i.e., the provider omitted both x-total-cost and streaming usage) and compute a fallback cost using the same pricing fallback logic used in _baseline_llm_caller (apply the model pricing estimation for active_model using uncached_prompt and state.turn_completion_tokens), assign that value to cost_usd and pass it into persist_and_record_usage so the persisted record includes an estimated cost.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/copilot/baseline/service.py`:
- Around line 418-423: The locals prompt_tokens_before and
completion_tokens_before must be initialized before calling
client.chat.completions.create(...) to avoid UnboundLocalError in the finally
block if the API call raises; set prompt_tokens_before =
state.turn_prompt_tokens and completion_tokens_before =
state.turn_completion_tokens (or sensible defaults) immediately before invoking
client.chat.completions.create so they are always defined, then proceed with the
API await and use those snapshots in the finally/fallback cost estimation logic.
---
Outside diff comments:
In `@autogpt_platform/backend/backend/copilot/baseline/service.py`:
- Around line 1373-1392: persist_and_record_usage is still called with
cost_usd=None when tokens were backfilled; detect when state.cost_usd is None
after you compute uncached_prompt (i.e., the provider omitted both x-total-cost
and streaming usage) and compute a fallback cost using the same pricing fallback
logic used in _baseline_llm_caller (apply the model pricing estimation for
active_model using uncached_prompt and state.turn_completion_tokens), assign
that value to cost_usd and pass it into persist_and_record_usage so the
persisted record includes an estimated cost.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c24b982e-afc3-4362-ade1-a6aa3e30f9c6
📒 Files selected for processing (2)
autogpt_platform/backend/backend/copilot/baseline/service.pyautogpt_platform/backend/backend/copilot/baseline/service_unit_test.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: integration_test
- GitHub Check: check API types
- GitHub Check: Seer Code Review
- GitHub Check: type-check (3.11)
- GitHub Check: test (3.13)
- GitHub Check: test (3.11)
- GitHub Check: test (3.12)
- GitHub Check: end-to-end tests
- GitHub Check: Check PR Status
- GitHub Check: Analyze (python)
🧰 Additional context used
📓 Path-based instructions (3)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.pyautogpt_platform/backend/backend/copilot/baseline/service.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.pyautogpt_platform/backend/backend/copilot/baseline/service.py
autogpt_platform/backend/**/*_test.py
📄 CodeRabbit inference engine (autogpt_platform/backend/AGENTS.md)
autogpt_platform/backend/**/*_test.py: Use pytest with snapshot testing for API responses
Colocate test files with source files using*_test.pynaming convention
Mock at boundaries — mock where the symbol is used, not where it's defined; after refactoring, update mock targets to match new module paths
UseAsyncMockfromunittest.mockfor async functions in tests
When writing tests, use Test-Driven Development (TDD): write failing tests marked with@pytest.mark.xfailbefore implementation, then remove the marker once the implementation is complete
When creating snapshots in tests, usepoetry run pytest path/to/test.py --snapshot-update; always review snapshot changes withgit diffbefore committing
Files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.py
🧠 Learnings (17)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12651
File: autogpt_platform/frontend/src/app/api/openapi.json:8653-8696
Timestamp: 2026-04-02T14:27:41.807Z
Learning: Repo: Significant-Gravitas/AutoGPT — Platform costs
The PlatformCostLog.duration is stored in DB but intentionally omitted from the CostLogRow API response to keep the raw logs compact. Do not flag this omission; suggest documenting the intent in the route description if needed.
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12284
File: autogpt_platform/frontend/src/app/api/openapi.json:11897-11900
Timestamp: 2026-03-04T23:58:18.476Z
Learning: Repo: Significant-Gravitas/AutoGPT — PR `#12284`
Backend/frontend OpenAPI codegen convention: In backend/api/features/store/model.py, the StoreSubmission and StoreSubmissionAdminView models define submitted_at: datetime | None, changes_summary: str | None, and instructions: str | None with no default. This is intentional to produce “required but nullable” fields in OpenAPI (properties appear in required[] and use anyOf [type, null]). This matches Prisma’s submittedAt DateTime? and changesSummary String?. Do not flag this as a required/nullable mismatch.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:12.953Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
📚 Learning: 2026-03-30T11:49:37.770Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12604
File: autogpt_platform/backend/backend/copilot/sdk/security_hooks.py:165-171
Timestamp: 2026-03-30T11:49:37.770Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/security_hooks.py`, the `web_search_count` and `total_tool_call_count` circuit-breaker counters in `create_security_hooks` are intentionally per-turn (closure-local), not per-session. Hooks are recreated per stream invocation in `service.py`, so counters reset each turn. This is an accepted v1 design: it caps a single runaway turn (incident d2f7cba3: 179 WebSearch calls, $20.66). True per-session persistence via Redis is deferred to a later iteration. Do not flag these as a per-session vs. per-turn mismatch bug.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.pyautogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.pyautogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.pyautogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-04-01T04:17:41.600Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12632
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-01T04:17:41.600Z
Learning: When reviewing AutoGPT Copilot tool implementations, accept that `readOnlyHint=True` (provided via `ToolAnnotations`) may be applied unconditionally to *all* tools—even tools that have side effects (e.g., `bash_exec`, `write_workspace_file`, or other write/save operations). Do **not** flag these tools for having `readOnlyHint=True`; this is intentional to enable fully-parallel dispatch by the Anthropic SDK/CLI and has been E2E validated. Only flag `readOnlyHint` issues if they conflict with the established `ToolAnnotations` behavior (e.g., missing/incorrect propagation relative to the intended annotation mechanism).
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.pyautogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.pyautogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.pyautogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.pyautogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-09T10:50:43.907Z
Learnt from: Bentlybro
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-03-09T10:50:43.907Z
Learning: Repo: Significant-Gravitas/AutoGPT — File: autogpt_platform/backend/backend/blocks/llm.py
For xAI Grok models accessed via OpenRouter, the API returns `null` for `max_completion_tokens`. The convention in this codebase is to use the model's context window size as the `max_output_tokens` value in ModelMetadata. For example, Grok 3 uses 131072 (128k) and Grok 4 uses 262144 (256k). Do not flag these as incorrect max output token values.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-13T15:49:44.961Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/rate_limit.py:0-0
Timestamp: 2026-03-13T15:49:44.961Z
Learning: In `autogpt_platform/backend/backend/copilot/rate_limit.py`, the original per-session token window (with a TTL-based reset) was replaced with fixed daily and weekly windows. `resets_at` is now derived from `_daily_reset_time()` (midnight UTC) and `_weekly_reset_time()` (next Monday 00:00 UTC) — deterministic fixed-boundary calculations that require no Redis TTL introspection.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-17T10:57:12.953Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:12.953Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-12T14:42:40.552Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/rate_limit.py:141-170
Timestamp: 2026-03-12T14:42:40.552Z
Learning: In Significant-Gravitas/AutoGPT, `check_rate_limit` in `autogpt_platform/backend/backend/copilot/rate_limit.py` is intentionally a "pre-turn soft check" (not a hard atomic reservation). Because LLM token counts are unknown before generation completes, a strict check-and-reserve is impractical. The TOCTOU race (two concurrent turns both passing the pre-check and both committing via `record_token_usage`) is an accepted trade-off. If stricter enforcement is ever needed, the approach is a Lua script doing GET+INCRBY atomically in Redis.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-10T08:39:22.025Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-17T06:48:26.471Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12445
File: autogpt_platform/backend/backend/copilot/sdk/service.py:1071-1072
Timestamp: 2026-03-17T06:48:26.471Z
Learning: In Significant-Gravitas/AutoGPT (autogpt_platform), the AI SDK enforces `z.strictObject({type, errorText})` on SSE `StreamError` responses, so additional fields like `retryable: bool` cannot be added to `StreamError` or serialized via `to_sse()`. Instead, retry signaling for transient Anthropic API errors is done via the `COPILOT_RETRYABLE_ERROR_PREFIX` constant prepended to persisted session messages (in `ChatMessage.content`). The frontend detects retryable errors by checking `markerType === "retryable_error"` from `parseSpecialMarkers()` — no SSE schema changes and no string matching on error text. This pattern was established in PR `#12445`, commit 64d82797b.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-01T07:59:02.311Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12213
File: autogpt_platform/frontend/src/app/api/openapi.json:10030-10037
Timestamp: 2026-03-01T07:59:02.311Z
Learning: Repo: Significant-Gravitas/AutoGPT PR: 12213 — For MCP manual token storage, backend model autogpt_platform/backend/backend/api/features/mcp/routes.py defines MCPStoreTokenRequest.token as Pydantic SecretStr with a min length constraint, which generates OpenAPI schema metadata (format: "password", writeOnly: true, minLength: 1) in autogpt_platform/frontend/src/app/api/openapi.json. Prefer SecretStr (with length constraints) for sensitive request fields so generated TS clients and docs treat them as secrets.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-04T23:58:18.476Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12284
File: autogpt_platform/frontend/src/app/api/openapi.json:11897-11900
Timestamp: 2026-03-04T23:58:18.476Z
Learning: Repo: Significant-Gravitas/AutoGPT — PR `#12284`
Backend/frontend OpenAPI codegen convention: In backend/api/features/store/model.py, the StoreSubmission and StoreSubmissionAdminView models define submitted_at: datetime | None, changes_summary: str | None, and instructions: str | None with no default. This is intentional to produce “required but nullable” fields in OpenAPI (properties appear in required[] and use anyOf [type, null]). This matches Prisma’s submittedAt DateTime? and changesSummary String?. Do not flag this as a required/nullable mismatch.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-16T07:34:53.523Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/rate_limit.py:0-0
Timestamp: 2026-03-16T07:34:53.523Z
Learning: In `autogpt_platform/backend/backend/copilot/rate_limit.py`, the `record_token_usage` Redis warning log omits `user_id` entirely. The final log message is `"Redis unavailable for recording token usage (tokens=%d)"` with only the token count — no user identifier (full or truncated) is included.
Applied to files:
autogpt_platform/backend/backend/copilot/baseline/service.py
🔇 Additional comments (1)
autogpt_platform/backend/backend/copilot/baseline/service_unit_test.py (1)
988-1052: Good regression coverage for the cumulative-turn pricing path.This test exercises the exact stateful failure mode that used to compound fallback cost across tool rounds, so it should be very effective at preventing that regression from coming back.
The backend added total_cache_read_tokens and total_cache_creation_tokens to UserCostSummary but the OpenAPI spec was not updated, causing frontend build failures.
majdyz
added a commit
that referenced
this pull request
Apr 13, 2026
Keep HEAD's pre-drain count logic for transcript loading and drain error handling, and merge incoming cache token extraction tests from PR #12762.
This was referenced Apr 13, 2026
…ndLocalError When client.chat.completions.create() raises (e.g. network timeout), the finally block referenced prompt_tokens_before/completion_tokens_before which were only assigned after the API call inside the try block, causing an UnboundLocalError that masked the original exception. Move the snapshots to before the try block so the finally block can always reference them safely even when the API call fails.
This was referenced Apr 13, 2026
Merged
Contributor
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
github-actions
Bot
added
conflicts
Contributor
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
…f hardcoded discount table Remove the hardcoded _CACHE_READ_DISCOUNT dict (Anthropic 10%, OpenAI 50%). Instead, extract the pricing.cache_read field from OpenRouter's /api/v1/models response and use it directly. Models where OpenRouter does not publish a cache_read rate fall back to the full input rate (safe over-estimate). Cache tuple type: (input_rate, output_rate) → (input_rate, output_rate, cache_read_rate | None)
Contributor
Author
|
🤖 Fixed in 012ea167d: Replaced the hardcoded |
…x/copilot-cost-tracking
…led StreamUsage Two gaps closed: 1. When a provider omits both `x-total-cost` and streaming `usage`, `stream_chat_completion_baseline` backfills token counts via tiktoken but previously left `state.cost_usd = None`. Now we call `await _estimate_cost_from_tokens(active_model, ...)` immediately after the tiktoken backfill so `persist_and_record_usage` receives a non-None cost when the model is in the OpenRouter pricing response. 2. The `StreamUsage` event was reporting raw `state.turn_prompt_tokens` (includes cached reads), making the frontend token count inconsistent with `cost_usd` which already applied the cache discount. Now we compute `billed_prompt = max(0, turn_prompt_tokens - turn_cache_read_tokens)` and yield that instead. Also removes unused `_ThinkingStripper` import from service_unit_test.py (caught by ruff) to fix the lint CI failure.
Contributor
Author
|
🤖 Fixed in 8420ad0c1: After the tiktoken token backfill (when a provider omits both |
…ing cache fetch _fetch_openrouter_pricing() called from the finally block can block for up to 10s on a cold cache, and concurrent callers all seeing an expired cache would each make separate HTTP requests simultaneously. Two fixes: 1. Add a lazy asyncio.Lock (_OPENROUTER_PRICING_LOCK) with double-checked locking so only one coroutine makes the HTTP request; others wait and reuse the result. 2. Update _OPENROUTER_PRICING_CACHE_FETCHED_AT *before* the HTTP request so that a failed or timed-out fetch advances the "next allowed fetch" window, creating a natural backoff instead of every subsequent request immediately retrying on OpenRouter outage.
…n x-total-cost absent
majdyz
changed the title
github-project-automation
Bot
moved this from 🆕 Needs initial review
to ✅ Done
in AutoGPT development kanban
This was referenced Apr 15, 2026
Merged
5 tasks
This was referenced May 4, 2026
Merged
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The baseline copilot path (OpenAI-compatible / OpenRouter) did not record any cost when the
x-total-costresponse header was absent, even though token counts were always available. The admin cost dashboard also lacked cache token columns.What
x-total-costheader extraction: Reads the OpenRouter cost header per LLM call in thefinallyblock (so cost is captured even when the stream errors mid-way). Accumulated across multi-round tool-calling turns.prompt_tokens_details.cached_tokensandcache_creation_input_tokensfrom streaming usage chunks and passescache_read_tokens/cache_creation_tokensthrough topersist_and_record_usagefor storage inPlatformCostLog.total_cache_read_tokensandtotal_cache_creation_tokenstoUserCostSummary.x-total-costis absent,cost_usdis left asNoneandpersist_and_record_usagerecords the entry undertracking_type="tokens". Token-based cost estimation was removed — the platform dashboard already handles per-token cost display, and estimates would introduce inaccuracy in the reported figures.How
_baseline_llm_caller: extract thex-total-costheader in thefinallyblock; accumulate tostate.cost_usd._BaselineStreamState: addturn_cache_read_tokens/turn_cache_creation_tokenscounters, populated from streaming usage chunks.persist_and_record_usage/record_cost_log: pass throughcache_read_tokensandcache_creation_tokenstoPlatformCostEntry.total_cache_read_tokens/total_cache_creation_tokensfields toUserCostSummaryand render them as columns in the cost dashboard.Test plan
x-total-costheader is presentcost_usdstaysNoneand token count is logged when header is absenttest_cost_usd_extracted_from_response_header,test_cost_usd_remains_none_when_header_missing,test_cache_tokens_extracted_from_usage_details