feat(platform/admin): cost tracking for system credentials by majdyz · Pull Request #12696 · Significant-Gravitas/AutoGPT

majdyz · 2026-04-07T11:57:32Z

Why

When system-managed credentials are used (AutoGPT pays the API bills), there was no visibility into which providers were being called, how much each costs, or which users were driving usage. This makes it impossible to set appropriate per-user limits or reconcile expenses with actual API invoices.

What

End-to-end platform cost tracking for all 22 system-credential providers + both copilot modes:

Every block execution that uses system credentials records a PlatformCostLog row (provider, cost, tokens, user, execution IDs)
Copilot turns (SDK + baseline) are tracked with model name, token counts, and actual USD cost
Admin dashboard at /admin/platform-costs shows cost breakdown by provider and user with date/provider/user filters and paginated raw logs
Admin API endpoints with 30s TTL cache: GET /platform-costs/dashboard and GET /platform-costs/logs

How

Core hook

cost_tracking.py calls log_system_credential_cost() after each block node execution. It reads NodeExecutionStats.provider_cost (set by merge_stats() inside each block) and dispatches a fire-and-forget INSERT via log_platform_cost_safe().

Per-block tracking

Each block calls self.merge_stats(NodeExecutionStats(provider_cost=..., provider_cost_type=...)):

Tracking type	Providers	Amount
`cost_usd`	OpenRouter, Exa	Actual USD from API response
`tokens`	OpenAI, Anthropic, Groq, Ollama, Jina	Token count from response.usage
`characters`	Unreal Speech, ElevenLabs, D-ID	Input text length
`sandbox_seconds`	E2B	Walltime
`walltime_seconds`	FAL, Revid, Replicate	Walltime
`per_run`	Google Maps, Apollo, SmartLead, etc.	1 per execution

OpenRouter cost: extracted via with_raw_response.create() and raw.headers.get("x-total-cost") with math.isfinite + >= 0 validation (replaces private _response access).

Copilot tracking

token_tracking.py writes a PlatformCostLog row per copilot LLM turn via an async fire-and-forget queue bounded by a Semaphore(50). SDK path uses sdk_msg.total_cost_usd; baseline path uses the x-total-cost header from OpenRouter streaming responses.

Executor drain

drain_pending_cost_logs() is called before executor.shutdown() using a module-level loop registry (_active_node_execution_loops) so that pending log tasks from each worker thread's event loop are awaited before the process exits. Tasks are filtered by task.get_loop() is current_loop to avoid cross-loop RuntimeError in Python ≥ 3.10.

CoPilot executor lifecycle

Worker threads connect Prisma on startup and disconnect on cleanup (even on failure). If db.connect() fails during @func_retry, the event loop is stopped and joined before re-raising so no loop is leaked across retry attempts.

Schema

model PlatformCostLog {
  id                  String   @id @default(uuid())
  createdAt           DateTime @default(now())
  userId              String?
  graphExecId         String?
  nodeExecId          String?
  blockName           String
  provider            String
  trackingType        String
  costMicrodollars    BigInt   @default(0)
  inputTokens         Int?
  outputTokens        Int?
  duration            Float?
  model               String?
}

Admin dashboard

React page with three tabs (By Provider / By User / Raw Logs) driven by two generated Orval hooks (useGetV2GetPlatformCostDashboard, useGetV2GetPlatformCostLogs). Filters are URL-based (searchParams) for bookmarkability. Pagination for raw logs. Per-provider estimated totals using configurable cost-per-unit multipliers.

Test plan

Migration applies cleanly
Block execution with system credentials creates PlatformCostLog row
Copilot conversation records cost log with tokens + model
/admin/platform-costs dashboard renders with correct data
Date/provider/user filters work correctly
Non-admin users get 403 on cost endpoints
Executor drain completes before process exit (no lost logs)

Track real API costs incurred when users consume system-managed credentials. Captures provider, tokens, duration, and model per block execution and surfaces an admin dashboard with provider/user aggregation and raw logs.

- Parameterize LIMIT/OFFSET in SQL queries to prevent injection - Only log platform cost on successful block execution - Convert model enum values to strings for proper logging - Add error handling with try/catch/finally in frontend useEffect - Drive filter state from URL params to prevent desync - Add dark mode support using design tokens - Return total_users count in dashboard for accurate reporting - Add credit_cost to metadata as cost proxy until per-token pricing

- Parallelize dashboard queries with asyncio.gather for ~3x speedup - Move json import to top-level - Use consistent p. table alias across all dashboard queries

- Remove duplicate block_usage_cost call from cost logging - Add case-insensitive provider filter using LOWER() - Add platform_cost_routes_test.py with basic endpoint tests

- Add tests for query parameter forwarding and pagination

- Replace ServerCrash icon with Receipt for Platform Costs sidebar

- CRITICAL: Use execute_raw_with_schema for INSERT (not query_raw) - Remove accidentally committed transcripts/ - Add dry_run guard to skip cost logging for simulated executions - Change onDelete: Cascade → SetNull to preserve cost history - Add standalone createdAt index for date-only queries - Add deterministic tiebreaker (id) to pagination ORDER BY - Update migration SQL to match schema changes

Include the block's credit cost (from block_cost_config) in the log metadata so every entry has a known cost proxy even when the provider doesn't expose actual dollar costs.

Make user_id Optional[str] in UserCostSummary and CostLogRow to handle cases where the referenced user has been deleted. Use .get() for safe access to user_id from query result rows. Regenerate OpenAPI schema.

- OpenRouter: Extract actual USD cost from x-total-cost response header - Exa (search, contents): Write cost_dollars.total to execution_stats - LLM blocks: Store provider_cost in stats when available - Add provider_cost field to NodeExecutionStats - Hook now converts provider_cost to costMicrodollars in PlatformCostLog - Metadata includes both credit_cost and provider_cost_usd when available

…adata Standardize cost tracking across providers: - cost_usd: actual dollar cost (OpenRouter, Exa) - tokens: total token count (LLM blocks) - duration_seconds: execution time (video gen, sandboxes) - per_run: flat per-request (all others)

…g.py Copilot uses OpenRouter via a separate code path (not through the block executor). This integrates PlatformCostLog into the shared persist_and_record_usage() function which is called by both SDK and baseline copilot paths, capturing: - Every LLM turn (main conversation, title gen, context compression) - Tokens (prompt + completion + cache) - Actual USD cost when available (SDK path provides cost_usd) - Session ID for correlation

…tMicrodollars to BigInt - NodeExecutionStats.__iadd__ was overwriting accumulated provider_cost with None when merging stats that lacked provider_cost (e.g. the final llm_call_count/llm_retry_count merge). Skip None values in __iadd__ so existing data is never erased. - Widen PlatformCostLog.costMicrodollars from Int (max ~$2,147) to BigInt to prevent theoretical overflow for high-cost aggregated node executions.

…trics Replace one-size-fits-all tracking cascade with provider-aware logic: - cost_usd: OpenRouter (x-total-cost header), Exa (cost_dollars) - tokens: OpenAI, Anthropic, Groq, Ollama (token counts) - characters: Unreal Speech, ElevenLabs (input text length) - api_calls: Google Maps (1 nearby + N detail calls) - sandbox_seconds: E2B (sandbox execution time) - generation_seconds: FAL, Revid, D-ID, Replicate (video/image gen time) - per_run: Apollo, SmartLead, ZeroBounce, Jina, etc.

…dex/platform-cost-tracking

…ed as NULL - Add null-safe optional chaining for user_id.slice() in LogsTable, displaying "Deleted user" when user_id is null to prevent frontend crash - Change `if cost_float` to `if cost_float is not None` in token_tracking.py so that a legitimate $0.00 cost is stored as 0 instead of NULL

- Fix ElevenLabs/D-ID field name: script -> script_input - Remove incorrect Google Maps api_calls formula, use per_run instead - Remove D-ID from generation_seconds (walltime includes polling) - Jina embeddings: extract total_tokens from response.usage - Simplify tracking types: cost_usd, tokens, characters, sandbox_seconds, walltime_seconds, per_run

Every block that uses system credentials now calls merge_stats with meaningful data after the API response: - Google Maps: output_size = number of places returned (= detail API calls) - Apollo people/org: output_size = results count - Apollo person: output_size = 1 per enrichment - SmartLead: output_size = leads added or 1 per operation - Ideogram: output_size = 1 per image - Replicate: output_size = 1 per prediction - Nvidia: output_size = 1 per inference - ScreenshotOne: output_size = 1 per screenshot - ZeroBounce: output_size = 1 per email validated - Mem0: output_size = 1 per memory operation

…, E2B, YouTube, Weather, TTS, Enrichlayer) Every system credential block now has explicit merge_stats tracking. No block relies on the generic fallback anymore.

The baseline copilot path uses the same OpenRouter API but wasn't extracting the x-total-cost header. Now extracts cost from the streaming response headers and passes it to persist_and_record_usage, giving us actual USD cost for both copilot modes.

Both SDK and Baseline copilot paths now set OpenTelemetry span attributes for cost tracking before the trace context closes: - gen_ai.usage.prompt_tokens - gen_ai.usage.completion_tokens - gen_ai.usage.cost_usd (when available) - gen_ai.usage.cache_read_tokens (SDK only) - gen_ai.usage.cache_creation_tokens (SDK only) Also extracts x-total-cost from OpenRouter response headers in the Baseline streaming path, giving actual USD cost for both modes. These attributes flow to Langfuse/any OTEL backend for cost dashboards.

… paths

…multi-round costs - Move x-total-cost header extraction to finally block so cost is captured even when stream errors mid-way (we already paid) - Accumulate cost across multi-round tool-calling turns instead of overwriting with last round only - Handle UnboundLocalError if response was never assigned

Both SDK and Baseline paths now pass config.model to persist_and_record_usage so PlatformCostLog records the actual model (e.g. anthropic/claude-sonnet-4) for filtering/grouping.

…dex/platform-cost-tracking

Backend: - ProviderCostSummary now includes tracking_type and total_duration_seconds - CostLogRow includes tracking_type and duration - SQL queries extract tracking_type from metadata JSON Frontend: - Replaced hand-written types/client with generated API client (orval) - Actions use getV2GetPlatformCostDashboard/getV2GetPlatformCostLogs - Provider table shows: Type badge, Usage metric, Known Cost, Estimated Cost - Per-run providers have editable $/run input with defaults - Summary cards show "Known Cost" vs "Estimated Total" - Logs table shows tracking_type badge + duration column - Color-coded badges: cost_usd(green), tokens(blue), duration(orange), characters(purple), per_run(gray)

- Import Pagination from generated client instead of hand-written types - Add DEFAULT_COST_PER_1K_TOKENS for OpenAI/Anthropic/Groq/Ollama - estimateCostForRow now computes cost from token count when provider doesn't report actual USD (tokens * rate_per_1k / 1000) - Added date comment for when default rates were checked

…, Literal type, merged migration - db_accessors.py: fix platform_cost_db() to follow the conditional db.is_connected() accessor pattern used by all other accessors. When DB is connected (API process) it uses platform_cost directly; when not (executor) it routes through DatabaseManagerAsyncClient. - platform_cost.py: convert log_platform_cost() from a raw SQL INSERT to a Prisma create call, removing execute_raw_with_schema and _json_or_none. Make block_id, block_name, credential_id optional (nullable) in PlatformCostEntry to match the DB schema. - platform_cost_test.py: update TestLogPlatformCost to mock PrismaLog.prisma instead of execute_raw_with_schema; remove TestJsonOrNone. - model.py: type subscription_tier as SubscriptionTier | None using the Prisma-generated enum (FREE, PRO, BUSINESS, ENTERPRISE). - migrations: fold trackingAmount DOUBLE PRECISION into the CREATE TABLE in 20260402120000_add_platform_cost_log and delete the now-redundant 20260405140000_add_platform_cost_tracking_amount migration. Also remove NOT NULL from blockId, blockName, credentialId for schema consistency. - schema.prisma: mark blockId, blockName, credentialId as String?.

…to codex/platform-cost-tracking

Pwuts

LGTM

autogpt-pr-reviewer-in-dev

📋 Automated Review — PR #12696

PR #12696 — feat(platform/admin): cost tracking for system credentials
Author: majdyz | Files: 53

🎯 Verdict: APPROVE

PR Description Quality

✅ Has Why + What + How — PR clearly explains the motivation (no visibility into platform-subsidized API costs), the architecture (block-level instrumentation → executor/copilot scheduling → DB → admin dashboard), and includes two detailed E2E test reports.

What This PR Does

Adds end-to-end cost tracking for API calls made using system-owned credentials (OpenAI, ElevenLabs, Exa, etc.) across both the block executor and copilot subsystems. Blocks report per-call costs via merge_stats(), the executor and copilot schedule fire-and-forget DB writes, and a new admin dashboard displays cost breakdowns by provider, user, and individual log entries with filtering, pagination, and inline rate estimation. This gives platform operators visibility into subsidized API spend that was previously invisible.

Specialist Findings

🛡️ Security ✅ — Well-secured with requires_admin_user on all API routes, withRoleAccess(["admin"]) on the frontend page, parameterized SQL throughout, email masking on responses, and math.isfinite() + >= 0 validation preventing NaN/Inf cost poisoning.

🟠 Undocumented _response private attribute access for OpenRouter cost extraction (blocks/llm.py:786, baseline/service.py:442) — fragile pattern that silently degrades if the OpenAI SDK changes internals. (Flagged by: security, architect, performance, quality — 4 specialists)
🟡 No RLS on PlatformCostLog table (migration.sql:2) — access is gated by admin routes, but direct DB access would expose all cost logs. Consider adding RLS policies matching other sensitive tables.
🟡 Unbounded metadata JSONB field (platform_cost.py:50) — no size validation on the dict. A Pydantic validator capping at ~10KB would prevent storage bloat.

🏗️ Architecture ✅ — Clean separation of concerns: blocks → stats → executor/copilot scheduling → platform_cost DB layer → admin API → React dashboard. Accessor pattern correctly used via platform_cost_db(). Database schema well-indexed with composite indexes for common query patterns.

🟠 Duplicated fire-and-forget infrastructure between cost_tracking.py:28-57 and token_tracking.py:27-43 — nearly identical task sets, locks, per-loop semaphores, and scheduling logic. (Flagged by: architect, quality — 2 specialists)
🟠 Cross-subsystem coupling via private imports (cost_tracking.py:9-12) — executor imports _pending_log_tasks and _pending_log_tasks_lock directly from copilot's token_tracking.py. (Flagged by: architect, quality — 2 specialists)
🟡 Module-level asyncio.Semaphore(50) in platform_cost.py:82 doesn't follow the per-loop pattern used correctly in the other two modules. Latent bug if ever called from multiple event loops. (Flagged by: architect, performance, quality — 3 specialists)

⚡ Performance ✅ — Proper concurrency controls: semaphore-bounded at 50 concurrent writes, 30s TTL cache on dashboard queries, asyncio.gather for parallel SQL execution, graceful drain on shutdown. No hot-path regressions.

🟡 No retention/partitioning strategy for PlatformCostLog table (migration.sql:2) — at high volumes, COUNT(*) and aggregation queries will degrade over time. Worth adding a TODO for time-based partitioning.
🟡 Dashboard cache has no max_size bound (platform_cost.py:195) — each unique filter combination creates a separate entry. Low risk for admin-only endpoint but worth bounding.

🧪 Testing ✅ — ~94.6% patch coverage (target 80%). Strong backend tests covering resolve_tracking exhaustively, merge_stats for multiple block types, auth guards, microdollar rounding precision, and OpenRouter cost extraction edge cases (inf/nan/negative).

🟠 No test for _schedule_log fire-and-forget task lifecycle (cost_tracking.py:119) — the core async machinery (task creation, done-callback removal, semaphore limiting) is completely untested at the unit level. (Flagged by: testing — 1 specialist)
🟠 No test for drain_pending_cost_logs timeout branch (cost_tracking.py:100) — the "still_pending" warning path is never exercised. (Flagged by: testing — 1 specialist)
🟡 usePlatformCostContent hook has no dedicated test — filter state, tab management, and URL param synchronization are untested. (Flagged by: testing — 1 specialist)
🔵 Duplicate test coverage between block_cost_tracking_test.py and exa/cost_tracking_test.py for ExaCodeContextBlock and ExaContentsBlock.

📖 Quality ⚠️ — Readability B+. Well-structured with clear naming and good defensive coding. Main concern is DRY violations that will diverge over time.

🟠 OpenRouter cost extraction duplicated inline in baseline/service.py:439 instead of calling the shared extract_openrouter_cost() from blocks/llm.py. The comment even acknowledges the duplication. (Flagged by: security, architect, performance, quality — 4 specialists)
🔵 last_attempt_cost naming in llm.py:1446 is ambiguous — reads as "cost of the last attempt" but means "cost to report on success."
🔵 Inconsistent export style: ProviderTable.tsx uses bottom export while PlatformCostContent.tsx uses export function.
🔵 _build_where in platform_cost.py:161 lacks a docstring despite being a non-trivial SQL builder with positional parameter indexing.

📦 Product ✅ — Complete implementation of the stated requirements. Dashboard has three functional tabs, filter panel, pagination, skeleton loaders, error/empty states, and proper ARIA roles.

🟡 Rate overrides are ephemeral with a tiny (unsaved) label at text-[10px] (ProviderTable.tsx:47) — admins will lose custom rates on refresh with no warning. Consider localStorage persistence.
🟡 Logs tab always fetches regardless of active tab (usePlatformCostContent.ts:63) — wastes bandwidth when admin is on Provider/User tab. Use React Query's enabled option.
🔵 Execution IDs truncated to 8 chars with no tooltip or copy (LogsTable.tsx:92).

📬 Discussion ✅ — All reviewer concerns systematically addressed across 4 fix commits. 48/48 CI checks pass. All comment threads resolved. CodeRabbit's critical finding (private _response access) was addressed by switching to with_raw_response.create(). Sentry bot's semaphore issue was fixed. PR merged with APPROVED status.

🔎 QA ⚠️ — Author posted two detailed E2E test reports covering dashboard API, cost logging, copilot tracking, and admin auth. No independent QA screenshots available from this review. Local CI checks show frontend lint and typecheck failures, but these appear to be environment-related (the PR's own CI passed all 48 checks).

🟠 Should Fix

Duplicated fire-and-forget task infrastructure (cost_tracking.py:28-57, token_tracking.py:27-43) — Nearly identical task sets, locks, per-loop semaphores, and scheduling logic in two modules. Extract a shared AsyncTaskPool class in backend/util/async_tasks.py. (Flagged by: architect, quality — 2 specialists)
Duplicated OpenRouter cost extraction (baseline/service.py:439, blocks/llm.py:786) — Same fragile _response.headers access pattern implemented independently in two locations with different error handling. Extract to a shared utility. (Flagged by: security, architect, performance, quality — 4 specialists)
Private cross-module imports (cost_tracking.py:9-12) — Importing _pending_log_tasks and _pending_log_tasks_lock from token_tracking.py breaks encapsulation. Expose via a public drain() function instead. (Flagged by: architect, quality — 2 specialists)
Missing test: async task lifecycle (cost_tracking.py:119) — _schedule_log is the core async machinery for cost logging but has zero unit test coverage for task creation, callback cleanup, or semaphore bounding. (Flagged by: testing — 1 specialist)
Missing test: drain timeout branch (cost_tracking.py:100) — The "still_pending" warning path in drain_pending_cost_logs is never exercised in tests. (Flagged by: testing — 1 specialist)

🟡 Nice to Have

RLS on PlatformCostLog table (migration.sql:2) — Add row-level security policies matching other sensitive tables. Currently safe via admin route guards. (security)
Metadata size validation (platform_cost.py:50) — Cap metadata JSONB at ~10KB via Pydantic validator. (security)
Table retention strategy (migration.sql:2) — Add TODO for time-based partitioning on createdAt. (performance)
Per-loop semaphore in platform_cost.py (platform_cost.py:82) — Module-level semaphore is a latent multi-loop bug. Match pattern from other modules. (architect, performance, quality)
Rate override persistence (ProviderTable.tsx:47) — Persist to localStorage so admin rates survive page refresh. (product)
Conditional logs fetch (usePlatformCostContent.ts:63) — Only fetch logs when logs tab is active using React Query enabled. (product)
usePlatformCostContent hook tests — Filter state, tab management, URL params untested. (testing)
Dashboard cache max_size (platform_cost.py:195) — Bound cache entries to prevent unbounded growth. (performance, architect)

🔵 Nits

Ambiguous variable name (llm.py:1446) — last_attempt_cost → final_provider_cost for clarity.
Inconsistent export style (ProviderTable.tsx:131) — Uses bottom export vs export function in other components.
Missing docstring (platform_cost.py:161) — _build_where needs a brief docstring explaining return format.
Truncated IDs (LogsTable.tsx:92) — Add title attr with full execution/user ID for tooltip.

Human Review Needed

YES — This PR includes a new database migration (PlatformCostLog table), new admin API routes with auth gates, and cross-cutting changes across 53 files touching the executor, copilot, and frontend subsystems. The DRY violations in the async task infrastructure are architectural debt that a human reviewer should weigh in on before it calcifies.

Risk Assessment

Merge risk: LOW | Rollback: EASY — Feature is admin-only, fire-and-forget cost logging won't affect user-facing flows on failure, and the migration is additive (new table, no schema changes to existing tables).

CI Status

⚠️ Local checks showed failures in frontend lint/typecheck/build, but these appear environment-related. The PR's own GitHub CI passed all 48/48 checks with 94.62% patch coverage. Backend lint passed locally.

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:03Z

+        try:
+            # Access undocumented _response attribute — same pattern as
+            # extract_openrouter_cost() in blocks/llm.py.
+            cost_header = response._response.headers.get("x-total-cost")  # type: ignore[attr-defined]


🤖 🟠 high (security/Fragile private API access)

Accesses undocumented response._response.headers to extract OpenRouter cost. If the OpenAI SDK changes internals, cost tracking silently breaks. Unlike the extract_openrouter_cost() function in blocks/llm.py, this code accesses the full chain response._response.headers.get(...) in a single expression where response may be None (initialized on line 357). The except (AttributeError, ValueError) catches it, but the pattern is fragile.

Suggestion: Extract a shared utility (reuse extract_openrouter_cost from blocks/llm.py or a new shared module) so both code paths use the same guarded extraction logic with explicit AttributeError handling and a warning log.

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:03Z

@@ -0,0 +1,43 @@
+-- CreateTable
+CREATE TABLE "PlatformCostLog" (


🤖 🟡 medium (security/Missing Row-Level Security)

PlatformCostLog table is created without RLS policies. Direct database access (Supabase client, PostgREST, compromised service) would expose all cost logs including user IDs, execution IDs, and provider metadata.

Suggestion: Add RLS policies: enable RLS on the table, deny all by default, and grant SELECT only to the admin/service role. This matches the pattern used by other sensitive tables in the platform.

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:03Z

+    model: str | None = None
+    tracking_type: str | None = None
+    tracking_amount: float | None = None
+    metadata: dict[str, Any] | None = None


🤖 🟡 medium (security/Unbounded metadata payload)

PlatformCostEntry.metadata accepts dict[str, Any] with no size validation. A buggy block or manipulated input could insert arbitrarily large JSONB payloads, causing storage bloat.

Suggestion: Add a Pydantic validator to cap metadata size (e.g., max 10KB serialized) or limit the number of keys.

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:03Z

+    or renames that attribute, the warning is visible in logs rather than
+    silently degrading to no cost tracking.
+    """
+    try:


🤖 🟡 medium (security/Duplicate cost extraction logic)

extract_openrouter_cost() accesses response._response (private OpenAI SDK attribute). This same logic is duplicated in baseline/service.py:442 with slightly different error handling. If one is fixed/updated, the other may be forgotten.

Suggestion: Move to a shared utility module (e.g., backend/util/openrouter.py) so both consumers use one implementation.

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:03Z

+    try:
+        async with _log_semaphore:
+            await log_platform_cost(entry)
+    except Exception:


🤖 🟢 low (security/Silent cost data loss)

log_platform_cost_safe() silently swallows all exceptions. Under sustained DB failures, significant cost data would be lost with no health signal beyond exception logs.

Suggestion: Consider adding a counter/metric (e.g., Prometheus gauge) for failed cost log writes so monitoring can detect systematic failures.

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:04Z

+                    : "-"}
+                </td>
+                <td className="px-3 py-2 text-xs text-muted-foreground">
+                  {log.graph_exec_id


🤖 🟢 low (product/UX: Truncated ID)

graph_exec_id is truncated to 8 chars with no way to see or copy the full value. Admins investigating a specific execution can't easily get the full ID.

Suggestion: Add a title attribute with the full ID, and/or wrap in a click-to-copy button. Consider linking to the execution detail page.

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:04Z

+                </td>
+                <td className="px-3 py-2 text-xs">
+                  {log.email ||
+                    (log.user_id ? String(log.user_id).slice(0, 8) : "-")}


🤖 🟢 low (product/UX: Truncated user ID)

User ID is truncated to 8 chars with no tooltip showing the full value. Makes it difficult to use the User ID filter with a value from the logs table.

Suggestion: Add title={log.user_id} to the td or wrap in a span with a tooltip so admins can copy the full user ID to paste into the filter.

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:04Z

+    data: logsResponse,
+    isLoading: logsLoading,
+    error: logsError,
+  } = useGetV2GetPlatformCostLogs(


🤖 🟢 low (product/Performance: unnecessary fetch)

The logs API query fires on every filter change regardless of which tab is active. This wastes bandwidth and backend resources when the admin is on the Provider or User tab.

Suggestion: Conditionally enable the logs query only when tab === 'logs' using React Query's enabled option: { query: { enabled: tab === 'logs', select: okData } }

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:04Z

+            {pagination.total_items} total)
+          </span>
+          <div className="flex gap-2">
+            <button


🤖 🟢 low (product/Accessibility: pagination)

Previous/Next pagination buttons lack aria-labels describing their action and current page context.

Suggestion: Add aria-label={Go to page ${pagination.current_page - 1}} on Previous and aria-label={Go to page ${pagination.current_page + 1}} on Next.

autogpt-pr-reviewer-in-dev · 2026-04-08T10:40:04Z

            state.text_started = False
            state.text_block_id = str(uuid.uuid4())
    finally:
+        # Extract OpenRouter cost from response headers (in finally so we


🤖 🟢 low (discussion/Stale automated review)

CodeRabbit paused its review due to rapid commits — the final code state (after all fixes) did not receive a full automated re-scan. Individual fix confirmations were provided per-thread, but a comprehensive re-review of interactions between fixes was not performed.

Suggestion: Consider triggering a fresh CodeRabbit review on the merged code if any regressions are suspected.

autogpt-pr-reviewer-in-dev

📋 Automated Review — PR #12696

PR #12696 — feat(platform/admin): cost tracking for system credentials
Author: majdyz | Files: 53

🎯 Verdict: REQUEST_CHANGES

PR Description Quality

✅ Has Why + What + How — comprehensive description covering motivation (reconcile platform expenses), implementation details (fire-and-forget logging, admin API, dashboard UI), and test plan with E2E verification.

What This PR Does

Adds end-to-end cost tracking for blocks that use system-managed API credentials (OpenRouter, Exa, etc.) and copilot modes. When a block executes using a system credential, the provider's reported cost (or a token/character-based estimate) is logged to a new PlatformCostLog table. An admin-only dashboard with three tabs (By Provider, By User, Raw Logs) lets platform operators view aggregated costs, filter by date/provider/user, and apply custom rate overrides for estimation. This is an admin-only, best-effort (fire-and-forget) system — cost logging failures never block block execution.

Specialist Findings

🛡️ Security ✅ — Admin routes properly gated with requires_admin_user dependency. SQL queries use parameterized values for all user-supplied inputs. Auth negative tests verify 403/401 for non-admin and unauthenticated users.

🟠 response._response private attribute access in blocks/llm.py:787 and copilot/baseline/service.py:441 — if OpenAI SDK changes internals, cost tracking silently breaks with only a debug log, potentially allowing unbilled usage to accumulate. (Flagged by: security, architect, quality, testing — 4 specialists)
🟡 platform_cost.py:230 — SQL queries use f-string interpolation for LIMIT/OFFSET constants. Safe today (hardcoded integers) but fragile pattern for future edits. (Flagged by: security, architect — 2 specialists)
🟡 platform_cost_routes.py:64 — user_id query param accepts any string without UUID validation. Low risk (admin-only, parameterized in SQL) but defense-in-depth is missing.

🏗️ Architecture ⚠️ — Clean separation of concerns overall (blocks → stats → cost_tracking → platform_cost → routes → frontend). However, significant DRY violations in the async task infrastructure.

🟠 Duplicated fire-and-forget task management (cost_tracking.py:43-57 and token_tracking.py:27-43) — identical _pending_log_tasks, lock, semaphore dict, getter, and scheduling functions copy-pasted across modules. (Flagged by: architect, quality — 2 specialists)
🟠 Cross-module private symbol import (cost_tracking.py:9-12) — drain_pending_cost_logs() reaches into token_tracking._pending_log_tasks and _pending_log_tasks_lock. If token_tracking restructures internals, this silently breaks. (Flagged by: architect, quality — 2 specialists)
🟡 platform_cost.py:82 — Module-level asyncio.Semaphore(50) bound to import-time event loop. Safe in single-loop REST server, but latent bug if ever called from executor worker threads.
🟡 CostLogRow.block_name typed as str (non-optional) but schema has blockName String? (nullable) — Pydantic validation error if null row encountered.

⚡ Performance ✅ — Well-designed with semaphore-bounded fire-and-forget writes (cap of 50 concurrent), 30s TTL cache on dashboard, proper composite indexes on (userId, createdAt), (provider, createdAt), etc.

🟠 usePlatformCostContent.ts:51-66 — Both dashboard and logs API calls fire on every render regardless of active tab, doubling backend load per page view. Fix: add enabled condition based on active tab.
🟡 cost_tracking.py:48 — _log_semaphores dict keyed by event loop never cleans up entries for closed loops (minor memory leak in long-running processes).

🧪 Testing ⚠️ — ~75% estimated new code coverage with strong backend route/auth tests and thorough LLM cost extraction boundary tests (inf, nan, negative). However, notable gaps exist.

🟠 block_cost_tracking_test.py and exa/cost_tracking_test.py — ~90% overlap testing the same Exa blocks with slightly different mocking. One should be consolidated. (Flagged by: testing, quality — 2 specialists)
🟠 No negative test verifying log_system_credential_cost is NOT called on FAILED status — the guard in manager.py is untested for non-COMPLETED paths.
🟡 actions.test.ts — only tests happy path + non-200; no test for fetch() throwing (network error/timeout).
🟡 Frontend patch coverage at 66.84%, below the 80% project target.

📖 Quality ⚠️ — Good naming throughout, proper hook separation (usePlatformCostContent.ts), helpers.ts for pure functions. Main issues are the duplicated OpenRouter cost extraction and cross-module encapsulation violation.

🟠 OpenRouter x-total-cost header extraction duplicated verbatim between blocks/llm.py:786-803 and copilot/baseline/service.py:435-444. Should be a shared utility. (Flagged by: security, architect, quality, testing — 4 specialists)
🟡 PlatformCostContent.tsx at 234 lines, above the 200-line frontend guideline. Filter bar is a natural extraction candidate.
🔵 ProviderTable.tsx, LogsTable.tsx, UserTable.tsx — use export { X } at bottom instead of export function X at declaration, inconsistent with codebase convention.

📦 Product ✅ — Feature-complete for an initial admin tool. Dashboard loads with summary cards, three working tabs, URL-based filter state for bookmarkability, proper accessibility foundations (ARIA roles, labels).

🟡 Rate overrides are session-only with a tiny "(unsaved)" label — admins adjusting 20+ providers lose all changes on refresh.
🟡 No CSV/JSON export despite PR goal mentioning "reconcile expenses with actual API invoices."
🟡 Provider filter is free-text requiring exact lowercase match; no dropdown from available providers.

📬 Discussion ⚠️ — 3 human reviewer comments from @Pwuts are completely unaddressed. Bot feedback was largely addressed across 90+ commits. No valid human approval exists (ntindle's was dismissed).

🟠 @Pwuts: db_accessors.py — "unconditional wrapper doesn't add value" — no response from author
🟠 @Pwuts: platform_cost.py — "Must this be a raw query? Should be done through Prisma" — no response from author
🟠 @Pwuts: model.py — "If there are 4 literal options, it should be typed as such" — no response from author
🟡 No RLS policy or retention strategy on PlatformCostLog table (unbounded growth).

🔎 QA ✅ — All endpoints verified working: dashboard aggregations, logs with pagination/filtering, auth protection (403/401/422), email masking, input validation. Frontend renders all three tabs with real data. No PR-related errors in logs.

🟠 Should Fix

Respond to @Pwuts' review comments — 3 substantive human reviewer comments about raw queries vs Prisma, unnecessary DB accessor wrapper, and weak typing have no author response. These are from the project's maintainer and must be acknowledged before merge. (Flagged by: discussion)
Deduplicate OpenRouter cost extraction (copilot/baseline/service.py:435-444 + blocks/llm.py:786-803) — Same response._response.headers.get('x-total-cost') + math.isfinite + >= 0 pattern duplicated verbatim. Extract to a shared utility (e.g., backend/util/openrouter.py). This was flagged by 4 of 5 specialists as a real maintenance risk. (Flagged by: security, architect, quality, testing — 4 specialists)
Consolidate duplicate test files (block_cost_tracking_test.py:37-163 + exa/cost_tracking_test.py:16-107) — ~90% overlap testing the same Exa blocks. Consolidate into one location. (Flagged by: testing, quality — 2 specialists)
Add negative test for FAILED execution status (manager_cost_tracking_test.py) — No test verifies log_system_credential_cost is NOT called when execution status is FAILED. This invariant in manager.py is untested. (Flagged by: testing)
Conditionally fetch data based on active tab (usePlatformCostContent.ts:51-66) — Both dashboard and logs queries fire regardless of which tab is visible, doubling API load per page view. Add enabled: tab !== 'logs' / enabled: tab === 'logs' to the respective query options. (Flagged by: performance)

🟡 Nice to Have

Extract shared AsyncTaskPool utility (cost_tracking.py:43-57, token_tracking.py:27-43) — The pending-tasks set, lock, per-loop semaphore, and scheduling boilerplate is duplicated. A shared utility would eliminate divergent maintenance. (architect, quality)
Expose public drain function from token_tracking (cost_tracking.py:9-12) — Instead of importing private _pending_log_tasks and _pending_log_tasks_lock across module boundaries. (architect, quality)
Add UUID validation on user_id query param (platform_cost_routes.py:64) — Defense-in-depth for admin endpoints. (security)
Guard against nullable blockName (platform_cost.py:147) — CostLogRow.block_name: str vs schema blockName String?. (architect)
CSV/JSON export for cost reconciliation (PlatformCostContent.tsx) — PR goal mentions invoice reconciliation but no export exists. (product)
Add RLS policy and retention strategy on PlatformCostLog table. (discussion)

🔵 Nits

Export style inconsistency (ProviderTable.tsx:131, LogsTable.tsx:140, UserTable.tsx:75) — export { X } at bottom instead of export function X at declaration.
_mask_email reveals full local part for short emails (platform_cost.py:112) — ab@corp.com → ab***@corp.com.
Pagination buttons lack aria-label context (LogsTable.tsx:119-130) — Screen readers hear "Previous"/"Next" without knowing what's being paginated.

QA Screenshots

Screenshot	Description
	Admin dashboard loads with summary cards, By Provider tab showing cost breakdown ✅
	By User tab shows masked emails and per-user cost aggregation ✅
	Raw Logs tab displays individual cost entries with pagination ✅

Human Review Needed

YES — This PR adds a new database table (PlatformCostLog) with a Prisma migration, modifies auth-gated admin routes, touches the executor hot path with fire-and-forget tasks, and has 3 unaddressed comments from a project maintainer (@Pwuts). The DB migration and raw SQL patterns require human sign-off.

Risk Assessment

Merge risk: LOW | Rollback: EASY

Cost tracking is admin-only and best-effort (fire-and-forget). Failures in cost logging never block block execution. The new PlatformCostLog table is append-only with no foreign keys to execution tables, so dropping it has zero impact on core functionality. Admin routes are behind requires_admin_user — no user-facing exposure.

CI Status

❌ 2/6 local quality checks passed — frontend lint, typecheck, tests, and build failed (likely environment/dependency issues); backend lint passed, backend tests failed. Note: PR's GitHub CI shows 50/50 checks green, suggesting local check failures are environment-specific rather than code issues.

autogpt-pr-reviewer-in-dev

📋 Automated Review — PR #12696

PR #12696 — feat(platform/admin): cost tracking for system credentials
Author: majdyz | Files: 53

🎯 Verdict: REQUEST_CHANGES

PR Description Quality

✅ Has Why + What + How — Thorough description covering motivation (cost visibility for system credentials), architecture (fire-and-forget logging, microdollars storage, admin API + dashboard), and test evidence (3 E2E test reports posted in comments).

What This PR Does

Adds end-to-end platform cost tracking for system-credential API providers (OpenRouter, Exa, Google Maps, etc.) and copilot usage. Blocks instrument their costs at execution time via fire-and-forget async tasks that write to a new PlatformCostLog table. A new admin-only API serves aggregated dashboard data (by provider, by user) and paginated raw logs. The frontend adds a 3-tab admin dashboard at /admin/platform-costs with summary cards, filtering by provider/user/date, and client-side cost estimation for token-based providers.

Specialist Findings

🛡️ Security ✅ — Solid posture. Admin auth enforced at both backend (FastAPI Security(requires_admin_user)) and frontend (Supabase middleware) layers. All SQL is parameterized via $N placeholders. Cost values validated against NaN/Inf/negative before storage. PII masked via _mask_email(). Page sizes bounded (le=200).

🟡 _response._response access for cost headers is an operational risk (silent failure if SDK changes) but has proper fallback logging and test coverage.
🟡 Nullable userId on PlatformCostLog means deleted users' cost records become orphaned — acceptable for v1 but worth documenting.

🏗️ Architecture ⚠️ — Clean separation of concerns (block instrumentation → fire-and-forget task → DB write → admin API → dashboard). Dual-mode DB access via db_accessors works in both local and executor contexts. However, significant code duplication exists.

🟠 Duplicated fire-and-forget scheduling infrastructure between executor/cost_tracking.py and copilot/token_tracking.py — identical semaphore/task-set/lock/schedule pattern in ~80 lines each. (Flagged by: architect, quality — 2)
🟠 Duplicated OpenRouter cost extraction logic between blocks/llm.py:787 and copilot/baseline/service.py:439 — same _response._response.headers access with slightly different error handling. Baseline version lacks the TypeError catch. (Flagged by: architect, performance, quality — 3)
🟠 Fragile cross-module coupling: executor/cost_tracking.py:9 imports private _pending_log_tasks and _pending_log_tasks_lock from copilot/token_tracking.py. Any refactor of token_tracking's task management silently breaks executor drain. (Flagged by: architect, quality — 2)

⚡ Performance ✅ — Fire-and-forget cost logging adds O(1) overhead per block execution (single INSERT, semaphore-bounded at 50 concurrent). Dashboard has 30s TTL cache. Response sizes bounded by MAX_PROVIDER_ROWS=500 and MAX_USER_ROWS=100.

🟡 COUNT(*) on every paginated logs request without caching will degrade as PlatformCostLog grows past millions of rows (platform_cost.py:319).
🟡 OFFSET-based pagination (platform_cost.py:348) degrades at high page numbers — cursor-based would be O(1).
🟡 Three parallel GROUP BY queries scan overlapping rows; GROUPING SETS could consolidate to one pass.

🧪 Testing ⚠️ — ~3000 lines of new tests with ~80% estimated coverage. Route tests cover auth (401/403), validation (422), filters, and pagination. Unit tests cover cost parsing, NaN/Inf rejection, retry accumulation, and model stats merging. However:

🟠 ~500 lines of duplicate test code: block_cost_tracking_test.py (712 lines) and exa/cost_tracking_test.py (575 lines) test the same Exa blocks with nearly identical scenarios. (Flagged by: testing, quality — 2)
🟠 Frontend PlatformCostContent.test.tsx: 14 tests but zero user interaction tests — no fireEvent.click on tabs, filters, or pagination. Component could render correctly but be non-functional. (PlatformCostContent.test.tsx:1)
🟡 Mock fidelity concern: service_unit_test.py:653 mocks response._response via MagicMock which auto-creates any attribute — won't catch SDK renames.

📖 Quality ✅ — Readability A-. Clean naming, good docstrings on backend functions, consistent formatting. Frontend follows component structure conventions.

🟠 Three independent copies of OpenRouter header parsing with math.isfinite + >= 0 validation — see Architecture finding above.
🔵 Inconsistent export style: ProviderTable.tsx:131 and LogsTable.tsx:140 use bottom-of-file export { X } while PlatformCostContent.tsx uses export function. Per CONTRIBUTING.md, prefer export function.
🔵 Magic semaphore limit 50 hardcoded independently in cost_tracking.py:55 and token_tracking.py:40.

📦 Product ✅ — Solid v1 delivering complete admin cost visibility. Summary cards, 3-tab navigation (provider/user/logs), date+provider+user filtering, pagination, skeleton loading states, and error alerts all functional.

🟡 Truncated IDs in LogsTable.tsx:66,93 — user/execution IDs sliced to 8 chars with no tooltip or copy button; admins need full IDs for debugging.
🟡 Rate overrides are session-only with a tiny text-[10px] "(unsaved)" label (ProviderTable.tsx:47) — easy to miss.
🟡 Hardcoded provider cost rates in helpers.ts:7-49 require frontend deploy to update pricing.

📬 Discussion ⚠️ — All CI checks pass (50/52, 2 skipped non-required). All 9 CodeRabbit comments and 1 Sentry comment addressed with code fixes. However:

🟠 3 unaddressed comments from maintainer @Pwuts — no author response at all:
1. db_accessors.py: "unconditional wrapper doesn't add value" — pattern consistency question
2. platform_cost.py: "Must this be a raw query? Should be done through Prisma" — architectural preference
3. model.py: "If there are 4 literal options, it should be typed as such" — type safety
⚠️ @Pwuts approved with "LGTM" 30 minutes after leaving these 3 comments, without receiving any response. The approval may expect post-merge follow-up.

🔎 QA ✅ — 15 test scenarios passed across API and UI: happy paths, filter combinations, pagination validation (422 for page=0, page_size=0, page_size=999), auth enforcement (401 no-auth, 403 non-admin), and all 3 dashboard tabs rendered with correct data.

🟠 Should Fix

Respond to @Pwuts' 3 open review comments (db_accessors.py, platform_cost.py, model.py) — The maintainer left substantive feedback about pattern consistency, raw SQL vs Prisma, and type safety. Even if the current approach is correct, a response explaining the rationale is needed. (Flagged by: discussion — 1)
Consolidate duplicate test files (blocks/block_cost_tracking_test.py:1 + blocks/exa/cost_tracking_test.py:1) — ~500 lines of identical Exa block test scenarios maintained in two files. Remove Exa tests from one file to eliminate the maintenance liability. (Flagged by: testing, quality — 2)
Add frontend interaction tests (PlatformCostContent.test.tsx:1) — 14 render-only tests with zero click/interaction coverage. Add at minimum: tab switching, Apply filter, and pagination navigation tests. Frontend patch coverage is 66.84%, below the 80% target. (Flagged by: testing — 1)
Extract shared OpenRouter cost extraction utility (blocks/llm.py:787, copilot/baseline/service.py:439) — Same _response._response.headers.get("x-total-cost") + validation logic duplicated with inconsistent error handling (baseline lacks TypeError catch). Single utility eliminates dual maintenance. (Flagged by: architect, performance, quality, security — 4)
Extract shared async task scheduling utility (executor/cost_tracking.py:43-57, copilot/token_tracking.py:27-42) — Identical semaphore+task-set+lock+schedule pattern duplicated across two modules. Extracting to a shared AsyncTaskPool also eliminates the private import coupling at cost_tracking.py:9. (Flagged by: architect, quality — 2)

🟡 Nice to Have

Cursor-based pagination for logs (platform_cost.py:348) — OFFSET pagination degrades at high page numbers. Keyset pagination on (createdAt, id) would be O(1) per page. (performance)
Cache for logs COUNT(*) (platform_cost.py:319) — Dashboard has 30s TTL but logs endpoint has none; a short TTL would reduce DB load during admin pagination. (performance)
Add title attr and click-to-copy for truncated IDs (LogsTable.tsx:66,93) — Admins need full IDs for debugging. (product)
Type provider_cost_type as Literal (model.py) — Per @Pwuts' comment; adds compile-time safety for tracking type strings. (discussion, quality)
Use WeakKeyDictionary for _log_semaphores (cost_tracking.py:48, token_tracking.py:34) — Prevents minor memory leak from dead event loop keys in long-running processes. (performance)

🔵 Nits

Inconsistent export style (ProviderTable.tsx:131, LogsTable.tsx:140) — Use export function declarations per CONTRIBUTING.md instead of bottom-of-file export { X }.
Hardcoded semaphore limit (cost_tracking.py:55, token_tracking.py:40) — Extract MAX_CONCURRENT_LOG_TASKS = 50 as a shared constant.
Token column header ambiguous (LogsTable.tsx:83) — X / Y format should be labeled "Tokens (In / Out)" for clarity.

QA Screenshots

Screenshot	Description
	Dashboard with summary cards and By Provider table showing exa/anthropic/openai rows ✅
	User cost breakdown tab renders correctly ✅
	Individual log entries table with pagination ✅

Human Review Needed

YES — This PR adds a DB migration (PlatformCostLog table), new admin auth routes, raw SQL queries, and touches 53 files across backend and frontend. The 3 unaddressed maintainer comments need author response. @Pwuts should confirm whether the open feedback is blocking or deferred.

Risk Assessment

Merge risk: LOW | Rollback: EASY — Cost tracking is additive (new table, new routes, new UI page). Fire-and-forget pattern means cost logging failures never impact block execution. Migration is forward-only but the table is new with no dependencies.

CI Status

❌ 2/6 local checks passed (backend lint ✅, frontend lint ❌, typecheck ❌, backend test ❌, frontend test ❌, frontend build ❌). Note: local check failures appear to be environment issues (0s runtime on 4 checks suggests setup failures, not code failures). GitHub CI shows 50/52 passing with 2 non-required skipped.

…a mismatch The autogpt-database-manager pod can run a stale Prisma client immediately after a schema migration (e.g. rolling deploy of PR #12696 that added PlatformCostLog). This caused every copilot token-tracking write to raise prisma.errors.DataError ('userId'/'metadata' field not found), which was caught by logger.exception() — firing Sentry events at ERROR level. Catch DataError specifically in both log_platform_cost_safe (platform_cost.py) and the _safe_log closure in token_tracking.py, and demote to WARNING so Sentry is not spammed during deploy windows. All other exceptions still escalate to ERROR/Sentry as before.

… DataError (#12713) ## Changes - Wrap `metadata` field in `SafeJson()` when calling `PrismaLog.prisma().create()` in `log_platform_cost` - Add `platform_cost_integration_test.py` with DB round-trip tests for the fix ## Why `PrismaLog.prisma().create()` was silently failing with a `DataError` because passing a plain Python `dict` to a `Json?`-typed Prisma field is not allowed: ``` DataError: Invalid argument type. `metadata` should be of type NullableJsonNullValueInput or Json ``` The error was swallowed silently by `logger.exception` in the background task, so **no rows ever landed in `PlatformCostLog`** — which is why the dev admin cost dashboard showed no data after #12696 was merged. ## How Wrap `entry.metadata` in `SafeJson()` (already used throughout the codebase, lives in `backend/util/json.py`) before passing it to the Prisma create call. `SafeJson` extends `prisma.Json`, sanitizes PostgreSQL-incompatible control characters, and handles Pydantic-model conversion. Add two integration tests in `platform_cost_integration_test.py` (following the `credit_integration_test.py` pattern) that write a record to a real DB and read it back — confirming both metadata round-trip and NULL metadata work correctly. ## Test plan - [x] Integration tests verify metadata persists/reads correctly via Prisma - [x] Unit tests updated: `isinstance(data["metadata"], Json)` confirms the field is wrapped - [x] Verified on dev executor pod: cost rows now appear in the admin dashboard after fix

sentry · 2026-04-10T01:58:24Z

Issues attributed to commits in this pull request

This pull request was merged and Sentry observed the following issues:

‼️ ValueError: <Token var=<ContextVar name='current_c... in app:local-behave:local
‼️ [SyntaxError: closing parenthesis ']' does not matc...](https://significant-gravitas.sentry.io/issues/7407588060/?referrer=github-pr-bot) in app:dev-behave:cloud
‼️ [SyntaxError: closing parenthesis ']' does not matc...](https://significant-gravitas.sentry.io/issues/7407588110/?referrer=github-pr-bot) in app:dev-behave:cloud
‼️ [SyntaxError: closing parenthesis ']' does not matc...](https://significant-gravitas.sentry.io/issues/7407588134/?referrer=github-pr-bot) in app:dev-behave:cloud
‼️ TypeError: ClaudeAgentOptions.init() got an un... in app:local-behave:local

majdyz added 30 commits April 2, 2026 15:42

feat(platform): add platform cost tracking for system credentials

1a6c50c

Track real API costs incurred when users consume system-managed credentials. Captures provider, tokens, duration, and model per block execution and surfaces an admin dashboard with provider/user aggregation and raw logs.

fix(platform): address PR review round 2

26b0c95

- Parallelize dashboard queries with asyncio.gather for ~3x speedup - Move json import to top-level - Use consistent p. table alias across all dashboard queries

fix(platform): address PR review round 3

072b623

- Remove duplicate block_usage_cost call from cost logging - Add case-insensitive provider filter using LOWER() - Add platform_cost_routes_test.py with basic endpoint tests

fix(platform): address PR review round 4

476d9dc

- Add tests for query parameter forwarding and pagination

fix(platform): address PR review round 5

f3b5d58

- Replace ServerCrash icon with Receipt for Platform Costs sidebar

fix: regenerate API schema and block docs

c054799

fix(platform): re-add credit_cost to platform cost log metadata

b933698

Include the block's credit cost (from block_cost_config) in the log metadata so every entry has a known cost proxy even when the provider doesn't expose actual dollar costs.

fix(backend): handle NULL userId in platform cost models and queries

7020ae2

Make user_id Optional[str] in UserCostSummary and CostLogRow to handle cases where the referenced user has been deleted. Use .get() for safe access to user_id from query result rows. Regenerate OpenAPI schema.

Merge branch 'dev' of github.com:Significant-Gravitas/AutoGPT into co…

a338c72

…dex/platform-cost-tracking

feat(platform): add merge_stats to remaining blocks (FAL, Revid, D-ID…

993c43b

…, E2B, YouTube, Weather, TTS, Enrichlayer) Every system credential block now has explicit merge_stats tracking. No block relies on the generic fallback anymore.

fix(platform): move opentelemetry import to top-level in both copilot…

9a08011

… paths

fix(platform): pass model name to copilot cost tracking

ab16e63

Both SDK and Baseline paths now pass config.model to persist_and_record_usage so PlatformCostLog records the actual model (e.g. anthropic/claude-sonnet-4) for filtering/grouping.

fix(backend): initialize response before try block to satisfy pyright

405403e

Merge branch 'dev' of github.com:Significant-Gravitas/AutoGPT into co…

61ef54a

…dex/platform-cost-tracking

fix(blocks): regenerate block docs after merge with dev

567bc73

Pwuts changed the title ~~feat(platform): platform cost tracking for system credentials~~ feat(platform/admin): cost tracking for system credentials Apr 8, 2026

majdyz added 2 commits April 8, 2026 16:48

Merge remote-tracking branch 'origin/codex/platform-cost-tracking' in…

fde3f6a

…to codex/platform-cost-tracking

Pwuts approved these changes Apr 8, 2026

View reviewed changes

github-project-automation Bot moved this from 🚧 Needs work to 👍🏼 Mergeable in AutoGPT development kanban Apr 8, 2026

majdyz added this pull request to the merge queue Apr 8, 2026

Merged via the queue into dev with commit ff8cdda Apr 8, 2026
41 checks passed

majdyz deleted the codex/platform-cost-tracking branch April 8, 2026 10:26

github-project-automation Bot moved this from 👍🏼 Mergeable to ✅ Done in AutoGPT development kanban Apr 8, 2026

github-project-automation Bot moved this to Done in Frontend Apr 8, 2026

autogpt-pr-reviewer-in-dev Bot approved these changes Apr 8, 2026

View reviewed changes

autogpt-pr-reviewer-in-dev Bot suggested changes Apr 8, 2026

View reviewed changes

github-project-automation Bot moved this from ✅ Done to 🚧 Needs work in AutoGPT development kanban Apr 8, 2026

autogpt-pr-reviewer-in-dev Bot suggested changes Apr 8, 2026

View reviewed changes

This was referenced Apr 9, 2026

feat(backend): add Graphiti temporal knowledge graph memory for CoPilot #12720

Merged

feat(platform/admin): per-model cost breakdown, cache token tracking, OrchestratorBlock cost fix #12726

Merged

This was referenced Apr 13, 2026

fix(copilot): baseline cost tracking and cache token display #12762

Merged

fix(copilot): strip <internal_reasoning> tags from Sonnet response stream #12763

Merged

[TMP] [TESTING] merge(preview): consolidated preview of all 14 active PRs #12773

Closed

		@@ -0,0 +1,43 @@
		-- CreateTable
		CREATE TABLE "PlatformCostLog" (

Uh oh!

Conversation

majdyz commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

How

Core hook

Per-block tracking

Copilot tracking

Executor drain

CoPilot executor lifecycle

Schema

Admin dashboard

Test plan

Uh oh!

Pwuts left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

autogpt-pr-reviewer-in-dev Bot left a comment

Choose a reason for hiding this comment

📋 Automated Review — PR #12696

PR Description Quality

What This PR Does

Specialist Findings

🟠 Should Fix

🟡 Nice to Have

🔵 Nits

Human Review Needed

Risk Assessment

CI Status

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

autogpt-pr-reviewer-in-dev Bot left a comment

Choose a reason for hiding this comment

📋 Automated Review — PR #12696

PR Description Quality

What This PR Does

Specialist Findings

🟠 Should Fix

🟡 Nice to Have

🔵 Nits

QA Screenshots

Human Review Needed

Risk Assessment

CI Status

Uh oh!

autogpt-pr-reviewer-in-dev Bot left a comment

Choose a reason for hiding this comment

📋 Automated Review — PR #12696

majdyz commented Apr 7, 2026 •

edited

Loading

sentry Bot commented Apr 10, 2026 •

edited

Loading