fix(block_cost_config): audit + correct stale LLM/block rates + migrate generic ReplicateModelBlock to COST_USD - #12912
Merged
Merged
Conversation
PR #12909 refresh set GPT-5 to 94/1500 cr/1M which corresponds to a $0.625/$5 provider rate — that's OpenAI's Batch API tier (50% off Sync). Most block calls go through the Sync API; the correct Standard rate is $1.25/$10 per 1M, which at our 1.5x margin = 188/1500 cr/1M. This was under-billing every GPT-5 call by 2x on input. Source: https://openai.com/api/pricing — GPT-5 Standard pricing.
github-project-automation
Bot
moved this to 🆕 Needs initial review
in AutoGPT development kanban
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughRecalibrated token and block credit rates across multiple LLM providers and blocks; expanded provider-specific LLM BlockCost mappings; switched several blocks from flat-run credits to USD-based billing and updated tests to match new per-second/per-unit rates. Replicate execution now uses predictions.async_create/async_wait and emits USD provider costs. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🔍 PR Overlap DetectionThis check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early. 🔴 Merge Conflicts DetectedThe following PRs have been tested and will have merge conflicts if merged after this PR. Consider coordinating with the authors.
🟢 Low Risk — File Overlap OnlyThese PRs touch the same files but different sections (click to expand)
Summary: 5 conflict(s), 0 medium risk, 2 low risk (out of 7 PRs with file overlap) Auto-generated on push. Ignores: |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #12912 +/- ##
==========================================
+ Coverage 68.21% 68.24% +0.03%
==========================================
Files 1959 1960 +1
Lines 149934 150048 +114
Branches 15606 15612 +6
==========================================
+ Hits 102273 102403 +130
+ Misses 44627 44610 -17
- Partials 3034 3035 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Beyond the GPT-5 Standard-vs-Batch fix, verified all TOKEN_COST entries against each provider's current pricing page. Additional corrections: - DEEPSEEK_CHAT: 42/63 -> 21/42 (provider unified deepseek-chat + deepseek-reasoner to deepseek-v4-flash $0.14/$0.28 in Sept 2025) - DEEPSEEK_R1_0528: 82/329 -> 21/42 (same v4-flash routing) - MISTRAL_LARGE_3: 300/900 -> 75/225 (Mistral dropped to $0.50/$1.50) - MISTRAL_NEMO: 3/6 -> 23/23 (was severely under-billing; provider is $0.15 flat for both input and output) - KIMI_K2_0905: 82/330 -> 90/375 (matches current K2-0905 $0.60/$2.50) - META_LLAMA_4_MAVERICK: 30/90 -> 75/116 (Groq prices $0.50/$0.77; note Groq deprecated this 2026-02-20 — consider retiring enum) Provider sources: openai.com/api/pricing, api-docs.deepseek.com, mistral.ai/pricing, platform.kimi.ai/docs/pricing, groq.com/pricing. Cross-verified via agent-browser for JS-rendered docs.x.ai + DeepSeek. All 40 cost-pipeline unit tests pass.
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Full audit against provider pricing pages uncovered 10 more stale entries beyond the LLM token rates: Under-billing (was losing money): - AIVideoGeneratorBlock (FAL): SECOND 3 -> 15 cr/s (provider is $0.05-$0.30/s depending on tier; 3 cr only covered $0.02/s models) - CreateTalkingAvatarVideoBlock (D-ID): RUN 15 -> 100 cr (D-ID charges $5.90/min; 15 cr was ~10x under for a median 10-sec clip at $0.98 real cost) - Nano Banana Pro / Nano Banana 2 (3 blocks each): RUN 14 -> 21 cr (provider $0.14/image, 14 cr was under cost-of-goods) Over-billing (normalizing margin to 1.5x baseline): - IdeogramModelBlock default: RUN 16 -> 12 cr - IdeogramModelBlock V_3: RUN 18 -> 14 cr - AIImageEditorBlock FLUX_KONTEXT_MAX: RUN 20 -> 12 cr - ValidateEmailsBlock (ZeroBounce): COST_USD 250 -> 150 cr/$ - SearchTheWebBlock (Jina): COST_USD 100 -> 150 cr/$ - GetLinkedinProfilePictureBlock: RUN 3 -> 1 cr Tests updated to match new FAL 15 cr/s rate (was 3 cr/s in 2 tests). Sources: replicate.com, fal.ai, d-id.com, ideogram.ai, zerobounce.net, jina.ai. Cross-verified via agent-browser for JS-rendered docs.x.ai (Grok prices already correct at 300/900 for Grok 4.20 @ $2/$6).
majdyz
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
autogpt_platform/backend/backend/data/block_cost_config.py (1)
1227-1243:⚠️ Potential issue | 🟡 MinorFAL per-second rate bump from 3 → 15 cr/s is a 5× user-facing price increase — confirm it's intentional.
Per prior learnings,
AIVideoGeneratorBlockis walltime-billed (FAL SDK doesn't surfaceprovider_cost/ output duration), so the credit cost scales linearly with whatever wall-clock time the block spends. The comment's justification (Lite tier$0.10/s @ 1.5×) is reasonable for the slow tiers, but fast Lite-tier calls that previously charged ~3 cr will now charge ~15 cr even when the actual provider cost is well under that. That's a significant live-customer price change that deserves a release note / changelog entry separate from the GPT-5 fix.Code-wise, the change and both test updates (
75cr @ 5s,120cr @ 8s) are internally consistent — no bug. Just flagging the pricing impact for sign-off.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/data/block_cost_config.py` around lines 1227 - 1243, AIVideoGeneratorBlock's per-second cost was increased from 3→15 (BlockCost(cost_amount=15, cost_type=BlockCostType.SECOND) with fal_credentials filter), which is a 5× user-facing price change — confirm this is intentional with product/finance and either revert to previous value or get sign-off; after sign-off, add a clear release-note/changelog entry describing the pricing change and rationale, and add an in-code comment next to AIVideoGeneratorBlock referencing the changelog entry and the fal_credentials-based walltime billing behavior so reviewers see this is deliberate; if reverting, update the tests that expect 75cr@5s and 120cr@8s accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@autogpt_platform/backend/backend/data/block_cost_config.py`:
- Around line 1227-1243: AIVideoGeneratorBlock's per-second cost was increased
from 3→15 (BlockCost(cost_amount=15, cost_type=BlockCostType.SECOND) with
fal_credentials filter), which is a 5× user-facing price change — confirm this
is intentional with product/finance and either revert to previous value or get
sign-off; after sign-off, add a clear release-note/changelog entry describing
the pricing change and rationale, and add an in-code comment next to
AIVideoGeneratorBlock referencing the changelog entry and the
fal_credentials-based walltime billing behavior so reviewers see this is
deliberate; if reverting, update the tests that expect 75cr@5s and 120cr@8s
accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d8d5ce75-26c2-4c95-82ce-f3d3db924e52
📒 Files selected for processing (3)
autogpt_platform/backend/backend/data/block_cost_config.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/executor/block_usage_cost_test.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
- GitHub Check: check API types
- GitHub Check: end-to-end tests
- GitHub Check: test (3.13)
- GitHub Check: type-check (3.12)
- GitHub Check: test (3.12)
- GitHub Check: test (3.11)
- GitHub Check: type-check (3.13)
- GitHub Check: type-check (3.11)
- GitHub Check: Seer Code Review
- GitHub Check: Analyze (typescript)
- GitHub Check: Analyze (python)
- GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (5)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/backend/**/*_test.py
📄 CodeRabbit inference engine (autogpt_platform/backend/AGENTS.md)
autogpt_platform/backend/**/*_test.py: Use pytest with snapshot testing for API responses
Colocate test files with source files using*_test.pynaming convention
Mock at boundaries — mock where the symbol is used, not where it's defined; after refactoring, update mock targets to match new module paths
UseAsyncMockfromunittest.mockfor async functions in tests
When writing tests, use Test-Driven Development (TDD): write failing tests marked with@pytest.mark.xfailbefore implementation, then remove the marker once the implementation is complete
When creating snapshots in tests, usepoetry run pytest path/to/test.py --snapshot-update; always review snapshot changes withgit diffbefore committing
Files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.py
autogpt_platform/backend/backend/data/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
All data access in backend requires user ID checks; verify this for any 'data/*.py' changes
Files:
autogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/**/data/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
For changes touching
data/*.py, validate user ID checks or explain why not needed
Files:
autogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
🧠 Learnings (18)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-23T00:07:27.117Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/openrouter_cost.py`, the `openrouter-cost-reconcile` Langfuse event carries `cost_source` ("openrouter" for authoritative OpenRouter-resolved cost, "fallback" for rate-card fallback) and `resolved_generation_id_count` alongside the reconciled cost and token/model/provider metadata. This lets operators distinguish authoritative reconciliations from fallbacks in Langfuse. Established in PR `#12889` commit 5ce3d0388.
📚 Learning: 2026-04-23T13:55:24.409Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T13:53:29.246Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-22T04:01:32.723Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T12:55:26.122Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T13:53:40.315Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-15T15:30:09.706Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-15T02:43:36.890Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-22T11:46:04.431Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/config.py:0-0
Timestamp: 2026-04-22T11:46:04.431Z
Learning: Do not flag the Claude Sonnet 4.6 model ID as incorrect when it uses the project’s established hyphenated convention: `anthropic/claude-sonnet-4-6`. This hyphen form is the intentional, production convention and should be treated as valid (including in files like llm.py, blocks tests, reasoning.py, `_is_anthropic_model` tests, and config defaults). Note that OpenRouter also accepts the dot variant `anthropic/claude-sonnet-4.6`, so either form may be tolerated, but `anthropic/claude-sonnet-4-6` should be considered the standard to match project usage.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-22T11:46:12.892Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/baseline/service.py:322-332
Timestamp: 2026-04-22T11:46:12.892Z
Learning: In this codebase (Significant-Gravitas/AutoGPT), OpenRouter-routed Anthropic model IDs should use the hyphen-separated convention (e.g., `anthropic/claude-sonnet-4-6`, `anthropic/claude-opus-4-6`). Although OpenRouter may accept both hyphen and dot variants, treat the hyphen-separated form as the intended, correct codebase-wide convention and do not flag it as an error. Only flag the dot-separated variant (e.g., `anthropic/claude-sonnet-4.6`) as incorrect when reviewing/validating model ID strings for OpenRouter-routed Anthropic models.
Applied to files:
autogpt_platform/backend/backend/executor/block_usage_cost_test.pyautogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-21T04:35:34.710Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12865
File: autogpt_platform/backend/backend/data/credit.py:1584-1584
Timestamp: 2026-04-21T04:35:34.710Z
Learning: When reviewing this codebase, don’t flag snake_case attribute names (e.g., `subscription_tier`, `stripe_customer_id`, `top_up_config`) on the app-layer Pydantic `User` model as “wrong” field names. These are correct for the app-layer model and are expected to be mapped from the Prisma-layer camelCase fields (e.g., `subscriptionTier`, `stripeCustomerId`) inside methods like `User.from_db()`. Only Prisma-returned/raw objects would use camelCase, but functions like `get_user_by_id(user_id: str)` are expected to return the Pydantic app-layer model.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config_test.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T00:07:27.117Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-23T00:07:27.117Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/openrouter_cost.py`, the `openrouter-cost-reconcile` Langfuse event carries `cost_source` ("openrouter" for authoritative OpenRouter-resolved cost, "fallback" for rate-card fallback) and `resolved_generation_id_count` alongside the reconciled cost and token/model/provider metadata. This lets operators distinguish authoritative reconciliations from fallbacks in Langfuse. Established in PR `#12889` commit 5ce3d0388.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-09T10:50:43.907Z
Learnt from: Bentlybro
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-03-09T10:50:43.907Z
Learning: Repo: Significant-Gravitas/AutoGPT — File: autogpt_platform/backend/backend/blocks/llm.py
For xAI Grok models accessed via OpenRouter, the API returns `null` for `max_completion_tokens`. The convention in this codebase is to use the model's context window size as the `max_output_tokens` value in ModelMetadata. For example, Grok 3 uses 131072 (128k) and Grok 4 uses 262144 (256k). Do not flag these as incorrect max output token values.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-08T17:27:07.646Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: classic/forge/CLAUDE.md:0-0
Timestamp: 2026-04-08T17:27:07.646Z
Learning: Applies to classic/forge/**/forge/llm/providers/**/*.py : Supported model names include OpenAI (GPT3, GPT3_16k, GPT4, GPT4_32k, GPT4_TURBO, GPT4_O), Anthropic (CLAUDE3_OPUS, CLAUDE3_SONNET, CLAUDE3_HAIKU, CLAUDE3_5_SONNET, CLAUDE3_5_SONNET_v2, CLAUDE3_5_HAIKU, CLAUDE4_SONNET, CLAUDE4_OPUS, CLAUDE4_5_OPUS), and Groq (LLAMA3_8B, LLAMA3_70B, MIXTRAL_8X7B)
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
🔇 Additional comments (5)
autogpt_platform/backend/backend/data/block_cost_config.py (3)
278-283: GPT-5 rate correction looks right.
input=188, output=1500matches OpenAI Standard (Sync) API pricing of$1.25 / $10 per 1Mat the file's documented150 cr/$ = 1.5× marginconvention (line 241-245), and is now consistent withGPT5_1above. The previous94/…implied Batch input pricing, which under-billed Sync-mode calls (the only path used by these blocks).
570-584: No action required —SearchTheWebBlockcorrectly emitsprovider_cost=0.01withprovider_cost_type="cost_usd"(lines 76–78 ofsearch.py), confirming theCOST_USDbilling path will function as designed. The 150 cr/$ multiplier matches the documented 1.5x margin baseline.
310-340: Collateral TOKEN_COST re-prices confirmed accurate against provider sources.Spot-checked the TOKEN_COST rows:
- GPT-5
188/1500:$1.25/$10.00 per 1M(Sync API, April 2026) — matches. ✓- DeepSeek
21/42:$0.14/$0.28 per 1M(deepseek-v4-flash, Sept 2025 unified pricing) — matches. ✓- Mistral Large 3
75/225:$0.50/$1.50 per 1M(confirmed via OpenRouter) — matches. ✓- Mistral Nemo, Kimi K2_0905, Meta Llama 4 Maverick, Perplexity Sonar tiers: consistent with surrounding family entries and the documented 1.5× margin over published rates.
The TOKEN_COST entries apply a uniform 1.5× margin over provider pricing, with de-marginization via
1 credit ≈ $0.01(lines 244–245). All spot-checked values are correct.Note: The PR description advertises a "one-line change" but the diff actually touches multiple TOKEN_COST rows (GPT-5, DeepSeek, Mistral, Cohere, Kimi, Perplexity, Meta Llama) and other BLOCK_COSTS entries — worth clarifying in the merge-log description for transparency, but the numerical values are correct.
autogpt_platform/backend/backend/data/block_cost_config_test.py (1)
143-161: Test update matches the new 15 cr/s rate.Math checks out: 5s × 15 cr/s = 75 credits post-flight; pre-flight remains 0. Docstring correctly reflects
cost_amount=15.autogpt_platform/backend/backend/executor/block_usage_cost_test.py (1)
200-214: Test update is consistent with the FAL rate change.
8s × 15 cr/s = 120credits; pre-flight remains 0. Function name and comment correctly reflect the new rate. No other assertions in this file depend on FAL pricing.
…k legacy doc - KIMI_K2_5: 90/450 -> 66/300 (OpenRouter pass-through $0.44/$2) - KIMI_K2_6: 143/600 -> 112/698 (OpenRouter pass-through $0.7448/$4.655) - UnrealTextToSpeechBlock: RUN 5 cr -> COST_USD 150 cr/$. Block now computes USD from len(text) * $0.000016 (Unreal Speech $16/1M chars) and emits cost_usd via merge_stats. Long narrations no longer under-bill. - Grok legacy (grok-3, grok-4-0709, grok-4-fast, grok-code-fast-1): rates were already correct at their launch pricing; added inline comment noting the docs.x.ai page no longer lists them publicly but the API + historical rates remain valid.
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
autogpt_platform/backend/backend/data/block_cost_config.py (1)
712-733:⚠️ Potential issue | 🟡 MinorInconsistent Nano Banana pricing across blocks —
AIImageEditorBlockstill charges 14 cr whileAIImageGeneratorBlock/AIImageCustomizerBlockwere raised to 21 cr.The audit raised Nano Banana Pro / Nano Banana 2 from 14 → 21 in
AIImageGeneratorBlock(lines 928, 939) andAIImageCustomizerBlock(lines 963, 974) with the justification$0.14/image at 2K @ 1.5x margin. The same underlying Replicate models are wired here at cost_amount=14 — roughly the old 1.0× rate, so they're under-billed by ~33% relative to the rest of the PR's 1.5× baseline.🛠 Suggested alignment
BlockCost( - cost_amount=14, # Nano Banana Pro + cost_amount=21, # Nano Banana Pro: $0.14/image @ 1.5x margin cost_filter={ "model": FluxKontextModelName.NANO_BANANA_PRO, ... }, ), BlockCost( - cost_amount=14, # Nano Banana 2 + cost_amount=21, # Nano Banana 2: same pricing tier as Pro cost_filter={ "model": FluxKontextModelName.NANO_BANANA_2, ... }, ),🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/data/block_cost_config.py` around lines 712 - 733, The BlockCost entries for Nano Banana models are underpriced at cost_amount=14; update the BlockCost instances that use FluxKontextModelName.NANO_BANANA_PRO and FluxKontextModelName.NANO_BANANA_2 (the ones referencing replicate_credentials.id/provider/type) to cost_amount=21 so the AIImageEditorBlock pricing aligns with the AIImageGeneratorBlock and AIImageCustomizerBlock changes; keep the same cost_filter structure and replicate_credentials references unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@autogpt_platform/backend/backend/data/block_cost_config.py`:
- Around line 712-733: The BlockCost entries for Nano Banana models are
underpriced at cost_amount=14; update the BlockCost instances that use
FluxKontextModelName.NANO_BANANA_PRO and FluxKontextModelName.NANO_BANANA_2 (the
ones referencing replicate_credentials.id/provider/type) to cost_amount=21 so
the AIImageEditorBlock pricing aligns with the AIImageGeneratorBlock and
AIImageCustomizerBlock changes; keep the same cost_filter structure and
replicate_credentials references unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 2758ec89-9e16-4690-a629-5341769a155a
📒 Files selected for processing (2)
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: check API types
- GitHub Check: Seer Code Review
- GitHub Check: test (3.13)
- GitHub Check: test (3.12)
- GitHub Check: test (3.11)
- GitHub Check: end-to-end tests
- GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (5)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/backend/backend/blocks/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/backend/blocks/**/*.py: Inherit from 'Block' base class with input/output schemas when adding new blocks in backend
Implement 'run' method with proper error handling in backend blocks
Generate block UUID using 'uuid.uuid4()' when creating new blocks in backend
Write tests alongside block implementation when adding new blocks in backend
autogpt_platform/backend/backend/blocks/**/*.py: For blocks handling files, usestore_media_file()withreturn_format="for_local_processing"when processing with local tools (ffmpeg, MoviePy, PIL)
For blocks handling files, usestore_media_file()withreturn_format="for_external_api"when sending content to external APIs (Replicate, OpenAI)
For blocks returning files, usestore_media_file()withreturn_format="for_block_output"to enable auto-adaptation to execution context (workspace:// in CoPilot, data URI in graphs)
When creating new blocks, inherit fromBlockbase class, define input/output schemas usingBlockSchema, implement asyncrunmethod, and generate unique block ID usinguuid.uuid4()
Files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/backend/backend/data/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
All data access in backend requires user ID checks; verify this for any 'data/*.py' changes
Files:
autogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/**/data/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
For changes touching
data/*.py, validate user ID checks or explain why not needed
Files:
autogpt_platform/backend/backend/data/block_cost_config.py
🧠 Learnings (24)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-23T00:07:27.117Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/openrouter_cost.py`, the `openrouter-cost-reconcile` Langfuse event carries `cost_source` ("openrouter" for authoritative OpenRouter-resolved cost, "fallback" for rate-card fallback) and `resolved_generation_id_count` alongside the reconciled cost and token/model/provider metadata. This lets operators distinguish authoritative reconciliations from fallbacks in Langfuse. Established in PR `#12889` commit 5ce3d0388.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12566
File: autogpt_platform/frontend/src/lib/autogpt-server-api/types.ts:968-974
Timestamp: 2026-03-26T00:32:06.673Z
Learning: In Significant-Gravitas/AutoGPT, the admin-facing methods in `autogpt_platform/frontend/src/lib/autogpt-server-api/client.ts` (e.g., `addUserCredits`, `getUsersHistory`, `getUserRateLimit`, `resetUserRateLimit`) intentionally follow the legacy `BackendAPI` pattern with manually defined types in `autogpt_platform/frontend/src/lib/autogpt-server-api/types.ts`. Migrating these admin endpoints to the generated OpenAPI hooks (`@/app/api/__generated__/endpoints/`) is a planned separate effort covering all admin endpoints together, not done piecemeal per PR. Do not flag individual admin type additions in `types.ts` as blocking issues.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12536
File: autogpt_platform/frontend/src/app/api/openapi.json:5770-5790
Timestamp: 2026-03-24T21:25:15.983Z
Learning: Repo: Significant-Gravitas/AutoGPT — PR `#12536`
File: autogpt_platform/frontend/src/app/api/openapi.json
Learning: The OpenAPI spec file is auto-generated; per established convention, endpoints generally declare only 200/201, 401, and 422 responses. Do not suggest adding explicit 403/404 response entries for single operations unless planning a repo-wide spec update. Prefer clarifying such behaviors in endpoint descriptions/docstrings instead of altering response maps.
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12284
File: autogpt_platform/frontend/src/app/api/openapi.json:11897-11900
Timestamp: 2026-03-04T23:58:18.476Z
Learning: Repo: Significant-Gravitas/AutoGPT — PR `#12284`
Backend/frontend OpenAPI codegen convention: In backend/api/features/store/model.py, the StoreSubmission and StoreSubmissionAdminView models define submitted_at: datetime | None, changes_summary: str | None, and instructions: str | None with no default. This is intentional to produce “required but nullable” fields in OpenAPI (properties appear in required[] and use anyOf [type, null]). This matches Prisma’s submittedAt DateTime? and changesSummary String?. Do not flag this as a required/nullable mismatch.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
📚 Learning: 2026-04-23T13:53:29.246Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-22T04:01:32.723Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T13:55:24.409Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T12:55:26.122Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: Cost billing via the cost(*costs) decorator is applied at input-evaluation time (before a block’s run() executes). Therefore, mutating input_data inside run() will not change billing. When a block’s billing depends on a field plus URL/sniff-derived signals, treat the explicitly declared billing field (e.g., is_video) as the only billing source—set it correctly before run() (or in the code path that occurs before the decorator evaluates input_data). This should be checked for all blocks under autogpt_platform/backend/backend/blocks/ so billing signals are not mistakenly assumed to update during run().
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.py
📚 Learning: 2026-04-23T13:53:40.315Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-02-05T04:11:00.596Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 11796
File: autogpt_platform/backend/backend/blocks/video/concat.py:3-4
Timestamp: 2026-02-05T04:11:00.596Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, when creating a new block, generate a UUID once with uuid.uuid4() and hard-code the resulting string as the block's id parameter. Do not call uuid.uuid4() at runtime; IDs must be constant across all imports and runs to ensure stability.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.py
📚 Learning: 2026-03-16T16:32:21.686Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:21.686Z
Learning: In autogpt_platform/backend/backend/blocks/, the Block base class execute() already wraps run() in a try/except to convert uncaught exceptions into BlockExecutionError/BlockUnknownError. Do not add per-block try/except in individual block run() methods, as this is not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Only use explicit try/except within blocks that need to distinguish between success and error yield paths inside a generator (e.g., attachment blocks). This guidance applies to all Python files under autogpt_platform/backend/backend/blocks/ and similar block implementations; avoid duplicating error handling in run() unless a block requires generator-based branching.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: In autogpt_platform/backend/backend/blocks/ (and related blocks under autogpt_platform/backend/backend/blocks/), do not add try/except blocks around a block's run() method for standard error propagation. The block executor framework (backend/executor/manager.py) catches uncaught exceptions from run() and emits them on the 'error' output. Only add explicit try/except blocks when you need to control partial outputs in failure cases (e.g., certain outputs must not be yielded on error, as in attachment blocks). This is the standard pattern across the codebase; apply it broadly to blocks' run() implementations.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.py
📚 Learning: 2026-03-16T16:30:23.196Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:23.196Z
Learning: In any Python file under autogpt_platform/backend/backend/blocks, do not add a try/except around run() solely for standard error handling. The block framework’s _execute() in _base.py already catches unhandled exceptions and re-raises as BlockExecutionError or BlockUnknownError. If you yield ("error", message), _execute() raises BlockExecutionError immediately, so the error port will not propagate downstream. Reserve explicit try/except for scenarios where you must control partial output (e.g., attachment blocks that must skip yielding content_base64 on failure).
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: Do not wrap synchronous AgentMail SDK calls with asyncio.to_thread() in blocks under autogpt_platform/backend/backend/blocks (and across the codebase). The block executor runs node execution in dedicated threads via asyncio.run_coroutine_threadsafe (see manager.py around lines ~745-752 and ~1079). The existing pattern avoids using asyncio.to_thread for SDK calls inside async run() methods, so maintain that approach and do not add to_thread usage in these code paths.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-15T02:43:36.890Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-22T11:46:04.431Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/config.py:0-0
Timestamp: 2026-04-22T11:46:04.431Z
Learning: Do not flag the Claude Sonnet 4.6 model ID as incorrect when it uses the project’s established hyphenated convention: `anthropic/claude-sonnet-4-6`. This hyphen form is the intentional, production convention and should be treated as valid (including in files like llm.py, blocks tests, reasoning.py, `_is_anthropic_model` tests, and config defaults). Note that OpenRouter also accepts the dot variant `anthropic/claude-sonnet-4.6`, so either form may be tolerated, but `anthropic/claude-sonnet-4-6` should be considered the standard to match project usage.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-22T11:46:12.892Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/baseline/service.py:322-332
Timestamp: 2026-04-22T11:46:12.892Z
Learning: In this codebase (Significant-Gravitas/AutoGPT), OpenRouter-routed Anthropic model IDs should use the hyphen-separated convention (e.g., `anthropic/claude-sonnet-4-6`, `anthropic/claude-opus-4-6`). Although OpenRouter may accept both hyphen and dot variants, treat the hyphen-separated form as the intended, correct codebase-wide convention and do not flag it as an error. Only flag the dot-separated variant (e.g., `anthropic/claude-sonnet-4.6`) as incorrect when reviewing/validating model ID strings for OpenRouter-routed Anthropic models.
Applied to files:
autogpt_platform/backend/backend/blocks/text_to_speech_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T12:55:26.122Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-09T10:50:43.907Z
Learnt from: Bentlybro
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-03-09T10:50:43.907Z
Learning: Repo: Significant-Gravitas/AutoGPT — File: autogpt_platform/backend/backend/blocks/llm.py
For xAI Grok models accessed via OpenRouter, the API returns `null` for `max_completion_tokens`. The convention in this codebase is to use the model's context window size as the `max_output_tokens` value in ModelMetadata. For example, Grok 3 uses 131072 (128k) and Grok 4 uses 262144 (256k). Do not flag these as incorrect max output token values.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T00:07:27.117Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-23T00:07:27.117Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/openrouter_cost.py`, the `openrouter-cost-reconcile` Langfuse event carries `cost_source` ("openrouter" for authoritative OpenRouter-resolved cost, "fallback" for rate-card fallback) and `resolved_generation_id_count` alongside the reconciled cost and token/model/provider metadata. This lets operators distinguish authoritative reconciliations from fallbacks in Langfuse. Established in PR `#12889` commit 5ce3d0388.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-08T17:27:07.646Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: classic/forge/CLAUDE.md:0-0
Timestamp: 2026-04-08T17:27:07.646Z
Learning: Applies to classic/forge/**/forge/llm/providers/**/*.py : Supported model names include OpenAI (GPT3, GPT3_16k, GPT4, GPT4_32k, GPT4_TURBO, GPT4_O), Anthropic (CLAUDE3_OPUS, CLAUDE3_SONNET, CLAUDE3_HAIKU, CLAUDE3_5_SONNET, CLAUDE3_5_SONNET_v2, CLAUDE3_5_HAIKU, CLAUDE4_SONNET, CLAUDE4_OPUS, CLAUDE4_5_OPUS), and Groq (LLAMA3_8B, LLAMA3_70B, MIXTRAL_8X7B)
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-15T15:30:09.706Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-21T04:35:34.710Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12865
File: autogpt_platform/backend/backend/data/credit.py:1584-1584
Timestamp: 2026-04-21T04:35:34.710Z
Learning: When reviewing this codebase, don’t flag snake_case attribute names (e.g., `subscription_tier`, `stripe_customer_id`, `top_up_config`) on the app-layer Pydantic `User` model as “wrong” field names. These are correct for the app-layer model and are expected to be mapped from the Prisma-layer camelCase fields (e.g., `subscriptionTier`, `stripeCustomerId`) inside methods like `User.from_db()`. Only Prisma-returned/raw objects would use camelCase, but functions like `get_user_by_id(user_id: str)` are expected to return the Pydantic app-layer model.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
🔇 Additional comments (4)
autogpt_platform/backend/backend/data/block_cost_config.py (3)
303-314: Nice explicit documentation of legacy Grok SKUs.Inlining that these rates are historical and that docs.x.ai no longer lists them prevents future reviewers from flagging the entries as stale without context. Good call retaining them since the models are still API-callable.
1239-1255: Walltime rate bump + honest caveat LGTM.15 cr/s (~$0.15/s) cleanly covers the FAL Lite tier at 1.5×, and the inline comment explicitly documents that Veo/Seedance tiers still slightly under-bill pending per-call
provider_costfrom the SDK. Matches the intent already established for this block (walltime is the best available signal until the SDK exposes spend).
763-775: Migration confirmed:provider_cost_type="cost_usd"convention is established and correctly resolved.The COST_USD handler in
executor/utils.py:219properly checks for this sentinel (if stats.provider_cost_type != "cost_usd"returns 0.0). Multiple blocks already emit this pattern in production (VideoNarration, ClaudeCode, Perplexity, Codex, Firecrawl, etc.), and the billing path is tested inblock_usage_cost_test.py:112-114andbilling_reconciliation_test.py:133. UnrealTextToSpeechBlock follows the established convention and mirrors the pairedprovider_cost_typeemission fromtext_to_speech_block.py:114.autogpt_platform/backend/backend/blocks/text_to_speech_block.py (1)
108-116: Clean swap to proportional USD-based billing.
len(input_data.text) * 0.000016matches Unreal Speech's published $16 / 1M char rate, and Python'slen()onstrreturns code points (which is what Unreal meters), so multi-byte UTF-8 characters won't be double-counted. Themerge_statscall is still post-call_unreal_speech_api, so failed API calls don't charge the wallet.The only thing load-bearing across both files is the
"cost_usd"string literal matching whatever the COST_USD resolver expects — I've left a verification script on the pairedblock_cost_config.pychange rather than duplicating it here.
…enRouter floor ReplicateModelBlock takes ANY model ref as input. Flat 10 cr/run was 10-500x under-billing long video/LLM runs ($1-$50+) and 20x over-billing tiny SDXL. Block now uses predictions.async_create + async_wait to read prediction.metrics.predict_time after completion, emits (predict_time * $0.0014/s) as provider_cost, billed at COST_USD 150 cr/$. $0.0014/s is the Nvidia L40S mid-tier rate where most popular public models run. Also: MISTRAL_LARGE_3 and MISTRAL_NEMO in TOKEN_COST are the safety floor for OpenRouter-routed calls (ModelMetadata.provider = 'open_router'). Rates now match OpenRouter's pass-through pricing instead of Mistral-direct's /v1/chat rates, which we never call. Addresses Sentry bug prediction on MISTRAL_NEMO being 'higher than actual cost from OpenRouter'. - ReplicateModelBlock: RUN 10 -> COST_USD 150 cr/$ (dynamic billing) - ReplicateFluxAdvancedModelBlock: unchanged (bounded to Flux models $0.04-$0.08, flat 10 cr stays within 1.25-2.5x margin) - MISTRAL_LARGE_3: 75/225 -> 300/900 (OpenRouter $2/$6) - MISTRAL_NEMO: 23/23 -> 5/5 (OpenRouter $0.035/$0.035)
majdyz
commented
Apr 24, 2026
majdyz
left a comment
majdyz
left a comment
Contributor
Author
There was a problem hiding this comment.
@coderabbitai Re: FAL AIVideoGeneratorBlock 3→15 cr/s sign-off — this is intentional. The prior 3 cr/s rate was set based on Lite-tier FAL models at $0.02/s, but in practice most users run Veo/Seedance at $0.25-$0.30/s. At 3 cr/s we were under-billing the common Veo case by ~10x and losing money per render. 15 cr/s ($0.15/s billed at $0.01/credit) covers the Lite tier with 1.5x margin (matches the platform-wide baseline) while still being ~50% under-bill for Veo — acceptable until the block migrates to COST_USD (blocked on FAL SDK not exposing per-call provider_cost). Flagged in the PR description's follow-up section.
majdyz
changed the title
…ling path Adds 7 unit tests for the refactored run_model: - Uses version= keyword when model_ref has ':' (pinned version) - Uses model= keyword otherwise (unpinned 'owner/name') - Emits provider_cost = predict_time * $0.0014/s via merge_stats - async_wait is awaited before reading metrics - Skips merge_stats when metrics missing OR predict_time is 0 (avoids silent wallet-free leak if SDK quirks return empty metrics) - Sanity-checks _REPLICATE_USD_PER_SEC is in the Replicate hardware tier range ($0.0005-$0.002/s) SDK surface confirmed against installed replicate==* package: - Predictions.async_create(model=, version=, input=) — matches - Prediction.metrics is Optional[Dict] — matches - Prediction.async_wait exists — matches
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
autogpt_platform/backend/backend/data/block_cost_config.py (2)
447-466:⚠️ Potential issue | 🟡 MinorOpenRouter COST_USD path relies on
provider_costbeing emitted — no fallback if x-total-cost is missing.With
cost_amount=150andcost_type=COST_USD, a pre-flight charge is 0 and the real charge isprovider_cost × 150. If OpenRouter ever omitsx-total-cost(documented as rare but possible — e.g., some free/promotional models, errors mid-stream) the block will be billed as 0 credits.For existing COST_USD blocks (Perplexity, ClaudeCode, VideoNarration) you've accepted this tradeoff, and the PR comment explicitly says "provider pricing drift is handled upstream." Just flagging that unlike the prior per-model TOKEN_COST rates, there is no floor here. Consider whether to add a non-zero
cost_amountminimum (e.g., via a second TOKENS BlockCost entry that acts as a floor) or keep a lightweight rate-card fallback in the LLM path whenx-total-costis null — theopenrouter-cost-reconcileLangfuse event already tracks this viacost_source=="fallback"per your prior learnings.Not blocking; just make sure the monitoring for
resolved_generation_id_count==0/cost_source=="fallback"stays actionable so a drop in x-total-cost coverage surfaces before it becomes a wallet-drain incident.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/data/block_cost_config.py` around lines 447 - 466, The OpenRouter COST_USD BlockCost (created in the list comprehension that builds BlockCost(cost_type=BlockCostType.COST_USD, cost_amount=150, cost_filter={...} for models where MODEL_METADATA[model].provider == "open_router")) has no fallback when the x-total-cost header is missing, causing zero billing; add a lightweight fallback floor by either (A) adding a second BlockCost entry for the same models with cost_type=BlockCostType.TOKEN_COST (or a small non-zero COST_USD) that will act as a minimum pre-flight charge, or (B) reduce reliance on cost_amount=150 by ensuring the LLM billing path uses a fallback rate-card when x-total-cost is null and emits the openrouter-cost-reconcile event with cost_source=="fallback" and resolved_generation_id_count==0; update the BlockCost list creation around BlockCost and MODEL_METADATA/ MODEL_COST references and ensure open_router_credentials is used consistently.
417-466:⚠️ Potential issue | 🔴 CriticalAdd ollama provider check to LLM_COST BlockCost list to prevent zero-credit charge fallthrough.
The new provider-grouping split inadvertently omits ollama. Five ollama models are defined in MODEL_METADATA with provider="ollama" and assigned costs in MODEL_COST (lines 141-145: 1 credit each), and are tested as billable (block_usage_cost_test.py line 168). Without an explicit check for
MODEL_METADATA[model].provider == "ollama", ollama models will fail all cost_filter matches and silently charge 0 credits—violating the documented invariant at lines 524-525 ("A missing entry here makes the block run for free...even when the upstream provider charges real USD").Add:
# Ollama Models + [ BlockCost( cost_type=BlockCostType.TOKENS, cost_filter={ "model": model, "credentials": { "id": ollama_credentials.id, }, }, cost_amount=cost, ) for model, cost in MODEL_COST.items() if MODEL_METADATA[model].provider == "ollama" ]after the groq block (around line 446) and ensure
ollama_credentialsis initialized above the LLM_COST list (similar to openai_credentials, etc.).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/data/block_cost_config.py` around lines 417 - 466, The LLM_COST construction omitted models with provider "ollama", causing those models to fall through to zero-cost; add an explicit Ollama Models list comprehension (same shape as the Groq/OpenAI blocks) that creates BlockCost(cost_type=BlockCostType.TOKENS, cost_filter={"model": model, "credentials": {"id": ollama_credentials.id}}, cost_amount=cost) for models where MODEL_METADATA[model].provider == "ollama", and insert it after the Groq block; also ensure ollama_credentials is created/initialized above the LLM_COST list (analogous to openai_credentials, groq_credentials, open_router_credentials) so the reference exists.
🧹 Nitpick comments (1)
autogpt_platform/backend/backend/blocks/replicate/replicate_block.py (1)
94-99: Test mock bypasses the new billing + status paths.The
test_mockreplacesrun_modelwholesale with a string-returning lambda, so neithermerge_stats(provider_cost=...)nor the terminal-status handling is exercised in tests. Given the PR migrates this block from flat RUN to COST_USD billing viaprediction.metrics.predict_time, it would be worth adding a dedicated test (or expandingtest_mock) that:
- Returns a fake
Prediction-shaped object with populatedmetrics.predict_timeand assertsmerge_statsfires with the expectedprovider_cost.- Covers a
status="failed"fixture (once the fix from the previous comment lands) to lock in the new error behavior.Not blocking, but the refactor introduces two new invariants (billing path + failure path) that aren't currently test-covered.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/blocks/replicate/replicate_block.py` around lines 94 - 99, The test mock currently replaces run_model with a simple string-returning lambda and bypasses the new billing and failure handling; update the test_mock used in tests for replicate_block to return a fake Prediction-shaped object (include prediction.metrics.predict_time populated) so that replicate_block.merge_stats is exercised and assert the expected provider_cost is merged, and add a second test fixture where the fake Prediction has status="failed" to validate the terminal-status/error handling paths in the run_model -> replicate block flow.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/blocks/replicate/replicate_block.py`:
- Around line 180-193: After awaiting prediction.async_wait(), check
prediction.status and if it equals "failed" or "canceled" raise an exception
(including prediction.error or prediction.output for context) before doing any
billing or calling extract_result; this ensures failures don't silently proceed
and bill. Update the status reporting code that currently hardcodes "succeeded"
to use prediction.status so the real state is surfaced. Keep billing in
merge_stats(NodeExecutionStats(...)) only after confirming prediction.status
indicates success, and reference prediction.metrics["predict_time"] and
_REPLICATE_USD_PER_SEC as before when computing provider_cost.
---
Outside diff comments:
In `@autogpt_platform/backend/backend/data/block_cost_config.py`:
- Around line 447-466: The OpenRouter COST_USD BlockCost (created in the list
comprehension that builds BlockCost(cost_type=BlockCostType.COST_USD,
cost_amount=150, cost_filter={...} for models where
MODEL_METADATA[model].provider == "open_router")) has no fallback when the
x-total-cost header is missing, causing zero billing; add a lightweight fallback
floor by either (A) adding a second BlockCost entry for the same models with
cost_type=BlockCostType.TOKEN_COST (or a small non-zero COST_USD) that will act
as a minimum pre-flight charge, or (B) reduce reliance on cost_amount=150 by
ensuring the LLM billing path uses a fallback rate-card when x-total-cost is
null and emits the openrouter-cost-reconcile event with cost_source=="fallback"
and resolved_generation_id_count==0; update the BlockCost list creation around
BlockCost and MODEL_METADATA/ MODEL_COST references and ensure
open_router_credentials is used consistently.
- Around line 417-466: The LLM_COST construction omitted models with provider
"ollama", causing those models to fall through to zero-cost; add an explicit
Ollama Models list comprehension (same shape as the Groq/OpenAI blocks) that
creates BlockCost(cost_type=BlockCostType.TOKENS, cost_filter={"model": model,
"credentials": {"id": ollama_credentials.id}}, cost_amount=cost) for models
where MODEL_METADATA[model].provider == "ollama", and insert it after the Groq
block; also ensure ollama_credentials is created/initialized above the LLM_COST
list (analogous to openai_credentials, groq_credentials,
open_router_credentials) so the reference exists.
---
Nitpick comments:
In `@autogpt_platform/backend/backend/blocks/replicate/replicate_block.py`:
- Around line 94-99: The test mock currently replaces run_model with a simple
string-returning lambda and bypasses the new billing and failure handling;
update the test_mock used in tests for replicate_block to return a fake
Prediction-shaped object (include prediction.metrics.predict_time populated) so
that replicate_block.merge_stats is exercised and assert the expected
provider_cost is merged, and add a second test fixture where the fake Prediction
has status="failed" to validate the terminal-status/error handling paths in the
run_model -> replicate block flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 8db81f37-3bcf-424f-bc7b-cdc3960183b7
📒 Files selected for processing (2)
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
- GitHub Check: check API types
- GitHub Check: test (3.12)
- GitHub Check: type-check (3.12)
- GitHub Check: type-check (3.13)
- GitHub Check: type-check (3.11)
- GitHub Check: test (3.11)
- GitHub Check: test (3.13)
- GitHub Check: lint
- GitHub Check: Seer Code Review
- GitHub Check: end-to-end tests
- GitHub Check: Analyze (python)
- GitHub Check: Check PR Status
- GitHub Check: Analyze (typescript)
🧰 Additional context used
📓 Path-based instructions (5)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/backend/backend/blocks/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/backend/blocks/**/*.py: Inherit from 'Block' base class with input/output schemas when adding new blocks in backend
Implement 'run' method with proper error handling in backend blocks
Generate block UUID using 'uuid.uuid4()' when creating new blocks in backend
Write tests alongside block implementation when adding new blocks in backend
autogpt_platform/backend/backend/blocks/**/*.py: For blocks handling files, usestore_media_file()withreturn_format="for_local_processing"when processing with local tools (ffmpeg, MoviePy, PIL)
For blocks handling files, usestore_media_file()withreturn_format="for_external_api"when sending content to external APIs (Replicate, OpenAI)
For blocks returning files, usestore_media_file()withreturn_format="for_block_output"to enable auto-adaptation to execution context (workspace:// in CoPilot, data URI in graphs)
When creating new blocks, inherit fromBlockbase class, define input/output schemas usingBlockSchema, implement asyncrunmethod, and generate unique block ID usinguuid.uuid4()
Files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/backend/backend/data/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
All data access in backend requires user ID checks; verify this for any 'data/*.py' changes
Files:
autogpt_platform/backend/backend/data/block_cost_config.py
autogpt_platform/**/data/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
For changes touching
data/*.py, validate user ID checks or explain why not needed
Files:
autogpt_platform/backend/backend/data/block_cost_config.py
🧠 Learnings (25)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-23T00:07:27.117Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/openrouter_cost.py`, the `openrouter-cost-reconcile` Langfuse event carries `cost_source` ("openrouter" for authoritative OpenRouter-resolved cost, "fallback" for rate-card fallback) and `resolved_generation_id_count` alongside the reconciled cost and token/model/provider metadata. This lets operators distinguish authoritative reconciliations from fallbacks in Langfuse. Established in PR `#12889` commit 5ce3d0388.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12566
File: autogpt_platform/frontend/src/lib/autogpt-server-api/types.ts:968-974
Timestamp: 2026-03-26T00:32:06.673Z
Learning: In Significant-Gravitas/AutoGPT, the admin-facing methods in `autogpt_platform/frontend/src/lib/autogpt-server-api/client.ts` (e.g., `addUserCredits`, `getUsersHistory`, `getUserRateLimit`, `resetUserRateLimit`) intentionally follow the legacy `BackendAPI` pattern with manually defined types in `autogpt_platform/frontend/src/lib/autogpt-server-api/types.ts`. Migrating these admin endpoints to the generated OpenAPI hooks (`@/app/api/__generated__/endpoints/`) is a planned separate effort covering all admin endpoints together, not done piecemeal per PR. Do not flag individual admin type additions in `types.ts` as blocking issues.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12284
File: autogpt_platform/frontend/src/app/api/openapi.json:11897-11900
Timestamp: 2026-03-04T23:58:18.476Z
Learning: Repo: Significant-Gravitas/AutoGPT — PR `#12284`
Backend/frontend OpenAPI codegen convention: In backend/api/features/store/model.py, the StoreSubmission and StoreSubmissionAdminView models define submitted_at: datetime | None, changes_summary: str | None, and instructions: str | None with no default. This is intentional to produce “required but nullable” fields in OpenAPI (properties appear in required[] and use anyOf [type, null]). This matches Prisma’s submittedAt DateTime? and changesSummary String?. Do not flag this as a required/nullable mismatch.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-23T00:07:27.117Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/service.py`, background tasks that persist cost or emit Langfuse backfill (e.g. the cost-reconcile task) must be anchored to `_background_tasks` using `_background_tasks.add(task)` and `task.add_done_callback(_background_tasks.discard)`, mirroring the existing pattern at lines 3063 / 4232 / 4256. This prevents the asyncio task from being garbage-collected before persistence or Langfuse emission completes. Do NOT flag the absence of this anchoring as acceptable in this file. Established in PR `#12889` commit 5ce3d0388.
📚 Learning: 2026-04-23T13:53:29.246Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1064-1111
Timestamp: 2026-04-23T13:53:29.246Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, the E2B sandbox blocks (`ExecuteCodeBlock`, `InstantiateCodeSandboxBlock`, `ExecuteCodeStepBlock`) are intentionally billed per-node-execution walltime (`BlockCostType.SECOND`, `cost_divisor=10`, 1 credit per 10s) using `stats.walltime` from the `async_time_measured` decorator in `manager.py::_on_node_execution`. Idle time between steps is deliberately absorbed by the platform — this makes per-step charges predictable and user-visible. Do NOT flag the absence of sandbox-lifetime billing as a bug. Future migration to raw sandbox uptime billing would require plumbing `provider_cost_type="sandbox_seconds"` from the E2B SDK into `NodeExecutionStats` and switching to `COST_USD`/`SECOND` against that field; until then, walltime-per-execution is the correct and intentional model. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-22T04:01:32.723Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12876
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:0-0
Timestamp: 2026-04-22T04:01:32.723Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/helpers.py`, the `run_block` copilot path charges ONLY the user credit wallet (via `spend_credits` / `_charge_block_credits`). The microdollar rate-limit counter (`record_cost_usage`) is NOT incremented for `run_block` block executions — the `_record_block_microdollar_cost` helper was explicitly reverted (commit 16ae0f7b5, PR `#12876`) to avoid double-accounting. Do NOT flag missing microdollar recording in `execute_block` as a bug; the credit wallet is the sole billing mechanism for copilot `run_block` calls.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T13:55:24.409Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:1112-1119
Timestamp: 2026-04-23T13:55:24.409Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `AIVideoGeneratorBlock` is intentionally billed by walltime seconds (`BlockCostType.SECOND`, `cost_amount=3`, no `cost_divisor`) using `fal_credentials`. The FAL SDK does not currently expose per-request `provider_cost` or output duration in the response path used; walltime is the best available billing signal. Do NOT flag the absence of `COST_USD` or output-duration billing as a bug. Migration to `BlockCostType.COST_USD` against FAL-reported spend would require an SDK upgrade or a new stats-scraping path and is deferred until the FAL SDK exposes that signal. Established in PR `#12894`.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-02-05T04:11:00.596Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 11796
File: autogpt_platform/backend/backend/blocks/video/concat.py:3-4
Timestamp: 2026-02-05T04:11:00.596Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, when creating a new block, generate a UUID once with uuid.uuid4() and hard-code the resulting string as the block's id parameter. Do not call uuid.uuid4() at runtime; IDs must be constant across all imports and runs to ensure stability.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.py
📚 Learning: 2026-03-16T16:32:21.686Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:21.686Z
Learning: In autogpt_platform/backend/backend/blocks/, the Block base class execute() already wraps run() in a try/except to convert uncaught exceptions into BlockExecutionError/BlockUnknownError. Do not add per-block try/except in individual block run() methods, as this is not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Only use explicit try/except within blocks that need to distinguish between success and error yield paths inside a generator (e.g., attachment blocks). This guidance applies to all Python files under autogpt_platform/backend/backend/blocks/ and similar block implementations; avoid duplicating error handling in run() unless a block requires generator-based branching.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.py
📚 Learning: 2026-04-23T12:55:26.122Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: Cost billing via the cost(*costs) decorator is applied at input-evaluation time (before a block’s run() executes). Therefore, mutating input_data inside run() will not change billing. When a block’s billing depends on a field plus URL/sniff-derived signals, treat the explicitly declared billing field (e.g., is_video) as the only billing source—set it correctly before run() (or in the code path that occurs before the decorator evaluates input_data). This should be checked for all blocks under autogpt_platform/backend/backend/blocks/ so billing signals are not mistakenly assumed to update during run().
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: In autogpt_platform/backend/backend/blocks/ (and related blocks under autogpt_platform/backend/backend/blocks/), do not add try/except blocks around a block's run() method for standard error propagation. The block executor framework (backend/executor/manager.py) catches uncaught exceptions from run() and emits them on the 'error' output. Only add explicit try/except blocks when you need to control partial outputs in failure cases (e.g., certain outputs must not be yielded on error, as in attachment blocks). This is the standard pattern across the codebase; apply it broadly to blocks' run() implementations.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.py
📚 Learning: 2026-03-16T16:30:23.196Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:23.196Z
Learning: In any Python file under autogpt_platform/backend/backend/blocks, do not add a try/except around run() solely for standard error handling. The block framework’s _execute() in _base.py already catches unhandled exceptions and re-raises as BlockExecutionError or BlockUnknownError. If you yield ("error", message), _execute() raises BlockExecutionError immediately, so the error port will not propagate downstream. Reserve explicit try/except for scenarios where you must control partial output (e.g., attachment blocks that must skip yielding content_base64 on failure).
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.py
📚 Learning: 2026-03-16T16:30:11.452Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:11.452Z
Learning: Do not wrap synchronous AgentMail SDK calls with asyncio.to_thread() in blocks under autogpt_platform/backend/backend/blocks (and across the codebase). The block executor runs node execution in dedicated threads via asyncio.run_coroutine_threadsafe (see manager.py around lines ~745-752 and ~1079). The existing pattern avoids using asyncio.to_thread for SDK calls inside async run() methods, so maintain that approach and do not add to_thread usage in these code paths.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-15T02:43:36.890Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-22T11:46:04.431Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/config.py:0-0
Timestamp: 2026-04-22T11:46:04.431Z
Learning: Do not flag the Claude Sonnet 4.6 model ID as incorrect when it uses the project’s established hyphenated convention: `anthropic/claude-sonnet-4-6`. This hyphen form is the intentional, production convention and should be treated as valid (including in files like llm.py, blocks tests, reasoning.py, `_is_anthropic_model` tests, and config defaults). Note that OpenRouter also accepts the dot variant `anthropic/claude-sonnet-4.6`, so either form may be tolerated, but `anthropic/claude-sonnet-4-6` should be considered the standard to match project usage.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-22T11:46:12.892Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/baseline/service.py:322-332
Timestamp: 2026-04-22T11:46:12.892Z
Learning: In this codebase (Significant-Gravitas/AutoGPT), OpenRouter-routed Anthropic model IDs should use the hyphen-separated convention (e.g., `anthropic/claude-sonnet-4-6`, `anthropic/claude-opus-4-6`). Although OpenRouter may accept both hyphen and dot variants, treat the hyphen-separated form as the intended, correct codebase-wide convention and do not flag it as an error. Only flag the dot-separated variant (e.g., `anthropic/claude-sonnet-4.6`) as incorrect when reviewing/validating model ID strings for OpenRouter-routed Anthropic models.
Applied to files:
autogpt_platform/backend/backend/blocks/replicate/replicate_block.pyautogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T13:53:40.315Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12894
File: autogpt_platform/backend/backend/data/block_cost_config.py:271-277
Timestamp: 2026-04-23T13:53:40.315Z
Learning: In `autogpt_platform/backend/backend/data/block_cost_config.py`, `compute_token_credits()` intentionally returns `MODEL_COST[model]` (the flat tier) on pre-flight (when `stats is None`) for the `TOKENS` billing path. Returning 0 pre-flight would allow a zero-balance user to bypass the credit gate and trigger an LLM call, with the insufficient-balance error only surfacing post-flight (a billing leak). The overcharge concern (actual token cost < MODEL_COST estimate) is handled by `_charge_reconciled_usage_sync` in `autogpt_platform/backend/backend/executor/billing.py`, which issues a negative-delta refund via `spend_credits(cost=negative)` when real usage falls below the pre-flight estimate. Do NOT flag the MODEL_COST pre-flight floor in this function as an overcharge bug; the refund path covers it.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T12:55:26.122Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12893
File: autogpt_platform/backend/backend/blocks/ayrshare/post_to_tiktok.py:24-24
Timestamp: 2026-04-23T12:55:26.122Z
Learning: In `Significant-Gravitas/AutoGPT`, the `cost(*costs)` decorator in `autogpt_platform/backend/backend/sdk/cost_integration.py` evaluates `input_data` **at input-evaluation time, before `run()` executes**. Mutating fields of `input_data` inside a block's `run()` method (e.g., `input_data.is_video = has_video`) has **no effect on billing** because the cost filter has already been applied. For blocks that derive a computed boolean from both an explicit field and URL-sniffing (like `has_video` in `PostToTikTokBlock`), the explicit field (e.g., `is_video`) is the sole billing signal; callers must set it correctly. This applies to all blocks under `autogpt_platform/backend/backend/blocks/`.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-23T00:07:27.117Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-04-23T00:07:27.117Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/openrouter_cost.py`, the `openrouter-cost-reconcile` Langfuse event carries `cost_source` ("openrouter" for authoritative OpenRouter-resolved cost, "fallback" for rate-card fallback) and `resolved_generation_id_count` alongside the reconciled cost and token/model/provider metadata. This lets operators distinguish authoritative reconciliations from fallbacks in Langfuse. Established in PR `#12889` commit 5ce3d0388.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-09T10:50:43.907Z
Learnt from: Bentlybro
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-03-09T10:50:43.907Z
Learning: Repo: Significant-Gravitas/AutoGPT — File: autogpt_platform/backend/backend/blocks/llm.py
For xAI Grok models accessed via OpenRouter, the API returns `null` for `max_completion_tokens`. The convention in this codebase is to use the model's context window size as the `max_output_tokens` value in ModelMetadata. For example, Grok 3 uses 131072 (128k) and Grok 4 uses 262144 (256k). Do not flag these as incorrect max output token values.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-13T15:49:44.961Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/rate_limit.py:0-0
Timestamp: 2026-03-13T15:49:44.961Z
Learning: In `autogpt_platform/backend/backend/copilot/rate_limit.py`, the original per-session token window (with a TTL-based reset) was replaced with fixed daily and weekly windows. `resets_at` is now derived from `_daily_reset_time()` (midnight UTC) and `_weekly_reset_time()` (next Monday 00:00 UTC) — deterministic fixed-boundary calculations that require no Redis TTL introspection.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-03-15T15:30:09.706Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-08T17:27:07.646Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: classic/forge/CLAUDE.md:0-0
Timestamp: 2026-04-08T17:27:07.646Z
Learning: Applies to classic/forge/**/forge/llm/providers/**/*.py : Supported model names include OpenAI (GPT3, GPT3_16k, GPT4, GPT4_32k, GPT4_TURBO, GPT4_O), Anthropic (CLAUDE3_OPUS, CLAUDE3_SONNET, CLAUDE3_HAIKU, CLAUDE3_5_SONNET, CLAUDE3_5_SONNET_v2, CLAUDE3_5_HAIKU, CLAUDE4_SONNET, CLAUDE4_OPUS, CLAUDE4_5_OPUS), and Groq (LLAMA3_8B, LLAMA3_70B, MIXTRAL_8X7B)
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
📚 Learning: 2026-04-21T04:35:34.710Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12865
File: autogpt_platform/backend/backend/data/credit.py:1584-1584
Timestamp: 2026-04-21T04:35:34.710Z
Learning: When reviewing this codebase, don’t flag snake_case attribute names (e.g., `subscription_tier`, `stripe_customer_id`, `top_up_config`) on the app-layer Pydantic `User` model as “wrong” field names. These are correct for the app-layer model and are expected to be mapped from the Prisma-layer camelCase fields (e.g., `subscriptionTier`, `stripeCustomerId`) inside methods like `User.from_db()`. Only Prisma-returned/raw objects would use camelCase, but functions like `get_user_by_id(user_id: str)` are expected to return the Pydantic app-layer model.
Applied to files:
autogpt_platform/backend/backend/data/block_cost_config.py
🔇 Additional comments (3)
autogpt_platform/backend/backend/data/block_cost_config.py (2)
280-354: Token rate recalibration checks out.Spot-checked the updated entries against the inline provider rates:
- GPT5 / GPT5_1 @ 188/1500 = $1.25/$10 × 150 cr/$ ✓
- DEEPSEEK_CHAT / R1_0528 @ 21/42 = $0.14/$0.28 × 150 ✓
- KIMI_K2_5 @ 66/300 = $0.44/$2.00 × 150 ✓
- KIMI_K2_6 @ 112/698 = $0.7448/$4.655 × 150 ✓ (698.25 → 698 floors, not ceils — trivial)
- META_LLAMA_4_MAVERICK @ 75/116 = $0.50/$0.77 × 150 ✓
- MISTRAL_NEMO @ 5/5 ≈ $0.03/$0.03 × 150 ✓
Note: the comment at lines 320–324 claims TOKEN_COST is a safety floor "when OpenRouter fails to return x-total-cost" for Mistral models, but since the OpenRouter group in
LLM_COSTonly emitsCOST_USDBlockCosts (notTOKENS), the Mistral TOKEN_COST entries aren't actually consulted at billing time forprovider=="open_router"models. Harmless (the entries are dormant data), just inaccurate documentation.
681-701: ReplicateModelBlock pricing change is sound; see the follow-up on the block itself.The $0.0014/s L40S mid-tier assumption with 1.5× margin is a reasonable interim; the follow-up ticket for generic Replicate migration noted in the PR description is the right resolution path. The actual billing correctness depends on
prediction.metrics.predict_timebeing populated — see separate comment onreplicate_block.pyrun_model regarding failed-prediction handling which affects this billing path.autogpt_platform/backend/backend/blocks/replicate/replicate_block.py (1)
30-35: L40S rate under-bills cheap L4 models noticeably; leaving as-is per PR scope is fine.Cheap L4 models run at ~$0.000275/s — the L40S rate of $0.001400/s over-bills those by ~5×, which the PR comment acknowledges. The inverse (A100 heavy models) under-bills marginally. Given the PR explicitly defers proper per-hardware resolution to the follow-up refactor ("separate PR"), this is an acceptable interim. Just noting the constant should be revisited alongside that refactor so it doesn't silently calcify.
…lling async_wait() returns normally regardless of prediction terminal status — only async_run raises ModelError on 'failed'. Without an explicit status check we'd bill partial compute time on a failed run, yield empty output via extract_result(None), and hardcode 'status: succeeded' hiding the failure. Check prediction.status after async_wait and raise before merge_stats so failures surface as exceptions (caught by run() and re-raised as BlockExecutionError). Also guard against output=None on succeeded predictions (type-narrowing for extract_result). Addresses CodeRabbit critical on #12912.
5 tasks
…ates Three tests on CI were still asserting old values: - UnrealTextToSpeech tests assumed provider_cost=len(text) with type 'characters'. Updated to assert provider_cost=len(text)*$0.000016 with type 'cost_usd' per the text_to_speech_block migration. - ZeroBounce ValidateEmailsBlock cost_amount test assumed 250, now 150 after the margin alignment in this PR.
github-project-automation
Bot
moved this from 🆕 Needs initial review
to ✅ Done
in AutoGPT development kanban
2 tasks
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
PR #12909's pricing refresh was sourced from aggregators (pricepertoken, blog mirrors) instead of provider pricing pages. Follow-up audit against official provider docs caught 22 stale entries — 9 LLM token rates + 12 non-LLM block rates + 1 block that needed a code refactor to bill dynamically. Also flagged by Sentry: Mistral models were sitting on the wrong provider's rate table.
Cross-verified JS-rendered pages (docs.x.ai, DeepSeek, Kimi) via agent-browser.
Corrections applied
LLM TOKEN_COST (9 entries)
GPT5DEEPSEEK_CHATDEEPSEEK_R1_0528MISTRAL_LARGE_3MISTRAL_NEMOKIMI_K2_0905KIMI_K2_5KIMI_K2_6META_LLAMA_4_MAVERICKNon-LLM BLOCK_COSTS — rate corrections (11 entries)
Under-billing fixes:
AIVideoGeneratorBlock(FAL) SECOND 3 → 15 cr/sCreateTalkingAvatarVideoBlock(D-ID) RUN 15 → 100 crUnrealTextToSpeechBlockRUN 5 → COST_USD 150 cr/$ (block now emitschars × $0.000016)Over-billing fixes:
IdeogramModelBlockdefault 16 → 12, V_3 18 → 14AIImageEditorBlockFLUX_KONTEXT_MAX 20 → 12ValidateEmailsBlock250 → 150 cr/$SearchTheWebBlock100 → 150 cr/$GetLinkedinProfilePictureBlock3 → 1 crNon-LLM BLOCK_COSTS — block refactored for dynamic billing (1 entry)
ReplicateModelBlock(the generic "run any Replicate model" wrapper) migrated from flat RUN 10 cr → COST_USD 150 cr/$. Block now usesclient.predictions.async_create + async_waitinstead ofasync_run(wait=False)so it can readprediction.metrics.predict_timeand billpredict_time × $0.0014/s(Nvidia L40S mid-tier, where most popular public models run).Additionally (addressing CodeRabbit's critical review on this refactor):
async_wait()returns normally regardless of terminal status — it doesn't raise onfailed/canceledlike the oldasync_rundid. The block now explicitly checksprediction.statusafterasync_wait()and raisesRuntimeErroronfailed(withprediction.erroras context) orcanceledbeforemerge_stats, so failed runs are never billed for partial compute time.Why this matters: flat 10 cr was 10–500× under-billing long video/LLM runs (users could wire in a $50/hr A100 Llama inference and pay us $0.10). It was also 20× over-billing trivial SDXL runs. Now scales with real compute time AND no longer bills failed predictions.
Documentation-only
ModelMetadata.provider = "open_router"for all Mistral entries.How
our_cr = round(1.5 × provider_usd × 100).UnrealTextToSpeechBlock,ReplicateModelBlock) from flat RUN to COST_USD — the Replicate migration touched the block's SDK interaction.3 cr/srate.characterscost_type) + 1 for ZeroBounce (old 250 cr).Known remaining risk (explicitly out of scope)
ReplicateFluxAdvancedModelBlocknot migrated — bounded to Flux models ($0.04–$0.08), flat 10 cr stays within 1.25–2.5× margin. Separate PR if desired.async_createversion-vs-model branching, metrics-based billing emission, failed/canceled raises, zero/missing-metrics no-emission,async_waitordering), and SDK signature confirmed viainspect.signature— but no real API call executed. A smoke test on a cheap model before merge is still recommended.Test plan
poetry run pytest backend/data/block_cost_config_test.py backend/executor/block_usage_cost_test.py backend/blocks/claude_code_cost_test.py backend/blocks/cost_leak_fixes_test.py backend/blocks/block_cost_tracking_test.py backend/copilot/tools/helpers_test.py backend/blocks/replicate/replicate_block_cost_test.py -q— all passing (80+ tests).predictions.async_create + async_wait + metrics.predict_timepath.