feat(platform): Add LLM registry public read API - #12371
Closed
Bentlybro wants to merge 10 commits into
Closed
Conversation
github-project-automation
Bot
moved this to 🆕 Needs initial review
in AutoGPT development kanban
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughA comprehensive LLM registry system is introduced with database tables, an in-memory async-refreshed cache, data models, and REST API endpoints for querying available models and providers. The registry integrates into the startup sequence and exposes v2 LLM API routes. Changes
Sequence Diagram(s)sequenceDiagram
participant Startup as Application Startup
participant Refresh as refresh_llm_registry
participant DB as Database
participant Cache as In-Memory Registry
participant Logger as Logger
Startup->>Refresh: Call async refresh at startup
Refresh->>DB: Fetch LlmModel, LlmProvider, LlmModelCreator, LlmModelCost
alt Success
DB-->>Refresh: Return model records with relations
Refresh->>Refresh: Parse costs, metadata, creator info
Refresh->>Refresh: Construct RegistryModel instances
Refresh->>Cache: Atomically swap _dynamic_models & _schema_options
Refresh->>Logger: Log success
else Database Error
DB-->>Refresh: Error
Refresh->>Logger: Log warning
Refresh->>Cache: Initialize with empty registry
end
Refresh-->>Startup: Complete
Startup->>Startup: Continue block initialization
sequenceDiagram
participant Client as Client
participant Router as API Router
participant Registry as In-Memory Registry
Client->>Router: GET /api/v2/llm/models?enabled_only=true
Router->>Registry: get_enabled_models()
Registry-->>Router: list[RegistryModel]
Router->>Router: Map RegistryModel → LlmModel (costs, creator, metadata)
Router-->>Client: LlmModelsResponse (models, total)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🔍 PR Overlap DetectionThis check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early. 🔴 Merge Conflicts DetectedThe following PRs have been tested and will have merge conflicts if merged after this PR. Consider coordinating with the authors.
🟡 Medium Risk — Some Line OverlapThese PRs have some overlapping changes:
🟢 Low Risk — File Overlap OnlyThese PRs touch the same files but different sections (click to expand)
Summary: 4 conflict(s), 2 medium risk, 4 low risk (out of 10 PRs with file overlap) Auto-generated on push. Ignores: |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (5)
autogpt_platform/backend/backend/data/llm_registry/registry.py (1)
215-217: Consider returning a copy to prevent external mutation.
get_schema_options()returns the internal_schema_optionslist reference. If a caller modifies this list, it will affect all subsequent callers until the next registry refresh.This is a minor concern since callers are expected to be read-only, but returning
list(_schema_options)would be more defensive.🛡️ Defensive copy
def get_schema_options() -> list[dict[str, str]]: """Get schema options for model selection dropdown (enabled models only).""" - return _schema_options + return list(_schema_options)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/data/llm_registry/registry.py` around lines 215 - 217, get_schema_options currently returns the internal _schema_options list by reference which allows external code to mutate shared state; change get_schema_options to return a defensive copy (e.g., a new list created from _schema_options) so callers get a snapshot rather than the original, preserving internal registry state and avoiding unintended mutations affecting subsequent callers; update the return in get_schema_options to return a shallow copy of _schema_options.autogpt_platform/backend/backend/server/v2/llm/routes.py (1)
52-77: Consider extracting model mapping to reduce duplication.The
RegistryModel→LlmModelmapping logic is duplicated betweenlist_models(Lines 52-77) andlist_providers(Lines 112-136). Extracting this to a helper function would improve maintainability.♻️ Proposed refactor
+def _map_model(model) -> llm_model.LlmModel: + """Convert registry model to API model.""" + return llm_model.LlmModel( + slug=model.slug, + display_name=model.display_name, + description=model.description, + provider_name=model.provider_display_name, + creator=_map_creator(model.creator), + context_window=model.metadata.context_window, + max_output_tokens=model.metadata.max_output_tokens, + price_tier=model.metadata.price_tier, + is_recommended=model.is_recommended, + capabilities=model.capabilities, + costs=[ + llm_model.LlmModelCost( + unit=cost.unit, + credit_cost=cost.credit_cost, + credential_provider=cost.credential_provider, + credential_id=cost.credential_id, + credential_type=cost.credential_type, + currency=cost.currency, + metadata=cost.metadata, + ) + for cost in model.costs + ], + ) + + `@router.get`("/models", response_model=llm_model.LlmModelsResponse) async def list_models( enabled_only: bool = fastapi.Query( default=True, description="Only return enabled models" ), ): # ... registry_models = get_enabled_models() if enabled_only else get_all_models() - models = [ - llm_model.LlmModel( - slug=model.slug, - # ... all the mapping logic - ) - for model in registry_models - ] + models = [_map_model(model) for model in registry_models] return llm_model.LlmModelsResponse(models=models, total=len(models))Similar simplification applies to
list_providers.Also applies to: 112-136
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/server/v2/llm/routes.py` around lines 52 - 77, The RegistryModel→LlmModel mapping is duplicated in list_models and list_providers; extract the transformation into a single helper (e.g. _map_registry_model_to_llm_model) that accepts a registry model and returns an llm_model.LlmModel, reusing the same construction of LlmModel and nested LlmModelCost (including _map_creator for creator) and replace the inline comprehensions in both list_models and list_providers to call this helper to eliminate duplication and centralize future changes.autogpt_platform/backend/backend/server/v2/llm/model.py (3)
57-61: Consider adding non-negative constraint tototal.For consistency with other validated fields,
totalshould usege=0.♻️ Suggested improvement
- total: int + total: int = pydantic.Field(ge=0)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/server/v2/llm/model.py` around lines 57 - 61, The LlmModelsResponse model's total field lacks a non-negative constraint; update the LlmModelsResponse pydantic model to declare total with a Field(..., ge=0) (or equivalent pydantic validator) so total must be >= 0; ensure the Field import from pydantic is present and update the total annotation in the LlmModelsResponse class accordingly.
13-13: Consider usingLiteraltype forunitfield.The comment indicates only "RUN" or "TOKENS" are valid. Using
Literalprovides better type safety and generates a more precise OpenAPI schema with enum constraints.♻️ Suggested improvement
-from typing import Any +from typing import Any, Literal ... - unit: str # "RUN" or "TOKENS" + unit: Literal["RUN", "TOKENS"]🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/server/v2/llm/model.py` at line 13, Replace the loose string annotation for the unit field with a Literal type to restrict values to "RUN" or "TOKENS": import Literal (from typing or typing_extensions depending on Python target), change the annotation for the unit field from unit: str to unit: Literal["RUN", "TOKENS"], and ensure any serializers/schemas (e.g., Pydantic model or dataclass) pick up the new type so OpenAPI generates an enum for this field; also run type checks and fix any callers that pass other strings to the unit field.
41-43: Add validation constraints for consistency withcredit_cost.
credit_costinLlmModelCostusesge=0, butcontext_window,max_output_tokens, andprice_tierlack constraints. Consider adding bounds for defensive validation and better API documentation.♻️ Suggested constraints
- context_window: int - max_output_tokens: int | None = None - price_tier: int # 1=cheapest, 2=medium, 3=expensive + context_window: int = pydantic.Field(gt=0) + max_output_tokens: int | None = pydantic.Field(default=None, gt=0) + price_tier: int = pydantic.Field(ge=1, le=3) # 1=cheapest, 2=medium, 3=expensive🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/server/v2/llm/model.py` around lines 41 - 43, The fields context_window, max_output_tokens, and price_tier need defensive validation similar to LlmModelCost.credit_cost (which uses ge=0): update the model definition to add Pydantic constraints (e.g. use pydantic.Field) so context_window is a positive integer (Field(..., ge=1)), max_output_tokens is either None or non-negative (Field(default=None, ge=0)), and price_tier is constrained to the allowed range (Field(..., ge=1, le=3)); import Field if missing and apply these constraints on the context_window, max_output_tokens, and price_tier declarations in the model class where they are defined.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@autogpt_platform/backend/backend/data/llm_registry/registry.py`:
- Around line 215-217: get_schema_options currently returns the internal
_schema_options list by reference which allows external code to mutate shared
state; change get_schema_options to return a defensive copy (e.g., a new list
created from _schema_options) so callers get a snapshot rather than the
original, preserving internal registry state and avoiding unintended mutations
affecting subsequent callers; update the return in get_schema_options to return
a shallow copy of _schema_options.
In `@autogpt_platform/backend/backend/server/v2/llm/model.py`:
- Around line 57-61: The LlmModelsResponse model's total field lacks a
non-negative constraint; update the LlmModelsResponse pydantic model to declare
total with a Field(..., ge=0) (or equivalent pydantic validator) so total must
be >= 0; ensure the Field import from pydantic is present and update the total
annotation in the LlmModelsResponse class accordingly.
- Line 13: Replace the loose string annotation for the unit field with a Literal
type to restrict values to "RUN" or "TOKENS": import Literal (from typing or
typing_extensions depending on Python target), change the annotation for the
unit field from unit: str to unit: Literal["RUN", "TOKENS"], and ensure any
serializers/schemas (e.g., Pydantic model or dataclass) pick up the new type so
OpenAPI generates an enum for this field; also run type checks and fix any
callers that pass other strings to the unit field.
- Around line 41-43: The fields context_window, max_output_tokens, and
price_tier need defensive validation similar to LlmModelCost.credit_cost (which
uses ge=0): update the model definition to add Pydantic constraints (e.g. use
pydantic.Field) so context_window is a positive integer (Field(..., ge=1)),
max_output_tokens is either None or non-negative (Field(default=None, ge=0)),
and price_tier is constrained to the allowed range (Field(..., ge=1, le=3));
import Field if missing and apply these constraints on the context_window,
max_output_tokens, and price_tier declarations in the model class where they are
defined.
In `@autogpt_platform/backend/backend/server/v2/llm/routes.py`:
- Around line 52-77: The RegistryModel→LlmModel mapping is duplicated in
list_models and list_providers; extract the transformation into a single helper
(e.g. _map_registry_model_to_llm_model) that accepts a registry model and
returns an llm_model.LlmModel, reusing the same construction of LlmModel and
nested LlmModelCost (including _map_creator for creator) and replace the inline
comprehensions in both list_models and list_providers to call this helper to
eliminate duplication and centralize future changes.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: dadbebbb-15c4-44ec-9ef9-a58efc76aa22
📒 Files selected for processing (9)
autogpt_platform/backend/backend/api/rest_api.pyautogpt_platform/backend/backend/data/llm_registry/__init__.pyautogpt_platform/backend/backend/data/llm_registry/model.pyautogpt_platform/backend/backend/data/llm_registry/registry.pyautogpt_platform/backend/backend/server/v2/llm/__init__.pyautogpt_platform/backend/backend/server/v2/llm/model.pyautogpt_platform/backend/backend/server/v2/llm/routes.pyautogpt_platform/backend/migrations/20260310_add_llm_registry_schema/migration.sqlautogpt_platform/backend/schema.prisma
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: types
- GitHub Check: Seer Code Review
- GitHub Check: test (3.12)
- GitHub Check: test (3.13)
- GitHub Check: test (3.11)
- GitHub Check: check-overlaps
- GitHub Check: Analyze (python)
- GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (8)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
Files:
autogpt_platform/backend/backend/server/v2/llm/routes.pyautogpt_platform/backend/backend/api/rest_api.pyautogpt_platform/backend/backend/server/v2/llm/model.pyautogpt_platform/backend/backend/server/v2/llm/__init__.pyautogpt_platform/backend/backend/data/llm_registry/registry.pyautogpt_platform/backend/backend/data/llm_registry/model.pyautogpt_platform/backend/backend/data/llm_registry/__init__.py
autogpt_platform/backend/**/*.{py,txt}
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use
poetry runprefix for all Python commands, including testing, linting, formatting, and migrations
Files:
autogpt_platform/backend/backend/server/v2/llm/routes.pyautogpt_platform/backend/backend/api/rest_api.pyautogpt_platform/backend/backend/server/v2/llm/model.pyautogpt_platform/backend/backend/server/v2/llm/__init__.pyautogpt_platform/backend/backend/data/llm_registry/registry.pyautogpt_platform/backend/backend/data/llm_registry/model.pyautogpt_platform/backend/backend/data/llm_registry/__init__.py
autogpt_platform/backend/backend/**/*.py
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings
Files:
autogpt_platform/backend/backend/server/v2/llm/routes.pyautogpt_platform/backend/backend/api/rest_api.pyautogpt_platform/backend/backend/server/v2/llm/model.pyautogpt_platform/backend/backend/server/v2/llm/__init__.pyautogpt_platform/backend/backend/data/llm_registry/registry.pyautogpt_platform/backend/backend/data/llm_registry/model.pyautogpt_platform/backend/backend/data/llm_registry/__init__.py
autogpt_platform/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/server/v2/llm/routes.pyautogpt_platform/backend/backend/api/rest_api.pyautogpt_platform/backend/backend/server/v2/llm/model.pyautogpt_platform/backend/backend/server/v2/llm/__init__.pyautogpt_platform/backend/backend/data/llm_registry/registry.pyautogpt_platform/backend/backend/data/llm_registry/model.pyautogpt_platform/backend/backend/data/llm_registry/__init__.py
autogpt_platform/backend/backend/api/**/*.py
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
autogpt_platform/backend/backend/api/**/*.py: Use FastAPI for building REST and WebSocket endpoints
Use JWT-based authentication with Supabase integration
Files:
autogpt_platform/backend/backend/api/rest_api.py
autogpt_platform/backend/schema.prisma
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Run database migrations with 'poetry run prisma migrate dev' and 'poetry run prisma generate' after schema changes in backend
Files:
autogpt_platform/backend/schema.prisma
autogpt_platform/backend/**/schema.prisma
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Define key database models (User, AgentGraph, AgentGraphExecution, AgentNode, StoreListing) in
schema.prisma
Files:
autogpt_platform/backend/schema.prisma
autogpt_platform/backend/backend/data/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
All data access in backend requires user ID checks; verify this for any 'data/*.py' changes
Files:
autogpt_platform/backend/backend/data/llm_registry/registry.pyautogpt_platform/backend/backend/data/llm_registry/model.pyautogpt_platform/backend/backend/data/llm_registry/__init__.py
🧠 Learnings (9)
📓 Common learnings
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12284
File: autogpt_platform/frontend/src/app/api/openapi.json:11897-11900
Timestamp: 2026-03-04T23:58:18.476Z
Learning: Repo: Significant-Gravitas/AutoGPT — PR `#12284`
Backend/frontend OpenAPI codegen convention: In backend/api/features/store/model.py, the StoreSubmission and StoreSubmissionAdminView models define submitted_at: datetime | None, changes_summary: str | None, and instructions: str | None with no default. This is intentional to produce “required but nullable” fields in OpenAPI (properties appear in required[] and use anyOf [type, null]). This matches Prisma’s submittedAt DateTime? and changesSummary String?. Do not flag this as a required/nullable mismatch.
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/api/features/**/*.py : Update routes in '/backend/backend/api/features/' and add/update Pydantic models in the same directory for API development
Applied to files:
autogpt_platform/backend/backend/server/v2/llm/routes.pyautogpt_platform/backend/backend/server/v2/llm/model.py
📚 Learning: 2026-02-04T16:50:20.508Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/CLAUDE.md:0-0
Timestamp: 2026-02-04T16:50:20.508Z
Learning: Applies to autogpt_platform/backend/backend/api/features/**/*.py : When modifying API routes, update corresponding Pydantic models in the same directory and write tests alongside the route file
Applied to files:
autogpt_platform/backend/backend/server/v2/llm/routes.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/server/v2/llm/routes.pyautogpt_platform/backend/backend/api/rest_api.pyautogpt_platform/backend/backend/server/v2/llm/model.pyautogpt_platform/backend/schema.prismaautogpt_platform/backend/backend/server/v2/llm/__init__.pyautogpt_platform/backend/backend/data/llm_registry/registry.pyautogpt_platform/backend/migrations/20260310_add_llm_registry_schema/migration.sqlautogpt_platform/backend/backend/data/llm_registry/model.pyautogpt_platform/backend/backend/data/llm_registry/__init__.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/server/v2/llm/routes.pyautogpt_platform/backend/backend/api/rest_api.pyautogpt_platform/backend/backend/server/v2/llm/model.pyautogpt_platform/backend/backend/server/v2/llm/__init__.pyautogpt_platform/backend/backend/data/llm_registry/registry.pyautogpt_platform/backend/backend/data/llm_registry/model.pyautogpt_platform/backend/backend/data/llm_registry/__init__.py
📚 Learning: 2026-02-04T16:50:20.508Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/CLAUDE.md:0-0
Timestamp: 2026-02-04T16:50:20.508Z
Learning: Applies to autogpt_platform/backend/**/schema.prisma : Define key database models (User, AgentGraph, AgentGraphExecution, AgentNode, StoreListing) in `schema.prisma`
Applied to files:
autogpt_platform/backend/schema.prismaautogpt_platform/backend/migrations/20260310_add_llm_registry_schema/migration.sql
📚 Learning: 2026-03-04T23:58:18.476Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12284
File: autogpt_platform/frontend/src/app/api/openapi.json:11897-11900
Timestamp: 2026-03-04T23:58:18.476Z
Learning: Repo: Significant-Gravitas/AutoGPT — PR `#12284`
Backend/frontend OpenAPI codegen convention: In backend/api/features/store/model.py, the StoreSubmission and StoreSubmissionAdminView models define submitted_at: datetime | None, changes_summary: str | None, and instructions: str | None with no default. This is intentional to produce “required but nullable” fields in OpenAPI (properties appear in required[] and use anyOf [type, null]). This matches Prisma’s submittedAt DateTime? and changesSummary String?. Do not flag this as a required/nullable mismatch.
Applied to files:
autogpt_platform/backend/schema.prisma
📚 Learning: 2026-03-05T00:13:52.412Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12284
File: autogpt_platform/backend/schema.prisma:284-284
Timestamp: 2026-03-05T00:13:52.412Z
Learning: In `autogpt_platform/backend/schema.prisma`, the `AgentGraph` ↔ `StoreListing` relation uses the pattern: `AgentGraph` declares `StoreListing? relation(fields: [id], references: [agentGraphId], onDelete: NoAction)` and `StoreListing` declares `AgentGraph AgentGraph[]` with `agentGraphId String unique`. This is intentional and valid because `AgentGraph` has a composite PK `@id([id, version])` (multiple rows per graph id, one per version), while `StoreListing.agentGraphId` is `unique` (one listing per graph id). The `fields: [id], references: [agentGraphId]` on the `AgentGraph` side joins `AgentGraph.id` against `StoreListing.agentGraphId`. Do NOT flag this as a cardinality mismatch or malformed relation — `prisma validate` passes cleanly.
Applied to files:
autogpt_platform/backend/schema.prisma
📚 Learning: 2026-03-09T10:50:43.907Z
Learnt from: Bentlybro
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-03-09T10:50:43.907Z
Learning: Repo: Significant-Gravitas/AutoGPT — File: autogpt_platform/backend/backend/blocks/llm.py
For xAI Grok models accessed via OpenRouter, the API returns `null` for `max_completion_tokens`. The convention in this codebase is to use the model's context window size as the `max_output_tokens` value in ModelMetadata. For example, Grok 3 uses 131072 (128k) and Grok 4 uses 262144 (256k). Do not flag these as incorrect max output token values.
Applied to files:
autogpt_platform/backend/backend/data/llm_registry/model.py
🔇 Additional comments (15)
autogpt_platform/backend/backend/data/llm_registry/model.py (1)
1-25: LGTM!Clean and well-documented metadata type. The
NamedTuplewith typed fields andLiteralconstraint forprice_tieris appropriate for immutable model metadata.autogpt_platform/backend/backend/data/llm_registry/registry.py (2)
69-72: LGTM - In-memory cache design is appropriate.The module-level globals with
asyncio.Lockprovide thread-safe refresh while allowing lock-free reads. This aligns with the PR's design goal of zero DB queries at read time.
75-180: LGTM - Robust registry refresh implementation.The refresh logic correctly handles:
- Serialization via
_lockto prevent concurrent refreshes- Graceful fallbacks for missing Provider/Creator relations
- Atomic swap pattern for in-memory cache update
- Proper error logging with re-raise for upstream handling
autogpt_platform/backend/backend/data/llm_registry/__init__.py (1)
1-31: LGTM!Clean package initializer with explicit
__all__for public API surface. The re-exports are organized logically (models, then functions).autogpt_platform/backend/schema.prisma (2)
1307-1458: LGTM - Well-structured LLM registry schema.The schema additions follow established patterns:
- Proper audit fields (
createdAt,updatedAt)- Clear relations with appropriate cascade rules
- Indexes for common query patterns
- Documentation of DB-level constraints (partial unique indexes, check constraints)
The separation of concerns between Provider (hosting) and Creator (training organization) is a good design choice for the model metadata.
1395-1400: Good documentation of partial index limitation.The comment correctly notes that Prisma doesn't support partial unique constraints natively. The DB will enforce uniqueness, but the Prisma client won't validate before insert. Consider catching unique constraint violations in any future admin write API that creates
LlmModelCostrecords.autogpt_platform/backend/migrations/20260310_add_llm_registry_schema/migration.sql (1)
1-148: LGTM - Comprehensive migration with proper constraints.The migration correctly implements:
- Partial unique indexes for cost overrides (Lines 112-116)
- Partial unique index preventing multiple active migrations per source (Line 128)
- Check constraints for
priceTier,creditCost,nodeCount, andcustomCreditCost(Lines 140-148)- Appropriate foreign key actions (RESTRICT for providers, SET NULL for creators, CASCADE for costs)
autogpt_platform/backend/backend/api/rest_api.py (2)
121-133: Appropriate graceful degradation for initial rollout.The try/except with warning log allows the server to start even if the LLM registry is unavailable (e.g., empty database, migration not run). The comment clearly documents that this should become a hard failure once blocks depend on the registry.
366-370: LGTM - Router registration consistent with existing patterns.The v2 LLM router is registered with appropriate tags and prefix, making endpoints available at
/api/llm/modelsand/api/llm/providers.autogpt_platform/backend/backend/server/v2/llm/__init__.py (1)
1-5: LGTM!Clean package initializer exposing only the router for external consumption.
autogpt_platform/backend/backend/server/v2/llm/routes.py (2)
13-17: LGTM - Authentication dependency correctly applied.The
fastapi.Security(autogpt_libs.auth.requires_user)at the router level ensures all endpoints require authentication without needing to inject the user object into individual handlers.
83-141: LGTM - Provider grouping logic is sound.The implementation correctly:
- Groups models by provider key
- Derives display name from first model in group
- Sorts providers alphabetically and models within providers by display name
Minor: Line 94's
dict[str, list]could bedict[str, list[RegistryModel]]for type clarity, but this is a nitpick.autogpt_platform/backend/backend/server/v2/llm/model.py (3)
22-30: LGTM!The
LlmModelCreatormodel is well-structured with appropriate optional fields and clear field naming.
49-54: LGTM!The
LlmProvidermodel correctly usesdefault_factoryfor the mutable list default.
64-67: LGTM!Clean response wrapper model for the providers endpoint.
This was referenced Mar 11, 2026
Bentlybro
marked this pull request as draft
March 11, 2026 11:40
5 tasks
github-actions
Bot
added
the
conflicts
Contributor
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
8 tasks
Bentlybro
force-pushed
the
feat/llm-public-api
branch
from
March 16, 2026 14:56
68fe429 to
10e84cd
Compare
8 tasks
Bentlybro
force-pushed
the
feat/llm-public-api
branch
2 times, most recently
from
March 16, 2026 15:44
737a875 to
957ec03
Compare
6 tasks
Bentlybro
added a commit
that referenced
this pull request
Mar 17, 2026
Add admin write API endpoints for LLM registry management:
- POST /api/llm/models - Create model
- PATCH /api/llm/models/{slug} - Update model
- DELETE /api/llm/models/{slug} - Delete model
- POST /api/llm/providers - Create provider
- PATCH /api/llm/providers/{name} - Update provider
- DELETE /api/llm/providers/{name} - Delete provider
All endpoints require admin authentication via requires_admin_user.
Request/response models defined in admin_model.py:
- CreateLlmModelRequest, UpdateLlmModelRequest
- CreateLlmProviderRequest, UpdateLlmProviderRequest
Implementation coming in follow-up commits (currently returns 501 Not Implemented).
This builds on:
- PR #12357: Schema foundation
- PR #12359: Registry core
- PR #12371: Public read API
Bentlybro
added a commit
that referenced
this pull request
Mar 17, 2026
Implement full CRUD operations for admin API: Database layer (db_write.py): - create_provider, update_provider, delete_provider - create_model, update_model, delete_model - refresh_runtime_caches - invalidates in-memory registry after mutations - Proper validation and error handling Admin routes (admin_routes.py): - All endpoints now functional (no more 501) - Proper error responses (400 for validation, 404 for not found, 500 for server errors) - Lookup by slug/name before operations - Cache refresh after all mutations Features: - Provider deletion blocked if models exist (FK constraint) - All mutations refresh registry cache automatically - Proper logging for audit trail - Admin auth enforced on all endpoints Based on original implementation from PR #11699 (upstream-llm branch). Builds on: - PR #12357: Schema foundation - PR #12359: Registry core - PR #12371: Public read API
This was referenced Mar 18, 2026
Bentlybro
force-pushed
the
feat/llm-public-api
branch
from
March 19, 2026 11:03
957ec03 to
cc60c3c
Compare
Bentlybro
added a commit
that referenced
this pull request
Apr 7, 2026
Add admin write API endpoints for LLM registry management:
- POST /api/llm/models - Create model
- PATCH /api/llm/models/{slug} - Update model
- DELETE /api/llm/models/{slug} - Delete model
- POST /api/llm/providers - Create provider
- PATCH /api/llm/providers/{name} - Update provider
- DELETE /api/llm/providers/{name} - Delete provider
All endpoints require admin authentication via requires_admin_user.
Request/response models defined in admin_model.py:
- CreateLlmModelRequest, UpdateLlmModelRequest
- CreateLlmProviderRequest, UpdateLlmProviderRequest
Implementation coming in follow-up commits (currently returns 501 Not Implemented).
This builds on:
- PR #12357: Schema foundation
- PR #12359: Registry core
- PR #12371: Public read API
Bentlybro
added a commit
that referenced
this pull request
Apr 7, 2026
Implement full CRUD operations for admin API: Database layer (db_write.py): - create_provider, update_provider, delete_provider - create_model, update_model, delete_model - refresh_runtime_caches - invalidates in-memory registry after mutations - Proper validation and error handling Admin routes (admin_routes.py): - All endpoints now functional (no more 501) - Proper error responses (400 for validation, 404 for not found, 500 for server errors) - Lookup by slug/name before operations - Cache refresh after all mutations Features: - Provider deletion blocked if models exist (FK constraint) - All mutations refresh registry cache automatically - Proper logging for audit trail - Admin auth enforced on all endpoints Based on original implementation from PR #11699 (upstream-llm branch). Builds on: - PR #12357: Schema foundation - PR #12359: Registry core - PR #12371: Public read API
Bentlybro
force-pushed
the
feat/llm-public-api
branch
from
April 8, 2026 14:22
90a6808 to
c74652d
Compare
Bentlybro
added a commit
that referenced
this pull request
Apr 8, 2026
Add admin write API endpoints for LLM registry management:
- POST /api/llm/models - Create model
- PATCH /api/llm/models/{slug} - Update model
- DELETE /api/llm/models/{slug} - Delete model
- POST /api/llm/providers - Create provider
- PATCH /api/llm/providers/{name} - Update provider
- DELETE /api/llm/providers/{name} - Delete provider
All endpoints require admin authentication via requires_admin_user.
Request/response models defined in admin_model.py:
- CreateLlmModelRequest, UpdateLlmModelRequest
- CreateLlmProviderRequest, UpdateLlmProviderRequest
Implementation coming in follow-up commits (currently returns 501 Not Implemented).
This builds on:
- PR #12357: Schema foundation
- PR #12359: Registry core
- PR #12371: Public read API
Bentlybro
added a commit
that referenced
this pull request
Apr 8, 2026
Implement full CRUD operations for admin API: Database layer (db_write.py): - create_provider, update_provider, delete_provider - create_model, update_model, delete_model - refresh_runtime_caches - invalidates in-memory registry after mutations - Proper validation and error handling Admin routes (admin_routes.py): - All endpoints now functional (no more 501) - Proper error responses (400 for validation, 404 for not found, 500 for server errors) - Lookup by slug/name before operations - Cache refresh after all mutations Features: - Provider deletion blocked if models exist (FK constraint) - All mutations refresh registry cache automatically - Proper logging for audit trail - Admin auth enforced on all endpoints Based on original implementation from PR #11699 (upstream-llm branch). Builds on: - PR #12357: Schema foundation - PR #12359: Registry core - PR #12371: Public read API
Bentlybro
force-pushed
the
feat/llm-registry-core
branch
from
April 13, 2026 14:47
3919423 to
ef30c1e
Compare
github-actions
Bot
added
the
conflicts
Contributor
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
Implements public GET endpoints for querying LLM models and providers - Part 3 of 6 in the incremental registry rollout. **Endpoints:** - GET /api/llm/models - List all models (filterable by enabled_only) - GET /api/llm/providers - List providers with their models **Design:** - Uses in-memory registry from PR 2 (no DB queries) - Fast reads from cache populated at startup - Grouped by provider for easy UI rendering **Response models:** - LlmModel - model info with capabilities, costs, creator - LlmProvider - provider with nested models - LlmModelsResponse - list + total count - LlmProvidersResponse - grouped by provider **Authentication:** - Requires user auth (requires_user dependency) - Public within authenticated sessions **Integration:** - Registered in rest_api.py at /api prefix - Tagged with v2 + llm for OpenAPI grouping **What's NOT included (later PRs):** - Admin write API (PR 4) - Block integration (PR 5) - Redis cache (PR 6) Lines: ~180 total Files: 4 (3 new, 1 modified) Review time: < 10 minutes
Add two new GET endpoints to the OpenAPI spec: /api/llm/models (with optional enabled_only query param, JWT auth) and /api/llm/providers (JWT auth). These endpoints expose the in-memory LLM registry: list of models and grouped providers with their enabled models. Also add related component schemas (LlmModel, LlmModelCost, LlmModelCreator, LlmModelsResponse, LlmProvider, LlmProvidersResponse) describing model metadata, costs, creators and response shapes.
Introduce an is_enabled: bool = True field to the LlmModel pydantic model to allow toggling model availability. Defaulting to True preserves backward compatibility and avoids breaking changes; can be used by APIs or UIs to filter or disable models without removing them.
Introduce a new boolean property `is_enabled` (default: true) into the OpenAPI schema in autogpt_platform/frontend/src/app/api/openapi.json next to `price_tier` and `is_recommended`. This exposes an enable/disable flag in the API model for consumers and defaults new entries to enabled.
routes_test.py (new, 8 tests): - GET /llm/models: enabled_only default, all, empty, creator, costs - GET /llm/providers: single provider, multiple sorted, empty
…ntation, add total to providers response - Extract _map_model() to eliminate ~25-line duplication between list_models and list_providers - Fix misaligned is_recommended field in list_providers - Remove duplicate tags=["llm"] from router definition - Add total field to LlmProvidersResponse for consistency with LlmModelsResponse - Tighten provider_map type annotation
Bentlybro
force-pushed
the
feat/llm-public-api
branch
from
April 13, 2026 14:52
c74652d to
5b2d459
Compare
Contributor
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
github-actions
Bot
removed
conflicts
Bentlybro
added a commit
that referenced
this pull request
Apr 13, 2026
Add admin write API endpoints for LLM registry management:
- POST /api/llm/models - Create model
- PATCH /api/llm/models/{slug} - Update model
- DELETE /api/llm/models/{slug} - Delete model
- POST /api/llm/providers - Create provider
- PATCH /api/llm/providers/{name} - Update provider
- DELETE /api/llm/providers/{name} - Delete provider
All endpoints require admin authentication via requires_admin_user.
Request/response models defined in admin_model.py:
- CreateLlmModelRequest, UpdateLlmModelRequest
- CreateLlmProviderRequest, UpdateLlmProviderRequest
Implementation coming in follow-up commits (currently returns 501 Not Implemented).
This builds on:
- PR #12357: Schema foundation
- PR #12359: Registry core
- PR #12371: Public read API
Bentlybro
added a commit
that referenced
this pull request
Apr 13, 2026
Implement full CRUD operations for admin API: Database layer (db_write.py): - create_provider, update_provider, delete_provider - create_model, update_model, delete_model - refresh_runtime_caches - invalidates in-memory registry after mutations - Proper validation and error handling Admin routes (admin_routes.py): - All endpoints now functional (no more 501) - Proper error responses (400 for validation, 404 for not found, 500 for server errors) - Lookup by slug/name before operations - Cache refresh after all mutations Features: - Provider deletion blocked if models exist (FK constraint) - All mutations refresh registry cache automatically - Proper logging for audit trail - Admin auth enforced on all endpoints Based on original implementation from PR #11699 (upstream-llm branch). Builds on: - PR #12357: Schema foundation - PR #12359: Registry core - PR #12371: Public read API
|
|
Member
|
Closing in favor of the LLM registry restack: the read surface (now a public catalog endpoint so self-hosted installs can sync) is now #13608 (fresh re-cut onto current dev — the original migrations and seed data had drifted ~4 months). The design and much of the code here carried over directly; @Bentlybro is credited as co-author on the carried commits. Full stack starts at #13605. Thanks for the groundwork — the reviewed schema and cache design survived contact with the restack almost unchanged. |
github-project-automation
Bot
moved this from 🚧 Needs work
to ✅ Done
in AutoGPT development kanban
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add LLM registry public read API - Part 3 of 3 in incremental rollout.
Builds on PR #12359 (registry core) to expose LLM model data via REST endpoints for frontend consumption.
Changes
REST API Endpoints (`backend/server/v2/llm/`)
Response Models
Design
Review Feedback Addressed
Testing
Stacked PRs