feat(backend/copilot): require library similarity check before create_agent - #13080
Merged
Conversation
…_agent Hybrid semantic + lexical search over the user's library agents runs before CoPilot's create_agent tool, so the LLM can recommend an existing agent instead of building a near-duplicate. A hard gate on create_agent mirrors require_guide_read and refuses unless find_library_agent has been called this session (or library_check_ack=true is passed after the user explicitly chooses to build new). Library agents are embedded (text-embedding-3-small, 1536d) into the existing UnifiedContentEmbedding table on create/update via a background asyncio.create_task. A new LibraryAgentHandler joins the unified backfill pipeline so existing rows get embedded too. Semantic-biased weights (0.85/0.10/0/0.05) compensate for LIBRARY_AGENT rows lacking categories and tsvector population in dev environments. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
github-project-automation
Bot
moved this to 🆕 Needs initial review
in AutoGPT development kanban
|
|
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds background embedding generation and a content handler for library agents, implements hybrid semantic+lexical search over library agents, integrates a pre-creation similarity gate into copilot tools (find/create agent) with helper guard logic, and updates tests and documentation. ChangesLibrary Agent Similarity Search and Creation Gate
Sequence DiagramsequenceDiagram
participant User
participant FindTool as FindLibraryAgentTool
participant Hybrid as hybrid_search_library_agents
participant DB as unified_hybrid_search
participant LibraryDB as LibraryAgent DB
participant Scheduler as schedule_library_agent_embedding
User->>FindTool: for_creation + goal_summary
FindTool->>Hybrid: hybrid_search_library_agents(query, user_id)
Hybrid->>DB: unified_hybrid_search(ContentType.LIBRARY_AGENT, weights, user_id, min_score)
DB-->>Hybrid: ranked_results
FindTool->>LibraryDB: get_library_agent(content_id) [per match]
LibraryDB-->>FindTool: agent rows
FindTool-->>User: AgentsFoundResponse / NoResultsResponse (with library_check_ack guidance)
LibraryDB->>Scheduler: schedule_library_agent_embedding(library_agent_id, user_id, graph) [on create/update]
Scheduler->>Scheduler: _build_searchable_text(graph)
Scheduler->>LibraryDB: ensure_content_embedding(ContentType.LIBRARY_AGENT, content_id, user_id, force=True, metadata)
LibraryDB-->>Scheduler: (async, best-effort)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🔍 PR Overlap DetectionThis check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early. 🔴 Merge Conflicts DetectedThe following PRs have been tested and will have merge conflicts if merged after this PR. Consider coordinating with the authors.
🟢 Low Risk — File Overlap OnlyThese PRs touch the same files but different sections (click to expand)
Summary: 4 conflict(s), 0 medium risk, 15 low risk (out of 19 PRs with file overlap) Auto-generated on push. Ignores: |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (5)
autogpt_platform/backend/backend/api/features/library/embeddings.py (1)
59-69: 💤 Low valueConsider tracking the returned task to prevent premature garbage collection.
The fire-and-forget pattern with
asyncio.create_taskis appropriate for this use case, but the returned task is not stored or tracked. While Python 3.11+ keeps strong references to tasks until completion, explicitly tracking background tasks in a module-level set (e.g.,_background_tasks.add(task); task.add_done_callback(_background_tasks.discard)) provides clearer lifecycle management and prevents reliance on implementation details.This is a recommended defensive practice rather than a current bug, since the embedding failures are already logged and swallowed.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@autogpt_platform/backend/backend/api/features/library/embeddings.py` around lines 59 - 69, The schedule_library_agent_embedding function currently calls asyncio.create_task(...) and returns the task without tracking it; add a module-level set (e.g., _background_tasks = set()) and when creating the task in schedule_library_agent_embedding add the task to that set and attach task.add_done_callback(_background_tasks.discard) so the task is kept alive until completion and automatically removed when done; keep returning the task and continue logging failures inside _run_embedding as before.autogpt_platform/backend/backend/copilot/tools/agent_search.py (2)
456-456: 💤 Low valueRemove redundant
float()cast.The
scorevariable is already afloat(frommatch.get("combined_score") or 0.0), so the explicitfloat(score)cast on line 456 is unnecessary.♻️ Suggested simplification
- percent = max(0, min(100, int(round(float(score) * 100)))) + percent = max(0, min(100, int(round(score * 100))))🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@autogpt_platform/backend/backend/copilot/tools/agent_search.py` at line 456, The percent calculation uses an unnecessary float() cast—update the assignment to remove the redundant conversion: replace the line that sets percent (currently using percent = max(0, min(100, int(round(float(score) * 100)))) ) so it instead uses score directly (int(round(score * 100))) while preserving the max/min bounds; reference the variables percent and score and the source match.get("combined_score") or 0.0 to locate the code in agent_search.py.
345-494: ⚡ Quick winConsider extracting the agent-loading loop to improve readability.
This function is 150 lines, significantly exceeding the 40-line guideline. While the logic flow is clear, extracting the agent-loading and description-formatting loop (lines 431-461) into a helper like
_load_and_format_matched_agentswould improve maintainability.As per coding guidelines: "Keep functions under ~40 lines; extract named helpers when a function grows longer"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@autogpt_platform/backend/backend/copilot/tools/agent_search.py` around lines 345 - 494, Extract the agent-loading and description-formatting loop in search_library_for_creation into a new helper function named _load_and_format_matched_agents(matches, user_id, lib_db) that returns list[AgentInfo]; move the logic that iterates matches, skips missing content_id, calls await lib_db.get_library_agent(content_id, user_id), continues on NotFoundError, re-raises DatabaseError, logs and continues on other exceptions, converts library_agent via _library_agent_to_info, computes score/percent/prefix from match["combined_score"], prepends the prefix to info.description (or uses prefix.strip() if empty), and appends to the result list; then replace the original loop in search_library_for_creation with a single await call to this helper and keep all return behaviors unchanged.autogpt_platform/backend/backend/copilot/tools/find_library_agent.py (1)
63-72: 💤 Low valueSchema and description mismatch for
goal_summaryrequirement.The description at lines 66-68 states
goal_summaryis "Required when for_creation=true", but the schema at line 72 does not includegoal_summaryin therequiredarray. This mismatch could confuse tool callers.However, the implementation handles this gracefully:
search_library_for_creationreturns aNoResultsResponsewhengoal_summaryis empty (lines 364-384 in agent_search.py), guiding the LLM to retry with the correct parameters. This soft-fail UX is appropriate, so the current schema may be intentional to avoid hard validation errors.Consider adding a comment explaining this design choice if it's intentional.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@autogpt_platform/backend/backend/copilot/tools/find_library_agent.py` around lines 63 - 72, The schema says "goal_summary" is required when for_creation=true but it's not listed in the "required" array; either make the schema reflect that conditional requirement or (preferable given current soft-fail handling) add an inline comment by the "goal_summary" schema in find_library_agent.py explaining that omission is intentional because search_library_for_creation in agent_search.py returns a NoResultsResponse when goal_summary is empty and the LLM is expected to retry, so we avoid hard validation errors; reference the goal_summary property name and the search_library_for_creation function in your comment.autogpt_platform/backend/backend/api/features/store/embeddings.py (1)
619-645: ⚖️ Poor tradeoffAdd
LIBRARY_AGENTto orphan cleanup coverage if soft-delete handling is needed.
cleanup_orphaned_embeddings()currently excludesLIBRARY_AGENTfrom its cleanup loop. SinceLibraryAgentrows support soft-delete (isDeleted) and hiding (isHidden), embeddings for deleted or hidden agents will persist inUnifiedContentEmbeddingand may still surface inhybrid_search_library_agentsresults unless explicitly removed.If soft-deleted/hidden agents should not return search results, add
LIBRARY_AGENTtocleanup_typesand computecurrent_idsfrom non-deleted, non-hidden agents—mirroring the filter inLibraryAgentHandler.get_missing_items().If persistence is intentional (e.g., via database trigger or handler-level deletion), document it with a comment for clarity.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@autogpt_platform/backend/backend/api/features/store/embeddings.py` around lines 619 - 645, cleanup_orphaned_embeddings currently omits ContentType.LIBRARY_AGENT so embeddings for soft-deleted/hidden LibraryAgent rows can remain; add ContentType.LIBRARY_AGENT to the cleanup_types list and ensure the handler lookup (via CONTENT_HANDLERS) computes current_ids using the same non-deleted/non-hidden filter as LibraryAgentHandler.get_missing_items (i.e., exclude isDeleted/isHidden rows) so UnifiedContentEmbedding rows for hidden/deleted agents are deleted and no longer surface in hybrid_search_library_agents; alternatively, if intentional, add a clear comment explaining that persistence is handled elsewhere.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@autogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py`:
- Line 127: Move the local import of DatabaseError out of the
test_for_creation_db_error_soft_fails function and add it at module scope with
the other imports; specifically, import DatabaseError from
backend.util.exceptions at the top of the file so the test function
(test_for_creation_db_error_soft_fails) can reference DatabaseError without
performing a runtime import inside the function.
In `@autogpt_platform/backend/backend/copilot/tools/helpers_test.py`:
- Around line 1338-1378: Hoist the repeated local imports to the module top:
move the imports for require_library_check, ErrorResponse, and make_session out
of the test functions (test_passes_when_tool_was_called_in_messages,
test_passes_when_tool_was_announced_inflight,
test_returns_error_when_not_called, test_bypassed_in_builder_context) and add
them to the existing top-of-file import block; ensure you keep the relative
single-dot import for ._test_data.make_session and remove the now-unnecessary
inner imports from each test function body.
---
Nitpick comments:
In `@autogpt_platform/backend/backend/api/features/library/embeddings.py`:
- Around line 59-69: The schedule_library_agent_embedding function currently
calls asyncio.create_task(...) and returns the task without tracking it; add a
module-level set (e.g., _background_tasks = set()) and when creating the task in
schedule_library_agent_embedding add the task to that set and attach
task.add_done_callback(_background_tasks.discard) so the task is kept alive
until completion and automatically removed when done; keep returning the task
and continue logging failures inside _run_embedding as before.
In `@autogpt_platform/backend/backend/api/features/store/embeddings.py`:
- Around line 619-645: cleanup_orphaned_embeddings currently omits
ContentType.LIBRARY_AGENT so embeddings for soft-deleted/hidden LibraryAgent
rows can remain; add ContentType.LIBRARY_AGENT to the cleanup_types list and
ensure the handler lookup (via CONTENT_HANDLERS) computes current_ids using the
same non-deleted/non-hidden filter as LibraryAgentHandler.get_missing_items
(i.e., exclude isDeleted/isHidden rows) so UnifiedContentEmbedding rows for
hidden/deleted agents are deleted and no longer surface in
hybrid_search_library_agents; alternatively, if intentional, add a clear comment
explaining that persistence is handled elsewhere.
In `@autogpt_platform/backend/backend/copilot/tools/agent_search.py`:
- Line 456: The percent calculation uses an unnecessary float() cast—update the
assignment to remove the redundant conversion: replace the line that sets
percent (currently using percent = max(0, min(100, int(round(float(score) *
100)))) ) so it instead uses score directly (int(round(score * 100))) while
preserving the max/min bounds; reference the variables percent and score and the
source match.get("combined_score") or 0.0 to locate the code in agent_search.py.
- Around line 345-494: Extract the agent-loading and description-formatting loop
in search_library_for_creation into a new helper function named
_load_and_format_matched_agents(matches, user_id, lib_db) that returns
list[AgentInfo]; move the logic that iterates matches, skips missing content_id,
calls await lib_db.get_library_agent(content_id, user_id), continues on
NotFoundError, re-raises DatabaseError, logs and continues on other exceptions,
converts library_agent via _library_agent_to_info, computes score/percent/prefix
from match["combined_score"], prepends the prefix to info.description (or uses
prefix.strip() if empty), and appends to the result list; then replace the
original loop in search_library_for_creation with a single await call to this
helper and keep all return behaviors unchanged.
In `@autogpt_platform/backend/backend/copilot/tools/find_library_agent.py`:
- Around line 63-72: The schema says "goal_summary" is required when
for_creation=true but it's not listed in the "required" array; either make the
schema reflect that conditional requirement or (preferable given current
soft-fail handling) add an inline comment by the "goal_summary" schema in
find_library_agent.py explaining that omission is intentional because
search_library_for_creation in agent_search.py returns a NoResultsResponse when
goal_summary is empty and the LLM is expected to retry, so we avoid hard
validation errors; reference the goal_summary property name and the
search_library_for_creation function in your comment.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: dee697ce-196d-4d21-b6b4-211e285f9571
📒 Files selected for processing (17)
autogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/sdk/agent_generation_guide.mdautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: check API types
- GitHub Check: end-to-end tests
- GitHub Check: Check PR Status
- GitHub Check: type-check (3.12)
- GitHub Check: test (3.11)
- GitHub Check: test (3.13)
- GitHub Check: type-check (3.13)
- GitHub Check: type-check (3.11)
- GitHub Check: test (3.12)
- GitHub Check: Analyze (typescript)
- GitHub Check: Analyze (python)
🧰 Additional context used
📓 Path-based instructions (6)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
autogpt_platform/backend/backend/api/features/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Update routes in '/backend/backend/api/features/' and add/update Pydantic models in the same directory for API development
Files:
autogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.py
autogpt_platform/backend/**/api/**/*.py
📄 CodeRabbit inference engine (autogpt_platform/backend/AGENTS.md)
autogpt_platform/backend/**/api/**/*.py: UseSecurity()instead ofDepends()for authentication dependencies to get proper OpenAPI security specification
Follow SSE (Server-Sent Events) protocol: usedata:lines for frontend-parsed events (must match Zod schema) and: commentlines for heartbeats/status
Files:
autogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.py
autogpt_platform/backend/**/*_test.py
📄 CodeRabbit inference engine (autogpt_platform/backend/AGENTS.md)
autogpt_platform/backend/**/*_test.py: Use pytest with snapshot testing for API responses
Colocate test files with source files using*_test.pynaming convention
Mock at boundaries — mock where the symbol is used, not where it's defined; after refactoring, update mock targets to match new module paths
UseAsyncMockfromunittest.mockfor async functions in tests
When writing tests, use Test-Driven Development (TDD): write failing tests marked with@pytest.mark.xfailbefore implementation, then remove the marker once the implementation is complete
When creating snapshots in tests, usepoetry run pytest path/to/test.py --snapshot-update; always review snapshot changes withgit diffbefore committing
Files:
autogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
autogpt_platform/backend/**/*.md
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Document agent responsibilities and interfaces in markdown files
Files:
autogpt_platform/backend/backend/copilot/sdk/agent_generation_guide.md
🧠 Learnings (13)
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/sdk/agent_generation_guide.mdautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-04-01T04:17:41.600Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12632
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-01T04:17:41.600Z
Learning: When reviewing AutoGPT Copilot tool implementations, accept that `readOnlyHint=True` (provided via `ToolAnnotations`) may be applied unconditionally to *all* tools—even tools that have side effects (e.g., `bash_exec`, `write_workspace_file`, or other write/save operations). Do **not** flag these tools for having `readOnlyHint=True`; this is intentional to enable fully-parallel dispatch by the Anthropic SDK/CLI and has been E2E validated. Only flag `readOnlyHint` issues if they conflict with the established `ToolAnnotations` behavior (e.g., missing/incorrect propagation relative to the intended annotation mechanism).
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-03-04T12:19:39.243Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12279
File: autogpt_platform/backend/backend/copilot/tools/base.py:184-188
Timestamp: 2026-03-04T12:19:39.243Z
Learning: In autogpt_platform/backend/backend/copilot/tools/, ensure that anonymous users always pass user_id=None to tool execution methods. The anon_ prefix (e.g., anon_123) is used only for PostHog/analytics distinct_id and must not be used as an actual user_id. Use a simple truthiness check on user_id (e.g., if user_id: ... else: ... or a dedicated is_authenticated flag) to distinguish anonymous from authenticated users, and review all tool execution call sites within this directory to prevent accidentally forwarding an anon_ user_id to tools.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-03-31T14:22:26.566Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12622
File: autogpt_platform/backend/backend/copilot/tools/agent_search.py:223-236
Timestamp: 2026-03-31T14:22:26.566Z
Learning: In files under autogpt_platform/backend/backend/copilot/tools/, ensure agent graph enrichment uses the typed Pydantic model `backend.data.graph.Graph` for `AgentInfo.graph` (i.e., `Graph | None`), not `dict[str, Any]`. When enriching with graph data (e.g., `_enrich_agents_with_graph`), prefer calling `graph_db().get_graph(graph_id, version=None, user_id=user_id)` directly to retrieve the typed `Graph` object rather than routing through JSON conversions like `get_agent_as_json()` / `graph_to_json()`.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-04-15T02:43:36.890Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-04-22T11:46:04.431Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/config.py:0-0
Timestamp: 2026-04-22T11:46:04.431Z
Learning: Do not flag the Claude Sonnet 4.6 model ID as incorrect when it uses the project’s established hyphenated convention: `anthropic/claude-sonnet-4-6`. This hyphen form is the intentional, production convention and should be treated as valid (including in files like llm.py, blocks tests, reasoning.py, `_is_anthropic_model` tests, and config defaults). Note that OpenRouter also accepts the dot variant `anthropic/claude-sonnet-4.6`, so either form may be tolerated, but `anthropic/claude-sonnet-4-6` should be considered the standard to match project usage.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-04-22T11:46:12.892Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12881
File: autogpt_platform/backend/backend/copilot/baseline/service.py:322-332
Timestamp: 2026-04-22T11:46:12.892Z
Learning: In this codebase (Significant-Gravitas/AutoGPT), OpenRouter-routed Anthropic model IDs should use the hyphen-separated convention (e.g., `anthropic/claude-sonnet-4-6`, `anthropic/claude-opus-4-6`). Although OpenRouter may accept both hyphen and dot variants, treat the hyphen-separated form as the intended, correct codebase-wide convention and do not flag it as an error. Only flag the dot-separated variant (e.g., `anthropic/claude-sonnet-4.6`) as incorrect when reviewing/validating model ID strings for OpenRouter-routed Anthropic models.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-05-07T18:48:14.242Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 13040
File: autogpt_platform/backend/backend/blocks/llm.py:0-0
Timestamp: 2026-05-07T18:48:14.242Z
Learning: In this repository, isort may split imports from the same module into separate blocks when some imports are aliased (e.g., `from module import X as Y`) and others are not. Preserve the two-block layout when it results from isort (such as keeping `from openai.types.chat import ChatCompletion as OpenAIChatCompletion` separate from non-aliased imports from `openai.types.chat`). Do not treat that split as a style issue during review; merging them into a single block can fail CI with `Imports are incorrectly sorted and/or formatted`.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/create_agent.pyautogpt_platform/backend/backend/api/features/library/embeddings.pyautogpt_platform/backend/backend/api/features/store/embeddings.pyautogpt_platform/backend/backend/copilot/tools/helpers_test.pyautogpt_platform/backend/backend/copilot/tools/create_agent_test.pyautogpt_platform/backend/backend/copilot/tools/_test_data.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/api/features/store/content_handlers.pyautogpt_platform/backend/backend/api/features/library/db.pyautogpt_platform/backend/backend/copilot/tools/agent_search.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent.pyautogpt_platform/backend/backend/api/features/store/content_handlers_test.pyautogpt_platform/backend/backend/api/features/library/search.pyautogpt_platform/backend/backend/api/features/library/search_test.pyautogpt_platform/backend/backend/api/features/library/embeddings_test.pyautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py
📚 Learning: 2026-03-15T15:30:02.282Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:02.282Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, inside execute_block: when InsufficientBalanceError occurs after post-execution credit charging (i.e., balance drained concurrently after pre-check passed), treat as a non-fatal billing leak. Log at ERROR level with structured JSON: {"billing_leak": True, "user_id": ..., "cost": ...} for monitoring/alerting, then return BlockOutputResponse normally (do not discard the output). Do not perform a second get_user_credit_model call; reuse the credit_model obtained during the pre-execution balance check (guarded by if cost > 0 and credit_model:). This guidance improves UX by not discarding results and provides observable billing leak signals.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
🔇 Additional comments (13)
autogpt_platform/backend/backend/api/features/library/db.py (2)
601-606: LGTM!The embedding scheduler is integrated cleanly into the agent creation flow. The fire-and-forget pattern is appropriate here, and the comment clearly explains the purpose of the background task.
768-770: LGTM!The re-embedding on version/settings update ensures the similarity search reflects the latest agent metadata. The placement after the settings update is correct.
autogpt_platform/backend/backend/api/features/library/search.py (1)
48-80: LGTM!The hybrid search implementation is clean and well-documented. The early return for empty queries avoids unnecessary database calls, and the parameter forwarding to
unified_hybrid_searchis correct. The use ofmax(1, limit)prevents invalid page_size values.autogpt_platform/backend/backend/api/features/library/search_test.py (1)
24-61: LGTM!The test coverage is comprehensive and validates the key behaviors: empty query short-circuit, correct parameter delegation, and min_score override. The mock patching follows best practices by patching where the symbol is used.
autogpt_platform/backend/backend/copilot/tools/helpers.py (1)
933-973: LGTM!The library-similarity gate implementation mirrors the existing
require_guide_readpattern, which provides good consistency. The builder-bound bypass is appropriate, and the error message clearly guides the LLM through the required flow.autogpt_platform/backend/backend/copilot/tools/_test_data.py (1)
45-77: LGTM!The test helper extension mirrors the existing
guide_readpattern, maintaining consistency. Defaultinglibrary_check=Trueis sensible for existing tests, while allowingFalsefor gate-specific test coverage.autogpt_platform/backend/backend/copilot/tools/create_agent.py (1)
107-110: LGTM — gate is correctly bypassed by explicit acknowledgement.The early-return shape mirrors the adjacent
require_guide_readgate, and theif not library_check_ack:short-circuit cleanly maps to the documented bypass contract inagent_generation_guide.md.autogpt_platform/backend/backend/copilot/tools/create_agent_test.py (1)
179-284: LGTM — gate behavior covered across the three branches.Tests cleanly cover: gate-error path (with required error-message tokens),
library_check_ack=Truebypass, and the builder-context bypass. Patching at the use site (backend.copilot.tools.create_agent.fix_validate_and_save) is correct per the mocking-at-boundaries convention.autogpt_platform/backend/backend/copilot/sdk/agent_generation_guide.md (1)
8-31: Clear and complete documentation of the new gate.The instructions correctly distinguish:
- The create-time similarity check (step 1) versus the sub-agent composition lookup (step 4)
- The
library_check_ack=trueescape hatch and its "only after the user has seen the matches" constraint- The automatic builder-context bypass
This matches the gate semantics enforced in
require_library_checkand the test cases increate_agent_test.py.autogpt_platform/backend/backend/api/features/store/content_handlers.py (1)
647-731: LGTM — user-scoped backfill semantics look correct.Key invariants checked:
- The
LEFT JOIN ... uce."userId" = la."userId"+uce."contentId" IS NULLfilter correctly scopes "missing" per (library_agent_id, user_id), matching the(contentType, contentId, userId)unique key documented in the class docstring.INNER JOIN AgentGraphon(agentGraphId, agentGraphVersion)ensures only rows whose pinned graph version still exists are considered (orphaned library agents are skipped silently).get_stats()uses the same join shape sototalandwith_embeddingsare computed against the same eligibility filter (isDeleted=false AND isHidden=false), sowithout_embeddings = total - with_embeddingscannot go negative under normal conditions.autogpt_platform/backend/backend/api/features/store/content_handlers_test.py (1)
558-626: LGTM — coverage for both handler methods and the registry entry.
test_library_agent_handler_emits_user_scoped_itemsprecisely pins the empty-field skipping behavior (description="",instructions=None→searchable_text == "Inbox Triage"), and the side-effect dispatch intest_library_agent_handler_stats("uce"/"UnifiedContentEmbedding"→ embedded count, else total) correctly disambiguates the two SQL paths inLibraryAgentHandler.get_stats.autogpt_platform/backend/backend/api/features/library/embeddings_test.py (1)
1-80: LGTM — solid coverage for the scheduler's contracts.Tests pin down the four behaviors that matter for production safety:
- empty-content short-circuit
force=Trueforwarding (so updates re-embed rather than keep stale vectors)- error swallowing (so a failing embed never breaks the library-write path)
- returns an
asyncio.Taskthat can be awaited by tests / background runnersautogpt_platform/backend/backend/copilot/tools/find_library_agent_test.py (1)
63-72: Good boundary mocking and async mock usage.Patching where symbols are used and using
AsyncMockon async paths keeps these tests stable and aligned with the tool contract.Also applies to: 164-177
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #13080 +/- ##
==========================================
+ Coverage 71.91% 72.05% +0.13%
==========================================
Files 2256 2261 +5
Lines 170413 171084 +671
Branches 17283 17368 +85
==========================================
+ Hits 122554 123272 +718
+ Misses 44205 44150 -55
- Partials 3654 3662 +8
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…ity gate - Hoist DatabaseError + require_library_check / make_session imports to module top in find_library_agent_test.py and helpers_test.py per the top-level-imports-only guideline. - Track schedule_library_agent_embedding tasks in a module-level _background_tasks set with add_done_callback discard, so the fire-and-forget tasks aren't reliant on CPython 3.11+ strong-reference behaviour. - Extract the agent-loading loop from search_library_for_creation into a _load_and_format_matched_agents helper (function was 150+ lines, guideline is ~40). - Drop a redundant float() cast on combined_score in agent_search.py. - Document that goal_summary is intentionally not in find_library_agent's required[] (soft-fail returns NoResultsResponse with a recovery hint). - Add LIBRARY_AGENT to cleanup_orphaned_embeddings using the same non-deleted/non-hidden filter as LibraryAgentHandler.get_missing_items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI lint failed on three files because my local `poetry run format` exited early on a pre-existing ruff F841 issue in `store/db_test.py` and never reached the Black step. Re-running Black directly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI lint round 2 caught search_test.py and content_handlers_test.py — same root cause as the previous formatting commit (local poetry run format exited early on a pre-existing ruff issue in store/db_test.py). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
isort --profile black wants two-name imports on a single line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI test_total_schema_char_budget caught my new tool docs pushing the combined tool-schema size from ~35.3k to ~36.4k chars, over the 35,500 limit. Trimmed find_library_agent / create_agent descriptions to keep only the LLM-decision-critical signal: what the new mode does and the ordering constraint vs create_agent. Long-form rationale lives in agent_generation_guide.md (read once per session via get_agent_building_guide), so the LLM still gets it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
goodluck1103
force-pushed
the
feat/copilot-library-similarity-gate
branch
from
May 11, 2026 21:11
8c9a490 to
d27b66f
Compare
…ty search
Live tracing on the 9 near-duplicate "YouTube Video Summarizer" agents
showed lexical_score = 0.000 across the board even though the tsvector
trigger was correctly populating UnifiedContentEmbedding.search and
ts_rank() reported ~0.97 for the right rows. Root cause:
plainto_tsquery AND-joins every word in the query, so a 14-word goal
("Summarize a YouTube video with timestamped bullet points and a topic
summary from a URL input") requires every stemmed term to appear in
the agent description, which never holds for natural-language descriptions
of a similar agent.
Add an optional ``lexical_query`` parameter to ``unified_hybrid_search``
(defaults to ``query`` — no behaviour change for store / block / doc
callers), and wire ``hybrid_search_library_agents`` to send the full
sentence to the embedding path and a stopword-stripped keyword form
(``"summarize youtube video timestamped bullet"``) to the lexical path.
Verified on the real DB: same goal, same user, now returns 5 matches
with lexical_score 0.889–1.000 and combined_score 0.78–0.83 (was 0.000
and ~0.43).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…orks The semantic-heavy 85/10/0/5 mix was a workaround while plainto_tsquery zeroed every lexical score. With the keyword-extracted lexical query landing 0.89–1.00 on real near-duplicates, that imbalance leaves accuracy on the table. Empirical sweep against the 9-YouTube-duplicate corpus: current 85/10/0/5 duplicates 0.85 unrelated 0.29-0.38 50/40/0/10 duplicates 0.91 unrelated 0.24-0.29 45/45/0/10 duplicates 0.92 unrelated 0.23-0.27 Picking 50/40/0/10 (closer to ``DEFAULT_UNIFIED_WEIGHTS`` minus ``category``, which library agents don't have) — duplicates up, unrelated down, separation gap widens 0.47 -> 0.62. Threshold of 0.55 stays since it now sits comfortably in the middle of that gap. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
ntindle
previously approved these changes
May 18, 2026
ntindle
left a comment
ntindle
left a comment
Member
There was a problem hiding this comment.
Please upload video of testing to test plan
github-project-automation
Bot
moved this from 🆕 Needs initial review
to 👍🏼 Mergeable
in AutoGPT development kanban
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
…llback The `for_creation=true` path used `goal_summary or query` so a call with only `query` would run the hybrid search but then fail the create_agent gate (which strictly requires non-empty `goal_summary`), forcing the LLM into a retry loop. Drop the fallback so the tool's contract matches the gate validator. Empty `goal_summary` still soft-fails to NoResultsResponse with a retry hint, so well-formed callers are unaffected. Addresses Sentry MEDIUM bug prediction on find_library_agent.py:83.
The unified_hybrid_search user filter was ``AND (uce."userId" = $N OR uce."userId" IS NULL)``, which is correct for public types (STORE_AGENT/BLOCK/DOCUMENTATION) but would leak LIBRARY_AGENT rows across users if a write-path bug or migration ever produced one with NULL userId. Add a defense-in-depth clause that explicitly excludes that combination. No-op for well-formed data; closes the leak path called out by Sentry HIGH on hybrid_search.py:259.
10 tasks
…changed text - Re-embed only when the existing UnifiedContentEmbedding row's searchableText differs from the new (name, description, instructions). Skips a wasted OpenAI call on settings-only version bumps. - Drop multi-paragraph rationale comments and Sentry-prediction citations across helpers.py, find_library_agent.py, hybrid_search.py.
…storage CHECK - Turn-scoped gate: ``require_library_check`` now reads only the current turn's in-flight find_library_agent call; a stale call from a prior turn against an unrelated goal_summary no longer satisfies the gate for a new create_agent request. - ``AgentInfo.match_score`` field replaces the description-prefix hack; LLM message instructs the model to format ``[N% match]`` itself, so downstream consumers of AgentInfo see clean text. - PostHog ``copilot_library_check_outcome`` event tracks matches_shown / no_matches / soft_failed / bypassed_ack so the gate's effectiveness (and the 0.55 threshold) can be evaluated. - Hybrid search queries at min_score=0 and filters in the wrapper, logging the top 5 raw scores so sub-threshold near-misses are visible for retuning. - Soft-fail paths log at ERROR (captured by Sentry's LoggingIntegration) so a flaky embedding service can't silently disable the gate. - SQL CHECK constraint enforces ``contentType != 'LIBRARY_AGENT' OR userId IS NOT NULL`` at the storage layer; the runtime SQL filter becomes redundancy rather than the only defense.
Contributor
Update: pushed fixes for all five items
Test status64 tests passing across the touched suites ( |
github-actions
Bot
added
platform/frontend
Contributor
|
@anvyle — design question worth your call before this lands: Is the hard gate the right shape, or should this be a soft suggestion + server-side hash dedupe instead? Current shape (hard gate, what's in this PR):
Alternative (soft + hash):
Tradeoff: soft+hash doesn't prevent the dupe — by the time the user sees the suggestion, the agent exists. They'd need a one-click "delete this, use that" action. The hard gate gets you stronger preventive UX if the LLM cooperates. The hard-gate shape only wins if the model can usefully judge "the user has a similar agent but it's broken, let me build new" — which is real but probably the minority case. I'm not in a position to switch shapes unilaterally — that's a real redesign touching |
majdyz
previously approved these changes
May 25, 2026
github-project-automation
Bot
moved this from ✅ Done
to 👍🏼 Mergeable
in AutoGPT development kanban
…ry gate The create-time library-similarity gate adds ``for_creation``, ``goal_summary``, and ``library_check_ack`` parameters across find_library_agent + create_agent. CI registers more env-flagged tools than local, so the bump absorbs ~270 extra chars on CI.
majdyz
approved these changes
May 25, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
May 25, 2026
github-project-automation
Bot
moved this from 👍🏼 Mergeable
to ✅ Done
in AutoGPT development kanban
14 tasks
psbuilds
pushed a commit
to psbuilds/AutoGPT
that referenced
this pull request
May 28, 2026
…_agent (Significant-Gravitas#13080) ### Why / What / How **Why.** When a user describes a goal to CoPilot ("build me an agent that summarises my Gmail every morning"), the LLM has been free to call `create_agent` immediately — even when the user already has a near-identical agent in their library. The result is clutter, wasted credits, and a worse experience than just running what they already have. A real-world example from this dev account: nine `YouTube Video Summarizer` agents accumulated over time before this gate existed. **What.** Before `create_agent` runs, CoPilot must search the user's library for a functionally similar agent (hybrid semantic + lexical) and surface any matches. A hard gate refuses `create_agent` until that check has happened. If the user has been shown the matches and explicitly chose to build new anyway, the LLM retries with `library_check_ack=true` to bypass. **How.** - **Embedding write hook.** A new `schedule_library_agent_embedding()` fires `asyncio.create_task(...)` from `create_library_agent` and `update_library_agent_version_and_settings` (mirrors the existing `add_generated_agent_image` pattern). Library-agent name + description + instructions are embedded with the existing `ensure_content_embedding(ContentType.LIBRARY_AGENT, ...)` into `UnifiedContentEmbedding`, scoped by `userId`. - **Backfill.** A new `LibraryAgentHandler` joins `CONTENT_HANDLERS`, and `LIBRARY_AGENT` is appended to `backfill_all_content_types` so existing library agents become discoverable on first run. - **Hybrid search wrapper.** `hybrid_search_library_agents()` in `backend/api/features/library/search.py` delegates to the existing `unified_hybrid_search()` via the `db_accessors.search()` shim (so it works whether Prisma is connected in-process or only via the database-manager RPC service — same path `find_block` / `search_docs` already use). Library-specific weights `(semantic=0.50, lexical=0.40, category=0.0, recency=0.10)` and threshold `0.55`; `category` is zeroed because LIBRARY_AGENT rows have no categories, and the lexical query is keyword-extracted before being fed to `plainto_tsquery` so its AND-of-terms doesn't zero out matches on long natural-language goals. - **Tool upgrade.** `find_library_agent` gains `for_creation: bool` and `goal_summary: str`. When `for_creation=true`, it returns matches as the existing `AgentsFoundResponse` with each description prefixed by `[N% match]` (using `combined_score`, not post-BM25 `relevance`, since BM25 goes negative for near-duplicate corpora). - **Gate.** `require_library_check(session, tool_name)` in `helpers.py` mirrors `require_guide_read`: bypassed in builder-bound sessions, satisfied once `find_library_agent` has been called this session, otherwise returns an `ErrorResponse` instructing the LLM to call it. - **Wire-up.** `create_agent` calls the gate immediately after `require_guide_read`, accepting an explicit `library_check_ack: bool` parameter to bypass after explicit user confirmation. The agent-generation guide (`agent_generation_guide.md`) documents the workflow as the new step 1. ### Changes 🏗️ - New file `backend/api/features/library/embeddings.py` — `schedule_library_agent_embedding()` fire-and-forget background task. - New file `backend/api/features/library/search.py` — `hybrid_search_library_agents()`, `LIBRARY_SIMILARITY_THRESHOLD = 0.55`, library-specific `UnifiedSearchWeights`. - New file `backend/copilot/tools/find_library_agent_test.py` — hybrid mode (ranked results, no-matches, soft-fails on missing goal or DB error, default substring path unchanged). - New `LibraryAgentHandler` in `backend/api/features/store/content_handlers.py` + registry entry; `LIBRARY_AGENT` added to `backfill_all_content_types`. - `backend/api/features/library/db.py` — schedules embedding on create + version update. - `backend/copilot/tools/find_library_agent.py` — new `for_creation` / `goal_summary` parameters. - `backend/copilot/tools/agent_search.py` — `search_library_for_creation()` helper; soft-fails (`NoResultsResponse`) on missing goal or backend errors so the chat UI never renders "Error finding agents". - `backend/copilot/tools/helpers.py` — `require_library_check()` gate. - `backend/copilot/tools/create_agent.py` — new `library_check_ack` parameter; gate call after the guide-read gate; updated tool description. - `backend/copilot/sdk/agent_generation_guide.md` — new step 1 documenting the create-time similarity check, distinguishing it from the sub-agent-composition use of `find_library_agent`, and fixing the pre-existing duplicate `8.` numbering. - `backend/copilot/tools/_test_data.py` — `make_session()` accepts `library_check=True/False` so tests can opt into exercising the gate. - Tests added/updated across the touched modules (32 focused tests passing locally). No frontend changes (existing `AgentsFoundResponse` SSE rendering is reused). No Prisma migration (`ContentType.LIBRARY_AGENT` and indices already exist on `UnifiedContentEmbedding`). ### Checklist 📋 #### For code changes: - [ ] I have clearly listed my changes in the PR description - [ ] I have made a test plan - [ ] I have tested my changes according to the test plan: - [ ] `poetry run pytest backend/api/features/library/search_test.py backend/api/features/library/embeddings_test.py backend/copilot/tools/find_library_agent_test.py backend/copilot/tools/create_agent_test.py backend/copilot/tools/helpers_test.py::TestRequireLibraryCheck backend/api/features/store/content_handlers_test.py` — 32 passed locally - [ ] `poetry run ruff check` clean on all touched files - [ ] Local CoPilot run: empty-library user asks to create an agent → `find_library_agent(for_creation=true)` returns `NoResultsResponse`, `create_agent` proceeds (gate satisfied by the call) - [ ] Local CoPilot run with 9 pre-existing `YouTube Video Summarizer` duplicates: same goal returns 5 matches at 75–76% combined score; LLM surfaces them via `AgentsFoundResponse` - [ ] Builder-bound session (`metadata.builder_graph_id` set): gate is bypassed - [ ] Backfill verified via `backfill_all_content_types(50)` → `get_embedding_stats()['by_type']['LIBRARY_AGENT'].coverage_percent` rises from 0 to 100 #### For configuration changes: - [ ] `.env.default` is updated or already compatible with my changes - [ ] `docker-compose.yml` is updated or already compatible with my changes - [ ] I have included a list of my configuration changes in the PR description (under **Changes**) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: majdyz <zamil.majdy@agpt.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why / What / How
Why. When a user describes a goal to CoPilot ("build me an agent that summarises my Gmail every morning"), the LLM has been free to call
create_agentimmediately — even when the user already has a near-identical agent in their library. The result is clutter, wasted credits, and a worse experience than just running what they already have. A real-world example from this dev account: nineYouTube Video Summarizeragents accumulated over time before this gate existed.What. Before
create_agentruns, CoPilot must search the user's library for a functionally similar agent (hybrid semantic + lexical) and surface any matches. A hard gate refusescreate_agentuntil that check has happened. If the user has been shown the matches and explicitly chose to build new anyway, the LLM retries withlibrary_check_ack=trueto bypass.How.
schedule_library_agent_embedding()firesasyncio.create_task(...)fromcreate_library_agentandupdate_library_agent_version_and_settings(mirrors the existingadd_generated_agent_imagepattern). Library-agent name + description + instructions are embedded with the existingensure_content_embedding(ContentType.LIBRARY_AGENT, ...)intoUnifiedContentEmbedding, scoped byuserId.LibraryAgentHandlerjoinsCONTENT_HANDLERS, andLIBRARY_AGENTis appended tobackfill_all_content_typesso existing library agents become discoverable on first run.hybrid_search_library_agents()inbackend/api/features/library/search.pydelegates to the existingunified_hybrid_search()via thedb_accessors.search()shim (so it works whether Prisma is connected in-process or only via the database-manager RPC service — same pathfind_block/search_docsalready use). Library-specific weights(semantic=0.50, lexical=0.40, category=0.0, recency=0.10)and threshold0.55;categoryis zeroed because LIBRARY_AGENT rows have no categories, and the lexical query is keyword-extracted before being fed toplainto_tsqueryso its AND-of-terms doesn't zero out matches on long natural-language goals.find_library_agentgainsfor_creation: boolandgoal_summary: str. Whenfor_creation=true, it returns matches as the existingAgentsFoundResponsewith each description prefixed by[N% match](usingcombined_score, not post-BM25relevance, since BM25 goes negative for near-duplicate corpora).require_library_check(session, tool_name)inhelpers.pymirrorsrequire_guide_read: bypassed in builder-bound sessions, satisfied oncefind_library_agenthas been called this session, otherwise returns anErrorResponseinstructing the LLM to call it.create_agentcalls the gate immediately afterrequire_guide_read, accepting an explicitlibrary_check_ack: boolparameter to bypass after explicit user confirmation. The agent-generation guide (agent_generation_guide.md) documents the workflow as the new step 1.Changes 🏗️
backend/api/features/library/embeddings.py—schedule_library_agent_embedding()fire-and-forget background task.backend/api/features/library/search.py—hybrid_search_library_agents(),LIBRARY_SIMILARITY_THRESHOLD = 0.55, library-specificUnifiedSearchWeights.backend/copilot/tools/find_library_agent_test.py— hybrid mode (ranked results, no-matches, soft-fails on missing goal or DB error, default substring path unchanged).LibraryAgentHandlerinbackend/api/features/store/content_handlers.py+ registry entry;LIBRARY_AGENTadded tobackfill_all_content_types.backend/api/features/library/db.py— schedules embedding on create + version update.backend/copilot/tools/find_library_agent.py— newfor_creation/goal_summaryparameters.backend/copilot/tools/agent_search.py—search_library_for_creation()helper; soft-fails (NoResultsResponse) on missing goal or backend errors so the chat UI never renders "Error finding agents".backend/copilot/tools/helpers.py—require_library_check()gate.backend/copilot/tools/create_agent.py— newlibrary_check_ackparameter; gate call after the guide-read gate; updated tool description.backend/copilot/sdk/agent_generation_guide.md— new step 1 documenting the create-time similarity check, distinguishing it from the sub-agent-composition use offind_library_agent, and fixing the pre-existing duplicate8.numbering.backend/copilot/tools/_test_data.py—make_session()acceptslibrary_check=True/Falseso tests can opt into exercising the gate.No frontend changes (existing
AgentsFoundResponseSSE rendering is reused). No Prisma migration (ContentType.LIBRARY_AGENTand indices already exist onUnifiedContentEmbedding).Checklist 📋
For code changes:
poetry run pytest backend/api/features/library/search_test.py backend/api/features/library/embeddings_test.py backend/copilot/tools/find_library_agent_test.py backend/copilot/tools/create_agent_test.py backend/copilot/tools/helpers_test.py::TestRequireLibraryCheck backend/api/features/store/content_handlers_test.py— 32 passed locallypoetry run ruff checkclean on all touched filesfind_library_agent(for_creation=true)returnsNoResultsResponse,create_agentproceeds (gate satisfied by the call)YouTube Video Summarizerduplicates: same goal returns 5 matches at 75–76% combined score; LLM surfaces them viaAgentsFoundResponsemetadata.builder_graph_idset): gate is bypassedbackfill_all_content_types(50)→get_embedding_stats()['by_type']['LIBRARY_AGENT'].coverage_percentrises from 0 to 100For configuration changes:
.env.defaultis updated or already compatible with my changesdocker-compose.ymlis updated or already compatible with my changes