Skip to content

Add tool approval integration for Vercel AI adapter - #1

Closed
bendrucker wants to merge 21 commits into
vercel-ai-data-chunk-fieldsfrom
vercel-ai-tool-approval
Closed

Add tool approval integration for Vercel AI adapter#1
bendrucker wants to merge 21 commits into
vercel-ai-data-chunk-fieldsfrom
vercel-ai-tool-approval

Conversation

@bendrucker

Copy link
Copy Markdown
Owner

Adds tool approval integration for the Vercel AI adapter, enabling human-in-the-loop workflows with the AI SDK.

This PR builds on pydantic#3760 and implements the basic plumbing for tool approval. The integration is functional for emitting approval requests and parsing approval responses, but does not yet automatically wire up the full flow.

Changes

  • Adds ToolApprovalRequested and ToolApprovalResponded types to represent approval state
  • Adds approval field to all ToolUIPart and DynamicToolUIPart variants
  • Emits ToolApprovalRequestChunk when agent returns DeferredToolRequests
  • Adds deferred_tool_results property on VercelAIAdapter to extract approval responses from incoming tool parts
  • Adds extract_deferred_tool_results() class method for parsing approvals

How It Works

Approval Request (server → client):

  1. Agent tool has requires_approval=True
  2. When model calls the tool, agent returns DeferredToolRequests
  3. VercelAIEventStream.handle_run_result() emits tool-approval-request chunk
  4. Client receives chunk and shows approval UI

Approval Response (client → server):

  1. User approves or denies in the UI
  2. Client updates the tool part's approval field with {id, approved, reason?}
  3. Next SubmitMessage includes tool parts with approval responses
  4. VercelAIAdapter.deferred_tool_results extracts these into DeferredToolResults

Open Questions

1. Automatic vs Manual Integration

Currently, users must manually wire up the deferred tool results:

adapter = VercelAIAdapter(agent, request)
deferred = adapter.deferred_tool_results

# User must pass deferred_tool_results to run_stream
events = adapter.run_stream(deferred_tool_results=deferred)

Question: Should run_stream() automatically use self.deferred_tool_results when not explicitly provided? This would make the integration seamless but less explicit.

2. Approval ID Tracking

We generate a UUID for approval_id in ToolApprovalRequestChunk, but the AI SDK uses this ID to track the approval lifecycle. The ToolApprovalResponded.id field should match the original approval_id.

Question: Do we need to store/track these approval IDs to validate they match on response? Currently we don't validate this.

3. ToolOutputDeniedChunk

The ToolOutputDeniedChunk type exists but isn't emitted anywhere. In Pydantic AI, denied tool results flow as regular ToolReturnPart with the denial message as content.

Question: Should we emit ToolOutputDeniedChunk when the deferred tool result is a ToolDenied? This would require tracking which tool calls were denied through the agent run.

4. Message History Handling

When continuing with deferred tool results, the message history needs to include the original messages. Currently the adapter's messages property processes all incoming messages, but:

Question: Should tool parts with pending approvals (only ToolApprovalRequested, not ToolApprovalResponded) be filtered out or handled specially when building message history?

Testing

  • Added test for ToolApprovalRequestChunk emission when requires_approval=True tool is called
  • Added tests for extract_deferred_tool_results() covering approved, denied, and no-approval cases

References

@bendrucker
bendrucker force-pushed the vercel-ai-tool-approval branch from ab6ca8a to 099f07a Compare December 19, 2025 05:03
@bendrucker
bendrucker force-pushed the vercel-ai-tool-approval branch from 099f07a to 1160591 Compare December 19, 2025 05:03
- Add test for from_request() with tool_approval parameter

- Add test verifying approval chunks not emitted when tool_approval=False

- Add test for deferred_tool_results fallback from instance field

- Add test for denied_tool_ids with ToolUIPart (builtin tools)

- Fix docs link to point to VercelAIAdapter.from_request

- Reword Tool Approval section to clarify AI SDK UI vs AI Elements
- Make _extract_deferred_tool_results private and inline into from_request
- Make _denied_tool_ids a private cached_property
- Simplify approval values to True/False (per reviewer suggestion)
- Add AI SDK v6 requirement note to documentation and docstrings
- Update tests for new API and simplified return values
- Inline extract_deferred_tool_results logic into from_request()
- Remove unit tests for private _extract_deferred_tool_results method
- Remove unit tests for private _denied_tool_ids property
- Update test_tool_output_denied_chunk_emission to use public interface
- Update test_tool_output_denied_chunk_emission to use from_request()
  with explicit type binding to test the full public interface
- Remove test_from_request_with_tool_approval_enabled (now redundant)
- Remove test_deferred_tool_results_fallback_from_instance (tested
  internal plumbing rather than observable behavior)
@bendrucker
bendrucker force-pushed the vercel-ai-tool-approval branch from 52dea3e to 676530b Compare December 20, 2025 09:52
@bendrucker
bendrucker force-pushed the vercel-ai-tool-approval branch from 2deeb25 to 9eebd80 Compare December 20, 2025 11:03
@bendrucker bendrucker closed this Dec 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants