Skip to content

feat(dojo): generative UI (A2UI 1.0) demos and e2e across 17 framework lanes - #3209

Closed
mme wants to merge 12 commits into
mainfrom
mme/dojo-generative-ui
Closed

mme wants to merge 12 commits into
mainfrom
mme/dojo-generative-ui

Conversation

@mme

@mme mme commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Two new dojo demos for CopilotKit's generative UI (A2UI 1.0). Each has an e2e spec per lane.

  • Generative UI (Markdown): the agent writes A2UI operations inside ```a2ui fences in its reply.
  • Generative UI (Tool): the runtime gives the agent a render_ui tool.

Both demos depend on one thing: the adapter has to put RunAgentInput.context into the model's prompt.

CopilotKitProvider generativeUI={{ ui: standardUI, transport }}
  → context: "Generative UI markdown guide" / "Generative UI guide" + component schema
  → adapter → model prompt
  → reply with a fence, or a render_ui call
  → surface renders in the chat
Lane What it needed
Claude Agent SDK (py/ts), AWS Strands (py/ts), AG2, Agno, Cloudflare Agents, LangGraph FastAPI nothing: agentic_chat already gets the context
Pydantic AI a route that passes the context as instructions
Mastra (remote and local) an agent whose instructions read requestContext.get("ag-ui")
ADK middleware, ADK JS an instruction provider that reads the context from session state
CrewAI Flows and Conversational Flows a flow that adds state.context to its system prompt
LangChain a chainFn that writes the context into a system message
AG-UI .NET SDK a dojo-endpoint opt-in that writes the context into the system prompt
MS Agent Framework .NET a delegating agent that reads it via TryGetRunAgentInput

The @copilotkit/* packages are pinned to pkg.pr.new previews of CopilotKit/CopilotKit#7819 (commit 47deca0). The dojo now also depends on maplibre-gl: it's an optional peer, but Next fails the build without it.

Evidence

One spec per lane, run on both transports. It sends "Show me a team plan picker", and aimock answers with a plan picker only if the guide is in the request. The spec then checks:

  • the surface renders

  • the "Seats" slider shows "5 seats selected"

  • ArrowRight changes it to "6 seats selected"

  • no render error appears

  • Before: no generative UI demos.
    After (locally, against aimock): 34 of 34 pass across 17 lanes.

  • Cloudflare Agents isn't covered locally: its worker doesn't run here, and its existing agentic chat spec fails locally too. Its CI lane will verify it.

  • The CrewAI route tests and crewai.test.ts are updated and pass.

Merge Danger

Door: two-way

Do not merge before CopilotKit releases these packages. The preview pins must first be replaced with a released version.

Blast Radius: dojo

Each lane gets two new menu entries and two new specs. The other changes are additions to example servers; no published package changes. One shared fixture now skips system messages after the user's turn: CopilotKit's LangGraph middleware puts its context message there.

Not included, with the reason for each:

  • LangGraph Platform (Python and TS): the TS client doesn't fill copilotkit.context. Filling it makes the guide reach the model, but it changes the golden event traces. It also moves CopilotKit's context message after tool results, which breaks the existing frontend-tool specs. That needs its own change.
  • Claude Managed Agents: its lanes run on a scripted fake, which would need generative UI support.
  • Antigravity, Omnara, LlamaIndex, Langroid, Spring AI, MAF Python, A2A, watsonx, Agent Spec: these adapters don't put the context into the prompt.

mme added 12 commits October 26, 2026 15:53
…sports

Two new features show CopilotKit's generative UI: the agent answers with
A2UI 1.0 from the standard catalog, either inside ```a2ui fences in its
reply (markdown) or through the render_ui tool the CopilotKit runtime adds
(tool). CopilotKit sends the guide and the component schema as context
entries, so an integration needs no agent code as long as its adapter
puts RunAgentInput.context into the prompt.

The demos are enabled for the Claude Agent SDK (Python and TypeScript),
whose adapters inject the context into the system prompt; both reuse the
plain chat agent. The runtime route turns generativeUI on for the tool
demo.

The e2e fixtures answer only when the guide reached the model's system
prompt, so a passing test proves the injection. The specs check the
surface renders, that the slider and its bound total work without a round
trip, and that nothing fell back to an error.

The dojo pins @copilotkit/* to the pkg.pr.new previews of CopilotKit PR
standard catalog's Map loads. Re-pin to a release before merging.
…lare Agents

These adapters already forward RunAgentInput.context to the model, so the
generative UI demos reuse each integration's plain chat agent:

- AWS Strands (Python and TypeScript): a transient before-model hook
- AG2: the system prompt
- Agno: the run's message context
- Cloudflare Agents: the system prompt

The fixtures now look for the guide in any message of the request, since
adapters place the context differently; it can only be there if the
adapter forwarded it. Verified locally for Strands (both), AG2 and Agno on
both transports; Cloudflare Agents runs in its CI lane (its existing demos
need wrangler's Worker environment).
CopilotKit's LangGraph middleware already writes the run's context into the
prompt, so the agentic_chat graph serves both demos. The middleware appends
its context as a system message after the latest turn, so the fixtures now
skip trailing system messages when they look for the user's prompt.
Pydantic AI's AGUIAdapter does not pass RunAgentInput.context to the model,
so a generative_ui route reads it from the run input and hands it to the
agent as instructions.
The AG-UI bridge puts RunAgentInput.context on Mastra's request context
under "ag-ui". A generative_ui agent, registered in both Mastra instances,
appends those entries to its instructions.
The middleware stores RunAgentInput.context in session state under
CONTEXT_STATE_KEY. A generative UI agent's instruction provider appends it.
The bridge stores RunAgentInput.context in session state under
AG_UI_CONTEXT_KEY. A generative UI agent's instruction provider appends it;
the dojo factory now accepts an instruction provider.
…lows

The endpoint puts RunAgentInput.context in the flow state. A generative UI
flow appends it to its system prompt and serves both demos, with a route
per feature so the route tables keep mirroring the feature names.
chainFn receives the run's context; a second chain writes it into a system
message. (The adapter's model+prompt pattern would do this itself, but its
system message is not a coercible message-like, so LangChain rejects it.)
The dojo server's endpoints can opt in to writing RunAgentInput.context into
the system prompt; a generative_ui route does, for both demos.
A delegating agent recovers the RunAgentInput from the chat options and
prepends its context, as the shared state demo does with the state.
@pkg-pr-new

pkg-pr-new Bot commented Oct 26, 2026

Copy link
Copy Markdown

Open in StackBlitz

@ag-ui/a2a-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2a-middleware@3209

@ag-ui/a2ui-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2ui-middleware@3209

@ag-ui/event-throttle-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/event-throttle-middleware@3209

@ag-ui/mcp-apps-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/mcp-apps-middleware@3209

@ag-ui/mcp-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/mcp-middleware@3209

@ag-ui/a2a

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2a@3209

@ag-ui/adk-js

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/adk-js@3209

@ag-ui/adk

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/adk@3209

@ag-ui/ag2

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/ag2@3209

@ag-ui/agno

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/agno@3209

@ag-ui/antigravity

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/antigravity@3209

@ag-ui/aws-strands

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/aws-strands@3209

@ag-ui/claude-agent-sdk

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/claude-agent-sdk@3209

@ag-ui/claude-managed-agents

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/claude-managed-agents@3209

@ag-ui/crewai

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/crewai@3209

@ag-ui/langchain

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/langchain@3209

@ag-ui/langgraph

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/langgraph@3209

@ag-ui/llamaindex

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/llamaindex@3209

@ag-ui/mastra

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/mastra@3209

@ag-ui/pydantic-ai

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/pydantic-ai@3209

@ag-ui/vercel-ai-sdk

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/vercel-ai-sdk@3209

@ag-ui/watsonx

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/watsonx@3209

@ag-ui/a2ui-toolkit

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2ui-toolkit@3209

create-ag-ui-app

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/create-ag-ui-app@3209

@ag-ui/client

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/client@3209

@ag-ui/core

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/core@3209

@ag-ui/encoder

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/encoder@3209

@ag-ui/proto

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/proto@3209

commit: 7762c17

@mme mme closed this Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant