Repository navigation
fix(mastra): vendor sync partial-JSON parser, drop @ai-sdk/ui-utils - #3224
contextablemark merged 3 commits into
Conversation
…-protocol#3096, ag-ui-protocol#2874) @ag-ui/mastra used one function from @ai-sdk/ui-utils, parsePartialJson, to read working memory out of streaming tool-call arguments. ui-utils is the AI SDK 4 package: its last release (1.2.11) pins @ai-sdk/provider-utils@2.2.8, affected by GHSA-866g-f22w-33x8 / CVE-2026-8769 with no patched 2.x, and peers zod ^3.23.8, which forces a nested Zod 3 copy into apps on Zod 4. Fix it the way Mastra fixed the same dependency in @mastra/client-js (mastra-ai/mastra#27297), using the same layout: src/utils/ai-sdk-v4/{fix-json,parse-partial-json,types,index}.ts, taken from Mastra's copy at c56d026c5194b1977e5321f429565eedbccf2c40 (Apache-2.0, attribution kept). The code is identical to Mastra's apart from prettier formatting; types.ts and index.ts keep only what this package uses (JSONValue, parsePartialJson). parsePartialJson parses with plain JSON.parse instead of provider-utils' safeParseJSON, so no AI SDK package is needed. The function stays synchronous, so parseStreamingWorkingMemoryUpdate and the chunk pipeline are unchanged; nothing in streaming or cancellation moves. This replaces the async `ai` approach of ag-ui-protocol#3007. - remove @ai-sdk/ui-utils from dependencies - widen zod to ^3.25.76 || ^4.0.0 - add src/utils/ai-sdk-v4/ai-sdk-v4.test.ts (Mastra's parse-state cases plus the working-memory envelope shapes) typecheck clean; 47 files / 891 tests pass; build and test:exports pass. A clean consumer install with @mastra/core 1.79.0 and @mastra/client-js 1.56.0 resolves no @ai-sdk/ui-utils or provider-utils 2.x.
contextablemark
left a comment
There was a problem hiding this comment.
The synchronous approach avoids the async pipeline and cancellation changes from #3007. One behavior regression needs addressing before merge: the previous ui-utils parser used secure-json-parse, which rejected proto keys and constructor objects containing a prototype property. Plain JSON.parse drops that protection, and the parsed working-memory objects are subsequently merged.
Please preserve those rejection rules and add regression tests for complete and repaired inputs, nested/Unicode-escaped keys, and ordinary constructor/prototype fields that should remain valid. Also retain the previous initial-BOM handling. The inline suggestion provides a local change to tryParse without adding a dependency or changing the streaming pipeline.
I checked the suggested helper with 15 bounded local assertions; I have not independently run this PR's full suite.
| text: string, | ||
| ): { success: true; value: JSONValue } | { success: false } { | ||
| try { | ||
| return { success: true, value: JSON.parse(text) as JSONValue }; |
There was a problem hiding this comment.
The previous parser rejected prototype-related keys. Could we preserve that protection here? This also retains its handling of an initial BOM. The existing catch still converts rejection into a failed parse.
| return { success: true, value: JSON.parse(text) as JSONValue }; | |
| const value = JSON.parse( | |
| text.charCodeAt(0) === 0xfeff ? text.slice(1) : text, | |
| (key, value: unknown) => { | |
| if ( | |
| key === "__proto__" || | |
| (key === "constructor" && | |
| value !== null && | |
| typeof value === "object" && | |
| Object.prototype.hasOwnProperty.call(value, "prototype")) | |
| ) { | |
| throw new SyntaxError( | |
| "Object contains forbidden prototype property", | |
| ); | |
| } | |
| return value; | |
| }, | |
| ) as JSONValue; | |
| return { success: true, value }; |
There was a problem hiding this comment.
Thanks, good catch. Applied in 7c3b21f: tryParse now uses your reviver as suggested, with the BOM strip, and the header comment says the vendored copy deliberately keeps secure-json-parse's rules where Mastra's doesn't.
I ran every new test input through the original @ai-sdk/ui-utils@1.2.11 parsePartialJson and the results match, with one intended difference: {"constructor":null}. secure-json-parse@2.7.0 rejects it only because hasOwnProperty.call(null, "prototype") throws a TypeError; your helper accepts it, and the test pins that as valid.
Regression tests (src/utils/ai-sdk-v4/ai-sdk-v4.test.ts):
__proto__rejected in complete and repaired input, nested (in objects and arrays) and Unicode-escapedconstructorwith ownprototyperejected, including escaped and repaired formsObject.prototypenot polluted- still valid:
constructoras a string,nullor an object withoutprototype; aprototypekey;proto-like keys;__proto__/constructoras string values - leading BOM, for complete and repaired input
Plus a streaming test in state-delta.test.ts, run for local and remote agents: a working-memory update whose args contain __proto__ must not reach STATE_SNAPSHOT/STATE_DELTA. Without the fix, the single-delta case put {"__proto__":{"polluted":true}} into the snapshot sent to the client.
Full package suite: 47 files, 904 tests; typecheck, build and test:exports pass. No dependency added, streaming pipeline unchanged.
@ai-sdk/ui-utils parsed through @ai-sdk/provider-utils' safeParseJSON, which uses secure-json-parse with its defaults: it strips a leading BOM and rejects any __proto__ key and any constructor object with its own prototype property. Mastra's vendored copy uses plain JSON.parse, which dropped those rules. tryParse now applies them with a JSON.parse reviver; a rejection is a failed parse, as before. No dependency is added and the streaming pipeline is unchanged. Regression tests cover complete and repaired input, nested and Unicode-escaped keys, valid constructor/prototype fields, the BOM, and a working-memory stream whose __proto__ key previously reached the client in STATE_SNAPSHOT.
|
I thought I'd do it the way mastra did and have that copied. Interesting to see they implemented it differently and with the points you mentioned... |
Python Preview PackagesVersion
Install with uvAdd the TestPyPI index to your [[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
explicit = trueThen install the packages you need: # Core SDK
uv add 'ag-ui-protocol==0.0.0.dev1793185113' --index testpypi
# Integrations (each already depends on the matching ag-ui-protocol preview)
uv add 'ag-ui-langgraph==0.0.0.dev1793185113' --index testpypi
uv add 'ag-ui-crewai==0.0.0.dev1793185113' --index testpypi
# NOTE: ag-ui-agent-spec depends on pyagentspec (git-only, not on PyPI).
# You will need to install pyagentspec separately from its git repo.
uv add 'ag-ui-agent-spec==0.0.0.dev1793185113' --index testpypi
uv add 'ag_ui_adk==0.0.0.dev1793185113' --index testpypi
uv add 'ag_ui_strands==0.0.0.dev1793185113' --index testpypiInstall with pippip install \
--index-url https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
ag-ui-protocol==0.0.0.dev1793185113
Commit: a775a52 |
@ag-ui/a2a-middleware
@ag-ui/a2ui-middleware
@ag-ui/event-throttle-middleware
@ag-ui/mcp-apps-middleware
@ag-ui/mcp-middleware
@ag-ui/a2a
@ag-ui/adk-js
@ag-ui/adk
@ag-ui/ag2
@ag-ui/agno
@ag-ui/antigravity
@ag-ui/aws-strands
@ag-ui/claude-agent-sdk
@ag-ui/claude-managed-agents
@ag-ui/crewai
@ag-ui/langchain
@ag-ui/langgraph
@ag-ui/llamaindex
@ag-ui/mastra
@ag-ui/pydantic-ai
@ag-ui/vercel-ai-sdk
@ag-ui/watsonx
@ag-ui/a2ui-toolkit
create-ag-ui-app
@ag-ui/client
@ag-ui/core
@ag-ui/encoder
@ag-ui/proto
commit: |
contextablemark
left a comment
There was a problem hiding this comment.
@TheSeydiCharyyev Yes, I'm not sure why they altered their approach. In any case, thanks for sticking with this. Approved.
Summary
@ag-ui/mastraused a single function from@ai-sdk/ui-utils:parsePartialJson, for streaming working-memory updates. That package pins@ai-sdk/provider-utils@2.2.8(GHSA-866g-f22w-33x8 / CVE-2026-8769, #3096) andzod@^3, which gives Zod 4 apps a second, nested Zod 3 copy (#2874).This PR fixes it the same way Mastra did in mastra-ai/mastra#27297 for
@mastra/client-js: it vendors the synchronous parser instead of depending on the AI SDK.fix-json.tsandparse-partial-json.ts(from commitc56d026c) intosrc/utils/ai-sdk-v4/, the same path Mastra uses. Both are originally from@ai-sdk/ui-utils@1.2.11; the Apache-2.0 attribution to Vercel is kept in the headers.parsePartialJsonstays synchronous, as suggested on [Bug]: @ag-ui/mastra: drop @ai-sdk/ui-utils, which pins vulnerable @ai-sdk/provider-utils 2.2.8 #3096. Like Mastra's copy it usesJSON.parse, so it needs nothing from@ai-sdk/provider-utils. Unlike Mastra's copy, it keeps the rulessafeParseJSONapplied throughsecure-json-parse's defaults: a leading BOM is stripped, and a__proto__key or aconstructorobject with its ownprototypeis rejected (a failed parse).mastra.tschanges only its import. The streaming pipeline and cancellation code are untouched.package.json: drops@ai-sdk/ui-utils, widenszodto^3.25.76 || ^4.0.0, adds no new dependency. The lockfile change is the removedui-utilsentry only.Comparison with Mastra's files
After formatting Mastra's copies with this repo's prettier settings:
fix-json.ts: identical code; only the header comment differs.parse-partial-json.ts: identical apart from the header andtryParse, which keeps thesecure-json-parserules (see above).types.ts: onlyJSONValue, identical to Mastra's. Their other types aren't used here.index.ts: exports onlyparsePartialJsonand the types. Mastra's also exports the data-stream helpers, which this adapter doesn't need.The parser isn't part of the package's public exports.
Why not
ai@5This replaces #3007, which imported the async
parsePartialJsonfromai@5. That needed an async chunk pipeline and a change to cancellation timing (#2288). It also turned out that everyai@5release pulls@ai-sdk/provider-utils@3.0.x, which depends onundici@^5.29.0(4 high, 9 moderate advisories in a consumer audit). The vendored sync parser avoids both.Testing
tsc --noEmitcleanpnpm test: 47 files, 904 tests passed, including new parser tests insrc/utils/ai-sdk-v4/ai-sdk-v4.test.ts(Mastra's case, the working-memory shapes the adapter parses, a check that the function is synchronous, and thesecure-json-parserules) and a working-memory streaming test instate-delta.test.tspnpm buildandtest:exports(publint, attw) pass; noaior@ai-sdk/ui-utilsimport indistpnpm install --frozen-lockfilepasses@mastra/core@1.79.0and@mastra/client-js@1.56.0: no@ai-sdk/ui-utils,@ai-sdk/provider-utils@2.xorundici; the adapter uses the app's Zod 4.Not run: the full monorepo test matrix.
Closes #2874
Closes #3096