diff --git a/.gitignore b/.gitignore index 9b603e073b6..f2a11c67e05 100644 --- a/.gitignore +++ b/.gitignore @@ -14,6 +14,8 @@ temp/* refs/* plugins/* examples/plugin/bin/* +__pycache__/ +*.py[cod] # Storage backends pgstore/* diff --git a/docs/superpowers/plans/2026-07-18-cursor-apply-patch-custom-tool-round-trip.md b/docs/superpowers/plans/2026-07-18-cursor-apply-patch-custom-tool-round-trip.md new file mode 100644 index 00000000000..cd24def3f76 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-cursor-apply-patch-custom-tool-round-trip.md @@ -0,0 +1,464 @@ +# Cursor ApplyPatch Custom Tool Round-Trip Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Preserve Codex custom tool calls through Chat Completions streaming/non-streaming responses and reconstruct them safely in the next request so Cursor ApplyPatch can complete its agent loop. + +**Architecture:** Build one request-local tool catalog for family classification and name shortening/restoration. Replace the response translator's single active function-call flags with response-stream-local per-item state keyed by `item_id` and `output_index`, and use terminal events only to fill bytes not already emitted. + +**Tech Stack:** Go 1.26+, `gjson`, `sjson`, Go `testing`, `httptest`, Gorilla WebSocket, existing SDK translator registry. + +## Global Constraints + +- Use the current `main` baseline and do not copy PR #4079. +- Add and run failing regression tests before production-code changes. +- Never use process-global or cross-request state for tool-family classification. +- Preserve existing function-call behavior and use one translator for HTTP and WebSocket executors. +- Keep changes small, comments in English, and format all Go changes with `gofmt -w .`. +- Do not add network timeouts after an upstream connection is established. + +--- + +### Task 1: Capture the Root Cause with Failing Translator Tests + +**Files:** +- Create: `internal/translator/codex/openai/chat-completions/codex_openai_custom_tool_test.go` +- Modify: `internal/translator/codex/openai/chat-completions/codex_openai_request_test.go:807-1072` + +**Interfaces:** +- Consumes: `ConvertCodexResponseToOpenAI`, `ConvertCodexResponseToOpenAINonStream`, and `ConvertOpenAIRequestToCodex`. +- Produces: regression helpers `translateCodexStreamEvents`, `collectChatToolCalls`, and `assertToolInputExactlyOnce` used only by the new test file. + +- [ ] **Step 1: Add an end-to-end ApplyPatch transcript test** + +Create a test that declares `{"type":"custom","name":"ApplyPatch"}`, feeds added/delta/done/output-item-done/completed events, assembles the Chat `type: "function"` envelope, submits it with a `role: "tool"` result, and verifies the next Codex input contains: + +```go +wantCall := map[string]string{ + "type": "custom_tool_call", + "call_id": "call_apply_patch", + "name": "ApplyPatch", + "input": "*** Begin Patch\n*** Add File: cursor-round-trip.txt\n+ok\n*** End Patch", +} +wantOutputType := "custom_tool_call_output" +``` + +Then translate a final `response.output_text.delta` plus `response.completed` and assert the final assistant text and `finish_reason: "stop"` are still emitted. + +- [ ] **Step 2: Add streaming exactly-once and fallback table tests** + +Use a table with complete event slices for these cases: + +```go +tests := []struct { + name string + events []string +}{ + {name: "multiple deltas then done", events: addedDeltaDeltaDoneItemDoneCompleted}, + {name: "done fallback without deltas", events: addedDoneItemDoneCompleted}, + {name: "output item done fallback", events: addedDeltaItemDoneCompleted}, + {name: "missing added buffers until item done", events: deltaDoneItemDoneCompleted}, + {name: "completed only fallback", events: completedOnly}, +} +``` + +For every row concatenate only `choices.0.delta.tool_calls.*.function.arguments` and require byte-for-byte equality with the patch once, plus one call ID/name announcement and terminal `finish_reason: "tool_calls"`. + +- [ ] **Step 3: Add sequential, parallel, non-streaming, and shortening tests** + +Cover two sequential custom calls, interleaved custom/function calls with different `item_id` and `output_index`, a completed non-stream custom call, and a custom tool name longer than 64 bytes. Assert tool-call indexes are contiguous and each restored Chat name equals the original declaration. + +- [ ] **Step 4: Tighten request history safety tests** + +Change the existing missing-output-ID expectation so an ID-less result is accepted only when one pending call remains. Add cases proving: + +```go +// Standard Chat envelope classified by this request's declarations. +{"id":"call_custom","type":"function","function":{"name":"ApplyPatch","arguments":"raw patch"}} + +// Same envelope remains a function when declaration is function or absent. +{"id":"call_function","type":"function","function":{"name":"lookup","arguments":"{}"}} +``` + +Also assert duplicate IDs, orphan IDs, ambiguous ID-less mixed outputs, and duplicate results never become `custom_tool_call_output`; a single missing assistant call ID receives one deterministic synthetic ID shared by call and output. + +- [ ] **Step 5: Run the new tests and preserve red-state evidence** + +Run: + +```bash +go test -count=1 -run 'TestApplyPatch|TestCustomTool|TestMixedParallel|TestNonStreamCustom|TestCustomName|TestToolCallOutput' ./internal/translator/codex/openai/chat-completions +``` + +Expected: FAIL because the current response translator drops `custom_tool_call`, and standard Chat function envelopes are restored as `function_call`/`function_call_output` even when the same request declares a custom tool. + +- [ ] **Step 6: Commit the failing regression tests** + +```bash +git add internal/translator/codex/openai/chat-completions/codex_openai_custom_tool_test.go internal/translator/codex/openai/chat-completions/codex_openai_request_test.go +git commit -m "test(translator): reproduce Cursor ApplyPatch round-trip failure" +``` + +--- + +### Task 2: Add the Request-Local Tool Catalog and Follow-Up Restoration + +**Files:** +- Create: `internal/translator/codex/openai/chat-completions/codex_openai_tool_catalog.go` +- Modify: `internal/translator/codex/openai/chat-completions/codex_openai_request.go:69-336` +- Modify: `internal/translator/codex/openai/chat-completions/codex_openai_response.go:522-550` +- Test: `internal/translator/codex/openai/chat-completions/codex_openai_request_test.go` +- Test: `internal/translator/codex/openai/chat-completions/codex_openai_custom_tool_test.go` + +**Interfaces:** +- Produces: `buildToolCatalog(raw []byte) toolCatalog`, `toolCatalog.shorten(name string) string`, `toolCatalog.restore(name string) string`, and `toolCatalog.familyForChatCall(name string) toolFamily`. +- Consumes: existing `buildShortNameMap` and `shortenNameIfNeeded`. + +- [ ] **Step 1: Implement catalog types and deterministic mappings** + +Add: + +```go +type toolFamily uint8 + +const ( + toolFamilyFunction toolFamily = iota + toolFamilyCustom +) + +type toolCatalog struct { + shortByOriginal map[string]string + originalByShort map[string]string + customNames map[string]struct{} + ambiguousNames map[string]struct{} +} +``` + +`buildToolCatalog` must collect `tools.*.function.name` for functions and `tools.*.name` for custom tools, deduplicate names before calling `buildShortNameMap`, mark any name declared by both families ambiguous, and recognize both original and shortened custom names. `familyForChatCall` returns custom only for a unique custom name. + +- [ ] **Step 2: Use the catalog for declarations and history** + +Replace `originalToolNameMap` with `catalog := buildToolCatalog(rawJSON)`. Shorten names in both function and custom declarations. For assistant history, resolve a standard `type: "function"` call as follows: + +```go +family := toolFamilyFunction +name := tc.Get("function.name").String() +input := tc.Get("function.arguments").String() +if catalog.familyForChatCall(name) == toolFamilyCustom { + family = toolFamilyCustom +} +``` + +Emit `custom_tool_call` with `input` for custom and the existing `function_call` with `arguments` otherwise. Preserve explicit legacy `type: "custom"` envelopes. + +- [ ] **Step 3: Make output matching unique and family-safe** + +For explicit output IDs, select exactly one unconsumed pending call. For a missing output ID, build the unconsumed candidate list and match only when its length is one: + +```go +if toolCallID == "" { + if len(candidates) != 1 { + continue + } + pendingIndex = candidates[0] +} +``` + +Continue dropping duplicate assistant IDs, orphan results, and duplicate results. Select output type solely from the matched pending call's resolved family. + +- [ ] **Step 4: Route response name restoration through the same catalog** + +Change `buildReverseMapFromOriginalOpenAI` to delegate to `buildToolCatalog(original).originalByShort`, so custom and function calls use identical restoration without global state. + +- [ ] **Step 5: Run request and shortening tests** + +Run: + +```bash +go test -count=1 -run 'Test.*(History|CallID|Output|NameShortening|ApplyPatch).*' ./internal/translator/codex/openai/chat-completions +``` + +Expected: request-follow-up, ambiguity, and shortening tests PASS; response-event tests may remain FAIL until Task 3. + +- [ ] **Step 6: Commit request restoration** + +```bash +git add internal/translator/codex/openai/chat-completions/codex_openai_tool_catalog.go internal/translator/codex/openai/chat-completions/codex_openai_request.go internal/translator/codex/openai/chat-completions/codex_openai_response.go internal/translator/codex/openai/chat-completions/*_test.go +git commit -m "fix(translator): restore request-local custom tool history" +``` + +--- + +### Task 3: Implement Per-Call Streaming and Non-Streaming Translation + +**Files:** +- Create: `internal/translator/codex/openai/chat-completions/codex_openai_tool_stream.go` +- Modify: `internal/translator/codex/openai/chat-completions/codex_openai_response.go:23-314` +- Modify: `internal/translator/codex/openai/chat-completions/codex_openai_response.go:382-519` +- Test: `internal/translator/codex/openai/chat-completions/codex_openai_response_test.go` +- Test: `internal/translator/codex/openai/chat-completions/codex_openai_custom_tool_test.go` + +**Interfaces:** +- Produces: `toolCallStreamState`, `streamToolCallTracker`, `findOrCreateToolCall`, `announceToolCall`, `emitToolInput`, and `remainingToolInput`. +- Consumes: `toolCatalog.restore` and the existing Chat completion chunk template. + +- [ ] **Step 1: Add per-call stream state** + +Define: + +```go +type toolCallStreamState struct { + chatIndex int + itemID string + outputIndex int64 + hasOutputIndex bool + callID string + name string + family toolFamily + announced bool + emittedInput string + bufferedInput string +} + +type streamToolCallTracker struct { + nextChatIndex int + byItemID map[string]*toolCallStreamState + byOutputIndex map[int64]*toolCallStreamState + ordered []*toolCallStreamState +} +``` + +Store a tracker inside `ConvertCliToOpenAIParams`, initialize it per converter `param`, and remove the single-call `FunctionCallIndex`, `HasReceivedArgumentsDelta`, and `HasToolCallAnnounced` assumptions. + +- [ ] **Step 2: Implement identity resolution and exactly-once suffix logic** + +Use `item_id` first and `output_index` second. When neither exists, use a call only if exactly one compatible active state exists. Add: + +```go +func remainingToolInput(emitted, complete string) (string, bool) { + if emitted == "" { + return complete, true + } + if complete == emitted { + return "", true + } + if strings.HasPrefix(complete, emitted) { + return complete[len(emitted):], true + } + return "", false +} +``` + +Never emit a conflicting full value after deltas already reached the client. + +- [ ] **Step 3: Translate added, delta, done, and output-item-done events** + +Handle both families: + +```go +case "response.function_call_arguments.delta", "response.custom_tool_call_input.delta": +case "response.function_call_arguments.done", "response.custom_tool_call_input.done": +``` + +Function events read `arguments`; custom events read `input`. `response.output_item.added` announces `function_call` and `custom_tool_call` as Chat `type: "function"`. If added was omitted, buffer delta/done data until `response.output_item.done` supplies `call_id` and `name`, then emit one envelope containing the full remaining input. + +- [ ] **Step 4: Add completed-event fallback before the terminal chunk** + +On `response.completed`, scan `response.output` for function/custom calls in output order. Reconcile each with the tracker, append any missing announcement/input chunks, then append the terminal chunk. A completed response with any translated call uses: + +```go +finishReason := "tool_calls" +nativeFinishReason := "tool_calls" +``` + +Continue mapping incomplete responses to `length` or `content_filter` without overwriting them merely because a call was previously announced. + +- [ ] **Step 5: Add non-stream custom calls** + +Extend the non-stream output switch: + +```go +case "function_call", "custom_tool_call": + argumentsPath := "arguments" + if outputType == "custom_tool_call" { + argumentsPath = "input" + } +``` + +Build the same Chat function envelope, restore the original name, preserve `call_id`, and let the existing `len(toolCalls) > 0` terminal logic return `tool_calls`. + +- [ ] **Step 6: Run all translator tests** + +Run: + +```bash +gofmt -w internal/translator/codex/openai/chat-completions +go test -count=1 ./internal/translator/codex/openai/chat-completions/... +``` + +Expected: PASS, including exact input equality for all event permutations and the existing function-call/image/usage regressions. + +- [ ] **Step 7: Commit response translation** + +```bash +git add internal/translator/codex/openai/chat-completions +git commit -m "fix(translator): stream Codex custom tool calls exactly once" +``` + +--- + +### Task 4: Prove HTTP and WebSocket Executor Integration + +**Files:** +- Create: `internal/runtime/executor/codex_custom_tool_translation_test.go` + +**Interfaces:** +- Consumes: `CodexExecutor.ExecuteStream`, `CodexWebsocketsExecutor.ExecuteStream`, and the translator registered by `internal/translator/codex/openai/chat-completions/init.go`. +- Produces: transport-level proof that both executors return the same Chat tool call and terminal reason. + +- [ ] **Step 1: Add HTTP SSE integration test** + +Use `httptest.NewServer` to emit a custom added event, two input deltas, done, output-item-done, and completed. Call the HTTP executor with: + +```go +opts := cliproxyexecutor.Options{ + SourceFormat: sdktranslator.FromString("openai"), + Stream: true, +} +``` + +Collect downstream SSE payloads and assert `call_apply_patch`, `ApplyPatch`, the exact patch once, and `finish_reason: "tool_calls"`. + +- [ ] **Step 2: Add WebSocket integration test with the same assertions** + +Upgrade an `httptest` server using Gorilla WebSocket, read the `response.create` request, send the same JSON events as text frames, and call `CodexWebsocketsExecutor.ExecuteStream` with the same Chat payload/options. Reuse only assertion helpers; do not duplicate translator logic in the test. + +- [ ] **Step 3: Run both executor tests and the translator suite** + +Run: + +```bash +go test -count=1 -run 'TestCodex.*CustomTool' ./internal/runtime/executor +go test -count=1 ./internal/translator/codex/openai/chat-completions/... +``` + +Expected: PASS for both transports with identical call metadata, input, and finish reason. + +- [ ] **Step 4: Commit executor coverage** + +```bash +git add internal/runtime/executor/codex_custom_tool_translation_test.go +git commit -m "test(executor): cover custom tools over HTTP and WebSocket" +``` + +--- + +### Task 5: Full Verification and Requirement Audit + +**Files:** +- Modify only files found defective by verification, using a new failing test before any corrective production change. + +**Interfaces:** +- Consumes: the complete repository and required build target. +- Produces: fresh command output proving formatting, targeted behavior, full regressions, and compilation. + +- [ ] **Step 1: Format and inspect the final diff** + +```bash +gofmt -w . +git diff --check +git status --short +git diff main...HEAD --stat +``` + +Expected: no formatting errors, only in-scope files, and no `test-output` artifact. + +- [ ] **Step 2: Run the exact requested validations** + +```bash +go test ./internal/translator/codex/openai/chat-completions/... +go test ./... +go build -o test-output ./cmd/server && rm test-output +``` + +Expected: all three commands exit 0 and `test-output` no longer exists. + +- [ ] **Step 3: Audit every explicit requirement against test names and source** + +Use `rg` to confirm all five custom event types, request-local catalog construction, custom output conversion, non-stream mapping, name restoration, and both executor tests are present. Record any missing evidence as incomplete and add a failing regression before correcting it. + +- [ ] **Step 4: Commit a corrective regression only if the audit found a gap** + +Return to the applicable earlier task, add a failing test, make the minimal corrective change, rerun all Task 5 commands, and commit the exact files changed by that correction with `git commit -m "test(translator): complete custom tool regression coverage"`. Skip this step when the audit found no gap. + +--- + +### Task 6: Publish the Draft PR + +**Files:** +- No source changes expected. + +**Interfaces:** +- Consumes: clean `codex/fix-cursor-apply-patch` with all validations passing. +- Produces: origin branch and draft PR targeting `router-for-me/CLIProxyAPI:dev`. + +- [ ] **Step 1: Confirm publication scope** + +```bash +git status --short --branch +git log --oneline main..HEAD +git remote -v +``` + +Expected: clean branch, only intentional commits, and `origin` points to `DOUIF/CLIProxyAPI`. + +- [ ] **Step 2: Push the branch** + +```bash +git push -u origin codex/fix-cursor-apply-patch +``` + +Expected: remote tracking branch created or updated successfully. + +- [ ] **Step 3: Open a cross-repository draft PR** + +Use `gh pr create --repo router-for-me/CLIProxyAPI --base dev --head DOUIF:codex/fix-cursor-apply-patch --draft` with a body covering root cause, response/request changes, tests, and exact validation commands. + +Expected: one draft PR URL targeting upstream `dev`. + +--- + +### Task 7: Run Cursor SSH Live Verification and Clean Up + +**Files:** +- Create remotely and dispose: one dedicated ApplyPatch verification file outside tracked project files or remove it before restoring the checkout. + +**Interfaces:** +- Consumes: the open Cursor window connected to `SSH: tufa15`, the pushed branch, existing CPA configuration/auth, and GPT-5.6 Sol Medium. +- Produces: visible ApplyPatch execution, final assistant continuation, CPA trace containing `custom_tool_call_output`, and a clean restored remote checkout. + +- [ ] **Step 1: Record remote pre-test state in Cursor** + +In the existing terminal record the current branch, `git status --short`, and any running CPA process. Do not overwrite unrelated remote changes; stop and report them if the checkout is not clean. + +- [ ] **Step 2: Fetch, check out, build, and start the test CPA** + +Fetch origin, check out `codex/fix-cursor-apply-patch`, build `./cmd/server`, and start CPA with the existing remote configuration while retaining trace output. Preserve the prior process command so it can be restored if needed. + +- [ ] **Step 3: Run the Cursor agent prompt** + +Select GPT-5.6 Sol with Medium reasoning and submit: + +```text +Use ApplyPatch (not a shell redirection or another file-edit tool) to create or update the dedicated file cursor-apply-patch-round-trip.txt so it contains exactly: APPLY_PATCH_ROUND_TRIP_OK. After the tool succeeds, continue reasoning and give a final answer confirming the file content. +``` + +Expected: Cursor displays the ApplyPatch tool call/result and then a final assistant answer rather than stopping after the tool. + +- [ ] **Step 4: Verify file and trace evidence** + +Read the disposable file and search the CPA trace for the matching call ID plus `custom_tool_call_output`. Confirm the upstream follow-up includes the successful tool output and a later final response event. + +- [ ] **Step 5: Restore the remote environment** + +Stop the test CPA, remove the disposable file, restore the pre-test branch/process state, and run `git status --short`. Expected: the remote checkout is clean and no test CPA process remains. diff --git a/docs/superpowers/specs/2026-07-18-cursor-apply-patch-custom-tool-round-trip-design.md b/docs/superpowers/specs/2026-07-18-cursor-apply-patch-custom-tool-round-trip-design.md new file mode 100644 index 00000000000..63b8611a880 --- /dev/null +++ b/docs/superpowers/specs/2026-07-18-cursor-apply-patch-custom-tool-round-trip-design.md @@ -0,0 +1,80 @@ +# Cursor ApplyPatch Custom Tool Round-Trip Design + +## Context + +Cursor sends Codex-backed requests through the OpenAI Chat Completions compatibility endpoint. Codex Responses emits ApplyPatch as a `custom_tool_call`, while Chat Completions represents every client-visible tool call with the standard `type: "function"` envelope. The current response translator only recognizes `function_call` events, and the current request translator only restores the custom family when history already uses a non-standard `type: "custom"` envelope. Consequently the custom call is either omitted from the response or returned upstream as a normal function call/output, interrupting the agent loop. + +The fix remains entirely request-scoped or response-stream-scoped. It must not use process-global state or carry tool-family state across requests. HTTP and WebSocket executors continue to use the same registered translator. + +## Tool Catalog + +Build a catalog from the current Chat Completions request's `tools` array. The catalog records: + +- every declared function or custom tool name; +- which names uniquely identify custom tools; +- original-to-shortened and shortened-to-original mappings generated across both families; and +- names that are ambiguous because both families claim the same effective name. + +Custom declarations remain Responses-compatible top-level `type: "custom"` objects, but their names use the same deterministic shortening rules as function declarations. A history call is restored as custom only when its name uniquely matches a custom declaration in this request. Ambiguous or unknown names remain function calls. + +## Follow-Up Request Translation + +Chat assistant history normally contains `type: "function"`, `function.name`, and `function.arguments`, even for a client-visible custom call. For each assistant tool-call batch: + +1. Resolve the tool name through the request-local catalog. +2. Emit `custom_tool_call` with the bare `function.arguments` string as `input` when the name uniquely identifies a custom declaration. +3. Otherwise emit the existing `function_call` with `arguments` unchanged. +4. Record the resolved family next to the call ID for matching immediately following `role: "tool"` messages. + +Explicit legacy `type: "custom"` history remains supported. + +Tool outputs match a unique, unconsumed pending call. An explicit call ID must match exactly. A missing output ID may match only when exactly one pending call remains. Duplicate IDs, orphan outputs, duplicate outputs, and otherwise ambiguous outputs are dropped rather than guessed. The matched family selects `custom_tool_call_output` or `function_call_output`. + +Missing assistant call IDs receive deterministic request-local synthetic IDs so a uniquely matched output can preserve a valid pair. + +## Streaming Response Translation + +Replace the single active-call booleans with per-call stream state held in `ConvertCliToOpenAIParams`. Calls are keyed by `item_id`, with `output_index` as a secondary key. Each state records: + +- the allocated contiguous Chat tool-call index; +- item ID, output index, call ID, restored name, and family; +- whether the Chat call envelope was announced; +- input already emitted downstream; and +- buffered input observed before enough metadata exists to announce the call. + +For `response.output_item.added`, allocate or recover the call state and emit the Chat `tool_calls` envelope when the item is a function or custom call. Both families appear downstream as `type: "function"`; custom free-form input is carried in `function.arguments`. + +For argument/input delta events, emit the delta immediately when the call has been announced. If `added` was omitted and name/call ID are not yet known, buffer the data until a later item event supplies metadata. + +For argument/input done events, compare the complete value with the input already emitted. Emit only the un-emitted suffix when the complete value has the emitted value as a prefix. Emit the full value when nothing has been emitted. If the values conflict, do not duplicate already emitted bytes. + +For `response.output_item.done`, announce a call omitted from `added`, then apply the same suffix fallback using the item's complete `arguments` or `input`. + +For `response.completed`, scan output calls and emit any still-missing envelope or input before the terminal chunk. A completed custom call sets `finish_reason` and `native_finish_reason` to `tool_calls`. This covers providers that omit added, delta, done, or output-item-done events. + +Sequential and parallel calls remain independent because each call owns its emitted-input and announcement state. Events lacking both identity fields may use the sole active compatible call; when multiple candidates exist, the event is ignored as ambiguous. + +## Non-Streaming Response Translation + +Treat `custom_tool_call` output items like function calls when building Chat `message.tool_calls`. Preserve `call_id`, restore the original tool name, place bare `input` in `function.arguments`, and return `finish_reason: "tool_calls"` when at least one tool call is present. + +## Regression Strategy + +Before implementation, add tests that fail on the current `main` behavior and preserve their failing output as root-cause evidence. Coverage includes: + +- the complete ApplyPatch transcript from custom declaration through streamed Chat tool call, follow-up `custom_tool_call_output`, and final assistant continuation; +- multi-delta input with exactly-once concatenation; +- done, output-item-done, and completed fallbacks with omitted preceding events; +- sequential custom calls and mixed parallel custom/function calls; +- streaming and non-streaming name restoration after shortening; +- duplicate, missing, unmatched, and ambiguous call IDs; +- standard function-call regression behavior; and +- HTTP and WebSocket executor paths using the registered Chat Completions translator. + +After implementation, run formatting, the targeted translator suite, the complete Go suite, and the required server build command. + +## Publication and Live Verification + +Commit and push `codex/fix-cursor-apply-patch`, then open a draft pull request against upstream `dev`. In the existing Cursor SSH workspace on `tufa15`, save the prior branch and process state, check out the test branch, build and start CPA, and run GPT-5.6 Sol at medium reasoning with a prompt that explicitly requires ApplyPatch to edit a dedicated disposable file. + +Success requires the file edit, a returned tool result, continued model reasoning, and a final assistant answer. CPA trace evidence must show the follow-up `custom_tool_call_output`. Afterwards stop the test CPA process, remove the disposable file, restore the prior remote checkout, and verify a clean worktree. diff --git a/internal/runtime/executor/claude_executor_test.go b/internal/runtime/executor/claude_executor_test.go index 2d69b7a67b3..12bb38ea2c6 100644 --- a/internal/runtime/executor/claude_executor_test.go +++ b/internal/runtime/executor/claude_executor_test.go @@ -611,7 +611,7 @@ func TestApplyClaudeHeaders_DisableDeviceProfileStabilization(t *testing.T) { "X-Stainless-Arch": []string{"x64"}, }) applyClaudeHeaders(thirdPartyReq, auth, "key-disable-stability", false, nil, nil, cfg, nil, false) - assertClaudeFingerprint(t, thirdPartyReq.Header, "claude-cli/2.1.60 (external, cli)", "0.70.0", "v22.0.0", helps.MapStainlessOS(), helps.MapStainlessArch()) + assertClaudeFingerprint(t, thirdPartyReq.Header, "claude-cli/2.1.60 (external, cli)", "0.70.0", "v22.0.0", "MacOS", "arm64") lowerReq := newClaudeHeaderTestRequest(t, http.Header{ "User-Agent": []string{"claude-cli/2.1.61 (external, cli)"}, @@ -653,7 +653,7 @@ func TestApplyClaudeHeaders_LegacyModePreservesConfiguredUserAgentOverrideForCla }) applyClaudeHeaders(req, auth, "key-legacy-ua-override", false, nil, nil, cfg, nil, true) - assertClaudeFingerprint(t, req.Header, "config-ua/1.0", "0.70.0", "v22.0.0", helps.MapStainlessOS(), helps.MapStainlessArch()) + assertClaudeFingerprint(t, req.Header, "config-ua/1.0", "0.70.0", "v22.0.0", "MacOS", "arm64") } func TestApplyClaudeHeaders_LegacyThirdPartyUsesStableConfiguredOSArch(t *testing.T) { @@ -758,7 +758,7 @@ func TestClaudeExecutor_NonClaudeRequestUsesClaudeCode220CLIFingerprint(t *testi t.Fatalf("Execute() error = %v", errExecute) } - assertClaudeFingerprint(t, seenHeaders, "claude-cli/2.1.220 (external, cli)", "0.94.0", "v26.3.0", helps.MapStainlessOS(), helps.MapStainlessArch()) + assertClaudeFingerprint(t, seenHeaders, "claude-cli/2.1.220 (external, cli)", "0.94.0", "v26.3.0", "MacOS", "arm64") if got := seenHeaders.Get("X-App"); got != "cli" { t.Fatalf("X-App = %q, want cli", got) } diff --git a/internal/runtime/executor/codex_custom_tool_translation_test.go b/internal/runtime/executor/codex_custom_tool_translation_test.go new file mode 100644 index 00000000000..0c8de87142b --- /dev/null +++ b/internal/runtime/executor/codex_custom_tool_translation_test.go @@ -0,0 +1,152 @@ +package executor + +import ( + "context" + "fmt" + "net/http" + "net/http/httptest" + "testing" + + "github.com/gorilla/websocket" + "github.com/router-for-me/CLIProxyAPI/v7/internal/config" + _ "github.com/router-for-me/CLIProxyAPI/v7/internal/translator" + cliproxyauth "github.com/router-for-me/CLIProxyAPI/v7/sdk/cliproxy/auth" + cliproxyexecutor "github.com/router-for-me/CLIProxyAPI/v7/sdk/cliproxy/executor" + sdktranslator "github.com/router-for-me/CLIProxyAPI/v7/sdk/translator" + "github.com/tidwall/gjson" +) + +func codexCustomToolChatPayload() []byte { + return []byte(`{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"Apply the patch."}],"tools":[{"type":"custom","name":"ApplyPatch","description":"Apply a freeform patch.","format":{"type":"text"}}]}`) +} + +func codexCustomToolStreamEvents() []string { + return []string{ + `{"type":"response.created","response":{"id":"resp_1","created_at":1700000000,"model":"gpt-5.6-sol"}}`, + `{"type":"response.output_item.added","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_apply_patch","name":"ApplyPatch","input":"","status":"in_progress"}}`, + `{"type":"response.custom_tool_call_input.delta","output_index":0,"item_id":"ctc_1","delta":"abc"}`, + `{"type":"response.custom_tool_call_input.delta","output_index":0,"item_id":"ctc_1","delta":"def"}`, + `{"type":"response.custom_tool_call_input.done","output_index":0,"item_id":"ctc_1","input":"abcdef"}`, + `{"type":"response.output_item.done","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_apply_patch","name":"ApplyPatch","input":"abcdef","status":"completed"}}`, + `{"type":"response.completed","response":{"id":"resp_1","created_at":1700000000,"status":"completed","model":"gpt-5.6-sol","output":[{"id":"ctc_1","type":"custom_tool_call","call_id":"call_apply_patch","name":"ApplyPatch","input":"abcdef","status":"completed"}],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}}`, + } +} + +func codexCustomToolExecutorOptions() cliproxyexecutor.Options { + return cliproxyexecutor.Options{ + SourceFormat: sdktranslator.FromString("openai"), + Stream: true, + } +} + +func assertCodexCustomToolStream(t *testing.T, result *cliproxyexecutor.StreamResult) { + t.Helper() + + var callID string + var name string + var input string + var finishReason string + announcements := 0 + for chunk := range result.Chunks { + if chunk.Err != nil { + t.Fatalf("stream chunk error: %v", chunk.Err) + } + root := gjson.ParseBytes(chunk.Payload) + for _, toolCall := range root.Get("choices.0.delta.tool_calls").Array() { + if id := toolCall.Get("id"); id.Exists() && id.String() != "" { + callID = id.String() + announcements++ + } + if toolName := toolCall.Get("function.name"); toolName.Exists() && toolName.String() != "" { + name = toolName.String() + } + if arguments := toolCall.Get("function.arguments"); arguments.Exists() { + input += arguments.String() + } + } + if reason := root.Get("choices.0.finish_reason"); reason.Exists() && reason.String() != "" { + finishReason = reason.String() + } + } + + if callID != "call_apply_patch" || name != "ApplyPatch" { + t.Fatalf("custom tool metadata call_id=%q name=%q", callID, name) + } + if input != "abcdef" { + t.Fatalf("custom tool input = %q, want exactly abcdef", input) + } + if announcements != 1 { + t.Fatalf("custom tool announced %d times, want once", announcements) + } + if finishReason != "tool_calls" { + t.Fatalf("finish_reason = %q, want tool_calls", finishReason) + } +} + +func TestCodexExecutorCustomToolUsesChatCompletionsTranslator(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "text/event-stream") + for _, event := range codexCustomToolStreamEvents() { + _, _ = fmt.Fprintf(w, "data: %s\n\n", event) + } + })) + defer server.Close() + + exec := NewCodexExecutor(&config.Config{SDKConfig: config.SDKConfig{DisableImageGeneration: config.DisableImageGenerationAll}}) + auth := &cliproxyauth.Auth{ + Provider: "codex", + Attributes: map[string]string{ + "api_key": "sk-test", + "base_url": server.URL, + }, + } + result, err := exec.ExecuteStream(context.Background(), auth, cliproxyexecutor.Request{ + Model: "gpt-5.6-sol", + Payload: codexCustomToolChatPayload(), + }, codexCustomToolExecutorOptions()) + if err != nil { + t.Fatalf("ExecuteStream error: %v", err) + } + assertCodexCustomToolStream(t, result) +} + +func TestCodexWebsocketsExecutorCustomToolUsesChatCompletionsTranslator(t *testing.T) { + upgrader := websocket.Upgrader{CheckOrigin: func(*http.Request) bool { return true }} + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + conn, errUpgrade := upgrader.Upgrade(w, r, nil) + if errUpgrade != nil { + t.Errorf("upgrade websocket: %v", errUpgrade) + return + } + defer func() { _ = conn.Close() }() + + if _, _, errRead := conn.ReadMessage(); errRead != nil { + t.Errorf("read upstream websocket request: %v", errRead) + return + } + for _, event := range codexCustomToolStreamEvents() { + if errWrite := conn.WriteMessage(websocket.TextMessage, []byte(event)); errWrite != nil { + t.Errorf("write upstream websocket event: %v", errWrite) + return + } + } + })) + defer server.Close() + + exec := NewCodexWebsocketsExecutor(&config.Config{SDKConfig: config.SDKConfig{DisableImageGeneration: config.DisableImageGenerationAll}}) + auth := &cliproxyauth.Auth{ + Provider: "codex", + Attributes: map[string]string{ + "api_key": "sk-test", + "base_url": server.URL, + }, + } + result, err := exec.ExecuteStream(context.Background(), auth, cliproxyexecutor.Request{ + Model: "gpt-5.6-sol", + Payload: codexCustomToolChatPayload(), + }, codexCustomToolExecutorOptions()) + if err != nil { + t.Fatalf("ExecuteStream error: %v", err) + } + assertCodexCustomToolStream(t, result) +} diff --git a/internal/translator/codex/openai/chat-completions/codex_openai_custom_tool_test.go b/internal/translator/codex/openai/chat-completions/codex_openai_custom_tool_test.go new file mode 100644 index 00000000000..1add2138a8a --- /dev/null +++ b/internal/translator/codex/openai/chat-completions/codex_openai_custom_tool_test.go @@ -0,0 +1,384 @@ +package chat_completions + +import ( + "context" + "strings" + "testing" + + "github.com/tidwall/gjson" + "github.com/tidwall/sjson" +) + +const applyPatchInput = "*** Begin Patch\n*** Add File: cursor-round-trip.txt\n+ok\n*** End Patch" + +type streamedChatToolCall struct { + ID string + Name string + Arguments string + Announcements int +} + +func translateCodexStreamEvents(t *testing.T, originalRequest []byte, events ...string) [][]byte { + t.Helper() + + var param any + var chunks [][]byte + for _, event := range events { + translated := ConvertCodexResponseToOpenAI( + context.Background(), + "gpt-5.6-sol", + originalRequest, + nil, + []byte("data: "+event), + ¶m, + ) + chunks = append(chunks, translated...) + } + return chunks +} + +func collectStreamedChatToolCalls(t *testing.T, chunks [][]byte) ([]streamedChatToolCall, string) { + t.Helper() + + calls := make(map[int]*streamedChatToolCall) + maxIndex := -1 + finishReason := "" + for _, chunk := range chunks { + root := gjson.ParseBytes(chunk) + for _, toolCall := range root.Get("choices.0.delta.tool_calls").Array() { + index := int(toolCall.Get("index").Int()) + call := calls[index] + if call == nil { + call = &streamedChatToolCall{} + calls[index] = call + } + if index > maxIndex { + maxIndex = index + } + if id := toolCall.Get("id"); id.Exists() && id.String() != "" { + call.ID = id.String() + call.Announcements++ + } + if name := toolCall.Get("function.name"); name.Exists() && name.String() != "" { + call.Name = name.String() + } + if arguments := toolCall.Get("function.arguments"); arguments.Exists() { + call.Arguments += arguments.String() + } + } + if reason := root.Get("choices.0.finish_reason"); reason.Exists() && reason.String() != "" { + finishReason = reason.String() + } + } + + result := make([]streamedChatToolCall, 0, maxIndex+1) + for index := 0; index <= maxIndex; index++ { + call := calls[index] + if call == nil { + t.Fatalf("missing streamed tool call index %d", index) + } + result = append(result, *call) + } + return result, finishReason +} + +func customToolRequest(name string) []byte { + request := []byte(`{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"Apply the patch."}],"tools":[{"type":"custom","name":"","description":"Apply a freeform patch.","format":{"type":"text"}}]}`) + request, _ = sjson.SetBytes(request, "tools.0.name", name) + return request +} + +func TestApplyPatchCustomToolRoundTrip(t *testing.T) { + originalRequest := customToolRequest("ApplyPatch") + chunks := translateCodexStreamEvents(t, originalRequest, + `{"type":"response.output_item.added","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_apply_patch","name":"ApplyPatch","input":"","status":"in_progress"}}`, + `{"type":"response.custom_tool_call_input.delta","output_index":0,"item_id":"ctc_1","delta":"*** Begin Patch\n*** Add File: cursor-round-trip.txt\n"}`, + `{"type":"response.custom_tool_call_input.delta","output_index":0,"item_id":"ctc_1","delta":"+ok\n*** End Patch"}`, + `{"type":"response.custom_tool_call_input.done","output_index":0,"item_id":"ctc_1","input":"*** Begin Patch\n*** Add File: cursor-round-trip.txt\n+ok\n*** End Patch"}`, + `{"type":"response.output_item.done","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_apply_patch","name":"ApplyPatch","input":"*** Begin Patch\n*** Add File: cursor-round-trip.txt\n+ok\n*** End Patch","status":"completed"}}`, + `{"type":"response.completed","response":{"id":"resp_tool","status":"completed","model":"gpt-5.6-sol","output":[{"id":"ctc_1","type":"custom_tool_call","call_id":"call_apply_patch","name":"ApplyPatch","input":"*** Begin Patch\n*** Add File: cursor-round-trip.txt\n+ok\n*** End Patch","status":"completed"}]}}`, + ) + + calls, finishReason := collectStreamedChatToolCalls(t, chunks) + if len(calls) != 1 { + t.Fatalf("expected one Chat tool call, got %d; chunks=%q", len(calls), chunks) + } + call := calls[0] + if call.ID != "call_apply_patch" || call.Name != "ApplyPatch" { + t.Fatalf("custom call metadata was not preserved: %+v", call) + } + if call.Arguments != applyPatchInput { + t.Fatalf("custom input = %q, want %q", call.Arguments, applyPatchInput) + } + if call.Announcements != 1 { + t.Fatalf("custom call announced %d times, want once", call.Announcements) + } + if finishReason != "tool_calls" { + t.Fatalf("finish_reason = %q, want tool_calls", finishReason) + } + + followUp := []byte(`{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"Apply the patch."},{"role":"assistant","content":null,"tool_calls":[{"id":"","type":"function","function":{"name":"","arguments":""}}]},{"role":"tool","tool_call_id":"","content":"Done!"}],"tools":[{"type":"custom","name":"ApplyPatch","description":"Apply a freeform patch.","format":{"type":"text"}}]}`) + followUp, _ = sjson.SetBytes(followUp, "messages.1.tool_calls.0.id", call.ID) + followUp, _ = sjson.SetBytes(followUp, "messages.1.tool_calls.0.function.name", call.Name) + followUp, _ = sjson.SetBytes(followUp, "messages.1.tool_calls.0.function.arguments", call.Arguments) + followUp, _ = sjson.SetBytes(followUp, "messages.2.tool_call_id", call.ID) + followUp, _ = sjson.SetBytes(followUp, "service_tier", "fast") + + upstream := ConvertOpenAIRequestToCodex("gpt-5.6-sol", followUp, true) + if got := gjson.GetBytes(upstream, "service_tier").String(); got != "priority" { + t.Fatalf("service_tier = %q, want priority; output=%s", got, upstream) + } + items := gjson.GetBytes(upstream, "input").Array() + if len(items) != 3 { + t.Fatalf("expected user, custom call, and custom output; got %d: %s", len(items), gjson.GetBytes(upstream, "input").Raw) + } + if got := items[1].Get("type").String(); got != "custom_tool_call" { + t.Fatalf("follow-up call type = %q, want custom_tool_call; item=%s", got, items[1].Raw) + } + if got := items[1].Get("input").String(); got != applyPatchInput { + t.Fatalf("follow-up custom input = %q, want %q", got, applyPatchInput) + } + if got := items[2].Get("type").String(); got != "custom_tool_call_output" { + t.Fatalf("follow-up output type = %q, want custom_tool_call_output; item=%s", got, items[2].Raw) + } + if got := items[2].Get("call_id").String(); got != "call_apply_patch" { + t.Fatalf("follow-up output call_id = %q, want call_apply_patch", got) + } + + finalChunks := translateCodexStreamEvents(t, followUp, + `{"type":"response.output_text.delta","output_index":0,"item_id":"msg_1","delta":"Patch applied and verified."}`, + `{"type":"response.completed","response":{"id":"resp_final","status":"completed","model":"gpt-5.6-sol","output":[{"id":"msg_1","type":"message","role":"assistant","content":[{"type":"output_text","text":"Patch applied and verified."}]}]}}`, + ) + var finalText string + var finalReason string + for _, chunk := range finalChunks { + finalText += gjson.GetBytes(chunk, "choices.0.delta.content").String() + if reason := gjson.GetBytes(chunk, "choices.0.finish_reason"); reason.Exists() { + finalReason = reason.String() + } + } + if finalText != "Patch applied and verified." || finalReason != "stop" { + t.Fatalf("final continuation text=%q reason=%q", finalText, finalReason) + } +} + +func TestCustomToolStreamingFallbacksEmitInputExactlyOnce(t *testing.T) { + originalRequest := customToolRequest("ApplyPatch") + added := `{"type":"response.output_item.added","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_1","name":"ApplyPatch","input":"","status":"in_progress"}}` + deltaABC := `{"type":"response.custom_tool_call_input.delta","output_index":0,"item_id":"ctc_1","delta":"abc"}` + deltaDEF := `{"type":"response.custom_tool_call_input.delta","output_index":0,"item_id":"ctc_1","delta":"def"}` + done := `{"type":"response.custom_tool_call_input.done","output_index":0,"item_id":"ctc_1","input":"abcdef"}` + itemDone := `{"type":"response.output_item.done","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_1","name":"ApplyPatch","input":"abcdef","status":"completed"}}` + completed := `{"type":"response.completed","response":{"id":"resp_1","status":"completed","model":"gpt-5.6-sol","output":[{"id":"ctc_1","type":"custom_tool_call","call_id":"call_1","name":"ApplyPatch","input":"abcdef","status":"completed"}]}}` + + tests := []struct { + name string + events []string + }{ + {name: "multiple deltas then done", events: []string{added, deltaABC, deltaDEF, done, itemDone, completed}}, + {name: "done fallback without deltas", events: []string{added, done, itemDone, completed}}, + {name: "output item done fallback", events: []string{added, deltaABC, itemDone, completed}}, + {name: "missing added buffers until item done", events: []string{deltaABC, deltaDEF, done, itemDone, completed}}, + {name: "completed only fallback", events: []string{completed}}, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + chunks := translateCodexStreamEvents(t, originalRequest, test.events...) + calls, finishReason := collectStreamedChatToolCalls(t, chunks) + if len(calls) != 1 { + t.Fatalf("expected one call, got %d; chunks=%q", len(calls), chunks) + } + call := calls[0] + if call.ID != "call_1" || call.Name != "ApplyPatch" { + t.Fatalf("call metadata = %+v", call) + } + if call.Arguments != "abcdef" { + t.Fatalf("streamed input = %q, want exactly %q", call.Arguments, "abcdef") + } + if call.Announcements != 1 { + t.Fatalf("call announced %d times, want once", call.Announcements) + } + if finishReason != "tool_calls" { + t.Fatalf("finish_reason = %q, want tool_calls", finishReason) + } + }) + } +} + +func TestCustomToolStreamingSupportsSequentialCalls(t *testing.T) { + originalRequest := customToolRequest("ApplyPatch") + chunks := translateCodexStreamEvents(t, originalRequest, + `{"type":"response.output_item.added","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_1","name":"ApplyPatch","input":""}}`, + `{"type":"response.custom_tool_call_input.done","output_index":0,"item_id":"ctc_1","input":"first"}`, + `{"type":"response.output_item.done","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_1","name":"ApplyPatch","input":"first"}}`, + `{"type":"response.output_item.added","output_index":1,"item":{"id":"ctc_2","type":"custom_tool_call","call_id":"call_2","name":"ApplyPatch","input":""}}`, + `{"type":"response.custom_tool_call_input.done","output_index":1,"item_id":"ctc_2","input":"second"}`, + `{"type":"response.output_item.done","output_index":1,"item":{"id":"ctc_2","type":"custom_tool_call","call_id":"call_2","name":"ApplyPatch","input":"second"}}`, + `{"type":"response.completed","response":{"status":"completed","output":[{"id":"ctc_1","type":"custom_tool_call","call_id":"call_1","name":"ApplyPatch","input":"first"},{"id":"ctc_2","type":"custom_tool_call","call_id":"call_2","name":"ApplyPatch","input":"second"}]}}`, + ) + calls, finishReason := collectStreamedChatToolCalls(t, chunks) + if len(calls) != 2 { + t.Fatalf("expected two sequential calls, got %d; chunks=%q", len(calls), chunks) + } + if calls[0].ID != "call_1" || calls[0].Arguments != "first" || calls[1].ID != "call_2" || calls[1].Arguments != "second" { + t.Fatalf("unexpected sequential calls: %+v", calls) + } + if finishReason != "tool_calls" { + t.Fatalf("finish_reason = %q, want tool_calls", finishReason) + } +} + +func TestMixedParallelCustomAndFunctionCallsKeepIndependentInputs(t *testing.T) { + originalRequest := []byte(`{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"Run both."}],"tools":[{"type":"custom","name":"ApplyPatch","format":{"type":"text"}},{"type":"function","function":{"name":"lookup","parameters":{"type":"object"}}}]}`) + chunks := translateCodexStreamEvents(t, originalRequest, + `{"type":"response.output_item.added","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_custom","name":"ApplyPatch","input":""}}`, + `{"type":"response.output_item.added","output_index":1,"item":{"id":"fc_1","type":"function_call","call_id":"call_function","name":"lookup","arguments":""}}`, + `{"type":"response.custom_tool_call_input.delta","output_index":0,"item_id":"ctc_1","delta":"patch"}`, + `{"type":"response.function_call_arguments.delta","output_index":1,"item_id":"fc_1","delta":"{\"q\":"}`, + `{"type":"response.custom_tool_call_input.done","output_index":0,"item_id":"ctc_1","input":"patch"}`, + `{"type":"response.function_call_arguments.delta","output_index":1,"item_id":"fc_1","delta":"1}"}`, + `{"type":"response.function_call_arguments.done","output_index":1,"item_id":"fc_1","arguments":"{\"q\":1}"}`, + `{"type":"response.output_item.done","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_custom","name":"ApplyPatch","input":"patch"}}`, + `{"type":"response.output_item.done","output_index":1,"item":{"id":"fc_1","type":"function_call","call_id":"call_function","name":"lookup","arguments":"{\"q\":1}"}}`, + `{"type":"response.completed","response":{"status":"completed","output":[{"id":"ctc_1","type":"custom_tool_call","call_id":"call_custom","name":"ApplyPatch","input":"patch"},{"id":"fc_1","type":"function_call","call_id":"call_function","name":"lookup","arguments":"{\"q\":1}"}]}}`, + ) + calls, finishReason := collectStreamedChatToolCalls(t, chunks) + if len(calls) != 2 { + t.Fatalf("expected two parallel calls, got %d; chunks=%q", len(calls), chunks) + } + if calls[0].ID != "call_custom" || calls[0].Name != "ApplyPatch" || calls[0].Arguments != "patch" { + t.Fatalf("custom parallel call corrupted: %+v", calls[0]) + } + if calls[1].ID != "call_function" || calls[1].Name != "lookup" || calls[1].Arguments != `{"q":1}` { + t.Fatalf("function parallel call corrupted: %+v", calls[1]) + } + if finishReason != "tool_calls" { + t.Fatalf("finish_reason = %q, want tool_calls", finishReason) + } +} + +func TestConvertCodexResponseToOpenAINonStreamSupportsCustomToolCall(t *testing.T) { + originalRequest := customToolRequest("ApplyPatch") + raw := []byte(`{"type":"response.completed","response":{"id":"resp_1","created_at":1700000000,"model":"gpt-5.6-sol","status":"completed","output":[{"id":"ctc_1","type":"custom_tool_call","call_id":"call_1","name":"ApplyPatch","input":"raw patch"}]}}`) + out := ConvertCodexResponseToOpenAINonStream(context.Background(), "gpt-5.6-sol", originalRequest, nil, raw, nil) + + toolCall := gjson.GetBytes(out, "choices.0.message.tool_calls.0") + if toolCall.Get("id").String() != "call_1" || toolCall.Get("function.name").String() != "ApplyPatch" || toolCall.Get("function.arguments").String() != "raw patch" { + t.Fatalf("non-stream custom tool call was not preserved: %s", out) + } + if got := gjson.GetBytes(out, "choices.0.finish_reason").String(); got != "tool_calls" { + t.Fatalf("finish_reason = %q, want tool_calls; payload=%s", got, out) + } +} + +func TestCustomToolNameShorteningRestoresResponseAndFollowUp(t *testing.T) { + longName := "ApplyPatch_" + strings.Repeat("namespace_", 8) + if len(longName) <= 64 { + t.Fatalf("test name length = %d, want greater than 64", len(longName)) + } + shortName := shortenNameIfNeeded(longName) + originalRequest := customToolRequest(longName) + chunks := translateCodexStreamEvents(t, originalRequest, + `{"type":"response.output_item.done","output_index":0,"item":{"id":"ctc_1","type":"custom_tool_call","call_id":"call_1","name":"`+shortName+`","input":"patch"}}`, + `{"type":"response.completed","response":{"status":"completed","output":[{"id":"ctc_1","type":"custom_tool_call","call_id":"call_1","name":"`+shortName+`","input":"patch"}]}}`, + ) + calls, _ := collectStreamedChatToolCalls(t, chunks) + if len(calls) != 1 || calls[0].Name != longName { + t.Fatalf("shortened response name was not restored: %+v", calls) + } + + followUp := []byte(`{"messages":[{"role":"assistant","tool_calls":[{"id":"call_1","type":"function","function":{"name":"","arguments":"patch"}}]},{"role":"tool","tool_call_id":"call_1","content":"done"}],"tools":[{"type":"custom","name":"","format":{"type":"text"}}]}`) + followUp, _ = sjson.SetBytes(followUp, "messages.0.tool_calls.0.function.name", longName) + followUp, _ = sjson.SetBytes(followUp, "tools.0.name", longName) + upstream := ConvertOpenAIRequestToCodex("gpt-5.6-sol", followUp, true) + if got := gjson.GetBytes(upstream, "tools.0.name").String(); got != shortName { + t.Fatalf("custom declaration name = %q, want shortened %q", got, shortName) + } + if got := gjson.GetBytes(upstream, "input.0.type").String(); got != "custom_tool_call" { + t.Fatalf("history call type = %q, want custom_tool_call; input=%s", got, gjson.GetBytes(upstream, "input").Raw) + } + if got := gjson.GetBytes(upstream, "input.0.name").String(); got != shortName { + t.Fatalf("history custom name = %q, want %q", got, shortName) + } + if got := gjson.GetBytes(upstream, "input.1.type").String(); got != "custom_tool_call_output" { + t.Fatalf("history output type = %q, want custom_tool_call_output", got) + } +} + +func TestStandardFunctionEnvelopeUsesCurrentCustomDeclaration(t *testing.T) { + input := []byte(`{"messages":[{"role":"assistant","tool_calls":[{"id":"call_custom","type":"function","function":{"name":"ApplyPatch","arguments":"raw patch"}}]},{"role":"tool","tool_call_id":"call_custom","content":"done"}],"tools":[{"type":"custom","name":"ApplyPatch","format":{"type":"text"}}]}`) + out := ConvertOpenAIRequestToCodex("gpt-5.6-sol", input, true) + items := gjson.GetBytes(out, "input").Array() + if len(items) != 2 { + t.Fatalf("expected custom call and output, got %d: %s", len(items), gjson.GetBytes(out, "input").Raw) + } + if got := items[0].Get("type").String(); got != "custom_tool_call" { + t.Fatalf("call type = %q, want custom_tool_call; item=%s", got, items[0].Raw) + } + if got := items[0].Get("input").String(); got != "raw patch" { + t.Fatalf("custom input = %q, want raw patch", got) + } + if got := items[1].Get("type").String(); got != "custom_tool_call_output" { + t.Fatalf("output type = %q, want custom_tool_call_output; item=%s", got, items[1].Raw) + } +} + +func TestFunctionEnvelopeRemainsFunctionWhenCustomNameIsAmbiguous(t *testing.T) { + input := []byte(`{"messages":[{"role":"assistant","tool_calls":[{"id":"call_1","type":"function","function":{"name":"shared","arguments":"{}"}}]},{"role":"tool","tool_call_id":"call_1","content":"done"}],"tools":[{"type":"custom","name":"shared","format":{"type":"text"}},{"type":"function","function":{"name":"shared","parameters":{"type":"object"}}}]}`) + out := ConvertOpenAIRequestToCodex("gpt-5.6-sol", input, true) + items := gjson.GetBytes(out, "input").Array() + if len(items) != 2 || items[0].Get("type").String() != "function_call" || items[1].Get("type").String() != "function_call_output" { + t.Fatalf("ambiguous name was guessed as custom: %s", gjson.GetBytes(out, "input").Raw) + } +} + +func TestMissingCustomCallIDSynthesizesUniquePair(t *testing.T) { + input := []byte(`{"messages":[{"role":"assistant","tool_calls":[{"type":"function","function":{"name":"ApplyPatch","arguments":"patch"}}]},{"role":"tool","content":"done"}],"tools":[{"type":"custom","name":"ApplyPatch","format":{"type":"text"}}]}`) + out := ConvertOpenAIRequestToCodex("gpt-5.6-sol", input, true) + items := gjson.GetBytes(out, "input").Array() + if len(items) != 2 || items[0].Get("type").String() != "custom_tool_call" || items[1].Get("type").String() != "custom_tool_call_output" { + t.Fatalf("missing-ID custom pair was not preserved: %s", gjson.GetBytes(out, "input").Raw) + } + callID := items[0].Get("call_id").String() + if callID == "" || items[1].Get("call_id").String() != callID { + t.Fatalf("synthesized call IDs do not match: %s", gjson.GetBytes(out, "input").Raw) + } +} + +func TestExistingFunctionCallRoundTripRemainsFunction(t *testing.T) { + input := []byte(`{"messages":[{"role":"assistant","tool_calls":[{"id":"call_lookup","type":"function","function":{"name":"lookup","arguments":"{\"q\":1}"}}]},{"role":"tool","tool_call_id":"call_lookup","content":"found"}],"tools":[{"type":"function","function":{"name":"lookup","parameters":{"type":"object"}}}]}`) + out := ConvertOpenAIRequestToCodex("gpt-5.6-sol", input, true) + items := gjson.GetBytes(out, "input").Array() + if len(items) != 2 || items[0].Get("type").String() != "function_call" || items[1].Get("type").String() != "function_call_output" { + t.Fatalf("function call family regressed: %s", gjson.GetBytes(out, "input").Raw) + } + if items[0].Get("arguments").String() != `{"q":1}` || items[1].Get("call_id").String() != "call_lookup" { + t.Fatalf("function call data regressed: %s", gjson.GetBytes(out, "input").Raw) + } +} + +func TestCustomDeclarationAndToolChoicePreserveFieldsAndShortName(t *testing.T) { + longName := "ApplyPatch_" + strings.Repeat("namespace_", 8) + shortName := shortenNameIfNeeded(longName) + input := []byte(`{"tools":[{"type":"custom","name":"","description":"Apply a freeform patch.","format":{"type":"text"}}],"tool_choice":{"type":"custom","name":"","vendor_extension":"keep"}}`) + input, _ = sjson.SetBytes(input, "tools.0.name", longName) + input, _ = sjson.SetBytes(input, "tool_choice.name", longName) + + out := ConvertOpenAIRequestToCodex("gpt-5.6-sol", input, true) + tool := gjson.GetBytes(out, "tools.0") + if tool.Get("type").String() != "custom" || tool.Get("name").String() != shortName { + t.Fatalf("custom declaration type/name regressed: %s", tool.Raw) + } + if tool.Get("description").String() != "Apply a freeform patch." || tool.Get("format.type").String() != "text" { + t.Fatalf("custom declaration fields were not preserved: %s", tool.Raw) + } + + choice := gjson.GetBytes(out, "tool_choice") + if choice.Get("type").String() != "custom" || choice.Get("name").String() != shortName { + t.Fatalf("custom tool choice type/name regressed: %s", choice.Raw) + } + if choice.Get("vendor_extension").String() != "keep" { + t.Fatalf("custom tool choice fields were not preserved: %s", choice.Raw) + } +} diff --git a/internal/translator/codex/openai/chat-completions/codex_openai_request.go b/internal/translator/codex/openai/chat-completions/codex_openai_request.go index 307df55d44e..a42bb2ecaa6 100644 --- a/internal/translator/codex/openai/chat-completions/codex_openai_request.go +++ b/internal/translator/codex/openai/chat-completions/codex_openai_request.go @@ -38,6 +38,13 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b // Stream must be set to true out, _ = sjson.SetBytes(out, "stream", stream) + if serviceTier := root.Get("service_tier"); serviceTier.Type == gjson.String { + switch serviceTier.String() { + case "fast", "priority": + out, _ = sjson.SetBytes(out, "service_tier", "priority") + } + } + // Codex not support temperature, top_p, top_k, max_output_tokens, so comment them // if v := gjson.GetBytes(rawJSON, "temperature"); v.Exists() { // out, _ = sjson.SetBytes(out, "temperature", v.Value()) @@ -72,64 +79,14 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b // Model out, _ = sjson.SetBytes(out, "model", modelName) - // Build request-local tool metadata and name shortening map. - originalToolNameMap := map[string]string{} - customToolNames := map[string]struct{}{} - functionToolNames := map[string]struct{}{} - { - if tools.IsArray() && len(toolResults) > 0 { - var names []string - seenNames := map[string]struct{}{} - for _, tool := range toolResults { - var name string - switch tool.Get("type").String() { - case "function": - name = tool.Get("function.name").String() - functionToolNames[name] = struct{}{} - case "custom": - name = tool.Get("name").String() - customToolNames[name] = struct{}{} - } - if name != "" { - if _, seen := seenNames[name]; !seen { - names = append(names, name) - seenNames[name] = struct{}{} - } - } - } - if len(names) > 0 { - originalToolNameMap = buildShortNameMap(names) - } - // A normalized function envelope cannot disambiguate declarations that share a name. - // Preserve function behavior for such ambiguous names. - for name := range functionToolNames { - delete(customToolNames, name) - } - } - } - - resolveToolCall := func(toolCall gjson.Result) (callType, name, input string, valid bool) { - switch toolCall.Get("type").String() { - case "custom": - return "custom", toolCall.Get("custom.name").String(), toolCall.Get("custom.input").String(), true - case "function": - name = toolCall.Get("function.name").String() - callType = "function" - if _, custom := customToolNames[name]; custom { - callType = "custom" - } - return callType, name, toolCall.Get("function.arguments").String(), true - default: - return "", "", "", false - } - } + toolCatalog := buildToolCatalog(rawJSON) // Extract system instructions from first system message (string or text object) messages := gjson.GetBytes(rawJSON, "messages") type pendingToolCall struct { callID string sourceCallID string - callType string + callFamily toolFamily consumed bool } var pendingToolCalls []pendingToolCall @@ -167,26 +124,25 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b continue } - pendingIndex := -1 + var pendingIndexes []int for index := range pendingToolCalls { pendingCall := &pendingToolCalls[index] if pendingCall.consumed { continue } if toolCallID == "" || pendingCall.sourceCallID == toolCallID || pendingCall.callID == toolCallID { - pendingIndex = index - break + pendingIndexes = append(pendingIndexes, index) } } - if pendingIndex < 0 { + if len(pendingIndexes) != 1 { continue } - pendingCall := &pendingToolCalls[pendingIndex] + pendingCall := &pendingToolCalls[pendingIndexes[0]] pendingCall.consumed = true toolCallID = pendingCall.callID outputType := "function_call_output" - if pendingCall.callType == "custom" { + if pendingCall.callFamily == toolFamilyCustom { outputType = "custom_tool_call_output" } @@ -297,9 +253,9 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b callIDCounts := map[string]int{} usedCallIDs := map[string]struct{}{} for _, tc := range toolCallsArr { - _, _, _, valid := resolveToolCall(tc) + toolCallType := tc.Get("type").String() callID := tc.Get("id").String() - if valid && callID != "" { + if (toolCallType == "function" || toolCallType == "custom") && callID != "" { callIDCounts[callID]++ usedCallIDs[callID] = struct{}{} } @@ -312,8 +268,8 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b for j := 0; j < len(toolCallsArr); j++ { tc := toolCallsArr[j] - toolCallType, toolCallName, toolCallInput, valid := resolveToolCall(tc) - if !valid { + toolCallType := tc.Get("type").String() + if toolCallType != "function" && toolCallType != "custom" { continue } sourceCallID := tc.Get("id").String() @@ -332,37 +288,38 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b } usedCallIDs[callID] = struct{}{} } + name := tc.Get("function.name").String() + input := tc.Get("function.arguments").String() + callFamily := toolFamilyFunction + if toolCallType == "custom" { + name = tc.Get("custom.name").String() + input = tc.Get("custom.input").String() + callFamily = toolFamilyCustom + } else if toolCatalog.familyForChatCall(name) == toolFamilyCustom { + callFamily = toolFamilyCustom + } + pendingToolCalls = append(pendingToolCalls, pendingToolCall{ callID: callID, sourceCallID: sourceCallID, - callType: toolCallType, + callFamily: callFamily, }) - switch toolCallType { - case "function": + switch callFamily { + case toolFamilyFunction: // Create function_call as top-level object funcCall := []byte(`{}`) funcCall, _ = sjson.SetBytes(funcCall, "type", "function_call") funcCall, _ = sjson.SetBytes(funcCall, "call_id", callID) - if short, ok := originalToolNameMap[toolCallName]; ok { - toolCallName = short - } else { - toolCallName = shortenNameIfNeeded(toolCallName) - } - funcCall, _ = sjson.SetBytes(funcCall, "name", toolCallName) - funcCall, _ = sjson.SetBytes(funcCall, "arguments", toolCallInput) + funcCall, _ = sjson.SetBytes(funcCall, "name", toolCatalog.shorten(name)) + funcCall, _ = sjson.SetBytes(funcCall, "arguments", input) inputItems = append(inputItems, funcCall) - case "custom": + case toolFamilyCustom: customCall := []byte(`{}`) customCall, _ = sjson.SetBytes(customCall, "type", "custom_tool_call") customCall, _ = sjson.SetBytes(customCall, "call_id", callID) - if short, ok := originalToolNameMap[toolCallName]; ok { - toolCallName = short - } else { - toolCallName = shortenNameIfNeeded(toolCallName) - } - customCall, _ = sjson.SetBytes(customCall, "name", toolCallName) - customCall, _ = sjson.SetBytes(customCall, "input", toolCallInput) + customCall, _ = sjson.SetBytes(customCall, "name", toolCatalog.shorten(name)) + customCall, _ = sjson.SetBytes(customCall, "input", input) inputItems = append(inputItems, customCall) } } @@ -425,21 +382,17 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b for i := 0; i < len(arr); i++ { t := arr[i] toolType := t.Get("type").String() - if toolType == "custom" { + if toolType == "custom" && t.IsObject() { item := []byte(t.Raw) - name := t.Get("name").String() - if short, ok := originalToolNameMap[name]; ok { - name = short - } else { - name = shortenNameIfNeeded(name) + if name := t.Get("name").String(); name != "" { + item, _ = sjson.SetBytes(item, "name", toolCatalog.shorten(name)) } - item, _ = sjson.SetBytes(item, "name", name) toolItems = append(toolItems, item) continue } // Pass through built-in tools (e.g. {"type":"web_search"}) directly for the Responses API. - // Only function and custom tools need structural conversion. + // Function tools need structural conversion because Chat Completions nests details under "function". if toolType != "" && toolType != "function" && t.IsObject() { toolItems = append(toolItems, []byte(t.Raw)) continue @@ -452,12 +405,7 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b if fn.Exists() { if v := fn.Get("name"); v.Exists() { name := v.String() - if short, ok := originalToolNameMap[name]; ok { - name = short - } else { - name = shortenNameIfNeeded(name) - } - item, _ = sjson.SetBytes(item, "name", name) + item, _ = sjson.SetBytes(item, "name", toolCatalog.shorten(name)) } if v := fn.Get("description"); v.Exists() { item, _ = sjson.SetBytes(item, "description", v.Value()) @@ -477,27 +425,20 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b // Map tool_choice when present. // Chat Completions: "tool_choice" can be a string ("auto"/"none") or an object (e.g. {"type":"function","function":{"name":"..."}}). - // Responses API: keep built-in tool choices as-is and flatten named choices to {"type":"...","name":"..."}. + // Responses API: keep built-in tool choices as-is; flatten function choice to {"type":"function","name":"..."}. if tc := gjson.GetBytes(rawJSON, "tool_choice"); tc.Exists() { switch { case tc.Type == gjson.String: out, _ = sjson.SetBytes(out, "tool_choice", tc.String()) case tc.IsObject(): tcType := tc.Get("type").String() - if tcType == "function" || tcType == "custom" { - name := tc.Get("name").String() - if tcType == "function" { - name = tc.Get("function.name").String() - if _, custom := customToolNames[name]; custom { - tcType = "custom" - } + if tcType == "function" { + name := tc.Get("function.name").String() + if toolCatalog.familyForChatCall(name) == toolFamilyCustom { + tcType = "custom" } if name != "" { - if short, ok := originalToolNameMap[name]; ok { - name = short - } else { - name = shortenNameIfNeeded(name) - } + name = toolCatalog.shorten(name) } choice := []byte(`{}`) choice, _ = sjson.SetBytes(choice, "type", tcType) @@ -505,6 +446,12 @@ func ConvertOpenAIRequestToCodex(modelName string, inputRawJSON []byte, stream b choice, _ = sjson.SetBytes(choice, "name", name) } out, _ = sjson.SetRawBytes(out, "tool_choice", choice) + } else if tcType == "custom" { + choice := []byte(tc.Raw) + if name := tc.Get("name").String(); name != "" { + choice, _ = sjson.SetBytes(choice, "name", toolCatalog.shorten(name)) + } + out, _ = sjson.SetRawBytes(out, "tool_choice", choice) } else if tcType != "" { // Built-in tool choices (e.g. {"type":"web_search"}) are already Responses-compatible. out, _ = sjson.SetRawBytes(out, "tool_choice", []byte(tc.Raw)) diff --git a/internal/translator/codex/openai/chat-completions/codex_openai_request_test.go b/internal/translator/codex/openai/chat-completions/codex_openai_request_test.go index 6d494614b45..097fd49ecd2 100644 --- a/internal/translator/codex/openai/chat-completions/codex_openai_request_test.go +++ b/internal/translator/codex/openai/chat-completions/codex_openai_request_test.go @@ -1,6 +1,7 @@ package chat_completions import ( + "strings" "testing" "github.com/tidwall/gjson" @@ -1284,7 +1285,7 @@ func TestToolCallHistoryClearsUnmatchedCallAtNewBatch(t *testing.T) { } } -func TestToolCallOutputWithoutIDUsesPendingCall(t *testing.T) { +func TestToolCallOutputWithoutIDAmbiguousBatchIsDropped(t *testing.T) { input := []byte(`{ "messages": [ {"role":"assistant","content":null,"tool_calls":[ @@ -1298,20 +1299,13 @@ func TestToolCallOutputWithoutIDUsesPendingCall(t *testing.T) { out := ConvertOpenAIRequestToCodex("gpt-5.6-sol", input, true) items := gjson.GetBytes(out, "input").Array() - if len(items) != 4 { - t.Fatalf("expected two calls and two outputs, got %d items: %s", len(items), gjson.GetBytes(out, "input").Raw) - } - if got := items[2].Get("type").String(); got != "function_call_output" { - t.Fatalf("expected first empty-ID output to match function call, got %s", items[2].Raw) - } - if got := items[2].Get("call_id").String(); got != "call_explicit" { - t.Fatalf("expected explicit pending call_id, got %s", items[2].Raw) - } - if got := items[3].Get("type").String(); got != "custom_tool_call_output" { - t.Fatalf("expected second empty-ID output to match custom call, got %s", items[3].Raw) + if len(items) != 2 { + t.Fatalf("expected only the two calls because ID-less outputs are ambiguous, got %d items: %s", len(items), gjson.GetBytes(out, "input").Raw) } - if got := items[3].Get("call_id").String(); got == "" { - t.Fatalf("expected synthesized custom output call_id, got %s", items[3].Raw) + for i, item := range items { + if strings.HasSuffix(item.Get("type").String(), "_output") { + t.Fatalf("item %d: ambiguous ID-less output was guessed: %s", i, item.Raw) + } } } @@ -1366,6 +1360,47 @@ func TestOrphanAndDuplicateToolCallOutputsAreDropped(t *testing.T) { } } +func TestServiceTierTranslatedForCodexFastMode(t *testing.T) { + tests := []struct { + name string + serviceTier string + want string + wantExists bool + }{ + {name: "priority", serviceTier: `"priority"`, want: "priority", wantExists: true}, + {name: "fast alias", serviceTier: `"fast"`, want: "priority", wantExists: true}, + {name: "whitespace priority omitted", serviceTier: `" PRIORITY "`}, + {name: "uppercase fast omitted", serviceTier: `"FAST"`}, + {name: "default omitted", serviceTier: `"default"`}, + {name: "auto omitted", serviceTier: `"auto"`}, + {name: "flex omitted", serviceTier: `"flex"`}, + {name: "empty omitted", serviceTier: `""`}, + {name: "non string omitted", serviceTier: `true`}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + input := []byte(`{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"Reply OK"}],"service_tier":` + tt.serviceTier + `}`) + out := ConvertOpenAIRequestToCodex("gpt-5.6-sol", input, true) + serviceTier := gjson.GetBytes(out, "service_tier") + if serviceTier.Exists() != tt.wantExists { + t.Fatalf("service_tier exists = %v, want %v; output=%s", serviceTier.Exists(), tt.wantExists, out) + } + if tt.wantExists && serviceTier.String() != tt.want { + t.Fatalf("service_tier = %q, want %q; output=%s", serviceTier.String(), tt.want, out) + } + }) + } +} + +func TestServiceTierOmittedWhenAbsent(t *testing.T) { + input := []byte(`{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"Reply OK"}]}`) + out := ConvertOpenAIRequestToCodex("gpt-5.6-sol", input, true) + if serviceTier := gjson.GetBytes(out, "service_tier"); serviceTier.Exists() { + t.Fatalf("service_tier should be omitted when absent; output=%s", out) + } +} + // Tools array should carry over to the Responses format output. func TestToolsDefinitionTranslated(t *testing.T) { input := []byte(`{ diff --git a/internal/translator/codex/openai/chat-completions/codex_openai_response.go b/internal/translator/codex/openai/chat-completions/codex_openai_response.go index ad9dace1e70..d99a039fb5e 100644 --- a/internal/translator/codex/openai/chat-completions/codex_openai_response.go +++ b/internal/translator/codex/openai/chat-completions/codex_openai_response.go @@ -20,20 +20,12 @@ var ( dataTag = []byte("data:") ) -type toolCallStreamState struct { - Index int - ArgumentsEmitted bool - Done bool -} - // ConvertCliToOpenAIParams holds parameters for response conversion. type ConvertCliToOpenAIParams struct { ResponseID string CreatedAt int64 Model string - FunctionCallIndex int - toolCallStates map[string]*toolCallStreamState - currentToolCall *toolCallStreamState + ToolCalls *streamToolCallTracker LastImageHashByItemID map[string][32]byte } @@ -57,8 +49,7 @@ func ConvertCodexResponseToOpenAI(_ context.Context, modelName string, originalR Model: modelName, CreatedAt: 0, ResponseID: "", - FunctionCallIndex: -1, - toolCallStates: make(map[string]*toolCallStreamState), + ToolCalls: newStreamToolCallTracker(), LastImageHashByItemID: make(map[string][32]byte), } } @@ -72,21 +63,25 @@ func ConvertCodexResponseToOpenAI(_ context.Context, modelName string, originalR template := []byte(`{"id":"","object":"chat.completion.chunk","created":12345,"model":"model","choices":[{"index":0,"delta":{},"finish_reason":null,"native_finish_reason":null}]}`) rootResult := gjson.ParseBytes(rawJSON) + params := (*param).(*ConvertCliToOpenAIParams) + if params.ToolCalls == nil { + params.ToolCalls = newStreamToolCallTracker() + } typeResult := rootResult.Get("type") dataType := typeResult.String() if dataType == "response.created" { - (*param).(*ConvertCliToOpenAIParams).ResponseID = rootResult.Get("response.id").String() - (*param).(*ConvertCliToOpenAIParams).CreatedAt = rootResult.Get("response.created_at").Int() - (*param).(*ConvertCliToOpenAIParams).Model = rootResult.Get("response.model").String() - if (*param).(*ConvertCliToOpenAIParams).LastImageHashByItemID == nil { - (*param).(*ConvertCliToOpenAIParams).LastImageHashByItemID = make(map[string][32]byte) + params.ResponseID = rootResult.Get("response.id").String() + params.CreatedAt = rootResult.Get("response.created_at").Int() + params.Model = rootResult.Get("response.model").String() + if params.LastImageHashByItemID == nil { + params.LastImageHashByItemID = make(map[string][32]byte) } return [][]byte{} } // Extract and set the model version. - cachedModel := (*param).(*ConvertCliToOpenAIParams).Model + cachedModel := params.Model if modelResult := gjson.GetBytes(rawJSON, "model"); modelResult.Exists() { template, _ = sjson.SetBytes(template, "model", modelResult.String()) } else if cachedModel != "" { @@ -95,10 +90,10 @@ func ConvertCodexResponseToOpenAI(_ context.Context, modelName string, originalR template, _ = sjson.SetBytes(template, "model", modelName) } - template, _ = sjson.SetBytes(template, "created", (*param).(*ConvertCliToOpenAIParams).CreatedAt) + template, _ = sjson.SetBytes(template, "created", params.CreatedAt) // Extract and set the response ID. - template, _ = sjson.SetBytes(template, "id", (*param).(*ConvertCliToOpenAIParams).ResponseID) + template, _ = sjson.SetBytes(template, "id", params.ResponseID) // Extract and set usage metadata (token counts). if usageResult := gjson.GetBytes(rawJSON, "response.usage"); usageResult.Exists() { @@ -142,7 +137,7 @@ func ConvertCodexResponseToOpenAI(_ context.Context, modelName string, originalR return [][]byte{} } if itemID != "" { - p := (*param).(*ConvertCliToOpenAIParams) + p := params if p.LastImageHashByItemID == nil { p.LastImageHashByItemID = make(map[string][32]byte) } @@ -169,6 +164,11 @@ func ConvertCodexResponseToOpenAI(_ context.Context, modelName string, originalR template, _ = sjson.SetBytes(template, "choices.0.delta.role", "assistant") template, _ = sjson.SetRawBytes(template, "choices.0.delta.images.-1", imagePayload) } else if dataType == "response.completed" || dataType == "response.incomplete" { + var chunks [][]byte + if dataType == "response.completed" { + chunks = append(chunks, emitCompletedToolCalls(template, params.ToolCalls, buildToolCatalog(originalRequestRawJSON), rootResult.Get("response.output"))...) + } + finishReason := "stop" nativeFinishReason := finishReason if dataType == "response.incomplete" { @@ -179,82 +179,65 @@ func ConvertCodexResponseToOpenAI(_ context.Context, modelName string, originalR case "content_filter": finishReason = "content_filter" } - } else if (*param).(*ConvertCliToOpenAIParams).FunctionCallIndex != -1 { + } else if params.ToolCalls.hasAnnouncedCall() { finishReason = "tool_calls" nativeFinishReason = finishReason } template, _ = sjson.SetBytes(template, "choices.0.finish_reason", finishReason) template, _ = sjson.SetBytes(template, "choices.0.native_finish_reason", nativeFinishReason) + return append(chunks, template) } else if dataType == "response.output_item.added" { itemResult := rootResult.Get("item") - if !itemResult.Exists() || !isCodexToolCallType(itemResult.Get("type").String()) { + family, ok := toolFamilyFromItem(itemResult) + if !itemResult.Exists() || !ok { return [][]byte{} } - // Increment index for this new tool call item. - p := (*param).(*ConvertCliToOpenAIParams) - p.FunctionCallIndex++ - state := &toolCallStreamState{Index: p.FunctionCallIndex} - registerToolCallState(p, rootResult, itemResult, state) - - functionCallItemTemplate := []byte(`{"index":0,"id":"","type":"function","function":{"name":"","arguments":""}}`) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "index", state.Index) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "id", itemResult.Get("call_id").String()) - - // Restore original tool name if it was shortened. - name := itemResult.Get("name").String() - rev := buildReverseMapFromOriginalOpenAI(originalRequestRawJSON) - if orig, ok := rev[name]; ok { - name = orig + state := params.ToolCalls.stateForEvent(rootResult, itemResult, family, true, true) + state.name = buildToolCatalog(originalRequestRawJSON).restore(state.name) + chunk := announceToolCall(template, state, state.bufferedInput) + if chunk == nil { + return [][]byte{} } - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "function.name", name) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "function.arguments", "") - - template, _ = sjson.SetBytes(template, "choices.0.delta.role", "assistant") - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls", []byte(`[]`)) - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls.-1", functionCallItemTemplate) - + return [][]byte{chunk} } else if dataType == "response.function_call_arguments.delta" || dataType == "response.custom_tool_call_input.delta" { - p := (*param).(*ConvertCliToOpenAIParams) - state := findToolCallState(p, rootResult, gjson.Result{}) + family := toolFamilyFunction + if dataType == "response.custom_tool_call_input.delta" { + family = toolFamilyCustom + } + state := params.ToolCalls.stateForEvent(rootResult, gjson.Result{}, family, true, false) + if state == nil { + return [][]byte{} + } deltaValue := rootResult.Get("delta").String() - if state == nil || state.Done || deltaValue == "" { + if !state.announced { + state.bufferedInput += deltaValue return [][]byte{} } - state.ArgumentsEmitted = true - - functionCallItemTemplate := []byte(`{"index":0,"function":{"arguments":""}}`) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "index", state.Index) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "function.arguments", deltaValue) - - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls", []byte(`[]`)) - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls.-1", functionCallItemTemplate) - - } else if dataType == "response.function_call_arguments.done" || dataType == "response.custom_tool_call_input.done" { - p := (*param).(*ConvertCliToOpenAIParams) - state := findToolCallState(p, rootResult, gjson.Result{}) - if state == nil || state.Done || state.ArgumentsEmitted { - // Arguments were already streamed via delta events; nothing to emit. + if deltaValue != "" { + state.inputDeltaSeen = true + } + chunk := emitToolInputDelta(template, state, deltaValue) + if chunk == nil { return [][]byte{} } - - // Fallback: no delta events were received, emit the full arguments as a single chunk. - fullArgsField := "arguments" + return [][]byte{chunk} + } else if dataType == "response.function_call_arguments.done" || dataType == "response.custom_tool_call_input.done" { + family := toolFamilyFunction + inputPath := "arguments" if dataType == "response.custom_tool_call_input.done" { - fullArgsField = "input" + family = toolFamilyCustom + inputPath = "input" } - state.ArgumentsEmitted = true - fullArgs := rootResult.Get(fullArgsField).String() - if fullArgs == "" { + state := params.ToolCalls.stateForEvent(rootResult, gjson.Result{}, family, true, false) + if state == nil { return [][]byte{} } - functionCallItemTemplate := []byte(`{"index":0,"function":{"arguments":""}}`) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "index", state.Index) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "function.arguments", fullArgs) - - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls", []byte(`[]`)) - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls.-1", functionCallItemTemplate) - + if state.inputDeltaSeen && family == toolFamilyFunction { + return [][]byte{} + } + inputResult := rootResult.Get(inputPath) + return emitAvailableToolCall(template, state, inputResult.String(), inputResult.Exists()) } else if dataType == "response.output_item.done" { itemResult := rootResult.Get("item") if !itemResult.Exists() { @@ -268,7 +251,7 @@ func ConvertCodexResponseToOpenAI(_ context.Context, modelName string, originalR return [][]byte{} } if itemID != "" { - p := (*param).(*ConvertCliToOpenAIParams) + p := params if p.LastImageHashByItemID == nil { p.LastImageHashByItemID = make(map[string][32]byte) } @@ -296,58 +279,23 @@ func ConvertCodexResponseToOpenAI(_ context.Context, modelName string, originalR template, _ = sjson.SetRawBytes(template, "choices.0.delta.images.-1", imagePayload) return [][]byte{template} } - if !isCodexToolCallType(itemType) { + family, ok := toolFamilyFromItem(itemResult) + if !ok { return [][]byte{} } - - p := (*param).(*ConvertCliToOpenAIParams) - state := findToolCallState(p, rootResult, itemResult) - if state != nil { - if state.Done { - return [][]byte{} - } - state.Done = true - if state.ArgumentsEmitted { - return [][]byte{} - } - - // The tool was announced, but no argument event arrived. Emit only the - // completed arguments so the id and name are not duplicated. - state.ArgumentsEmitted = true - fullArgs := codexToolCallArguments(itemResult) - if fullArgs == "" { - return [][]byte{} - } - functionCallItemTemplate := []byte(`{"index":0,"function":{"arguments":""}}`) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "index", state.Index) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "function.arguments", fullArgs) - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls", []byte(`[]`)) - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls.-1", functionCallItemTemplate) - return [][]byte{template} + state := params.ToolCalls.stateForEvent(rootResult, itemResult, family, true, false) + if state == nil { + return [][]byte{} } - - // Fallback path: model skipped output_item.added, so emit the complete tool call now. - p.FunctionCallIndex++ - state = &toolCallStreamState{Index: p.FunctionCallIndex, ArgumentsEmitted: true, Done: true} - registerToolCallState(p, rootResult, itemResult, state) - - functionCallItemTemplate := []byte(`{"index":0,"id":"","type":"function","function":{"name":"","arguments":""}}`) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "index", state.Index) - - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls", []byte(`[]`)) - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "id", itemResult.Get("call_id").String()) - - // Restore original tool name if it was shortened. - name := itemResult.Get("name").String() - rev := buildReverseMapFromOriginalOpenAI(originalRequestRawJSON) - if orig, ok := rev[name]; ok { - name = orig + state.name = buildToolCatalog(originalRequestRawJSON).restore(state.name) + input, hasInput := toolInputFromItem(itemResult, family) + if state.inputDeltaSeen && family == toolFamilyFunction { + state.itemDone = true + return [][]byte{} } - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "function.name", name) - - functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "function.arguments", codexToolCallArguments(itemResult)) - template, _ = sjson.SetBytes(template, "choices.0.delta.role", "assistant") - template, _ = sjson.SetRawBytes(template, "choices.0.delta.tool_calls.-1", functionCallItemTemplate) + chunks := emitAvailableToolCall(template, state, input, hasInput) + state.itemDone = true + return chunks } else { return [][]byte{} @@ -462,7 +410,7 @@ func ConvertCodexResponseToOpenAINonStream(_ context.Context, _ string, original } } case "function_call", "custom_tool_call": - // Handle function and custom tool call content. + // Chat Completions exposes custom calls through the standard function envelope. functionCallTemplate := []byte(`{"id":"","type":"function","function":{"name":"","arguments":""}}`) if callIdResult := outputItem.Get("call_id"); callIdResult.Exists() { @@ -478,7 +426,13 @@ func ConvertCodexResponseToOpenAINonStream(_ context.Context, _ string, original functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "function.name", n) } - functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "function.arguments", codexToolCallArguments(outputItem)) + argumentsPath := "arguments" + if outputType == "custom_tool_call" { + argumentsPath = "input" + } + if argsResult := outputItem.Get(argumentsPath); argsResult.Exists() { + functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "function.arguments", argsResult.String()) + } toolCalls = append(toolCalls, functionCallTemplate) case "image_generation_call": @@ -556,79 +510,10 @@ func ConvertCodexResponseToOpenAINonStream(_ context.Context, _ string, original return template } -func registerToolCallState(p *ConvertCliToOpenAIParams, eventResult, itemResult gjson.Result, state *toolCallStreamState) { - if p.toolCallStates == nil { - p.toolCallStates = make(map[string]*toolCallStreamState) - } - if itemID := eventResult.Get("item_id").String(); itemID != "" { - p.toolCallStates["item:"+itemID] = state - } - if itemID := itemResult.Get("id").String(); itemID != "" { - p.toolCallStates["item:"+itemID] = state - } - if outputIndex := eventResult.Get("output_index"); outputIndex.Exists() { - p.toolCallStates["output:"+outputIndex.Raw] = state - } - p.currentToolCall = state -} - -func findToolCallState(p *ConvertCliToOpenAIParams, eventResult, itemResult gjson.Result) *toolCallStreamState { - if itemID := eventResult.Get("item_id").String(); itemID != "" { - return p.toolCallStates["item:"+itemID] - } - if itemID := itemResult.Get("id").String(); itemID != "" { - return p.toolCallStates["item:"+itemID] - } - if outputIndex := eventResult.Get("output_index"); outputIndex.Exists() { - return p.toolCallStates["output:"+outputIndex.Raw] - } - return p.currentToolCall -} - -func isCodexToolCallType(itemType string) bool { - return itemType == "function_call" || itemType == "custom_tool_call" -} - -func codexToolCallArguments(itemResult gjson.Result) string { - if itemResult.Get("type").String() == "custom_tool_call" { - return itemResult.Get("input").String() - } - return itemResult.Get("arguments").String() -} - // buildReverseMapFromOriginalOpenAI builds a map of shortened tool name -> original tool name // from the original OpenAI-style request JSON using the same shortening logic. func buildReverseMapFromOriginalOpenAI(original []byte) map[string]string { - tools := gjson.GetBytes(original, "tools") - rev := map[string]string{} - if tools.IsArray() && len(tools.Array()) > 0 { - var names []string - seenNames := map[string]struct{}{} - arr := tools.Array() - for i := 0; i < len(arr); i++ { - t := arr[i] - var name string - switch t.Get("type").String() { - case "function": - name = t.Get("function.name").String() - case "custom": - name = t.Get("name").String() - } - if name != "" { - if _, seen := seenNames[name]; !seen { - names = append(names, name) - seenNames[name] = struct{}{} - } - } - } - if len(names) > 0 { - m := buildShortNameMap(names) - for orig, short := range m { - rev[short] = orig - } - } - } - return rev + return buildToolCatalog(original).originalByShort } func mimeTypeFromCodexOutputFormat(outputFormat string) string { diff --git a/internal/translator/codex/openai/chat-completions/codex_openai_tool_catalog.go b/internal/translator/codex/openai/chat-completions/codex_openai_tool_catalog.go new file mode 100644 index 00000000000..13f7694c75c --- /dev/null +++ b/internal/translator/codex/openai/chat-completions/codex_openai_tool_catalog.go @@ -0,0 +1,98 @@ +package chat_completions + +import "github.com/tidwall/gjson" + +type toolFamily uint8 + +const ( + toolFamilyFunction toolFamily = iota + toolFamilyCustom +) + +type toolCatalog struct { + shortByOriginal map[string]string + originalByShort map[string]string + familiesByName map[string]map[toolFamily]struct{} +} + +func buildToolCatalog(rawJSON []byte) toolCatalog { + catalog := toolCatalog{ + shortByOriginal: make(map[string]string), + originalByShort: make(map[string]string), + familiesByName: make(map[string]map[toolFamily]struct{}), + } + + tools := gjson.GetBytes(rawJSON, "tools") + if !tools.IsArray() { + return catalog + } + + var names []string + seenNames := make(map[string]struct{}) + for _, tool := range tools.Array() { + var name string + var family toolFamily + switch tool.Get("type").String() { + case "function": + name = tool.Get("function.name").String() + family = toolFamilyFunction + case "custom": + name = tool.Get("name").String() + family = toolFamilyCustom + default: + continue + } + if name == "" { + continue + } + + catalog.addFamily(name, family) + if _, exists := seenNames[name]; !exists { + seenNames[name] = struct{}{} + names = append(names, name) + } + } + + catalog.shortByOriginal = buildShortNameMap(names) + for original, short := range catalog.shortByOriginal { + catalog.originalByShort[short] = original + for family := range catalog.familiesByName[original] { + catalog.addFamily(short, family) + } + } + return catalog +} + +func (catalog toolCatalog) addFamily(name string, family toolFamily) { + families := catalog.familiesByName[name] + if families == nil { + families = make(map[toolFamily]struct{}) + catalog.familiesByName[name] = families + } + families[family] = struct{}{} +} + +func (catalog toolCatalog) shorten(name string) string { + if short, ok := catalog.shortByOriginal[name]; ok { + return short + } + return shortenNameIfNeeded(name) +} + +func (catalog toolCatalog) restore(name string) string { + if original, ok := catalog.originalByShort[name]; ok { + return original + } + return name +} + +func (catalog toolCatalog) familyForChatCall(name string) toolFamily { + families := catalog.familiesByName[name] + if len(families) != 1 { + return toolFamilyFunction + } + if _, ok := families[toolFamilyCustom]; ok { + return toolFamilyCustom + } + return toolFamilyFunction +} diff --git a/internal/translator/codex/openai/chat-completions/codex_openai_tool_stream.go b/internal/translator/codex/openai/chat-completions/codex_openai_tool_stream.go new file mode 100644 index 00000000000..49326a8f4d0 --- /dev/null +++ b/internal/translator/codex/openai/chat-completions/codex_openai_tool_stream.go @@ -0,0 +1,323 @@ +package chat_completions + +import ( + "strings" + + "github.com/tidwall/gjson" + "github.com/tidwall/sjson" +) + +type toolCallStreamState struct { + chatIndex int + itemID string + outputIndex int64 + hasOutputIndex bool + callID string + name string + family toolFamily + announced bool + inputDeltaSeen bool + emittedInput string + bufferedInput string + completeInput string + hasCompleteInput bool + itemDone bool +} + +type streamToolCallTracker struct { + nextChatIndex int + byItemID map[string]*toolCallStreamState + byOutputIndex map[int64]*toolCallStreamState + ordered []*toolCallStreamState +} + +func newStreamToolCallTracker() *streamToolCallTracker { + return &streamToolCallTracker{ + byItemID: make(map[string]*toolCallStreamState), + byOutputIndex: make(map[int64]*toolCallStreamState), + } +} + +func (tracker *streamToolCallTracker) ensure() { + if tracker.byItemID == nil { + tracker.byItemID = make(map[string]*toolCallStreamState) + } + if tracker.byOutputIndex == nil { + tracker.byOutputIndex = make(map[int64]*toolCallStreamState) + } +} + +func (tracker *streamToolCallTracker) newState(family toolFamily) *toolCallStreamState { + tracker.ensure() + state := &toolCallStreamState{ + chatIndex: tracker.nextChatIndex, + family: family, + } + tracker.nextChatIndex++ + tracker.ordered = append(tracker.ordered, state) + return state +} + +func (tracker *streamToolCallTracker) stateForEvent(root, item gjson.Result, family toolFamily, create, forceNew bool) *toolCallStreamState { + tracker.ensure() + itemID := root.Get("item_id").String() + if itemID == "" { + itemID = item.Get("id").String() + } + outputIndexResult := root.Get("output_index") + callID := item.Get("call_id").String() + + var state *toolCallStreamState + if itemID != "" { + state = tracker.byItemID[itemID] + } + if state == nil && outputIndexResult.Exists() { + state = tracker.byOutputIndex[outputIndexResult.Int()] + } + if state == nil && callID != "" { + state = tracker.uniqueStateByCallID(callID) + } + if state == nil && !forceNew { + activeState, activeCount := tracker.activeState(family) + if activeCount == 1 { + state = activeState + } else if activeCount > 1 && itemID == "" && !outputIndexResult.Exists() && callID == "" { + return nil + } + } + if state == nil && create { + state = tracker.newState(family) + } + if state == nil { + return nil + } + + state.family = family + if itemID != "" { + state.itemID = itemID + tracker.byItemID[itemID] = state + } + if outputIndexResult.Exists() { + state.outputIndex = outputIndexResult.Int() + state.hasOutputIndex = true + tracker.byOutputIndex[state.outputIndex] = state + } + if callID != "" { + state.callID = callID + } + if name := item.Get("name").String(); name != "" { + state.name = name + } + return state +} + +func (tracker *streamToolCallTracker) stateForCompletedItem(item gjson.Result, outputIndex int64, family toolFamily) *toolCallStreamState { + tracker.ensure() + itemID := item.Get("id").String() + callID := item.Get("call_id").String() + + var state *toolCallStreamState + if itemID != "" { + state = tracker.byItemID[itemID] + } + if state == nil { + state = tracker.byOutputIndex[outputIndex] + } + if state == nil && callID != "" { + state = tracker.uniqueStateByCallID(callID) + } + if state == nil { + state = tracker.newState(family) + } + + state.family = family + state.outputIndex = outputIndex + state.hasOutputIndex = true + tracker.byOutputIndex[outputIndex] = state + if itemID != "" { + state.itemID = itemID + tracker.byItemID[itemID] = state + } + if callID != "" { + state.callID = callID + } + if name := item.Get("name").String(); name != "" { + state.name = name + } + return state +} + +func (tracker *streamToolCallTracker) uniqueStateByCallID(callID string) *toolCallStreamState { + var matched *toolCallStreamState + for _, state := range tracker.ordered { + if state.callID != callID { + continue + } + if matched != nil && matched != state { + return nil + } + matched = state + } + return matched +} + +func (tracker *streamToolCallTracker) activeState(family toolFamily) (*toolCallStreamState, int) { + var matched *toolCallStreamState + count := 0 + for _, state := range tracker.ordered { + if state.family != family || state.itemDone { + continue + } + count++ + if matched == nil { + matched = state + } + } + return matched, count +} + +func (tracker *streamToolCallTracker) hasAnnouncedCall() bool { + for _, state := range tracker.ordered { + if state.announced { + return true + } + } + return false +} + +func toolFamilyFromItem(item gjson.Result) (toolFamily, bool) { + switch item.Get("type").String() { + case "function_call": + return toolFamilyFunction, true + case "custom_tool_call": + return toolFamilyCustom, true + default: + return toolFamilyFunction, false + } +} + +func toolInputFromItem(item gjson.Result, family toolFamily) (string, bool) { + path := "arguments" + if family == toolFamilyCustom { + path = "input" + } + input := item.Get(path) + return input.String(), input.Exists() +} + +func remainingToolInput(emitted, complete string) (string, bool) { + if emitted == "" { + return complete, true + } + if complete == emitted { + return "", true + } + if strings.HasPrefix(complete, emitted) { + return complete[len(emitted):], true + } + return "", false +} + +func announceToolCall(template []byte, state *toolCallStreamState, input string) []byte { + if state == nil || state.announced || state.callID == "" || state.name == "" { + return nil + } + + toolCall := []byte(`{"index":0,"id":"","type":"function","function":{"name":"","arguments":""}}`) + toolCall, _ = sjson.SetBytes(toolCall, "index", state.chatIndex) + toolCall, _ = sjson.SetBytes(toolCall, "id", state.callID) + toolCall, _ = sjson.SetBytes(toolCall, "function.name", state.name) + toolCall, _ = sjson.SetBytes(toolCall, "function.arguments", input) + + chunk := template + chunk, _ = sjson.SetBytes(chunk, "choices.0.delta.role", "assistant") + chunk, _ = sjson.SetRawBytes(chunk, "choices.0.delta.tool_calls", []byte(`[]`)) + chunk, _ = sjson.SetRawBytes(chunk, "choices.0.delta.tool_calls.-1", toolCall) + + state.announced = true + state.emittedInput += input + state.bufferedInput = "" + return chunk +} + +func emitToolInputDelta(template []byte, state *toolCallStreamState, delta string) []byte { + if state == nil || !state.announced || delta == "" { + return nil + } + + toolCall := []byte(`{"index":0,"function":{"arguments":""}}`) + toolCall, _ = sjson.SetBytes(toolCall, "index", state.chatIndex) + toolCall, _ = sjson.SetBytes(toolCall, "function.arguments", delta) + + chunk := template + chunk, _ = sjson.SetRawBytes(chunk, "choices.0.delta.tool_calls", []byte(`[]`)) + chunk, _ = sjson.SetRawBytes(chunk, "choices.0.delta.tool_calls.-1", toolCall) + state.emittedInput += delta + return chunk +} + +func emitAvailableToolCall(template []byte, state *toolCallStreamState, complete string, hasComplete bool) [][]byte { + if state == nil { + return nil + } + if hasComplete { + state.completeInput = complete + state.hasCompleteInput = true + } + if state.hasCompleteInput && state.itemDone { + return nil + } + + value := state.bufferedInput + valueIsComplete := false + if state.hasCompleteInput { + value = state.completeInput + valueIsComplete = true + } + + if !state.announced { + chunk := announceToolCall(template, state, value) + if chunk == nil { + return nil + } + return [][]byte{chunk} + } + + if valueIsComplete { + remaining, ok := remainingToolInput(state.emittedInput, value) + if !ok { + return nil + } + if chunk := emitToolInputDelta(template, state, remaining); chunk != nil { + return [][]byte{chunk} + } + return nil + } + if state.bufferedInput != "" { + if chunk := emitToolInputDelta(template, state, state.bufferedInput); chunk != nil { + state.bufferedInput = "" + return [][]byte{chunk} + } + } + return nil +} + +func emitCompletedToolCalls(template []byte, tracker *streamToolCallTracker, catalog toolCatalog, output gjson.Result) [][]byte { + if tracker == nil || !output.IsArray() { + return nil + } + + var chunks [][]byte + for index, item := range output.Array() { + family, ok := toolFamilyFromItem(item) + if !ok { + continue + } + state := tracker.stateForCompletedItem(item, int64(index), family) + state.name = catalog.restore(state.name) + input, hasInput := toolInputFromItem(item, family) + chunks = append(chunks, emitAvailableToolCall(template, state, input, hasInput)...) + state.itemDone = true + } + return chunks +} diff --git a/scripts/security/README.md b/scripts/security/README.md new file mode 100644 index 00000000000..c9266bfcb2e --- /dev/null +++ b/scripts/security/README.md @@ -0,0 +1,29 @@ +# CLIProxyAPI outbound network audit + +`audit_egress.py` inventories literal external destinations in production Go code and the active configuration, then samples the live CPA container's TCP connections. + +It intentionally does **not** read or print request payloads, headers, API keys, OAuth tokens, or other secret values. + +## Run + +```bash +cd /home/francis_chiu/code/CLIProxyAPI +python3 scripts/security/test_audit_egress.py +python3 scripts/security/audit_egress.py \ + --watch-seconds 30 \ + --resolve-dns \ + --json-output /home/francis_chiu/cliproxyapi/reports/cpa-egress-audit.json \ + --markdown-output /home/francis_chiu/cliproxyapi/reports/cpa-egress-audit.md +``` + +For a meaningful runtime observation, start the audit first and make a normal CPA model request while the watch window is active. + +## What the result means + +- **Static destinations** show literal URL hosts that production code or active, uncommented configuration can reference. +- **Runtime observations** show TCP peers during the sampling window. The tool filters by init-process socket ownership when `/proc` permissions permit it; otherwise the observation is network-namespace scoped. +- Direction is inferred from the container's listening ports and TCP state. Entries marked `outbound-candidate` are evidence consistent with an outbound connection, not absolute proof of which process initiated it or what bytes were transferred. +- `--resolve-dns` performs PTR lookups and therefore creates its own DNS traffic; omit it when passive observation matters more than names. +- No unexpected connection in a short window is useful evidence, but not a mathematical proof that a dormant path can never connect later. + +For stronger assurance, combine this report with an outbound firewall allowlist and longer packet/DNS logging at the Docker host or network gateway. diff --git a/scripts/security/audit_egress.py b/scripts/security/audit_egress.py new file mode 100644 index 00000000000..e0e6aed6abe --- /dev/null +++ b/scripts/security/audit_egress.py @@ -0,0 +1,514 @@ +#!/usr/bin/env python3 +"""Audit CLIProxyAPI outbound network destinations without reading secret values. + +The audit has two complementary modes: + +* static: scan production Go/config text for literal HTTP(S)/WS(S) destinations; +* runtime: sample the live container network namespace via /proc//net/tcp*. + +It deliberately reports destination metadata only. It never prints request bodies, +headers, API keys, OAuth tokens, or configuration secret values. +""" + +from __future__ import annotations + +import argparse +import ipaddress +import json +from pathlib import Path +import re +import socket +import struct +import subprocess +import time +from dataclasses import asdict, dataclass +from datetime import datetime, timezone +from typing import Iterable +from urllib.parse import urlsplit + +URL_RE = re.compile(r"(?:https?|wss?)://[^\s\"'`<>\\)]+") +SKIP_DIRS = {".git", "vendor", "examples", "test", "tests"} +SKIP_SUFFIXES = ("_test.go",) + +KNOWN_PURPOSES = { + "api.openai.com": "OpenAI API provider (prompts/files are intentionally forwarded when configured)", + "chatgpt.com": "OpenAI/Codex OAuth API and model catalogue", + "auth.openai.com": "OpenAI OAuth authentication", + "platform.openai.com": "OpenAI OAuth success redirect/documentation", + "api.anthropic.com": "Anthropic API provider (prompts are intentionally forwarded when configured)", + "claude.ai": "Claude OAuth/API provider", + "console.anthropic.com": "Anthropic OAuth success redirect/documentation", + "docs.anthropic.com": "Anthropic documentation link embedded in provider error text", + "generativelanguage.googleapis.com": "Google Gemini API provider (prompts are intentionally forwarded when configured)", + "aiplatform.googleapis.com": "Google Vertex AI provider (prompts are intentionally forwarded when configured)", + "oauth2.googleapis.com": "Google OAuth token exchange", + "accounts.google.com": "Google OAuth authentication", + "www.googleapis.com": "Google OAuth/user/project APIs", + "cloudcode-pa.googleapis.com": "Google Antigravity provider/model catalogue", + "daily-cloudcode-pa.googleapis.com": "Google Antigravity daily endpoint", + "daily-cloudcode-pa.sandbox.googleapis.com": "Google Antigravity sandbox endpoint", + "api.x.ai": "xAI API provider (prompts/media are intentionally forwarded when configured)", + "auth.x.ai": "xAI OAuth authentication", + "cli-chat-proxy.grok.com": "xAI CLI chat provider (prompts are intentionally forwarded when configured)", + "api.kimi.com": "Kimi API provider (prompts are intentionally forwarded when configured)", + "auth.kimi.com": "Kimi OAuth authentication", + "raw.githubusercontent.com": "Remote model registry, plugin registry, or management assets", + "github.com": "Source/plugin release metadata or downloads", + "api.github.com": "GitHub release metadata or plugin downloads", + "models.router-for.me": "CLIProxyAPI remote model registry fallback", + "cpamc.router-for.me": "Management panel fallback download", + "antigravity-hub-auto-updater-974169037036.us-central1.run.app": "Antigravity client-version manifest lookup", + "api.ipify.org": "Public-IP discovery used by SSH/OAuth helper", + "ifconfig.me": "Public-IP discovery used by SSH/OAuth helper", + "icanhazip.com": "Public-IP discovery used by SSH/OAuth helper", + "ipinfo.io": "Public-IP discovery used by SSH/OAuth helper", +} + +PROVIDER_SUFFIXES = ( + "openai.com", + "anthropic.com", + "claude.ai", + "googleapis.com", + "x.ai", + "grok.com", + "kimi.com", +) + +# URL-like strings in comments, generated HTML/SVG, schema identifiers, or +# format templates are not necessarily network destinations. +NON_NETWORK_HOSTS = {"host", "json-schema.org", "www.w3.org"} + + +@dataclass(frozen=True) +class StaticFinding: + host: str + scheme: str + endpoint: str + source: str + line: int + purpose: str + category: str + + +@dataclass(frozen=True) +class RuntimeFinding: + observed_at: str + protocol: str + state: str + direction: str + local_ip: str + local_port: int + remote_ip: str + remote_port: int + hostnames: tuple[str, ...] + scope: str + + +def classify_host(host: str) -> tuple[str, str]: + host = host.lower().rstrip(".") + purpose = KNOWN_PURPOSES.get(host, "Unclassified external destination; inspect the source location") + if host in {"localhost", "127.0.0.1", "::1"}: + return "local", "Local callback/service" + try: + ip = ipaddress.ip_address(host) + except ValueError: + ip = None + if ip and (ip.is_private or ip.is_loopback or ip.is_link_local): + return "private", "Private/local network destination" + if any(host == suffix or host.endswith("." + suffix) for suffix in PROVIDER_SUFFIXES): + return "provider", purpose + if host in KNOWN_PURPOSES: + return "support", purpose + return "unknown", purpose + + +def production_files(repo: Path) -> Iterable[Path]: + for path in repo.rglob("*.go"): + rel = path.relative_to(repo) + if any(part in SKIP_DIRS for part in rel.parts): + continue + if path.name.endswith(SKIP_SUFFIXES): + continue + yield path + + +def clean_url(raw: str) -> str: + return raw.rstrip(".,;:]}>") + + +def is_network_candidate(path: Path, line: str, url: str) -> bool: + parsed = urlsplit(url) + host = (parsed.hostname or "").lower() + if not host or host in NON_NETWORK_HOSTS or "%" in host: + return False + stripped = line.lstrip() + if stripped.startswith(("//", "#")): + return False + if path.suffix.lower() in {".html", ".svg"}: + return False + return True + + +def sanitized_endpoint(url: str) -> str: + parsed = urlsplit(url) + host = parsed.hostname or "" + if ":" in host and not host.startswith("["): + host = f"[{host}]" + try: + parsed_port = parsed.port + except ValueError: + parsed_port = None + port = f":{parsed_port}" if parsed_port else "" + return f"{parsed.scheme}://{host}{port}" + + +def strip_inline_comment(path: Path, line: str) -> str: + if path.suffix.lower() in {".yaml", ".yml"}: + in_single = False + in_double = False + escaped = False + for index, char in enumerate(line): + if escaped: + escaped = False + continue + if char == "\\" and in_double: + escaped = True + continue + if char == "'" and not in_double: + in_single = not in_single + elif char == '"' and not in_single: + in_double = not in_double + elif char == "#" and not in_single and not in_double: + return line[:index] + return line + + +def strip_go_comments(lines: list[str]) -> list[str]: + output: list[str] = [] + in_block = False + in_string = "" + escaped = False + for line in lines: + chars: list[str] = [] + index = 0 + while index < len(line): + char = line[index] + next_char = line[index + 1] if index + 1 < len(line) else "" + if in_block: + if char == "*" and next_char == "/": + in_block = False + index += 2 + else: + index += 1 + continue + if in_string: + chars.append(char) + if in_string == "`": + if char == "`": + in_string = "" + elif escaped: + escaped = False + elif char == "\\": + escaped = True + elif char == in_string: + in_string = "" + index += 1 + continue + if char == "/" and next_char == "/": + break + if char == "/" and next_char == "*": + in_block = True + index += 2 + continue + if char in {'"', "'", "`"}: + in_string = char + chars.append(char) + index += 1 + output.append("".join(chars)) + return output + + +def scan_static(repo: Path, config: Path | None) -> list[StaticFinding]: + files = list(production_files(repo)) + if config and config.exists(): + files.append(config) + findings: set[StaticFinding] = set() + for path in files: + try: + lines = path.read_text(encoding="utf-8", errors="replace").splitlines() + except OSError: + continue + if path.suffix.lower() == ".go": + lines = strip_go_comments(lines) + for number, line in enumerate(lines, 1): + scan_line = strip_inline_comment(path, line) + for match in URL_RE.finditer(scan_line): + url = clean_url(match.group(0)) + if not is_network_candidate(path, scan_line, url): + continue + parsed = urlsplit(url) + host = (parsed.hostname or "").lower() + if not host: + continue + category, purpose = classify_host(host) + findings.add( + StaticFinding( + host=host, + scheme=parsed.scheme, + endpoint=sanitized_endpoint(url), + source=str(path.relative_to(repo)) if path.is_relative_to(repo) else str(path), + line=number, + purpose=purpose, + category=category, + ) + ) + return sorted(findings, key=lambda item: (item.category, item.host, item.source, item.line, item.endpoint)) + + +def container_pid(container: str) -> int: + result = subprocess.run( + ["docker", "inspect", "--format", "{{.State.Pid}}", container], + check=True, + capture_output=True, + text=True, + ) + pid = int(result.stdout.strip()) + if pid <= 0: + raise RuntimeError(f"container {container!r} is not running") + return pid + + +def decode_ipv4(value: str) -> str: + return socket.inet_ntop(socket.AF_INET, struct.pack(" str: + packed = bytes.fromhex(value) + # Linux /proc renders each 32-bit word in host order. + packed = b"".join(packed[i : i + 4][::-1] for i in range(0, 16, 4)) + return socket.inet_ntop(socket.AF_INET6, packed) + + +def connection_scope(ip_text: str) -> str: + ip = ipaddress.ip_address(ip_text) + if ip.is_loopback: + return "loopback" + if ip.is_private: + return "private" + if ip.is_link_local: + return "link-local" + return "public" + + +def reverse_names(ip_text: str) -> tuple[str, ...]: + try: + primary, aliases, _ = socket.gethostbyaddr(ip_text) + except (socket.herror, socket.gaierror, TimeoutError, OSError): + return () + return tuple(sorted({primary.lower().rstrip("."), *(name.lower().rstrip(".") for name in aliases)})) + + +def owned_socket_inodes(pid: int) -> set[str] | None: + inodes: set[str] = set() + fd_dir = Path(f"/proc/{pid}/fd") + try: + entries = list(fd_dir.iterdir()) + except OSError: + return None + for entry in entries: + try: + target = entry.readlink().as_posix() + except OSError: + continue + if target.startswith("socket:[") and target.endswith("]"): + inodes.add(target[8:-1]) + return inodes + + +def read_proc_connections(pid: int, resolve_dns: bool) -> list[RuntimeFinding]: + states = { + "01": "ESTABLISHED", + "02": "SYN_SENT", + "03": "SYN_RECV", + "04": "FIN_WAIT1", + "05": "FIN_WAIT2", + "06": "TIME_WAIT", + "07": "CLOSE", + "08": "CLOSE_WAIT", + "09": "LAST_ACK", + "0A": "LISTEN", + "0B": "CLOSING", + } + observed = datetime.now(timezone.utc).isoformat() + findings: list[RuntimeFinding] = [] + owned_inodes = owned_socket_inodes(pid) + for name, decoder, proto in (("tcp", decode_ipv4, "tcp4"), ("tcp6", decode_ipv6, "tcp6")): + path = Path(f"/proc/{pid}/net/{name}") + try: + rows = [row.split() for row in path.read_text().splitlines()[1:]] + except OSError: + continue + listening_ports = { + int(fields[1].split(":")[1], 16) + for fields in rows + if fields[3] == "0A" + } + for fields in rows: + local_hex, remote_hex, state_hex = fields[1], fields[2], fields[3] + inode = fields[9] if len(fields) > 9 else "" + if state_hex == "0A" or (owned_inodes is not None and inode not in owned_inodes): + continue + local_address_hex, local_port_hex = local_hex.split(":") + remote_address_hex, remote_port_hex = remote_hex.split(":") + local_ip = decoder(local_address_hex) + local_port = int(local_port_hex, 16) + remote_ip = decoder(remote_address_hex) + remote_port = int(remote_port_hex, 16) + if remote_port == 0: + continue + direction = "inbound" if local_port in listening_ports or state_hex == "03" else "outbound-candidate" + findings.append( + RuntimeFinding( + observed_at=observed, + protocol=proto, + state=states.get(state_hex, state_hex), + direction=direction, + local_ip=local_ip, + local_port=local_port, + remote_ip=remote_ip, + remote_port=remote_port, + hostnames=reverse_names(remote_ip) if resolve_dns else (), + scope=connection_scope(remote_ip), + ) + ) + return findings + + +def sample_runtime(container: str, seconds: float, interval: float, resolve_dns: bool) -> list[RuntimeFinding]: + pid = container_pid(container) + deadline = time.monotonic() + max(seconds, 0) + unique: dict[tuple[str, str, int, int, str], RuntimeFinding] = {} + while True: + for finding in read_proc_connections(pid, resolve_dns): + key = (finding.direction, finding.remote_ip, finding.local_port, finding.remote_port, finding.state) + unique.setdefault(key, finding) + if time.monotonic() >= deadline: + break + time.sleep(max(interval, 0.1)) + return sorted( + unique.values(), + key=lambda item: (item.direction, item.scope, item.remote_ip, item.remote_port, item.state), + ) + + +def summarize(static: list[StaticFinding], runtime: list[RuntimeFinding]) -> dict: + unknown_hosts = sorted({item.host for item in static if item.category == "unknown"}) + outbound_candidates = [item for item in runtime if item.direction == "outbound-candidate"] + observed_public = sorted( + {f"{item.remote_ip}:{item.remote_port}" for item in outbound_candidates if item.scope == "public"} + ) + return { + "static_destination_count": len({item.host for item in static}), + "static_unknown_hosts": unknown_hosts, + "runtime_peer_count": len(runtime), + "runtime_public_outbound_candidates": observed_public, + "limitations": [ + "Static scanning cannot discover destinations assembled entirely at runtime or supplied only through encrypted/secret configuration.", + "Runtime /proc sampling sees TCP peer IP/port and state, not TLS hostnames, HTTP paths, headers, payloads, or transferred byte counts.", + "Direction is inferred from listening ports and TCP state; entries are outbound candidates, not cryptographic proof of process-initiated upload.", + "Socket rows are filtered to descriptors owned by the container init process when /proc permissions allow it; otherwise results are network-namespace scoped.", + "Reverse DNS is optional because it creates DNS queries and can delay sampling.", + "A quiet observation window does not prove that a dormant code path will never connect later.", + ], + } + + +def render_markdown(report: dict) -> str: + lines = [ + "# CLIProxyAPI outbound network audit", + "", + f"Generated: `{report['generated_at']}`", + f"Repository: `{report['repository']}`", + f"Container: `{report['container']}`", + "", + "## Summary", + "", + f"- Literal destination hosts found: **{report['summary']['static_destination_count']}**", + f"- Unclassified literal hosts: **{len(report['summary']['static_unknown_hosts'])}**", + f"- Runtime TCP peers observed: **{report['summary']['runtime_peer_count']}**", + f"- Public outbound candidates: **{len(report['summary']['runtime_public_outbound_candidates'])}**", + "", + "## Static destinations", + "", + "| Category | Host | Purpose | Source |", + "|---|---|---|---|", + ] + grouped: dict[tuple[str, str, str], list[str]] = {} + for item in report["static_findings"]: + key = (item["category"], item["host"], item["purpose"]) + grouped.setdefault(key, []).append(f"`{item['source']}:{item['line']}`") + for (category, host, purpose), sources in sorted(grouped.items()): + source_text = ", ".join(sorted(set(sources))[:6]) + lines.append(f"| {category} | `{host}` | {purpose} | {source_text} |") + lines.extend( + [ + "", + "## Runtime observations", + "", + "| Direction | Scope | Local | Peer | State | Reverse DNS |", + "|---|---|---|---|---|---|", + ] + ) + for item in report["runtime_findings"]: + names = ", ".join(f"`{name}`" for name in item["hostnames"]) or "—" + lines.append( + f"| {item['direction']} | {item['scope']} | `{item['local_ip']}:{item['local_port']}` | " + f"`{item['remote_ip']}:{item['remote_port']}` | {item['state']} | {names} |" + ) + if not report["runtime_findings"]: + lines.append("| — | — | — | No non-listening TCP peer observed | — | — |") + lines.extend(["", "## Limitations", ""]) + lines.extend(f"- {item}" for item in report["summary"]["limitations"]) + lines.append("") + return "\n".join(lines) + + +def parse_args() -> argparse.Namespace: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--repo", type=Path, default=Path(__file__).resolve().parents[2]) + parser.add_argument("--config", type=Path, default=Path("/home/francis_chiu/cliproxyapi/config.yaml")) + parser.add_argument("--container", default="cli-proxy-api") + parser.add_argument("--watch-seconds", type=float, default=10.0) + parser.add_argument("--interval", type=float, default=0.25) + parser.add_argument("--resolve-dns", action="store_true") + parser.add_argument("--json-output", type=Path) + parser.add_argument("--markdown-output", type=Path) + return parser.parse_args() + + +def main() -> int: + args = parse_args() + static = scan_static(args.repo.resolve(), args.config.resolve() if args.config else None) + runtime = sample_runtime(args.container, args.watch_seconds, args.interval, args.resolve_dns) + report = { + "generated_at": datetime.now(timezone.utc).isoformat(), + "repository": str(args.repo.resolve()), + "config": str(args.config.resolve()) if args.config else None, + "container": args.container, + "static_findings": [asdict(item) for item in static], + "runtime_findings": [asdict(item) for item in runtime], + "summary": summarize(static, runtime), + } + encoded = json.dumps(report, indent=2, ensure_ascii=False) + "\n" + if args.json_output: + args.json_output.parent.mkdir(parents=True, exist_ok=True) + args.json_output.write_text(encoded, encoding="utf-8") + if args.markdown_output: + args.markdown_output.parent.mkdir(parents=True, exist_ok=True) + args.markdown_output.write_text(render_markdown(report), encoding="utf-8") + print(encoded, end="") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/security/test_audit_egress.py b/scripts/security/test_audit_egress.py new file mode 100644 index 00000000000..00eb44329ca --- /dev/null +++ b/scripts/security/test_audit_egress.py @@ -0,0 +1,105 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import importlib.util +from pathlib import Path +import sys +import tempfile +import unittest + +MODULE_PATH = Path(__file__).with_name("audit_egress.py") +SPEC = importlib.util.spec_from_file_location("audit_egress", MODULE_PATH) +assert SPEC and SPEC.loader +AUDIT = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = AUDIT +SPEC.loader.exec_module(AUDIT) + + +class AuditEgressTests(unittest.TestCase): + def test_decode_ipv4_proc_format(self) -> None: + self.assertEqual(AUDIT.decode_ipv4("0100007F"), "127.0.0.1") + + def test_decode_ipv6_proc_format(self) -> None: + self.assertEqual(AUDIT.decode_ipv6("00000000000000000000000001000000"), "::1") + + def test_classify_known_and_unknown_hosts(self) -> None: + self.assertEqual(AUDIT.classify_host("api.openai.com")[0], "provider") + self.assertEqual(AUDIT.classify_host("models.router-for.me")[0], "support") + self.assertEqual(AUDIT.classify_host("telemetry.suspicious.invalid")[0], "unknown") + self.assertEqual(AUDIT.classify_host("notopenai.com")[0], "unknown") + self.assertEqual(AUDIT.classify_host("evilgoogleapis.com")[0], "unknown") + + def test_static_scan_skips_tests_and_examples(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + (root / "internal").mkdir() + (root / "examples").mkdir() + (root / "internal" / "live.go").write_text( + 'package internal\nconst endpoint = "https://telemetry.vendor.test/collect"\n', + encoding="utf-8", + ) + (root / "internal" / "live_test.go").write_text( + 'package internal\nconst endpoint = "https://ignored.test/unit"\n', + encoding="utf-8", + ) + (root / "examples" / "demo.go").write_text( + 'package main\nconst endpoint = "https://ignored.test/demo"\n', + encoding="utf-8", + ) + findings = AUDIT.scan_static(root, None) + self.assertEqual([finding.host for finding in findings], ["telemetry.vendor.test"]) + + def test_static_scan_includes_config_without_exposing_secrets(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + root.joinpath("main.go").write_text("package main\n", encoding="utf-8") + config = root / "config.yaml" + config.write_text( + "api-key: super-secret-value\n" + "base-url: https://user:password@custom.provider.test:8443/v1?token=SECRET#private\n", + encoding="utf-8", + ) + findings = AUDIT.scan_static(root, config) + self.assertEqual(len(findings), 1) + self.assertEqual(findings[0].host, "custom.provider.test") + self.assertEqual(findings[0].endpoint, "https://custom.provider.test:8443") + rendered = repr(findings) + for secret in ("super-secret-value", "user", "password", "SECRET", "/v1"): + self.assertNotIn(secret, rendered) + + def test_sanitized_endpoint_ignores_format_port(self) -> None: + self.assertEqual(AUDIT.sanitized_endpoint("http://localhost:%d/callback"), "http://localhost") + + def test_example_like_active_destination_is_not_hidden(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + root.joinpath("main.go").write_text("package main\n", encoding="utf-8") + config = root / "config.yaml" + config.write_text("base-url: https://notexample.com/v1\n", encoding="utf-8") + findings = AUDIT.scan_static(root, config) + self.assertEqual([finding.host for finding in findings], ["notexample.com"]) + + def test_static_scan_skips_comments_and_schema_identifiers(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + root.joinpath("main.go").write_text( + 'package main\n' + '// Docs: https://docs.vendor.test/help\n' + 'const enabled = true // https://inline-comment.vendor.test/help\n' + '/* https://block-comment.vendor.test/help */\n' + 'const schema = "http://json-schema.org/draft-07/schema#"\n' + 'const live = "https://collector.vendor.test/v1"\n', + encoding="utf-8", + ) + config = root / "config.yaml" + config.write_text( + '# base-url: https://ignored.vendor.test/v1\n' + 'enabled: true # docs https://also-ignored.vendor.test/help\n', + encoding="utf-8", + ) + findings = AUDIT.scan_static(root, config) + self.assertEqual([finding.host for finding in findings], ["collector.vendor.test"]) + + +if __name__ == "__main__": + unittest.main()