Maintainer-level notes on how the Codex provider is bundled, and how to update it.
The Codex provider relies on OpenAI's Codex Python SDK (openai_codex) plus the Codex CLI binary it drives over JSON-RPC.
- The SDK is vendored from the
openai/codexrepository at tagrust-v0.169.0. A PyPI release (openai-codex) exists but currently pins an older runtime version, so we stay on the vendored source to ride a known-good combination with the matching upstream tag. - The CLI binary is downloaded at first launch from the matching GitHub Release (
openai-codex-cli-binwheel), extracted into a shared cache, and pointed at viaCodexConfig(codex_bin=…). It is no longer a PyPI dependency: OpenAI stopped publishing stableopenai-codex-cli-binwheels on PyPI after0.136.0(the wheel is ~122 MB, above practical PyPI quotas), but every tagged stable still ships the same manylinux / macOS wheels as GitHub Release assets. Provisioning lives insrc/twicc/providers/codex/runtime.py; the download is triggered unconditionally in the background at startup (OrchestratorRegistry.start_all) and on demand bymake_codex_config.
The cache is shared by every checkout on the machine, so bumping CODEX_VERSION in a worktree would prune the version the main instance is running on. TWICC_NO_CODEX_RUNTIME_CLEANUP=1 (set by devctl in worktree mode) downloads without ever pruning, and provisioning re-checks the store on every call so a runtime deleted under a live process is downloaded again.
| Path | Origin |
|---|---|
src/openai_codex/ |
Vendored SDK source. Treat as read-only — edits land upstream, then we re-sync — except the local patches listed below. |
src/twicc/providers/codex/runtime.py |
Downloads + caches the Codex CLI binary wheel from the GitHub Release into ~/.cache/twicc/codex-runtime/<version>/ (honours $XDG_CACHE_HOME). Holds the pinned CODEX_VERSION / CODEX_RELEASE_TAG and the 4 wheel sha256. |
src/twicc/providers/codex/sdk_wrappers.py |
TwiCC subclasses (TwiccAsyncCodex, TwiccAsyncThread) that expose *_with_policy methods so we can keep our 5 fine-grained presets — the upstream SDK now only exposes the coarse ApprovalMode. |
src/twicc/providers/codex/bin.py |
Single entry point: resolve_bundled_binary() (sync, non-downloading) + make_codex_config() (async, ensures the runtime and builds a CodexConfig that also puts rg on PATH). |
The extracted tree is the whole codex_cli_bin/ package (not just the codex binary), so its sibling resources ship too: codex-resources/bwrap (Linux sandbox helper) and codex-resources/zsh are found by codex relative to its own path, and codex-path/rg (ripgrep) is put on PATH by make_codex_config (the SDK only does that itself when codex_bin is auto-resolved, which is never our case).
Interactive agents pass disable_async_questions=True to make_codex_config.
interactive_catalog.py extracts the pinned binary's catalogue with
codex debug models --bundled. It removes only request_user_input_async and
its legacy name send_user_message_async from each model's
experimental_supported_tools. Synchronous question widgets retain their
existing question_widget configuration.
The generated file lives under <data_dir>/cache/interactive-codex-catalog-*.json.
Its name includes the CLI version, filtered content hash, and transformation
version. Writes are atomic; missing or damaged cache files regenerate on launch.
Unknown model fields are preserved. An incompatible tools schema fails startup
instead of silently restoring async questions.
Each catalogue check also removes unused older interactive catalogue files and
their .lease sidecars. A backend holds a shared OS file lock for every catalogue
it uses until it exits. Cleanup takes a nonblocking exclusive lock and skips
catalogues retained by any running backend, including idle agents. A directory
lock serializes lease acquisition, atomic writes, and cleanup across processes.
Interactive cleanup leaves hermetic catalogues and other cache files untouched. After an older
backend exits, its files become eligible for cleanup on the next agent launch.
Hermetic calls use the same lease and cleanup implementation in catalog_cache.py,
with a separate directory lock and the hermetic-codex-catalog-* filename family.
Each call checks the cache and removes unused hermetic files and orphan .lease
sidecars, including when it reuses a valid cached catalogue. Every model and
variant used by a running backend remains protected until that backend exits.
The next hermetic call can then remove those files. Interactive files are untouched.
The process-level model_catalog_json override applies to new and resumed
interactive agents and their native subagents. It does not modify provider
configuration, provider caches, standalone CLI invocations, or hermetic calls.
This catalogue is static: remote model additions and instruction changes take
effect only after a binary update. A new binary version regenerates the file
automatically. Restart existing agent processes to load a new catalogue.
This is intentional: the catalogue follows the pinned runtime. Do not refresh
it from remote APIs independently of a runtime update.
On each Codex update, verify that the catalogue names still control async tool
registration and synchronous request_user_input remains independent. Run
tests/test_codex_interactive_catalog.py and the thread configuration tests.
Run TWICC_CODEX_INTEGRATION=1 uv run pytest tests/test_codex_interactive_catalog_integration.py
with the runtime already downloaded. The local Responses endpoint verifies
new threads, app-server restarts with durable thread resume, native subagents,
and every bundled model advertising async questions, without any remote model call.
Deliberate divergences from the pristine upstream tree. Re-apply them after every re-vendor (a diff -rq <pristine> src/openai_codex against the current version's pristine tree finds them all), and drop each one once upstream ships the fix.
None right now. The SubAgentActivityKind patch carried from 2026-08-27 was dropped at the rust-v0.155.1 re-vendor: upstream now declares completed itself. The vendored tree is identical to the pristine one, so the diff -rq above must come back empty.
Assumes the new version is published as a rust-vX.Y.Z GitHub tag/Release with the openai-codex-cli-bin wheels attached (they still are, even though PyPI stopped receiving stable ones).
- Pick the release tag matching the upstream version you want, e.g.
rust-v0.169.0. Verify thatsdk/python/src/openai_codex/exists at that tag and the GitHub Release carries the 4 platform wheels. - Re-vendor the SDK source (extract the
openai_codexpackage from the tarball):Then re-apply the local patches still needed at the new tag.rm -rf src/openai_codex mkdir -p /tmp/codex-sdk curl -sL "https://codeload.github.com/openai/codex/tar.gz/refs/tags/<new_tag>" -o /tmp/codex.tar.gz tar -xzf /tmp/codex.tar.gz -C /tmp/codex-sdk "codex-<new_tag>/sdk/python/src/openai_codex" mv /tmp/codex-sdk/codex-<new_tag>/sdk/python/src/openai_codex src/openai_codex
- Bump
CODEX_VERSIONandCODEX_RELEASE_TAGinsrc/twicc/providers/codex/runtime.py, and recompute the 4 wheel sha256:Paste the digests intoTAG=rust-vX.Y.Z; V=X.Y.Z for w in manylinux_2_17_x86_64 manylinux_2_17_aarch64 macosx_11_0_arm64 macosx_10_9_x86_64; do f=openai_codex_cli_bin-$V-py3-none-$w.whl echo "$f $(curl -sL https://github.com/openai/codex/releases/download/$TAG/$f | sha256sum | cut -d' ' -f1)" done
_WHEELS. - Diff the new SDK's
pyproject.tomlagainst ours — copy any new runtime dependency over (today onlypydantic>=2.12is shared; the SDK's ownopenai-codex-cli-binpin is deliberately ignored — that's the whole point of the runtime download). - Run the checklist from the
reference_codex_sdk_update_procedure.mdmemory: verify the monkey-patch path_client._sync._approval_handler, thatThreadStartParams/TurnStartParamsstill acceptapproval_policy/approvals_reviewer/sandbox(_policy), that the SDK subclassing insdk_wrappers.pystill compiles, that the_inputs._normalize_run_input/_to_wire_inputhelpers still exist, etc. Note the lazyfrom codex_cli_bin import bundled_codex_pathinclient.pyis fine to leave — it's only reached whencodex_binisNone, which we never do, and it degrades to aFileNotFoundErrorif the package is absent. - Re-verify the TwiCC MCP server's per-thread config keys (
src/twicc/providers/codex/agent/manager.py,_twicc_mcp_server_config):url,http_headers,default_tools_approval_mode,tool_timeout_sec,startup_timeout_secagainstcodex-rs/config/src/mcp_types.rs; that the streamable-HTTP MCP client stays un-gated (codex-rs/codex-mcp/src/connection_manager.rs); and thetool_search_always_defer_mcp_toolsfeature key (codex-rs/features/src/lib.rs) —Stage::Removedwithdefault_enabled: truesince at least 0.144.6, so Codex silently ignores the key and always defers._apply_codex_mcp_context_modeis therefore a harmless no-op: flipTWICC_MCP_CODEX_DEFER=Falseinsrc/twicc/mcp/__init__.py(eager fallback) only if deferral itself regresses, not because the flag disappeared. - Run
uv lock, then./scripts/build-release.shand check the resulting singlepy3-none-anywheel installs and runs locally (first launch downloads the runtime). - Run the hermetic diagnostic:
uv run python scripts/diagnose_hermetic_llm.py --provider all --live --yes. Do not ship on aFAILorINCONCLUSIVE.
The PyPI package openai-codex exists but currently pins an older runtime than the wheel we want to ship, and TwiCC reaches into private SDK attributes:
_client._sync._approval_handlerfor the approval bridge (CodexAgent.__init__)_client._sync._procfor the subprocess PID (CodexAgent.get_pid)_inputs._normalize_run_input/_inputs._to_wire_inputfor the*_with_policywrappers
Vendoring keeps the SDK pinned to the same commit as the binary we depend on, and makes any upstream refactor of those private attributes visible in the diff (during the next vendor refresh) rather than as a runtime AttributeError.
If/when the PyPI SDK catches up and TwiCC migrates off those private attributes (e.g. when OpenAI exposes a public approval-handler hook), we can drop the vendoring entirely and depend on openai-codex from PyPI like any other library.