H-6763: Migrate voice interviews to GPT-Realtime-2

apps/petrinaut-website/README.md

@@ -39,16 +39,16 @@ provides a fake optimizer for isolated UI development.
 
 ## Environment variables
 
-| Name                             | Required         | Used by          | Notes                                                     |
-| -------------------------------- | ---------------- | ---------------- | --------------------------------------------------------- |
-| `OPENAI_API_KEY`                 | for chat to work | `api/chat.ts`    | OpenAI key the function uses to call `streamText`.        |
-| `OPENAI_VOICE_API_KEY`           | for voice        | voice API        | Dedicated OpenAI key used by Realtime and Speech proxies. |
-| `PETRINAUT_OPENAI_VOICE_ENABLED` | no               | voice API        | Set to `true` to enable voice outside production.         |
-| `PETRINAUT_AI_MODEL`             | no               | `api/chat.ts`    | Overrides the default OpenAI model id.                    |
-| `PETRINAUT_OPT_ORIGIN`           | no               | `vite.config.ts` | Overrides the local optimizer proxy target.               |
-| `VITE_BRUNCH_CHAT_ENDPOINT`      | for voice input  | website          | Full Brunch Petrinaut chat endpoint used by the panel.    |
-| `VITE_PETRINAUT_OPT_PROVIDER`    | no               | website          | Set to `service` to enable the optimization route.        |
-| `SENTRY_DSN`                     | no               | `vite.config.ts` | Wired into the bundle via `__SENTRY_DSN__` at build time. |
+| Name                             | Required         | Used by          | Notes                                                      |
+| -------------------------------- | ---------------- | ---------------- | ---------------------------------------------------------- |
+| `OPENAI_API_KEY`                 | for chat to work | `api/chat.ts`    | OpenAI key the function uses to call `streamText`.         |
+| `OPENAI_VOICE_API_KEY`           | for voice        | voice API        | Dedicated OpenAI key used to create Realtime WebRTC calls. |
+| `PETRINAUT_OPENAI_VOICE_ENABLED` | no               | voice API        | Set to `true` to enable voice outside production.          |
+| `PETRINAUT_AI_MODEL`             | no               | `api/chat.ts`    | Overrides the default OpenAI model id.                     |
+| `PETRINAUT_OPT_ORIGIN`           | no               | `vite.config.ts` | Overrides the local optimizer proxy target.                |
+| `VITE_BRUNCH_CHAT_ENDPOINT`      | for voice input  | website          | Full Brunch Petrinaut chat endpoint used by the panel.     |
+| `VITE_PETRINAUT_OPT_PROVIDER`    | no               | website          | Set to `service` to enable the optimization route.         |
+| `SENTRY_DSN`                     | no               | `vite.config.ts` | Wired into the bundle via `__SENTRY_DSN__` at build time.  |
 
 Local values live in `.env.local`; Vite's `loadEnv` (see [`vite.config.ts`](vite.config.ts)) copies them into `process.env` for both the dev server and the API functions. In production, set these in the Vercel project settings.
 
@@ -58,23 +58,33 @@ Voice input is disabled by default and always unavailable when `VERCEL_ENV` is
 `production`. To exercise the preview locally or in a Vercel preview, set a
 real `VITE_BRUNCH_CHAT_ENDPOINT`, `PETRINAUT_OPENAI_VOICE_ENABLED=true`, and a
 dedicated `OPENAI_VOICE_API_KEY`. The browser sends its SDP offer to this app;
-the server initializes an OpenAI transcription-only Realtime session and keeps
-the provider key, model, language, and vocabulary policy private. The session
-uses `gpt-live-transcribe`'s default server VAD because OpenAI's unified call
-currently times out when explicit turn detection is included during setup.
-
-Only finalized transcripts enter the existing Petrinaut composer and Brunch AI
-SDK transport. Partial transcripts remain display-only. The preview derives a
-stable conversation id from the locally saved net; it is diagnostic identity,
-not production authentication or conversation authority.
-
-While voice is active, finalized assistant text and validated structured Brunch
-questions are spoken with OpenAI's dedicated Speech API. The server fixes the
-model and voice and forwards the selected canonical text without rewriting it;
-Realtime remains transcription-only. The microphone stays closed while Brunch
-is working and while AI-generated speech is being synthesized or played. The UI
-discloses that the voice is AI-generated. Ending voice cancels playback, and a
-speech failure leaves the exact response visible for reading.
+the server initializes a trusted `gpt-realtime-2` audio-input/audio-output
+session through OpenAI's unified Realtime call endpoint. The provider key,
+model, instructions, tools, language and vocabulary policy stay server-side.
+The session uses semantic VAD with low eagerness so natural thinking pauses are
+less likely to end an answer early.
+
+Realtime is the disposable media plane: it carries continuous microphone and
+remote audio, detects complete turns and handles barge-in. Brunch remains the
+control plane and sole authority for interview questions, captures, state,
+completion and durable history. The browser bridge accepts only the configured
+`continue_interview` function, validates and serializes its arguments, rejects
+duplicate or stale calls and submits the answer through Petrinaut's existing
+composer path with pending-`brunch_ask` correlation.
+
+The bridge waits for the correlated Brunch turn before returning canonical
+speech segments to Realtime. It then requests audio with tools disabled and
+instructs Realtime to speak only those segments. Generated audio is not treated
+as a verbatim record: canonical Brunch text remains visible and authoritative.
+Partial input transcripts are display-only and never submit or disable capture.
+
+The microphone remains active while the interviewer speaks and while Brunch is
+working. Speaking over assistant audio interrupts playback automatically;
+WebRTC truncates provider-side unheard audio without changing Brunch history.
+**Pause** temporarily turns off input, while **End interview**, failure and
+reconnect release or replace all media and invalidate events from old
+connections. There is no separate Speech API, MP3 playback queue, required
+**Done speaking** action or manual **Interrupt and speak** action.
 
 The Brunch deployment must allow the website origin through its
 `BRUNCH_PETRINAUT_ORIGINS` setting. Starting voice input requests browser
@@ -86,19 +96,18 @@ permission, microphone device, interrupted request, network, timeout, invalid
 response, and unavailable/disabled failures. Permission and device failures
 identify what to fix; network, timeout, and interrupted requests offer a
 reconnect; invalid responses include a diagnostic reference for an operator;
-and unavailable voice leaves the text composer as the fallback. Speech failures
+and unavailable voice leaves the text composer as the fallback. Voice failures
 always leave the canonical response visible to read.
 
-Realtime connection and Speech requests carry a random `x-request-id` through
-the browser and server route, and the existing Brunch transport sends the same
-header on each chat request so Brunch's privacy-safe request inspection can
-correlate that boundary. Browser and server diagnostics report operation,
-stage, outcome, duration, request ID, and—where applicable—status or sanitized
-error code. Voice responses also expose privacy-safe `Server-Timing` metrics.
-These diagnostics never record audio, SDP, transcript or prompt contents,
-canonical speech text, credentials, or provider response bodies. This
-controlled-preview evidence does not enable production: production remains
-unconditionally disabled by the server policy.
+Realtime connection, transcription and canonical speech timings use random
+request IDs, and the existing Brunch transport provides its own request
+correlation. Browser and server diagnostics report only operation, stage,
+outcome, duration, request ID and—where applicable—status or a sanitized error
+code. Voice responses also expose privacy-safe `Server-Timing` metrics. These
+diagnostics never record audio, SDP, transcript or prompt contents, canonical
+speech text, credentials or provider response bodies. This controlled-preview
+evidence does not enable production: production remains unconditionally
+disabled by the server policy.
 
 ## Testing the API against the built output