feat(simulation): pin text simulations to the low inference class

[theomonnom]

Text simulations run as batch load: a run fans out many jobs at once and nobody is waiting on the answers. Those requests currently reach the inference gateway at the same class as live user traffic. This pins every LiveKit Inference request made from a text simulation to inference class low.

How

JobContext.inference_headers is the extra headers a job puts on every LiveKit Inference request it makes. A text simulation contributes the low class; an ordinary job contributes nothing.

@property
def inference_headers(self) -> dict[str, str]:
    headers: dict[str, str] = {}
    sim = self.simulation_context()
    if sim is not None and sim.simulation_mode == SimulationMode.SIMULATION_MODE_TEXT:
        headers[HEADER_INFERENCE_PRIORITY] = INFERENCE_CLASS_LOW
    return headers

get_inference_headers() merges it last, so what the job asserts about itself outranks what an individual model was configured with:

mode	configured	sent
text simulation	anything, incl. priority	low
audio simulation	priority	priority
production	priority	priority
production	nothing	omitted

The inference layer never names a simulation. It already reads the job context for the room/job/agent headers, so this adds no new dependency, just one more thing to ask the job for. There is no job-runner hook and no process state: the property is read at request time off the job context that is already in scope.

Because it lands in the shared header builder, every LiveKit Inference model is covered: LLM, STT/TTS, avatar, interruption detection, and the end-of-turn transports. That includes models an AgentSession never sees, which is why it is resolved per request rather than set on the session's models at start: per-agent LLMs from a handoff, an AgentTask's own LLM, and any llm.chat() a tool calls directly would each need their own injection otherwise.

Audio simulations are deliberately excluded. They run in real time against the audio pipeline, so their latency has to stay representative of production. The gate is SIMULATION_MODE_TEXT only.

current_simulation() is added next to get_job_context in job.py: the job-context-to-simulation lookup AgentSession._text_only was doing inline.

Note on the gateway side

The gateway today recognizes only standard and priority on X-LiveKit-Inference-Priority and maps anything else to standard (ComputeInferenceTiersWithServed). So low is currently equivalent to standard, i.e. this is a no-op on billing and scheduling until a low tier lands gateway-side. It is deliberately sent now so the agents side is already correct when that happens, and so simulations stop being able to request priority. low is intentionally kept out of the public InferenceClass literal until the gateway honors it.

Testing

• make type-check clean (629 source files), make lint and format check clean
• uv run pytest --unit: 1415 passed (the 9 teardown errors are pre-existing OTel exporter noise, present on main without this change)
• verified end to end against a real inference.LLM: a real SimulationDispatch proto on a real agent.Job into a real JobContext, through get_inference_headers() and the openai client, to a local HTTP server recording what actually arrived on the wire. All rows of the table above match, plus SIMULATION_MODE_UNSPECIFIED (treated as text, so also demoted) and the no-argument call site STT/TTS/avatar/eot use. The room and job identification headers still flow unchanged.

tests/test_inference_utils.py is removed. It asserted dict keys against hand-rolled JobContext duck types, so every time the header builder asks the job for one more thing the fakes have to grow another stub. The guards it covered are untouched in the source: the isconnected() check before touching local_participant (from #5947) and the isinstance sid checks. Restore with git checkout origin/main -- tests/test_inference_utils.py if you'd rather keep it.