feat(evals): pin judge calls to the low inference class

[theomonnom]

Stacked on #6721 (needs INFERENCE_CLASS_LOW from it). Base is theo/sim-text-low-inference-class, so review the last commit only; retarget to main once #6721 merges.

Follow-up to #6721, same rationale applied to judges. Judging is batch load: an eval suite fans out many judgments at once, they run after the conversation they grade, and nobody is waiting on the verdict. They should not compete with live traffic for gateway capacity.

How

_judge_chat_kwargs() returns the chat() kwargs for a judgment call:

def _judge_chat_kwargs(llm: LLM) -> dict[str, Any]:
    if isinstance(llm, InferenceLLM):
        return {"inference_class": INFERENCE_CLASS_LOW}
    return {}

Applied at both judge call sites:

• evals/judge.py::_evaluate_with_llm, which every built-in judge goes through: accuracy, task_completion, handoff, tool_use, safety, relevancy, coherence, conciseness, whether reached via JudgeGroup or by calling a judge directly
• voice/run_result.py::ChatMessageAssert.judge, the test-assertion judge

It overrides the class the judge LLM was configured with, so an eval cannot claim priority capacity, matching how a text simulation behaves in #6721. The isinstance guard keeps it empty for a plugin LLM, which has no LiveKit Inference class to set; the existing test_judge.py suite runs its judges on a plain llm.LLM subclass, so that branch is already covered.

What is deliberately not changed

The agent under test. It is the thing being measured, and its latency should stay representative of production, so an eval only demotes the grading calls, not the conversation being graded.

low stays out of the public InferenceClass literal. The kwargs helper returns dict[str, Any], so nothing here needs the literal widened while the gateway still maps anything that is not priority to standard.

Note on the gateway side

As in #6721: the gateway recognizes only standard and priority on X-LiveKit-Inference-Priority today, so low is currently equivalent to standard. This is a no-op on billing and scheduling until a low tier lands gateway-side; the immediate effect is that evals can no longer request priority.

Testing

• make type-check clean (629 source files), make lint and format check clean
• uv run pytest --unit: 1415 passed, identical to the baseline
• verified end to end against a local HTTP server recording what actually arrived on the wire, with no job context in scope (so this is independent of the simulation path):

judge path	judge LLM configured	sent
JudgeGroup + accuracy_judge	nothing	low
JudgeGroup + task_completion_judge	priority	low
accuracy_judge(llm=...) called directly	priority	low
ChatMessageAssert.judge	priority	low

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.