fix(chat): normalize LangChain json_schema response_format - #156
cestercian wants to merge 1 commit into
Conversation
![orcacode-review[bot]](https://avatars.githubusercontent.com/in/4254738?s=60&v=4){kind=small}
There was a problem hiding this comment.
🐳 OrcaCode Review
✅ No findings — nothing to flag in this PR. Great work!
OrcaCode Review — Route Smarter. Ship Safer. Spend Less.
Engine-reported: 167 calls · 6M tokens · 92% cached
❤️ Share · Install OrcaCode Review
Free on GitHub — the review runs on your own OrcaRouter key. If it helped, a shout-out goes a long way.
Share: X · Reddit · LinkedIn
Follow: X · Discord · LinkedIn · OrcaRouter
|
Verified end-to-end — the normalization is correct. It needs a rebase: #147 landed on What I verified (after resolving locally on top of
|
| client sends | main forwards |
this PR forwards |
|---|---|---|
LangChain shape (strict at top level, parameters) |
top-level strict=True, no inner strict, key=parameters, no additionalProperties |
inner strict=True, key=schema, additionalProperties=False |
strict at top level + schema |
top-level strict=True, no additionalProperties |
inner strict=True, additionalProperties=False |
| already-canonical | unchanged | unchanged |
{"type":"json_object"} |
unchanged | unchanged |
All three fixes land (strict moved inward, parameters → schema, additionalProperties filled in) and nothing over-normalizes.
The conflict
Only two files, both tests:
tests/integration/test_chat_completion.py
tests/unit/test_litellm_client.py
app/response_format.py, app/schemas.py and packages/litellm_adapter/client.py all auto-merge cleanly — don't touch those by hand. I made that mistake first: checking schemas.py / client.py out wholesale from this branch clobbered parallel_tool_calls (#147), logit_bias (#149) and the hosted-fallback work (#153) that are now on main, and 7 tests went red. Let git do those.
The two test files conflict because #147 added a test function at the same position as yours, and they share enough boilerplate (client, fake = chat_client, the await client.post(..., json={...}) scaffold) that git interleaves them into one broken function instead of keeping two.
Resolution: keep main's version of both files and append your four new functions whole —
test_chat_completion_forwards_langchain_json_schematest_chat_completion_normalizes_misplaced_json_schema_stricttest_acompletion_forwards_json_schema_response_formattest_openai_deployments_advertise_response_schema
I applied exactly that and re-ran:
ruff check .— cleanpytest— 714 passed, twice- the table above still holds
parallel_tool_calls=Falseandlogit_bias={'1234': 5.0}still reach the upstream, so fix(chat): forward parallel_tool_calls to upstream providers #147 / fix(chat): forward logit_bias so token bias reaches LiteLLM #149 are intact
Push the rebase and I'll re-verify and merge.
d09b238 to
9770f53
Compare
…AI-Corp#132) Normalize response_format so json_schema (including misplaced strict and the parameters alias) is a valid OpenAI structured-outputs block before LiteLLM sees it. Mark OpenAI-compatible deployments as supporting response_schema so LiteLLM does not re-wrap the payload.
9770f53 to
0ae8f9b
Compare
There was a problem hiding this comment.
⚠️ Outdated review — the PR head changed from 9770f53 to 0ae8f9b during the review; these findings describe a diff the PR no longer has.
These findings describe9770f531d4d2. Re-run the review to check the PR as it stands now.
🐳 OrcaCode Review
✅ No findings — nothing to flag in this PR. Great work!
OrcaCode Review — Route Smarter. Ship Safer. Spend Less.
Engine-reported: 189 calls · 7.2M tokens · 93% cached
❤️ Share · Install OrcaCode Review
Free on GitHub — the review runs on your own OrcaRouter key. If it helped, a shout-out goes a long way.
Share: X · Reddit · LinkedIn
Follow: X · Discord · LinkedIn · OrcaRouter
There was a problem hiding this comment.
🐳 OrcaCode Review
✅ No findings — nothing to flag in this PR. Great work!
OrcaCode Review — Route Smarter. Ship Safer. Spend Less.
Engine-reported: 184 calls · 6.7M tokens · 93% cached
❤️ Share · Install OrcaCode Review
Free on GitHub — the review runs on your own OrcaRouter key. If it helped, a shout-out goes a long way.
Share: X · Reddit · LinkedIn
Follow: X · Discord · LinkedIn · OrcaRouter
This branch has not been deployed
2 participants
Orca-Code-Review — push 2
✅ no blocking findings
Summary
Normalize OpenAI-compatible
response_formatjson_schema shapes (misplacedstrict,parametersvsschema,additionalPropertiesunderstrict) before LiteLLM, and advertisesupports_response_schemaon OpenAI-compatible deployments so LangChainwith_structured_output(method=\"json_schema\")works through the router.Fixes #132
Test plan