feat: add LiteLLM as a first-class chat model provider

[prodmanpd]

OpenJudge Version

0.2.0

Description

Background and purpose

OpenJudge ships first class chat providers as BaseChatModel subclasses (OpenAIChatModel, MiniMaxChatModel, QiniuChatModel, QwenVLModel), each selected by instantiating the class directly. This PR adds LiteLLMChatModel, a first class provider backed by the LiteLLM SDK, so a single provider class reaches 100+ LLM backends (OpenAI, Anthropic, Gemini, Bedrock, Vertex, Groq, Mistral, DeepSeek, etc.).

Calling the SDK directly lets LiteLLM use each provider's native authentication (Bedrock SigV4, Vertex ADC, Azure AD), not just an OpenAI style bearer token.

Changes

• openjudge/models/litellm_chat_model.py: new LiteLLMChatModel(OpenAIChatModel). LiteLLM returns OpenAI shaped responses, so it reuses OpenAIChatModel's response handling and only swaps the transport to litellm.acompletion. Notable behavior:
  • drop_params=True by default, so params a provider rejects (Anthropic seed, Gemini response_format) are dropped, not errored.
  • Credentials omitted when unset, so LiteLLM uses each provider's own env var; set api_key / base_url for a proxy.
  • Same achat(...) signature and tool_choice validation as the OpenAI provider; structured_model falls back to json_object for qwen, gemini, and pai-judge.
  • No persistent client (LiteLLM routes per call).
• openjudge/models/__init__.py: export LiteLLMChatModel.
• pyproject.toml: optional litellm extra (litellm>=1.89.0,<2.0.0), lazy imported inside achat.
• tests/models/test_litellm_chat_model.py: 11 unit tests.

Checklist

Please check the following items before code is ready to be reviewed.

• Code has been formatted with pre-commit run --all-files command
• All tests are passing
• Docstrings are in Google style
• Related documentation has been updated (e.g. links, examples, etc.)
• Code is ready for review

Tests

1. Unit tests (11, stubbed litellm so the suite needs no network):

$ pytest tests/models/test_litellm_chat_model.py -q
...........                                                              [100%]
11 passed

Covers: inheritance and export; no persistent client built; defaults (drop_params=True, blank creds); dispatch sets drop_params and omits blank creds; proxy creds plus num_retries / timeout forwarded when set; structured_model sets response_format and drops tools; gemini uses json_object; invalid tool_choice raises; a messages value that isn't a list raises.

2. Full pre-commit gate, all hooks pass (black, isort, flake8, pylint 10/10 on the new module, pyroma, and the repo's local pytest hook):

black....................................................................Passed
isort....................................................................Passed
flake8...................................................................Passed
pylint...................................................................Passed
Run pytest...............................................................Passed

Sibling provider suites (test_openai_chat_model.py, test_minimax_chat_model.py, test_qiniu_chat_model.py) still pass (49 passed, 3 skipped), no regressions.

3. Live E2E through the real LiteLLMChatModel.achat (LiteLLM SDK to a local LiteLLM proxy to two different upstreams), exercising non streaming, structured output, and streaming:

[non-stream openai/gpt-4.1-mini]     role='assistant' content='4'
[non-stream openai/gemini-2.5-flash] role='assistant' content='4'
[structured]                         parsed={'answer': 4}
[stream]                             joined='streaming works'

One LiteLLMChatModel reached both an OpenAI model and a Google model unchanged, proving cross provider routing plus the reused non streaming, streaming, and structured parsing paths end to end.

4. Dependency pin verified: litellm>=1.89.0,<2.0.0 resolves cleanly (installed 1.100.0).

Example usage

import asyncio
from openjudge.models import LiteLLMChatModel

async def main():
    # Uses ANTHROPIC_API_KEY from the environment (no key passed here):
    model = LiteLLMChatModel(model="anthropic/claude-sonnet-4-6")
    resp = await model.achat(messages=[{"role": "user", "content": "What is 2+2?"}])
    print(resp.content)

asyncio.run(main())

Risk / Compatibility

• Additive only; existing providers untouched, and all existing model tests pass.
• litellm is an optional extra and is lazy imported inside achat, so from openjudge.models import LiteLLMChatModel works without it installed; only a live call requires the package.

[liuzhao1225]

I'm concerned about the supply-chain boundary of this LiteLLM integration. The upstream incident report confirms compromised PyPI releases 1.82.7/1.82.8 in March 2026; those releases were removed.

At reviewed head 7ecfc3bee7, this PR includes litellm>=1.89.0,<2.0.0 in pyproject.toml.

That upper bound permits later 1.x releases beyond a single tested version when dependencies are resolved afresh or upgraded. Please document tested versions, artifact verification, and whether installation of the SDK is opt-in. A disabled application feature cannot prevent an installed malicious Python-startup hook.

This is one of 49 observed LiteLLM integration PRs from the same account, whose author acknowledged the cross-project effort. The pattern prompted this review; it does not establish malicious intent or connect the author to the incident. The consolidated questions and corrections are in YouDub #130; this PR's review snapshot preserves the revision and scope.