[zby]

Motivation

• Provide an example of how to register and use a custom provider that does not depend on any LLM SDKs, demonstrating how to integrate plain HTTP endpoints with llm-do and PydanticAI.
• Supply a detailed implementation guide so contributors can implement their own SDK-free providers reliably.

Description

• Added SimpleHTTPProvider and SimpleHTTPChatModel to examples/custom_provider/providers.py that send/receive OpenAI-style chat payloads via raw HTTP using httpx.
• Registered the new factory under the prefix http-json via register_model_factory("http-json", build_http_json_model).
• Updated examples/custom_provider/README.md to document the new provider usage.
• Added examples/custom_provider/GUIDE.md, a detailed guide describing required responsibilities (provider, model, message mapping, request/response mapping, limitations, and extension points) for SDK-free custom providers.

Testing

• Ran lint/format checks with uv run ruff check . (auto-fix applied where needed) and the command completed successfully.
• Ran type checking with uv run mypy llm_do and it reported no issues.
• Ran the test suite with uv run pytest and all tests passed (315 passed).

---

Codex Task

[gemini-code-assist]

Summary of Changes

Hello @zby, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the flexibility of llm-do by providing a robust example and detailed guide for creating custom LLM providers that communicate directly with HTTP endpoints, bypassing the need for specific LLM SDKs. This allows users to integrate with a wider range of models and services, particularly those offering OpenAI-compatible APIs, using a lightweight and customizable approach.

Highlights

• New SDK-Free Custom Provider Example: Introduced a new example demonstrating how to register and use a custom LLM provider that does not rely on any external LLM SDKs.
• Plain HTTP Endpoint Integration: The example showcases how to integrate plain HTTP endpoints (specifically OpenAI-compatible ones) with llm-do and PydanticAI using raw HTTP requests via httpx.
• Detailed Implementation Guide: A comprehensive guide (GUIDE.md) has been added to provide contributors with detailed instructions on implementing their own SDK-free providers, covering responsibilities, message mapping, and extension points.
• New Provider Classes: Implemented SimpleHTTPProvider and SimpleHTTPChatModel in providers.py to handle OpenAI-style chat payloads over raw HTTP.
• Updated Documentation: The README.md for custom providers has been updated to document the new http-json provider usage and reference the new guide.

Changelog

• examples/custom_provider/GUIDE.md
  • Added a new file, GUIDE.md, which serves as a detailed guide for building SDK-free custom providers.
  • The guide outlines the responsibilities of provider and model classes, message mapping, request construction, response parsing, and potential extension points for advanced features.
• examples/custom_provider/README.md
  • Updated the README to announce the addition of the http-json provider.
  • Added a reference to the newly created GUIDE.md for in-depth information.
  • Included new instructions and an example command for running the http-json provider.
• examples/custom_provider/providers.py
  • Introduced SimpleHTTPProvider to encapsulate base URL, API key, and an httpx.AsyncClient for HTTP requests.
  • Implemented SimpleHTTPChatModel, a custom pydantic_ai.models.Model subclass, to handle OpenAI-compatible chat requests and responses.
  • Added logic within SimpleHTTPChatModel for message mapping, payload construction, and response parsing, supporting text-only output.
  • Registered the build_http_json_model factory under the http-json prefix, enabling its use via LLM_DO_MODEL environment variable.
  • Added necessary imports for httpx, UserError, and various PydanticAI model and message components.

Activity

• Lint and format checks were run using uv run ruff check ., and auto-fixes were applied successfully.
• Type checking was performed with uv run mypy llm_do, reporting no issues.
• The entire test suite was executed using uv run pytest, with all 315 tests passing successfully.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds a valuable example of an SDK-free custom provider, complete with comprehensive documentation in a new guide. The implementation is clear and serves as an excellent template for users. My review includes two suggestions for the providers.py file. First, a high-severity fix for a potential IndexError in the response parsing logic that could occur with valid API responses. Second, a medium-severity improvement to the model factory to reuse the HTTP client for better performance, which aligns with best practices mentioned in the new guide.

[gemini-code-assist]

The current logic for extracting the response content is not robust. If the API returns a response with an empty choices list (e.g., {"choices": []}), which is a valid scenario, your code will raise an IndexError. This can happen, for example, if content filtering is triggered by the model provider.

Suggested change

	data.get("choices", [{}])[0]
	(data.get("choices") or [{}])[0]

[gemini-code-assist]

The current implementation of build_http_json_model creates a new SimpleHTTPProvider instance on every call. Since SimpleHTTPProvider instantiates an httpx.AsyncClient, this leads to creating a new HTTP client and connection pool for each model resolution, which is inefficient. It's better to create a single provider instance at the module level and reuse it. This also aligns with the GUIDE.md's mention of a 'reusable async HTTP client'.

Suggested change

	def build_http_json_model(model_name: str) -> SimpleHTTPChatModel:
	return SimpleHTTPChatModel(
	model_name,
	provider=SimpleHTTPProvider(
	base_url="http://127.0.0.1:8000/v1",
	api_key=None,
	),
	)
	_http_json_provider = SimpleHTTPProvider(
	base_url="http://127.0.0.1:8000/v1",
	api_key=None,
	)
	def build_http_json_model(model_name: str) -> SimpleHTTPChatModel:
	return SimpleHTTPChatModel(
	model_name,
	provider=_http_json_provider,
	)