Motivation
ChatOCIModelDeployment targets OCI Data Science Model Deployment endpoints (typically vLLM/TGI serving open models). Two capability gaps vs the rest of the package and vs peer packages:
- Structured output:
with_structured_output supports only method="json_mode" and raises ValueError for function_calling and json_schema (libs/oci/langchain_oci/chat_models/oci_data_science.py, with_structured_output), even though the class already implements bind_tools — so tool-based structured output is feasible today. vLLM also supports guided/JSON-schema decoding for a native json_schema path.
- Multimodal: no image content-block handling, although vLLM deployments commonly serve vision models (Llama 3.2/4 Vision, etc.) via OpenAI-compatible
image_url content parts.
Proposed scope
- Add
method="function_calling" (via existing bind_tools + tool-output parsing) and method="json_schema" (vLLM guided decoding / response_format={"type": "json_schema", ...}) to with_structured_output.
- Pass through OpenAI-style multimodal content blocks (
image_url, base64 data URIs) in _construct_json_body for chat requests.
Acceptance criteria
with_structured_output(schema, method=...) works for all three methods against a vLLM deployment; clear error only where the serving backend genuinely lacks support.
- Vision input round-trips against a vision model deployment (integration test, credential-gated).
Motivation
ChatOCIModelDeploymenttargets OCI Data Science Model Deployment endpoints (typically vLLM/TGI serving open models). Two capability gaps vs the rest of the package and vs peer packages:with_structured_outputsupports onlymethod="json_mode"and raisesValueErrorforfunction_callingandjson_schema(libs/oci/langchain_oci/chat_models/oci_data_science.py,with_structured_output), even though the class already implementsbind_tools— so tool-based structured output is feasible today. vLLM also supports guided/JSON-schema decoding for a nativejson_schemapath.image_urlcontent parts.Proposed scope
method="function_calling"(via existingbind_tools+ tool-output parsing) andmethod="json_schema"(vLLM guided decoding /response_format={"type": "json_schema", ...}) towith_structured_output.image_url, base64 data URIs) in_construct_json_bodyfor chat requests.Acceptance criteria
with_structured_output(schema, method=...)works for all three methods against a vLLM deployment; clear error only where the serving backend genuinely lacks support.