Skip to content

feat(oci): ChatOCIModelDeployment — structured output beyond json_mode and multimodal input support #264

Description

@fede-kamel

Motivation

ChatOCIModelDeployment targets OCI Data Science Model Deployment endpoints (typically vLLM/TGI serving open models). Two capability gaps vs the rest of the package and vs peer packages:

  1. Structured output: with_structured_output supports only method="json_mode" and raises ValueError for function_calling and json_schema (libs/oci/langchain_oci/chat_models/oci_data_science.py, with_structured_output), even though the class already implements bind_tools — so tool-based structured output is feasible today. vLLM also supports guided/JSON-schema decoding for a native json_schema path.
  2. Multimodal: no image content-block handling, although vLLM deployments commonly serve vision models (Llama 3.2/4 Vision, etc.) via OpenAI-compatible image_url content parts.

Proposed scope

  • Add method="function_calling" (via existing bind_tools + tool-output parsing) and method="json_schema" (vLLM guided decoding / response_format={"type": "json_schema", ...}) to with_structured_output.
  • Pass through OpenAI-style multimodal content blocks (image_url, base64 data URIs) in _construct_json_body for chat requests.

Acceptance criteria

  • with_structured_output(schema, method=...) works for all three methods against a vLLM deployment; clear error only where the serving backend genuinely lacks support.
  • Vision input round-trips against a vision model deployment (integration test, credential-gated).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions