Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/scheduled-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ jobs:
uv run --frozen pytest tests/ --asyncio-mode=auto -n auto \
--ignore=tests/coverage \
--ignore=tests/test_batch_processor_coverage.py \
-m 'not llm' \
-k 'not test_core_providers and not test_openai and not test_anthropic and not test_gemini and not test_genai and not test_writer and not test_vertexai and not docs'

- name: Build and validate distributions
Expand Down
8 changes: 2 additions & 6 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,17 +30,12 @@ jobs:
uv run --frozen pytest tests/ --asyncio-mode=auto -n auto
--ignore=tests/coverage
--ignore=tests/test_batch_processor_coverage.py
-m 'not llm'
-k 'not test_core_providers and not test_openai and not test_anthropic
and not test_gemini and not test_genai and not test_writer and not
test_vertexai and not docs'
env:
INSTRUCTOR_ENV: CI
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}

# Offline coverage tests must not be filtered by provider names.
offline-coverage-tests:
Expand Down Expand Up @@ -129,6 +124,7 @@ jobs:
uv run --frozen coverage erase
uv run --frozen coverage run --branch -m pytest tests/ \
--asyncio-mode=auto --strict-config --strict-markers \
-m 'not llm' \
-k 'not docs'
uv run --frozen coverage report --show-missing --fail-under=99
uv run --frozen coverage json -o "${RUNNER_TEMP}/coverage.json"
Expand Down
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,12 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html)

## [Unreleased]

## [1.15.5] - 2026-08-02
## [1.15.5] - 2026-08-07

### Fixed
- **Remote multimodal fetches**: Apply the existing 30-second request timeout to image, audio, and PDF downloads so an unresponsive URL cannot block a caller indefinitely. ([#2507](https://github.com/567-labs/instructor/pull/2507))
- **OpenAI streaming retries**: Keep TOOLS, JSON, JSON_SCHEMA, and MD_JSON retries on the streaming parser after the one-shot model marker is consumed, allowing corrected streamed responses to validate successfully. ([#2508](https://github.com/567-labs/instructor/pull/2508))
- **Iterable streaming unions**: Parse PEP 604 unions (`create_iterable(response_model=Weather | GoogleSearch)`, `Iterable[Weather | GoogleSearch]`) member by member instead of calling `model_validate_json` on `types.UnionType`. ([#2509](https://github.com/567-labs/instructor/pull/2509))
- **Package metadata**: Point the published distribution's repository URL at the current `567-labs/instructor` organization and validate it before release.
- **Retry usage accounting**: Accumulate nested and newly added numeric usage fields across OpenAI and Anthropic retries, including prediction, cache-write, cache-creation, and server-tool counters, without treating boolean metadata as billable usage. ([#2493](https://github.com/567-labs/instructor/issues/2493), [#2500](https://github.com/567-labs/instructor/pull/2500))
- **OpenAI Responses reask**: Add a fallback correction message when a `RESPONSES_TOOLS` response contains no tool calls (e.g. reasoning-only output), so retries carry validation feedback instead of resending the identical request. ([#2498](https://github.com/567-labs/instructor/pull/2498))
Expand Down Expand Up @@ -42,6 +45,7 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
- **OpenAI Responses API**: Align `RESPONSES_TOOLS` `text.format` with the forced tool schema and add targeted retry guidance when tool calls return empty `{}` arguments. ([#2300](https://github.com/567-labs/instructor/issues/2300), [#2304](https://github.com/567-labs/instructor/pull/2304))

### Tests / CI
- **Fork-safe contributor checks**: Mark auto-client network tests explicitly and exclude them from core, coverage, and release lanes so fork PRs without provider secrets do not fail with empty authorization headers.
- **Coverage and test quality**: Run the complete offline suite on Python 3.9-3.13, enforce fork-safe statement and branch coverage plus supported-version type checks in pull-request CI, add strict resource and thread warning checks, and provide a manual retry-mutation workflow. Consolidate typed response, stream, and SDK fixtures; remove duplicate tests and unreachable provider paths; and replace coverage-only stubs with meaningful edge-case and transport-backed provider checks.
- **Release safety**: Validate the declared source, lockfile, changelog, tag, and built artifacts before any publication step; require an explicit version confirmation and publish opt-in; and publish the exact tested assets instead of rebuilding from a moving branch.

Expand Down
8 changes: 4 additions & 4 deletions instructor/v2/core/multimodal.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ def from_url(cls, url: str) -> Image:

if not media_type:
try:
response = requests.head(url, allow_redirects=True)
response = requests.head(url, allow_redirects=True, timeout=30)
media_type = response.headers.get("Content-Type")
except requests.RequestException as e:
raise ValueError(f"Failed to fetch image from URL") from e
Expand Down Expand Up @@ -233,7 +233,7 @@ def from_path(cls, path: Union[str, Path]) -> Image: # noqa: UP007
@lru_cache
def url_to_base64(url: str) -> str:
"""Cachable helper method for getting image url and encoding to base64."""
response = requests.get(url)
response = requests.get(url, timeout=30)
response.raise_for_status()
return base64.b64encode(response.content).decode("utf-8")

Expand Down Expand Up @@ -324,7 +324,7 @@ def from_url(cls, url: str) -> Audio:
"""Create an Audio instance from a URL."""
if url.startswith("gs://"):
return cls.from_gs_url(url)
response = requests.get(url)
response = requests.get(url, timeout=30)
content_type = response.headers.get("content-type")
if content_type not in VALID_AUDIO_MIME_TYPES:
raise ValueError(
Expand Down Expand Up @@ -591,7 +591,7 @@ def from_url(cls, url: str) -> PDF:

if not media_type:
try:
response = requests.head(url, allow_redirects=True)
response = requests.head(url, allow_redirects=True, timeout=30)
media_type = response.headers.get("Content-Type")
except requests.RequestException as e:
raise ValueError("Failed to fetch PDF from URL") from e
Expand Down
3 changes: 2 additions & 1 deletion instructor/v2/dsl/iterable.py
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,8 @@ def extract_cls_task_type(
**kwargs: Any,
):
assert cls.task_type is not None
if get_origin(cls.task_type) is Union:
# PEP 604 unions use types.UnionType rather than typing.Union as their origin.
if get_origin(cls.task_type) in _UNION_ORIGINS:
union_members = get_args(cls.task_type)
for member in union_members:
try:
Expand Down
2 changes: 1 addition & 1 deletion instructor/v2/providers/anthropic/multimodal.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def pdf_to_anthropic(pdf: Any) -> dict[str, Any]:
):
return {"type": "document", "source": {"type": "url", "url": pdf.source}}
if not pdf.data:
pdf.data = requests.get(str(pdf.source)).content
pdf.data = requests.get(str(pdf.source), timeout=30).content
pdf.data = base64.b64encode(pdf.data).decode("utf-8")
return {
"type": "document",
Expand Down
4 changes: 2 additions & 2 deletions instructor/v2/providers/genai/multimodal.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ def image_to_genai(image: Any) -> Any:
("http://", "https://")
):
return types.Part.from_bytes(
data=requests.get(image.source).content,
data=requests.get(image.source, timeout=30).content,
mime_type=image.media_type,
)
if image.data or image.is_base64(str(image.source)):
Expand All @@ -55,7 +55,7 @@ def pdf_to_genai(pdf: Any) -> Any:
and pdf.source.startswith(("http://", "https://"))
and not pdf.data
):
data = requests.get(pdf.source).content
data = requests.get(pdf.source, timeout=30).content
encoded = base64.b64encode(data).decode("utf-8")
return types.Part.from_bytes(
data=base64.b64decode(encoded),
Expand Down
40 changes: 21 additions & 19 deletions instructor/v2/providers/openai/handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -414,6 +414,19 @@ def _consume_streaming_flag(
return True
return False

def _should_parse_streaming(
self,
response_model: type[BaseModel] | ParallelBase | None,
stream: bool,
) -> bool:
"""Return whether a response needs DSL streaming parsing."""
registered = self._consume_streaming_flag(response_model)
return bool(
inspect.isclass(response_model)
and issubclass(response_model, (IterableBase, PartialBase))
and (stream or registered)
)

def extract_streaming_json(
self, completion: TypingIterable[Any]
) -> Generator[str, None, None]:
Expand Down Expand Up @@ -713,15 +726,12 @@ def parse_response(
response_model: type[BaseModel],
validation_context: dict[str, Any] | None = None,
strict: bool | None = None,
stream: bool = False, # noqa: ARG002
stream: bool = False,
is_async: bool = False, # noqa: ARG002
) -> Any:
"""Parse tool call response."""
# Check for streaming
consume_streaming = isinstance(
response_model, type
) and self._consume_streaming_flag(response_model)
if consume_streaming:
if self._should_parse_streaming(response_model, stream):
return self._parse_streaming_response(
response_model,
response,
Expand Down Expand Up @@ -810,16 +820,12 @@ def parse_response(
response_model: type[BaseModel],
validation_context: dict[str, Any] | None = None,
strict: bool | None = None,
stream: bool = False, # noqa: ARG002
stream: bool = False,
is_async: bool = False, # noqa: ARG002
) -> Any:
"""Parse JSON schema response."""
# Check for streaming
if (
isinstance(response_model, type)
and (stream or self._consume_streaming_flag(response_model))
and issubclass(response_model, (IterableBase, PartialBase))
):
if self._should_parse_streaming(response_model, stream):
return self._parse_streaming_response(
response_model,
response,
Expand Down Expand Up @@ -902,12 +908,10 @@ def parse_response(
response_model: type[BaseModel],
validation_context: dict[str, Any] | None = None,
strict: bool | None = None,
stream: bool = False, # noqa: ARG002
stream: bool = False,
is_async: bool = False, # noqa: ARG002
) -> Any:
if isinstance(response_model, type) and self._consume_streaming_flag(
response_model
):
if self._should_parse_streaming(response_model, stream):
return self._parse_streaming_response(
response_model,
response,
Expand Down Expand Up @@ -1002,14 +1006,12 @@ def parse_response(
response_model: type[BaseModel],
validation_context: dict[str, Any] | None = None,
strict: bool | None = None,
stream: bool = False, # noqa: ARG002
stream: bool = False,
is_async: bool = False, # noqa: ARG002
) -> Any:
"""Parse JSON from markdown code block in response."""
# Check for streaming
if isinstance(response_model, type) and self._consume_streaming_flag(
response_model
):
if self._should_parse_streaming(response_model, stream):
return self._parse_streaming_response(
response_model,
response,
Expand Down
2 changes: 1 addition & 1 deletion instructor/v2/providers/openai/multimodal.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ def pdf_to_openai(pdf: Any, mode: Mode) -> dict[str, Any]:
and pdf.source.startswith(("http://", "https://"))
and not pdf.data
):
response = requests.get(pdf.source)
response = requests.get(pdf.source, timeout=30)
data = base64.b64encode(response.content).decode("utf-8")
if mode in RESPONSES_MODES:
return {
Expand Down
8 changes: 4 additions & 4 deletions tests/coverage/test_anthropic_support_coverage.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,10 +184,10 @@ def test_anthropic_multimodal_encodes_remote_image_and_local_pdf(
assert image_calls == ["https://example.test/diagram.png"]
assert image.data == "aW1hZ2U="

requests: list[str] = []
requests: list[tuple[str, int]] = []

def fake_get(url: str) -> Any:
requests.append(url)
def fake_get(url: str, *, timeout: int) -> Any:
requests.append((url, timeout))
return SimpleNamespace(content=b"%PDF-1.7\nexample")

monkeypatch.setattr(multimodal.requests, "get", fake_get)
Expand All @@ -203,7 +203,7 @@ def fake_get(url: str) -> Any:
"data": expected,
},
}
assert requests == ["/tmp/example.pdf"]
assert requests == [("/tmp/example.pdf", 30)]
assert pdf.data == expected


Expand Down
18 changes: 10 additions & 8 deletions tests/coverage/test_core_multimodal_coverage.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,18 +143,18 @@ def test_image_raw_base64_rejects_non_webp_riff_data() -> None:


def test_image_url_to_base64_is_cached(monkeypatch: pytest.MonkeyPatch) -> None:
calls: list[str] = []
calls: list[tuple[str, int]] = []

def get(url: str) -> requests.Response:
calls.append(url)
def get(url: str, *, timeout: int) -> requests.Response:
calls.append((url, timeout))
return response(b"image-bytes", "image/png")

monkeypatch.setattr(requests, "get", get)
first = Image.url_to_base64("https://example.test/photo.png")
second = Image.url_to_base64("https://example.test/photo.png")

assert first == second == base64.b64encode(b"image-bytes").decode()
assert calls == ["https://example.test/photo.png"]
assert calls == [("https://example.test/photo.png", 30)]


def test_audio_autodetects_url_gcs_string_path_and_path(
Expand Down Expand Up @@ -260,10 +260,12 @@ def test_pdf_autodetects_gcs_path_and_raw_base64(
def test_pdf_autodetects_url_from_response_media_type(
monkeypatch: pytest.MonkeyPatch,
) -> None:
calls: list[tuple[str, bool]] = []
calls: list[tuple[str, bool, int]] = []

def head(url: str, allow_redirects: bool = False) -> requests.Response:
calls.append((url, allow_redirects))
def head(
url: str, allow_redirects: bool = False, *, timeout: int
) -> requests.Response:
calls.append((url, allow_redirects, timeout))
return response(b"", "application/pdf")

monkeypatch.setattr(requests, "head", head)
Expand All @@ -273,7 +275,7 @@ def head(url: str, allow_redirects: bool = False) -> requests.Response:
assert pdf.source == "https://example.test/reports/latest"
assert pdf.media_type == "application/pdf"
assert pdf.data is None
assert calls == [("https://example.test/reports/latest", True)]
assert calls == [("https://example.test/reports/latest", True, 30)]


def test_pdf_rejects_raw_base64_with_non_pdf_content() -> None:
Expand Down
10 changes: 5 additions & 5 deletions tests/coverage/test_openai_support_coverage.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,10 +237,10 @@ def test_openai_multimodal_encoders_cover_response_and_error_paths(
with pytest.raises(ValueError, match="Responses doesn't support audio"):
audio_to_openai(audio, Mode.RESPONSES_TOOLS_WITH_INBUILT_TOOLS)

requested: list[str] = []
requested: list[tuple[str, int]] = []

def fetch(url: str) -> requests.Response:
requested.append(url)
def fetch(url: str, *, timeout: int) -> requests.Response:
requested.append((url, timeout))
response = requests.Response()
response.status_code = 200
response._content = b"%PDF-1.7\ncoverage" # real response body, no network call
Expand All @@ -261,8 +261,8 @@ def fetch(url: str) -> requests.Response:
},
}
assert requested == [
"https://cdn.example.invalid/report.pdf",
"https://cdn.example.invalid/report.pdf",
("https://cdn.example.invalid/report.pdf", 30),
("https://cdn.example.invalid/report.pdf", 30),
]

pdf_data = PDF.from_base64("data:application/pdf;base64,cGRm")
Expand Down
32 changes: 32 additions & 0 deletions tests/multimodal/test_multimodal.py
Original file line number Diff line number Diff line change
Expand Up @@ -751,3 +751,35 @@ def test_pdf_to_bedrock_missing_data_no_source():
match="PDF data is missing. Provide base64-encoded data or use an s3:// source.",
):
pdf.to_bedrock()


def test_url_to_base64_passes_timeout():
"""Remote fetches must pass a request timeout so a slow or unresponsive host
cannot hang the caller indefinitely (uncontrolled resource consumption, CWE-400)."""
with patch("instructor.v2.core.multimodal.requests") as mock_requests:
mock_response = MagicMock()
mock_response.content = b"fake image bytes"
mock_requests.get.return_value = mock_response

Image.url_to_base64("https://example.com/timeout-regression.jpg")

mock_requests.get.assert_called_once()
_, kwargs = mock_requests.get.call_args
assert kwargs["timeout"] == 30


def test_image_from_url_head_passes_timeout():
"""The HEAD probe used to sniff an extensionless URL's content type must also
pass a timeout so it cannot hang the caller indefinitely (CWE-400)."""
with patch("instructor.v2.core.multimodal.requests") as mock_requests:
mock_requests.RequestException = Exception
mock_response = MagicMock()
mock_response.headers = {"Content-Type": "image/jpeg"}
mock_requests.head.return_value = mock_response

# URL has no file extension, forcing the HEAD content-type probe.
Image.from_url("https://example.com/no-extension")

mock_requests.head.assert_called_once()
_, kwargs = mock_requests.head.call_args
assert kwargs["timeout"] == 30
3 changes: 3 additions & 0 deletions tests/providers/test_auto_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ def test_validation_error_with_api_key_is_not_skipped() -> None:


@pytest.mark.parametrize("provider_string", PROVIDERS)
@pytest.mark.llm
def test_user_extraction_sync(provider_string):
"""Test user extraction for each provider (sync)."""

Expand Down Expand Up @@ -200,6 +201,7 @@ def test_user_extraction_sync(provider_string):

@pytest.mark.parametrize("provider_string", PROVIDERS)
@pytest.mark.asyncio
@pytest.mark.llm
async def test_user_extraction_async(provider_string):
"""Test user extraction for each provider (async)."""

Expand Down Expand Up @@ -270,6 +272,7 @@ def builder(**kwargs):
]


@pytest.mark.llm
def test_additional_kwargs_passed():
"""Test that additional kwargs are passed to provider."""
import instructor
Expand Down
Loading
Loading