feat: schema complexity analyzer + automatic mode selection

[mimran-khan]

What this does

I noticed that picking the right mode is a common friction point, especially for users with complex nested schemas. This PR adds two utilities that solve this at the library level:

analyze_schema(response_model) - inspects a Pydantic model before any API call and reports:

• Complexity score (0-100)
• Depth, field count, recursion detection
• Large enum warnings
• Actionable suggestions for schema restructuring

select_mode(response_model, provider) - picks the optimal extraction mode by cross-referencing schema complexity against provider capabilities:

• Simple schemas -> TOOLS (leverages native function calling)
• Deep schemas -> JSON_SCHEMA (gives the LLM explicit structure)
• Recursive schemas -> MD_JSON/JSON (avoids tool-schema limitations)
• Always respects what the provider actually supports

Why

I ran into this myself while testing extraction on deeply nested models. The retry loop burned tokens because the schema was too complex for TOOLS mode, but there was no guidance about that. Checking the issues list, I found similar confusion across multiple discussions.

These utilities are pure functions with no side effects, no network calls, and zero additional dependencies. They are lazy-loaded so they don't affect import performance.

Files changed

• instructor/v2/core/schema_analyzer.py - the complexity analysis engine
• instructor/v2/core/auto_mode.py - mode selection logic using provider specs
• instructor/__init__.py - top-level exports (analyze_schema, SchemaAnalysis, select_mode)
• instructor/v2/core/__init__.py - v2 core exports
• tests/test_schema_analyzer.py - 28 tests for the analyzer
• tests/test_auto_mode.py - 31 tests for mode selection

Usage

import instructor
from pydantic import BaseModel

class Invoice(BaseModel):
    items: list["LineItem"]
    # ... deeply nested

# Check complexity before calling
analysis = instructor.analyze_schema(Invoice)
print(f"Complexity: {analysis.complexity_score}/100")
for finding in analysis.findings:
    print(f"  [{finding.severity.value}] {finding.message}")

# Auto-select the best mode
mode = instructor.select_mode(Invoice, instructor.Provider.OPENAI)
client = instructor.from_openai(openai.OpenAI(), mode=mode)

Test results

All 59 new tests pass. Broader suite (1537 tests) has no regressions.

Closes #2393

Checklist before requesting a review

• I have performed a self-review of my code
• If it is a core feature, I have added thorough tests.
• If it is a core feature, I have added documentation.

[jxnl]

Reviewed for merge readiness. I would keep this open for #2393, but not merge it yet: it adds new public API (analyze_schema, select_mode) and hard-coded provider/mode heuristics, so this needs maintainer design approval rather than being treated as a small fix. A couple of things to tighten before review: document whether select_mode is advisory only or intended to become part of from_provider, and clean up the tests so model definitions do not rely on forward references before their classes exist / duplicate class definitions. CI should run on the refreshed branch before this gets reviewed for API shape.

[mimran-khan]

Thanks for the feedback. I've addressed both points:

1. Advisory-only documentation: Updated the module docstring and select_mode docstring to explicitly state this is advisory-only. It does not get called internally by from_provider or any client factory. Users opt in by passing the returned mode manually. The intent is a starting point for users unsure which mode fits their schema, not an automatic override.

2. Test cleanup: Reordered all model definitions in both test files so dependencies are defined before usage (no forward references). Removed the duplicate ModelWithLargeEnum class that existed in test_schema_analyzer.py.

Branch is force-pushed so CI should pick it up fresh.

[jxnl]

1.16 triage update: keeping this open, not merging it in the maintenance release. The analyzer may be separable, but select_mode still codifies product policy without provider/model outcome data, and this branch's Bedrock assumptions are already stale relative to fully green draft #2515 (which adds explicit Bedrock JSON_SCHEMA/TOOLS_STRICT support). Please treat a schema-analyzer-only API and mode recommendation policy as separate review decisions after the release.