fix(v2): accumulate all billable usage fields across retries (#2493)

[mbsdeepak]

Addresses #2493.

Problem

update_total_usage enumerated the token fields to sum by hand, once per provider, so every counter a provider SDK added afterwards was silently dropped:

• Anthropic (instructor/v2/providers/anthropic/usage.py) — the nested cache_creation and server_tool_use sub-models were never accumulated. initialize_usage() leaves them None and the old loop summed only four flat fields, so the ephemeral cache-write and web-search/fetch breakdowns stayed None for the whole run.
• OpenAI (instructor/v2/core/usage.py) — the accumulator's *_tokens_details objects (which only carried audio/reasoning/cached) were model_copy'd back onto the response, overwriting accepted_prediction_tokens / rejected_prediction_tokens with None.

Nothing raised, and input_tokens / prompt_tokens stayed correct, so the under-count only surfaced when reconciling against a provider invoice.

Fix

Replace both hand-written lists with a single generic accumulator (accumulate_usage) that walks the pydantic usage model and sums every numeric field, recursing into nested sub-models and adopting a zeroed copy when the accumulator's sub-model is None. Non-numeric fields (service_tier, inference_geo) take the latest reported value. New SDK counters are now picked up automatically — the "generic" shape suggested in the issue. Both provider paths route through it, and the running total is mirrored back onto the response in place, so a CompletionUsage subclass keeps its type.

Before / after

Anthropic, three identical retries, on the repo's pinned anthropic==0.93.0:

before:  total.cache_creation is None       -> AttributeError reading ephemeral_5m_input_tokens
after:   input=300   ephemeral_5m=1500   web_search=6

Tests

tests/v2/test_usage_accumulation.py reproduces both provider bugs (pure accounting logic, no API keys or network), plus a subclass-preservation test and a version-independent test of the accumulation rules (nesting, None handling, non-numeric last-wins).

Full suite locally: 1656 passed, 164 skipped; ruff check clean.

[ErenAta16]

I filed #2493, so treat this as an interested party. The generic walk is the right shape and this version has a design detail the other two do not. It also has one gap that sits directly under the claim in its own docstring.

The gap: the walk only visits declared fields, so a counter the SDK has not modelled yet is dropped. The SDK usage models are extra="allow", so a field the API starts returning before Stainless regenerates lands in model_extra rather than in model_fields. Measured on real anthropic==0.93.0, three attempts each reporting a counter the pinned SDK does not declare:

Usage.model_config["extra"]              -> 'allow'
Usage.model_validate(raw).model_extra    -> {'brand_new_billable_tokens': 7}
"brand_new_billable_tokens" in Usage.model_fields -> False

after 3 attempts, walking model_fields only:
  input_tokens=300  ephemeral_5m=60  brand_new_billable_tokens=None   (expected 21)

Declared fields all accumulate correctly, so this is narrow. But it is the same failure mode the module docstring here promises to end: "new billable fields are picked up automatically rather than needing a matching edit in two places every time." SDK lag is exactly when that matters, and it is a real window; output_tokens_details did not appear in anthropic until well after the API was returning the data. Walking set(type(m).model_fields) | set(m.model_extra or {}) in _accumulate_into and _zero_numeric closes it, and the same union in _sync_into mirrors it back.

The design detail this version gets right and I would not want lost. _sync_into mutates an existing sub-model in place instead of replacing it:

existing = getattr(response, name, None)
if isinstance(existing, BaseModel):
    _sync_into(existing, value)
else:
    setattr(response, name, value.model_copy(deep=True))

Replacing with a copy of the accumulator's object means the response ends up holding whatever concrete class the accumulator was built with. That is fine while both sides are the same class, but _initialize_usage hands the OpenAI accumulator to every non-Anthropic provider, and any provider shipping a CompletionUsage subclass with extra fields would silently have them swapped out for the base type. Keeping the response's own instance and writing into it avoids that entirely. Worth a comment saying that is deliberate, because it reads like an unnecessary branch.

Also correct, and easy to get wrong. Handling bool before (int, float) in both _zero_numeric and _accumulate_into. bool is an int subclass, so without that ordering a flag would be zeroed and then summed into 2, 3, 4. Two of the three implementations under review got this right and it is not obvious from reading; the explanatory comment earns its place.

Verified separately, and it constrains all three PRs. pyproject.toml pins anthropic==0.93.0, which has no output_tokens_details and no such module. The docstring here lists output_tokens_details (thinking_tokens) as one of the accumulated sub-models; that is true on a newer SDK and a no-op on the pinned one, which the generic walk handles silently and correctly. That is worth saying explicitly in the docstring, because it is the property that makes this approach necessary rather than merely tidier: #2499 names the same sub-model directly and raises ImportError on 0.93.0 on every call.

One correction to my own issue. I wrote #2493 against anthropic==0.116.0 and listed thinking_tokens among the fields left stale. It does not exist on the version instructor installs, so that third of the issue does not apply to the pinned SDK. The cache_creation and server_tool_use halves do, and the OpenAI half does; I measured cache_write_tokens landing as None on main against openai==2.52.0 where the provider had sent it three times.

[jxnl]

Superseded by #2502, now merged to main. The canonical implementation also walks Pydantic model_extra fields, which this patch omitted, so newly added SDK counters remain cumulative. Closing this overlapping source PR in favor of the consolidated fix.