Skip to content

Add Typesafe wrangles for structured AI answers - #1202

Merged
ebhills merged 3 commits into
mainfrom
codex/ai-typesafe
Sep 26, 2026
Merged

ebhills merged 3 commits into
mainfrom
codex/ai-typesafe

Conversation

@ebhills

@ebhills ebhills commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Linked issue

Closes #1201

What changes

Provide structured answers to common types of questions about each input row through the new ai namespace on the Python, recipe, and DataFrame surfaces. ai.choose answers which option best fits, ai.score answers where input falls along ordered criteria, and ai.true_false answers how likely a statement is to be true. ai.answers answers any combination of these question types in one Typesafe request; all four wrangles accept multiple named questions.

  • Generate output columns from question labels or complete positional overrides. Preserve choice labels, native fractional scores, provider confidence, and probability maps keyed by descriptive criteria. ai.true_false returns probability true and the supplied true criterion.
  • Integrate Typesafe through the shared model catalog/configuration, with a shared transport, response validation, transient retries, bounded concurrency, and credential-scoped caching.
  • Recognize nested outputs in filtered and concurrent recipes, including empty selections and intentional column replacement.
  • Document each wrangle around the question it answers, with input tables, annotated YAML, complete illustrative outputs, and Typesafe reference links in AI answers.

How it was verified

  • Credential-free local suite: .\scripts\test-local.ps1 — 2,508 passed, 6 skipped, 139 deselected.
  • Focused rename validation: 163 tests passed, including absence of the old alias/catalog role, dataframe access, schema discovery, and existing mixed-question request/output behavior.
  • Direct offline coverage includes configuration, malformed responses, retries, cache behavior, multiple questions/rows, output naming/collisions, filtering, concurrent execution, and recipe-to-HTTP integration with mocked provider responses.
  • All four documentation examples validated against the implemented schemas, input tables, and output mappings; the single Python example passed syntax validation.
  • git diff --check passed; branch verified current with refreshed origin/main.
  • Live check: Eric confirmed that the documented direct Python ai.choose call succeeded using TYPESAFE_API_KEY and cache=False. This is user-reported provider connectivity/function evidence, not a measured accuracy evaluation.
  • Live ai.score, ai.true_false, ai.answers, and hosted recipe execution have not been verified. GitHub CI results are pending; this PR remains Draft.

Compatibility and risk

  • The composite wrangle is ai.answers; the earlier unreleased draft name has no compatibility alias. Preview callers must update the function/recipe name and catalog operation/model role. The questions input mapping, provider payload, and result fields are unchanged.
  • extract.ai and its support-module imports remain in place; no ai.extract alias or module reorganization is included.
  • New wrangles require a Typesafe credential. Keys remain outside recipes/catalog values through environment or managed-secret substitution.
  • Version-1 AI configurations use packaged defaults for the new wrangles. Version-2 replacement configurations must add the applicable provider and operation entries explicitly.
  • Score criteria are unique strings and retain native 0..N-1 semantics. There is no rescaling or Boolean thresholding.
  • New WRANGLES_AI_CACHE_* controls are separate from extraction's existing cache controls.
  • No deployment or external data migration is included. Rollback is to revert this PR and stop using the newly introduced wrangles.

Ready-for-review checklist

  • One human delivery owner is assigned
  • The linked issue and intended milestone are correct (no release milestone selected)
  • The branch is current with main and has no merge conflicts
  • Focused tests pass
  • New or changed behavior has direct test coverage
  • Documentation/schema/configuration is updated where applicable
  • The PR contains no unrelated changes
  • The PR description reflects the branch's current scope and latest validation
  • One primary reviewer is requested only when this PR is ready

See the pull request workflow.

@ebhills ebhills self-assigned this Sep 26, 2026
@ebhills ebhills changed the title Add Typesafe AI question wrangles Add Typesafe wrangles for structured AI answers Sep 26, 2026
@ebhills
ebhills marked this pull request as ready for review September 26, 2026 23:41
@ebhills
ebhills merged commit 6db2a11 into main Sep 26, 2026
18 checks passed
@ebhills
ebhills deleted the codex/ai-typesafe branch September 26, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add ai namespace and Typesafe choose, score, true_false, and answers wrangles

1 participant