Skip to content

Apply the AO structured-output workaround during install - #407

Open
l3acon wants to merge 2 commits into
ansible:mainfrom
l3acon:fix/ao-structured-output-workaround
Open

l3acon wants to merge 2 commits into
ansible:mainfrom
l3acon:fix/ao-structured-output-workaround

Conversation

@l3acon

@l3acon l3acon commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #405

Related: AAP-99739 and upstream Syntara PR #1193. The upstream PR is open. After it merges, the workaround remains useful until a product release containing the fix is available.

Solution

Apply the temporary structured-output workaround automatically from the existing Infrastructure | Automation Orchestrator | Install job, using its existing OpenShift credential. For object response schemas on Task Agents without tools, the workaround prompts for JSON text, validates it against the schema, and retries up to three times. If no response is valid, the step fails. Other agent configurations retain their existing path.

What changed

  • Wait for the background worker, then apply or verify the overlay as part of Install. The successful workaround path reports workshop_prompt_json in its metadata.
  • Pin the patch to the tested backend image, original source, and generated overlay hashes. An unsupported image fails preflight rather than receiving an unreviewed patch.
  • Enable launch-time extra variables so that, after a fixed product release is available, launching Install with ao_workaround_state: absent removes the overlay. Once deployed overlays have been removed, the isolated workaround block and directory can be deleted from this project.
  • Reuse the Install job's kubeconfig and kubectl; no standalone rollout inventory, playbook, or extra execution environment is needed. Document the lifecycle and retain focused integration tests.

Validation

  • The same generated overlay was previously applied to five workshop instances. Three isolated no-tool Task Agent runs per instance produced schema-valid output with final-answer traces: 15/15. This is a small sample, not a proven long-run success rate or a downstream AAP workflow test.
  • An earlier APD canary returned schema-valid output in 10/10 sequential isolated runs after five empty successful responses before patching. An unsatisfiable schema failed after the configured retries.
  • Four simulated tests of the actual Install workaround tasks passed both locally and inside the AO Execution Environment image. They cover apply, idempotence, removal after a simulated backend upgrade, check mode, version guards, conflicting configuration, and invalid state.
  • A read-only check inside the AO execution environment against an existing workshop detected the installed overlay successfully. The full modified Install job has not yet been run end to end.
  • ansible-playbook -i localhost, infrastructure/ao/install.yml --syntax-check, ansible-lint infrastructure/ao/install.yml infrastructure/setup.yml, and Black checks passed.

Limits

This is an operator-unmanaged overlay, not a product fix. It is pinned to the tested AO backend image and source; upgrades or reconciliation can remove or invalidate it. Rollout and removal restart workers. Schema validity does not guarantee factual correctness, and sequential tests do not establish behavior under load.

@l3acon l3acon changed the title Add an opt-in AO structured-output workshop workaround Apply the AO structured-output workaround during install Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provide a temporary AO structured-output workaround for workshops

1 participant