Skip to content

Align Case eval criteria across live and SDK authoring loops #2910

Description

@tmatup

Several Maestro Case tasks in the 2026-08-30 same-ground run grade live-skill implementation details instead of the requested artifact semantics:

  • optional stages/tasks are rejected when isRequired is omitted, although omission is the SDK canonical/default false representation
  • HITL result-downstream requires tasks/tasks.md even when the SDK writes an equivalent root tasks.md and the compiled caseplan contains the required binding
  • the SDD-only registry handoff checker accepts only the live skill's top-level list, although both arms emitted semantically complete audit envelopes with different keys
  • the logging task asks for the live skill's private journal contract without stating that output contract in the prompt, leaving the SDK arm to invent a different issue-log format
  • project/solution discovery rejects a substantive Case project when a generated trigger-only duplicate scaffold is also present

Update prompts/checkers so both authoring loops receive the same observable requirements and equivalent representations are accepted. Preserve negative controls for genuinely missing optionality, binding, resource identity, issue-log evidence, and distinct project ambiguity.

Evidence: exact cohort from adhoc-2026-08-30_04-43-27, with fresh matched reruns under SDK 3.27.6 and gpt-5.6-luna.

🤖 Generated with Claude Code
Co-Authored-By: Claude

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions