Skip to content

test: add frozen recognition challenge - #183

Merged
akcorca merged 1 commit into
mainfrom
codex/issue-178-recognition-challenge
Aug 10, 2026
Merged

test: add frozen recognition challenge#183
akcorca merged 1 commit into
mainfrom
codex/issue-178-recognition-challenge

Conversation

@akcorca

@akcorca akcorca commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Closes #178

Outcome

  • Add a strict, frozen 24-case synthetic holdout independent from generated law/diversity fixtures.
  • Cover all six owned layers (syntax, binding, constraint, pack, resolution, presentation), all eight quality metrics, and positive/refusal outcomes for every layer.
  • Parse untrusted fixture JSON through one exact schema and score each dimension separately with pure functions.
  • Extract one shared native semantic-evaluation adapter instead of duplicating foundation process/I/O code.
  • Keep actual challenge execution manual while validating its schema and coverage matrix in fast unit tests.

Baseline

22/24 cases pass. The two preserved failures are actionable inputs to later ordered children rather than fixture edits:

The strict bun run challenge gate will turn green before parent #176 closes; challenge:baseline writes the current attributed report without weakening that final gate.

Verification

  • bun run check
  • bun run foundation (118/118 after runner extraction)
  • bun run challenge:baseline (22/24, exactly two attributed gaps)
  • awiki lint -r

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b92ce859-b93a-41da-99ae-f94b7e15efd4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@akcorca
akcorca merged commit 5570db2 into main Aug 10, 2026
2 checks passed
@akcorca
akcorca deleted the codex/issue-178-recognition-challenge branch August 10, 2026 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v0.18 2/5: Establish an independent synthetic recognition challenge

1 participant