Phase 0 reconciliation: the submission guard now reads the submission

.github/CODEOWNERS

@@ -0,0 +1,30 @@
+# Every path needs an owner that GitHub can actually resolve.
+#
+# A handle that does not resolve makes the whole rule silently inert: GitHub does
+# not warn, it just never requests the review. PLAN.md lists @pratik for
+# submissions/**, but that is a placeholder, not a real handle. It is deliberately
+# omitted until confirmed, and added at the Phase 12 transfer alongside the
+# required-review rule.
+#
+# Note also that on a personal-account repository a code owner must have write
+# access, and teams cannot be granted access at all. So @org/team entries would
+# be inert here even if written correctly. They become usable after transfer to
+# drexel-ice, not before.
+
+* @JiwaniZakir
+
+# This is the untrusted-input boundary. Everything under here arrives from
+# outside the lab: checkpoints are pickles and predict.py is arbitrary Python.
+/submissions/                 @JiwaniZakir
+
+# The workflows are the executable surface of this repository. A change here can
+# turn a read-only validation job into one holding a write token, which is
+# exactly the failure mode CODEOWNERS exists to slow down.
+/.github/                     @JiwaniZakir
+
+# TODO(phase-12): once real handles are confirmed, restore these AND enable
+# require_code_owner_reviews with required_approving_review_count: 1. Until both
+# are on, CODEOWNERS is advisory only.
+#
+#   /submissions/             @pratik-handle
+#   /.github/                 @JiwaniZakir @second-reviewer

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,9 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Report a security vulnerability
+    url: https://github.com/JiwaniZakir/eda-schema-experiments/security/advisories/new
+    about: Please report privately rather than in a public issue.
+
+  - name: Something wrong with the leaderboard site
+    url: https://github.com/JiwaniZakir/eda-schema-leaderboard/issues/new/choose
+    about: Wrong numbers, broken pages and rendering issues belong in the site repo.

.github/ISSUE_TEMPLATE/ingest-failure.yml

@@ -0,0 +1,53 @@
+name: My submission fails to ingest
+description: Derivation broke on a folder that follows the documented layout
+title: "[ingest]: "
+labels: ["parser"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        **This is our bug, not yours.**
+
+        Nothing is asked of submitters. Folders are ingested exactly as they come
+        off the cluster, and every field is derived. If derivation fails on your
+        folder, the parser needs fixing - please do not add a metadata file to
+        work around it, because that defeats the point.
+
+  - type: input
+    id: path
+    attributes:
+      label: Path to the folder
+      placeholder: submissions/your-name/total_area_prediction/...
+    validations:
+      required: true
+
+  - type: textarea
+    id: error
+    attributes:
+      label: What CI reported
+      description: Paste the failing job output. It will be formatted as a code block.
+      render: shell
+    validations:
+      required: true
+
+  - type: dropdown
+    id: where
+    attributes:
+      label: Which check failed
+      options:
+        - schema
+        - size
+        - Both
+        - Neither, the numbers came out wrong
+    validations:
+      required: true
+
+  - type: textarea
+    id: layout
+    attributes:
+      label: Directory layout
+      description: >-
+        The output of `find <your folder> -maxdepth 4 -type d`. Stage names
+        contain underscores and PDK directory names are uppercase, and most
+        parse failures come down to one of those two.
+      render: shell

.github/dependabot.yml

@@ -0,0 +1,17 @@
+version: 2
+updates:
+  # Actions only. This repository holds data and submissions, not a Python
+  # project - there is no pyproject.toml or lockfile here, and the validator it
+  # runs comes from the site repo, which has its own Dependabot config.
+  #
+  # This matters for the SHA-pinned workflows: pinning is only safe to maintain
+  # because Dependabot updates pins. Without this file the pins would rot.
+  - package-ecosystem: github-actions
+    directory: "/"
+    schedule:
+      interval: weekly
+    commit-message:
+      prefix: chore
+    groups:
+      actions:
+        patterns: ["*"]

.github/pull_request_template.md

@@ -0,0 +1,34 @@
+<!--
+Submitting a model? Keep the checklist. Fixing a typo? Delete it.
+
+Nothing is asked of submitters beyond the folder as it came off the cluster.
+Every field on the leaderboard is derived - from paths, eval.log, run.log,
+hparams.yaml, one checkpoint, and git log. If derivation fails on your folder,
+that is a parser bug on our side, not a request for you to add metadata.
+-->
+
+## What this changes
+
+## Submission checklist
+
+- [ ] The folder is exactly as it came off the cluster, with nothing added
+- [ ] No file exceeds 1 MB (CI blocks it; checkpoints are around 32 KB)
+- [ ] No tfevents (`events.out.tfevents*`) - the targets in them are z-scored
+- [ ] No PNGs - these belong in a Release and are referenced by URL
+- [ ] Directory names follow the documented layout, with uppercase PDK names
+
+## Division
+
+- [ ] **Open** - any training regime
+- [ ] **Closed** - the canonical leave-one-circuit-out split, canonical features
+      and target
+
+If you are unsure, leave both unchecked. Division is derived from the tree shape
+rather than declared, and a per-circuit training set resolves to Open
+automatically. You cannot misclassify yourself into Closed.
+
+## For maintainers
+
+- [ ] `schema` and `size` both green, and `schema` reported a non-zero submission count
+- [ ] Nothing under `.github/` changed, or the change was reviewed as a
+      permissions change

.github/workflows/codeql.yml

@@ -0,0 +1,43 @@
+name: codeql
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+  schedule:
+    - cron: "0 7 * * 1"
+
+permissions: {}
+
+# Actions are pinned to commit SHAs because the analyze job holds
+# `security-events: write`. Same rule as the site repo: any workflow with a write
+# scope is pinned, so a retagged upstream cannot reach it.
+#
+# This repository holds data and submissions rather than application code, so
+# `actions` is the language that matters most here - the workflows themselves are
+# the executable surface. Python is included because submissions carry
+# predict.py, which guard layer 4 will execute.
+
+jobs:
+  analyze:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      security-events: write
+    strategy:
+      fail-fast: false
+      matrix:
+        language: [actions, python]
+    steps:
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
+        with:
+          persist-credentials: false
+
+      - uses: github/codeql-action/init@c4dd10e44af883a891fe31ced449bcb4a6728b9b # v3
+        with:
+          languages: ${{ matrix.language }}
+          queries: security-extended
+
+      - uses: github/codeql-action/analyze@c4dd10e44af883a891fe31ced449bcb4a6728b9b # v3
+        with:
+          category: /language:${{ matrix.language }}

.github/workflows/notify-site.yml

@@ -6,25 +6,45 @@ on:
 
 permissions: {}
 
+# This workflow has never actually dispatched. SITE_DISPATCH_TOKEN was unset, so
+# it printed a notice and exited 0 - reporting success on every push to main
+# while doing nothing, which left the site's repository_dispatch path entirely
+# untested behind a green tick.
+#
+# Now a repository VARIABLE gates the job. `secrets` is not available in a
+# job-level `if:` but `vars` is, so when dispatch is not turned on the job does
+# not run and the workflow reports SKIPPED rather than success. When it IS turned
+# on and the credential is missing, that is a hard failure, because at that point
+# a silent no-op is a broken deploy chain rather than an unconfigured one.
+#
+# TO ENABLE:
+#   1. Create a fine-grained PAT scoped to JiwaniZakir/eda-schema-leaderboard
+#      only, with Contents: read and write. Nothing else.
+#   2. Store it as the secret SITE_DISPATCH_TOKEN.
+#   3. Set the repository variable SITE_DISPATCH_ENABLED to "true".
+#
+# This is the one sanctioned stored credential in either repository. GITHUB_TOKEN
+# cannot dispatch across repositories, and the workload identity federation used
+# for the Claude action covers the Anthropic side only, not GitHub's.
+
 jobs:
   dispatch:
+    if: vars.SITE_DISPATCH_ENABLED == 'true'
     runs-on: ubuntu-latest
     permissions: {}
     steps:
       - name: Tell the site to rebuild
         env:
-          # A fine-grained PAT with contents:write on the site repo only.
-          # GITHUB_TOKEN cannot dispatch across repositories.
-          #
-          # No payload is sent. The site's deploy workflow deliberately ignores
-          # client_payload and always builds its own main, so nothing here can
-          # influence what gets published.
           TOKEN: ${{ secrets.SITE_DISPATCH_TOKEN }}
         run: |
           if [ -z "$TOKEN" ]; then
-            echo "::notice::SITE_DISPATCH_TOKEN is not set; skipping rebuild trigger."
-            exit 0
+            echo "::error::SITE_DISPATCH_ENABLED is true but SITE_DISPATCH_TOKEN is not set"
+            exit 1
           fi
+
+          # No payload is sent. The site's deploy workflow deliberately ignores
+          # client_payload and always builds its own main, so nothing here can
+          # influence what gets published.
           curl -sSf -X POST \
             -H "Accept: application/vnd.github+json" \
             -H "Authorization: Bearer $TOKEN" \

.github/workflows/validate-submission.yml

@@ -23,6 +23,9 @@ jobs:
     steps:
       - uses: actions/checkout@v4
         with:
+          # Full history so the base commit is present and the diff below can be
+          # computed locally rather than trusting an event field for it.
+          fetch-depth: 0
           persist-credentials: false
 
       - uses: astral-sh/setup-uv@v5
@@ -36,11 +39,42 @@ jobs:
           path: .site
           persist-credentials: false
 
+      # Whether the pull request touched submissions/ decides whether an empty
+      # scan is acceptable. Computed here because only the workflow knows what
+      # the PR changed; the validator cannot infer it.
+      #
+      # BASE_SHA comes from the event payload and is a hex SHA produced by
+      # GitHub, but it is passed through env rather than interpolated into the
+      # script, per the rule that no event field is ever spliced into a run:.
+      - id: touched
+        env:
+          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+        run: |
+          changed=$(git diff --name-only "$BASE_SHA"...HEAD -- submissions/ | wc -l)
+          echo "count=$changed" >> "$GITHUB_OUTPUT"
+          echo "the pull request changes $changed file(s) under submissions/"
+
       - name: JSON Schema and guard layers 1 to 5
         working-directory: .site
+        env:
+          # Absolute, because the validator runs from .site and a relative path
+          # would resolve against the wrong tree - which is the bug this whole
+          # step replaces.
+          SUBMISSIONS: ${{ github.workspace }}/submissions
+          REQUIRE: ${{ steps.touched.outputs.count != '0' && '--require-nonempty' || '' }}
         run: |
+          # This previously ran `uv run eda-validate` with no argument, from
+          # inside .site. That validated the LEADERBOARD's own registries and
+          # baseline, reported success, and never opened the submission at all.
+          # A submission could have contained anything and this check would have
+          # been green.
+          #
+          # --submissions points it at the actual submission tree, and it prints
+          # the number of records it examined. --require-nonempty makes an empty
+          # scan a failure when the PR did touch submissions/, so "found nothing"
+          # can no longer be mistaken for "found nothing wrong".
           uv sync --all-extras
-          uv run eda-validate
+          uv run eda-validate --submissions "$SUBMISSIONS" $REQUIRE
 
   size:
     runs-on: ubuntu-latest
@@ -58,8 +92,10 @@ jobs:
           # -z because git quotes non-ASCII paths, and a quoted path silently
           # fails [ -f ] and gets skipped.
           fail=0
+          seen=0
           limit=$((MAX_FILE_MB * 1024 * 1024))
           while IFS= read -r -d '' f; do
+            seen=$((seen + 1))
             [ -f "$f" ] || continue
             case "$f" in
               *events.out.tfevents*)
@@ -72,4 +108,11 @@ jobs:
               fail=1
             fi
           done < <(git ls-files -z)
+          echo "checked $seen tracked files against the ${MAX_FILE_MB} MB cap"
+
+          # A scan that walked zero files is a broken scan, not a clean repo.
+          if [ "$seen" -eq 0 ]; then
+            echo "::error::no tracked files were scanned; the guard did not run"
+            exit 1
+          fi
           exit $fail

submissions/README.md

@@ -0,0 +1,50 @@
+# Submissions
+
+One directory per submission: `submissions/<name>/`.
+
+This directory is committed rather than left empty on purpose. CI validates
+submissions by pointing the leaderboard's validator at this path, and a missing
+directory is reported as a failure rather than as "nothing to check" - so the
+directory existing is what keeps the guard honest before the first submission
+lands.
+
+## What to send
+
+The folder exactly as it came off the cluster. Nothing added.
+
+Every leaderboard field is derived - from the directory path, `eval.log`,
+`run.log`, `hparams.yaml`, one checkpoint per architecture, and `git log`. There
+is no metadata file to fill in, and adding one will not help. If derivation fails
+on your folder, that is a parser bug on our side: open an issue with the
+"My submission fails to ingest" template.
+
+Two fields are genuinely underivable. `seed` and raw predictions emit `null` and
+set the badges `reproducibility: unverified` and `plots: static`. Neither blocks
+a submission.
+
+## What CI enforces
+
+- **`schema`** validates every `.json`, `.yaml` and `.yml` record under this
+  directory against the submission schema, and reports how many it examined. A
+  pull request that touches `submissions/` and yields zero records fails, because
+  "found nothing" must not read as "found nothing wrong".
+- **`size`** rejects any file over 1 MB and any `events.out.tfevents*`.
+
+Checkpoints are around 32 KB and belong here. PNGs are roughly 380 KB, are
+regenerable, and belong in a Release referenced by URL.
+
+## Two things that bite
+
+**Stage names contain underscores**, so `global_place` and `detailed_place` do
+not survive splitting a directory name on `_`. Parsing is anchored against the
+registry vocabulary instead.
+
+**PDK directory names are uppercase** - `default_config_ASAP7_cts`, not `asap7`.
+Registry IDs are lowercase and parsing normalizes to them.
+
+## Divisions
+
+Division follows from the shape of your tree and is never declared. A
+single-circuit-sized training set per leaf directory resolves to `open`, because
+the canonical Closed split is leave-one-circuit-out. You cannot misclassify
+yourself into Closed.