Skip to content

feat: list published pages a run never reached in the functional report - #92

Merged
katodea merged 2 commits into
trunkfrom
add/report-published
Nov 19, 2026
Merged

katodea merged 2 commits into
trunkfrom
add/report-published

Conversation

@katodea

@katodea katodea commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

what

  • scripts/page-capture.js
    • the merge reads published again after a run held to the site and writes it into the report ({ source, total, notReached } or { error }); a run refused for its wall never asks the site
    • a run with no capture drops any published the model wrote; a correction restores it from the archived copy (RUN_FIELDS) instead of refetching
    • the merge prints Published pages not reached: N of M, or why they couldn't be read
    • the CLI catches a failed merge and exits 1, since the merge is async now and --unhandled-rejections=warn would let it exit 0
  • scripts/generate-report.js adds "Published pages not reached" under Methodology: the list, a "loaded each one" line (saying "each of the 100 kosh read" when the REST list was capped) or the error
  • scripts/validate-report.js refuses a functional report holding published or a non-empty linkChecks[] with no captureSha256
  • schemas/qa-report-functional-schema.json adds optional published, with a non-empty error
  • tests/no-network.js replaces fetch in the scripts tests/run-qa-report.test.js spawns, so its merges never ask a real site; a stub-site test covers the real fetch
  • AGENTS.md covers the above

why

inventory already shows the model the site's published pages nothing links to, but the report didn't. on a second test site, one run saw 13 unreached pages, opened 1 more and left 12. its testMethodology didn't mention them. those pages held stale event sign-ups and a garbled fees section that another run reported as highs. with this change, the report lists them whatever the model does with the inventory.

the validator guard closes the other side: a report that skipped the merge could carry a hand-written list rendered as "read by kosh".

notes

  • a run whose browser recorded nothing but checked links is now refused. one with no link checks still renders, and whether no capture should always refuse is still open
  • the sitemap source reads only page 1 (up to 2,000 URLs), and total is counted before the off-host filter. both only matter on unusual sites, so they're deferred
  • rendering a functional report now makes up to two requests to the site, so it can wait up to 20 s on a hung site
  • at 375px the Exploration pass cards make the report scroll sideways; that was there before and belongs to the report re-style

co-authored by claude opus 5.5 🧭

@katodea katodea self-assigned this Nov 6, 2026
@t51eng-poseidon

t51eng-poseidon Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

🔱 Poseidon started reviewing this PR — central run.

@t51eng-poseidon t51eng-poseidon Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔱 Poseidon Review

Summary — The functional merge now re-reads the site's published pages after a wall-held run and records them as published, the renderer shows them under Methodology, and the validator refuses published/linkChecks[] without a capture hash. Risk is low: the fetch reuses the vetted published() path, it's skipped for wall-refused runs, and the tests cover both branches.

Walkthrough

  • ✨ New — published added to the functional schema, merge (scripts/page-capture.js), RUN_FIELDS restore and the HTML report's "Published pages not reached" block
  • 🐛 Fix — merge is now async, and the CLI catches a rejection and sets exit code 1, so run-qa-report.sh still stops on a failed merge
  • 🧪 Tests — tests/no-network.js preload keeps wrapper tests offline; a stub-site test covers the real fetch; validator, renderer and correction tests extended
  • 📝 Docs — AGENTS.md describes the merge's published and the new validator refusal

💡 Nitpicks (1)
  • scripts/validate-report.js:L88 — The new linkChecks?.length && !captureSha256 refusal also applies to archives made before this PR: the old merge wrote linkChecks[] even when no capture existed. Re-rendering or correcting such an archive now fails, which goes against the "an archive from before the wall still re-renders" rule elsewhere. If that's intended, add a sentence to AGENTS.md; if not, apply the check only to published, or only on a fresh merge.

🤖 Prompt for AI agents — fix all findings
Verify each finding against the current code and only fix it if
needed.

Nitpicks:
In `scripts/validate-report.js`:
- Line 88: The new branch `type === 'functional' &&
  !report.provenance?.captureSha256 && (report.published ||
  report.linkChecks?.length)` in `checkReport` refuses any functional
  report with non-empty `linkChecks[]` and no capture hash. Before
  this PR, `merge` in scripts/page-capture.js set `report.linkChecks`
  whether or not a capture existed. So an archived functional report
  from a run with no capture now fails validation when it is
  re-rendered with generate-report.js or corrected through
  run-qa-report.sh. That breaks the stated rule that old archives
  still re-render. Decide whether this is intended. If it is, say so
  in the AGENTS.md paragraph on the validator. If it isn't, narrow
  the `linkChecks` half of the condition so old archives keep
  rendering, e.g. check only `report.published`, or refuse
  `linkChecks[]` in the merge itself. Keep the `published` refusal
  and the existing `rows?.length` branch above it unchanged.

ℹ️ Review info
  • Commit: 8f7aa98
  • Files reviewed: AGENTS.md, schemas/qa-report-functional-schema.json, scripts/generate-report.js, scripts/page-capture.js, scripts/validate-report.js, tests/generate-report.test.js, tests/no-network.js, tests/page-capture.test.js, tests/run-qa-report.test.js, tests/validate-report.test.js
  • Files skipped: none
  • Model: claude-opus-5-5

@t51eng-poseidon t51eng-poseidon Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔱 Poseidon Review

No new issues found on this pass. The only change since the last review is the merge of trunk (#91, the capture guard's shell pre-filter), and the PR's own changes are the same as before.


♻️ Previously flagged

Still outstanding

  • ⚠️ scripts/validate-report.js:L88 — report.linkChecks?.length && !captureSha256 still refuses functional archives from before this PR that have link checks but no capture, which conflicts with the rule that old archives still re-render. AGENTS.md now says a fresh run like this is refused, but it says nothing about existing archives.

ℹ️ Review info
  • Commit: 7a8b6e2
  • Files reviewed: AGENTS.md, schemas/qa-report-functional-schema.json, scripts/generate-report.js, scripts/page-capture.js, scripts/validate-report.js, tests/generate-report.test.js, tests/no-network.js, tests/page-capture.test.js, tests/run-qa-report.test.js, tests/validate-report.test.js
  • Files skipped: none
  • Model: claude-opus-5-5

@katodea
katodea merged commit 3daccf9 into trunk Nov 19, 2026
3 checks passed
@katodea
katodea deleted the add/report-published branch November 19, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant