feat(ci): hash publication becomes a dispatch; flux CI loses write access to fluxhashes

[MorningLightMountain713]

Background

fluxbench verifies each node's FluxOS tree hash against the list served from RunOnFlux/fluxhashes. Today this workflow writes that list directly: it appends its computed hash and pushes into fluxhashes with a contents-write PAT exposed to every branch build. RunOnFlux/fluxhashes#2 signs the list — but as long as this repository can write it, the signature only attests "this was in the repo when the signer ran": a compromised token here could rewrite the list and have the rewrite signed.

Related PRs and merge order:

1. Publish the hash list signed fluxhashes#2 — the signer becomes the single writer, deriving every hash itself. Merges first; works standalone.
2. This PR — merges second. Once it is in, the fluxhashes master ruleset goes on and the old bot PAT (API_TOKEN_GITHUB) is revoked outright — a classic token with org-wide repo scope, and this PR removes the last reference to it. There is no narrower grant to leave behind: classic tokens carry no per-repo scoping.
3. feat(ci): gate release PRs on the signed hash list, and cut the tag after merge #1788 — the release gate; its provenance step becomes the same dispatch.

Supersedes #1785 (closed with explanation there).

What changes

• The clone/edit/push publish body is deleted. This repository holds no write credential to fluxhashes at all.
• In its place, a publish job computes the tree hash from a pristine checkout and dispatches "commit X exists, I computed H" to the fluxhashes signer. The credential is the flux-hashlist-dispatch GitHub App — Actions permission on fluxhashes and nothing else, verified able to trigger workflows and refused by every write API. The workflow mints a short-lived installation token per run, so there is no long-lived secret to leak and nothing that expires. The docs push moves to the flux-docs-push app the same way (Contents on fluxjsdocs only), which retires API_TOKEN_GITHUB entirely.
• The signer fetches this commit from this public repository and computes the hash itself. The dispatched claimed_hash is a tripwire: a mismatch is a red signing run, never a listed value. Note what that does and does not buy: the dispatch names a commit rather than a hash, but a fork network shares one object store, so a dispatched commit may be any commit ever pushed to this repository or a public fork of it, and the signer derives whatever it is pointed at. The credential is reachable by anyone who can land a workflow change on any branch here — who can already get a hash listed by pushing. Same trust boundary, not a defence against it. The tripwire also catches runner-environment drift, which today would silently publish a hash no node matches.
• publish is a separate job from build, so a publication failure never blocks the test suite; forks skip it entirely.
• The docs push is gated on development. It carried no ref condition, so every branch build raced for fluxjsdocs master and the published JSDocs were whichever branch happened to build last — three separate feature branches overwrote it within ten seconds on 2026-08-25, and it was serving an unmerged branch's services tree when this was found. Both docs steps now require development; the token mint is gated too, since there is no reason to mint a credential for a push that will not happen.
• Publication semantics are unchanged: every push on every branch is still published, and the list still means "genuine FluxOS tree" — the gated master decides what ships.

Note for reviewers: the four app secrets are now set on this repository and the flux-docs-push installation is approved (both 2026-08-25). build is green, and the docs steps correctly skip on this branch under the new development gate. publish mints its token successfully and stays red only until fluxhashes#2 merges, because the signer workflow it dispatches to does not exist before then — that one red is expected and is the last thing this PR is waiting on.

Verification

Full sandbox rehearsal on real GitHub 2026-08-24: this exact dispatch step drove the signer end to end, including a cross-environment claimed-hash match and bootstrap from a production-shaped state. The signer side carries a 37-case local harness; an independent implementation review found nothing above cosmetic.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SFrS7Q3JwuPj6Yr4vALnwp

[Cabecinha84]

Conditions on the ACK

1. Merge order is a real availability dependency, not a nicety. If #1789 lands before fluxhashes#2, hash publication
   stops dead: the old push path is deleted and sign-hashlist.yml doesn't exist on fluxhashes master, so every dispatch
   404s. fluxbench cannot verify any new FluxOS tree until #2 lands. Currently: fluxhashes#2 is OPEN, #1788 is OPEN. The
   PR body states the order correctly — just make it an explicit merge condition rather than a note.

2. The security payoff is entirely operational and outside this diff. Removing the fluxhashes grant from
   API_TOKEN_GITHUB is what actually closes the hole. Until that happens, this PR only removes the use of write access,
   not the access. And that PAT is shared — the Push docs to other repo step (nodejs.yml, still in build) uses the same
   secret for fluxjsdocs, so the grant must be trimmed, not the secret revoked.

---

Things I'd raise (none blocking)

1. No permissions: block — worth fixing in a security PR. Neither the workflow nor the publish job declares one, so
   GITHUB_TOKEN inherits the repo default (possibly read-write). publish needs contents: read and nothing else. One line.
   The dispatch step correctly scopes GH_TOKEN to itself, so gh won't fall back to GITHUB_TOKEN — that part is right.

2. Permanent red X on every push until the secret exists. Deliberate per the description, and nothing is blocked by
   it. But if you'd rather not train people to ignore a red check, a job-level env gate is the clean way:

publish:
env:
GH_TOKEN: ${{ secrets.FLUXHASHES_DISPATCH_TOKEN }}
steps:
- name: Request hash publication
if: env.GH_TOKEN != ''

Counter-argument, which I think is the stronger one: this same gate would later hide an expired token silently.
Loud-and-red is defensible. Raise it as a question, not a demand.

3. The claimed_hash tripwire is a cheap DoS for a compromised dispatch token — and it blocks, not just annoys. The
   description says a bad dispatch "can cause noise, not list content." Accurate on content, but understated on impact:
   sign-hashlist.js:294 throws, which aborts the entire run — so nothing publishes for anyone, not just the poisoned
   commit. Repeated dispatches stall publication for the whole fleet. Self-heals on the next clean run, and the fix (skip
   the offending commit, keep reconciling) belongs in fluxhashes#2, not here. Worth stating plainly rather than leaving
   it implied.

4. "The token can trigger that workflow and nothing else" is slightly stronger than the permission grants.
   Actions:write on fluxhashes also permits dispatching any workflow there, cancelling/re-running runs, and deleting logs
   and artifacts. GitHub offers no finer granularity, so there's nothing to change — but the comment in the YAML
   overstates the containment.

5. "Forks publish nothing" is only half-true. The fork guard on publish is correct and is a genuine improvement (forks
   previously went red at the hash push). But build's Push docs to other repo step still runs unconditionally with
   API_TOKEN_GITHUB, so fork builds stay red at the last step. Out of scope; the same if: github.repository ==
   'RunOnFlux/flux' would finish the job if you want it.

6. No timeout-minutes on publish. A hung gh call occupies a runner for the 6h default. Trivial.

[MorningLightMountain713]

Sharp review — two of its points were overtaken by the commit that landed just after it was written (664df60e7: both cross-repo credentials became GitHub Apps minting per-run tokens, and the docs push moved to its own app). Point by point:

Merge order — agreed it's a real availability dependency. This PR doesn't get the "ready to merge" label until fluxhashes#2 is in; the body carries the explicit order. One nuance: publication doesn't stop entirely in the wrong order — stale branches keep publishing through their old workflow copies until the old credential is revoked; it's new pushes on this line that would lose coverage.

Trimming the PAT vs revoking — overtaken twice. Nothing on this branch references API_TOKEN_GITHUB anymore (docs push uses the flux-docs-push app, Contents on fluxjsdocs only). And we fingerprinted the token behind that secret: it's a classic PAT with org-wide repo, workflow, write:packages scope, so there was never a per-repo grant to trim. The migration step is full revocation with nothing to replace.

permissions block and timeout-minutes — both right, both in (contents: read workflow-wide; 5 minutes on publish).

The red X — keeping loud-and-red, per your own counter-argument: a gate that skips on a missing secret also skips silently on a broken one. The app tokens removed the expiry case that made the gate tempting.

claimed_hash as a blocking DoS — your impact statement is accurate and better than our wording: a poisoned dispatch aborts the whole run, so sustained spam stalls publication, not just one commit. We're keeping the abort rather than skip-and-continue, deliberately: a mismatch is indistinguishable from signer-side environment drift, and drift that publishes "just the other commits" is drift publishing silently — the one failure that must never be quiet, since it signs hashes no node can match. The stall needs a flux-collaborator-level credential, gains no content, and self-heals on the next clean run. We'll state this plainly in fluxhashes#2 and the design doc rather than leaving it implied.

"and nothing else" overstates Actions:write — right; the comment now says what the permission actually grants (start/cancel/re-run/delete logs — availability, never content).

Fork guard on the docs step — already in 664df60e7.

Two asks while you're here, both single actions: approve the flux-docs-push installation request pending on @RunOnFlux (Contents on fluxjsdocs only — it's what retires the classic PAT), and grant MorningLightMountain713 admin on RunOnFlux/flux so the four app secrets (FLUXHASHES_APP_ID/KEY, FLUXJSDOCS_APP_ID/KEY) can be set — same setup role as on fluxhashes earlier today.

[MorningLightMountain713]

Both of those landed — thank you. Recording the result so the state of this PR is unambiguous.

The flux-docs-push installation is approved (Contents on fluxjsdocs, nothing else) and the four app secrets are set here. I checked both rather than assuming: the docs app mints an installation token that reaches RunOnFlux/fluxjsdocs and no other repository, and the dispatch app's token reaches RunOnFlux/fluxhashes with actions:write and is refused by the write APIs, as designed.

Uploading a secret says nothing about whether a multi-line PEM survived the round trip, so I re-ran this PR's publish job — it writes nothing. Mint the dispatch token now succeeds where it failed before the secrets existed. The job still ends red one step later, on gh: Not Found (HTTP 404) from the dispatch itself, and that 404 is worth disambiguating because GitHub returns the same code for "absent" and "forbidden": sign-hashlist.yml does not exist on fluxhashes master yet — that repository currently carries only validate.yml — and the token independently demonstrates actions:write there. So it is the missing workflow, and it clears the moment fluxhashes#2 merges. That is the only thing this PR is now waiting on.

One correction I owe you. You raised trimming API_TOKEN_GITHUB rather than revoking it, and I answered that a classic PAT has no per-repo grants to trim. That was right about the token — but this PR's own body, and fluxhashes#2's and #1788's, all still carried a pre-merge instruction to "remove the fluxhashes grant and keep the fluxjsdocs one". So our bodies were instructing the exact thing I'd told you was impossible. All three are fixed, and they now state the actual plan: full revocation, once fluxhashes#2 and this PR are in and the fluxhashes master ruleset is on.

Two other body corrections while I was in there: #1788 was citing the closed #1785 and had the merge order backwards, and fluxhashes#2 now discloses that key 1 was regenerated on the 24th — the pinned hex you verified in your review of that PR has changed, so that check is worth redoing. There is a longer note on fluxhashes#2 explaining that it was rewritten in place after your review.

Nothing further is needed from you except the re-reviews. Merge order unchanged: fluxhashes#2 first, then this.

[Cabecinha84]

Non-blocking findings

1. Check Hash has no failure guard — and its failure mode is fleet-wide, not local. newhash=$(find ./ZelBack … |
   md5sum | …) takes the exit status of the last pipeline element, so if ./ZelBack ever moves or disappears, find
   errors to stderr, the step exits 0, and newhash becomes d41d8cd98f00b204e9800998ecf8427e (md5 of empty input).
   That bogus claim then dispatches, and sign-hashlist.js:295 throws — which aborts the whole signing run, so
   nothing publishes for anyone until someone intervenes. The signer defends its own side of this (:167 rejects a
   non-md5 pipeline result); the flux side has no equivalent. One line fixes it:
   [ -d ./ZelBack ] || { echo "no ZelBack in this checkout"; exit 1; }
   Cheap, and it converts a fleet-wide publication stall into a red job in the repo that caused it.
2. cpina/github-action-push-to-another-repository@main is still unpinned, and now receives a freshly minted app
   token. Strictly better than before (the token it gets went from org-wide repo to Contents-on-fluxjsdocs), so
   this PR improves the situation rather than creating it. But in a PR whose whole thesis is credential
   containment, an unpinned third-party action tracking main is the residual hole: whoever controls that repo can
   take the token on any push. Worth a follow-up to pin it to a SHA — not a reason to hold this one.
3. Cosmetic: actions/create-github-app-token@v1 emits a Node 20 deprecation warning on the runner (v2 targets
   Node 24). Also, publish has timeout-minutes: 5 while build still has none.

Cross-repo caveat — belongs to fluxhashes#2, but affects how you time this one

The signer is level-triggered off git ls-remote of the flux refs, which recovers ref tips, not intermediate
commits. Two consequences worth naming when you write your ACK:

• Merge order is a hard gate, not a nicety. If #1789 lands before fluxhashes#2, every push in the gap 404s. The
  daily sweep then reconciles current tips — but any commit that was a tip only briefly during the gap is never
  listed, and never will be. So out-of-order costs permanently unlisted trees, not just a delay.
• The same shape exists at steady state. concurrency: {group: sign-hashlist, cancel-in-progress: false} means a
  newer dispatch cancels an older pending one; the surviving run reconciles refs, so a rapid A-then-B push to one
  branch can leave A's tree unlisted. Lower stakes than the gap case (intermediate branch commits rarely get
  deployed), and out of scope here — but it's the honest reading of "repaired by whichever run survives."

[MorningLightMountain713]

Heads-up that the diff moved since the last comment — one commit, 13ef13e11, five lines of nodejs.yml.

The docs push had no ref condition. It only ever checked the repository, so every branch build raced for fluxjsdocs master and the published JSDocs ended up being whichever branch happened to build last. On the 25th three separate feature branches overwrote it within ten seconds — chore/eslint-cleanup, fix/restart-pacing-crash-only, fix/external-endpoints-configurable — and it was serving an unmerged branch's services tree at the point I looked. That is pre-existing behaviour, not something this PR introduced; this PR was already adding a fork guard to those two steps, so the ref condition rides along in the same place rather than arriving as a second PR that would conflict on the same lines.

Both docs steps now require refs/heads/development. The token mint is gated as well as the push — no reason to mint a credential for a push that will not happen.

Verified rather than assumed, on this branch: the run at 13ef13e11 has build green with Mint the docs token and Push docs to other repo both skipped, and fluxjsdocs received no commit from it. Before the gate, a re-run of the same job did push — that one was deliberate and safe, since this branch's ZelBack/src/services is byte-identical to development's, so it wrote development's docs. It also served as the end-to-end proof of the flux-docs-push app, which had not previously pushed anything in anger.

publish is still red on the expected HTTP 404 — the signer workflow does not exist on fluxhashes master until fluxhashes#2 merges. That remains the only outstanding red.

[MorningLightMountain713]

All three findings taken, and the third turned into a removal rather than a pin. Tip is now b5571ab9b, five commits past the 2d70812e3 you reviewed — your review landed about forty seconds before the first of them, so none of this was in front of you.

Finding 1, Check Hash has no failure guard — fixed (fd184c6af), and your analysis of the mechanism was exactly right: the pipeline's status is awk's, so find errors to stderr while the step exits 0 with newhash set to the md5 of an empty stream.

I did not take the suggested [ -d ./ZelBack ], because it only covers half of it. A ZelBack that exists but holds no regular files produces the same bad hash with no error at all — find exits 0 and emits nothing. So the guard is on the value, which covers both, plus set -eo pipefail so the absent case dies on find's own message rather than being laundered into a hash comparison. Verified by extracting the step's script verbatim from the workflow and running it against three fixtures: absent exits 1, empty exits 1, a real tree returns the hash it always did.

One correction to the impact, which does not change the fix: it is not a stall "until someone intervenes". The next clean push reconciles everything, and the daily sweep does it unattended — every run reconciles the full delta rather than just what it was told about. The matching guard is on the signer side too (RunOnFlux/fluxhashes#2, d8238412), so the empty hash cannot reach the list even if a claim carrying it arrives from a stale branch.

Finding 2, the unpinned action — removed instead of pinned (4e9fda8c9). You were right to raise it and right that it was the residual hole in a PR arguing for containment. Reading it, though, the action was doing nine lines of work: clone the destination shallow, wipe the target directory, copy the source over it, commit if anything changed, push. The other 166 lines are options we never pass, error strings and git-lfs setup. Pinning would have frozen a dependency we do not need, so it is now plain git and nothing outside Flux touches the token at all.

Two things came out better than the action managed, rather than merely equal to it: fluxjsdocs is public, so the clone is anonymous and the token authenticates only the push — the action put the token in the clone URL and therefore in the checkout's stored config — and git add -A services scopes the commit to the directory we publish where the action did git add . across the whole destination. Behaviour is otherwise deliberately identical, including the Update from <origin commit url> message so the fluxjsdocs history stays uniform.

Verified by extracting the step verbatim and pointing only the two URLs at local repositories: a changed file updates, a file deleted upstream disappears downstream, nested directories copy, an unrelated directory in the destination is untouched, author and message match what the action produced, and a second run with identical content prints docs unchanged and pushes nothing — no empty commit. The credential form was proven separately with a dry-run push, since the step is gated to development and so cannot exercise itself on this branch.

Finding 3, the deprecation warning — done (b5571ab9b). actions/create-github-app-token v1 → v3. Neither major break reaches us: v2 removed the underscore input spellings and this workflow uses the hyphenated ones, and v3 removed custom proxy handling and raised the self-hosted runner floor, while these are GitHub-hosted with no proxy. Confirmed minting on a real run rather than assumed.

codecov/codecov-action stays on v3 deliberately: v4 moved to the CLI uploader and dropped tokenless uploading for non-fork runs, so bumping it needs a CODECOV_TOKEN secret first or coverage upload starts failing. Unrelated to this PR, and I did not want to break coverage in passing.

On timeout-minutes for build: I have left it, since it is the pre-existing test job rather than anything this PR introduces, and a five-minute cap made sense on publish because it is a single API call. Happy to add one if you would rather it be uniform.

The cross-repo caveat — your reading is sharper than what the body said, and the body now says yours. You are right that ls-remote recovers tips and not intermediate commits, so out-of-order merging costs permanently unlisted trees rather than a delay, and that the same shape exists at steady state through the concurrency collapse. "Repaired by whichever run survives" was true of the ref delta and quietly untrue of a tree that stopped being a tip. The merge order is unchanged and is stated in all three bodies: fluxhashes#2 first, this second.

One unrelated thing this review caused me to find. The docs push had no ref condition at all — only a repository check — so every branch build raced for fluxjsdocs master and the published JSDocs were whichever branch happened to build last. On the 25th three separate feature branches overwrote it within ten seconds, and it was serving an unmerged branch's services tree when I looked. Pre-existing, not introduced here, but this PR was already touching those two steps to add the fork guard, so the ref condition rides along (13ef13e11) rather than arriving as a second PR conflicting on the same lines. Both docs steps now require development, mint included.

Also corrected in this range (30c66fa00): the publish job's comment claimed a hash value could not enter the list through the dispatch credential. That is false — a fork network shares one object store — and the same wrong claim was in this body, fluxhashes#2's body and header, and #1788. All five now state what the credential is actually trusted to name. Detail on fluxhashes#2.

[Cabecinha84]

ack