Skip to content

ci(#23): add required PR-time Go/race/security gate - #34

Merged
BorisTyshkevich merged 18 commits into
mainfrom
ci/pr-gate
Aug 30, 2026
Merged

BorisTyshkevich merged 18 commits into
mainfrom
ci/pr-gate

Conversation

@BorisTyshkevich

@BorisTyshkevich BorisTyshkevich commented Aug 29, 2026 •

Copy link
Copy Markdown
Collaborator

What this does

Adds .github/workflows/pr-gate.yml — this repository's first PR-time verification
gate
. Workflow name PR gate, single job Required PR gate, triggered on every
pull request (unfiltered) and every push to main, running exactly the five commands
issue #23 specifies:

go build ./...
go vet ./...
go test -race ./...
go test -tags phase5release ./internal/securitytest -count=1
bash integration/clickhouse/tests/lib-tests.sh

Until now the repo had no PR-time CI at all: the two existing workflows only build and
push images on push to main. This gate is a hard prerequisite for #33 (the ADR #32 LDAP
parser replacement), whose security-relevant changes must not depend on a coordinator
running local commands by hand.

Part of #23.

Design decisions

One job, five sequential steps — not multiple jobs plus an aggregator. This yields
exactly one merge-relevant check name and no needs graph. The decisive reason: a job
skipped by a job-level condition reports success, so an aggregator is a
silent-waiver shape rather than a fail-closed one.

Required PR gate is an externally consumed interface. It is the exact check-run
context branch protection on main will require, so renaming the job means migrating
branch protection in the same change. That is now machine-enforced, not just documented.

Minimal authority over untrusted PR code. pull_request (never
pull_request_target), permissions: contents: read, persist-credentials: false,
cache: false, no secrets, no registry login, nothing published. The explicit
permissions: block is load-bearing: this repository's default_workflow_permissions
is currently write.

Action pins. Both actions are pinned to full commit SHAs, each annotated with the
exact release tag it resolves to (actions/checkout@3d3c42e5… = v7.0.1,
actions/setup-go@b7ad1dad… = v7.0.0; both verified against the GitHub API), and both
are at or above the first release declaring the node24 runtime. Review pass 1
caught that the original pins (checkout@v4.4.0, setup-go@v5.6.0) both declare
using: node20, which currently runs only because the runners re-execute node20 actions
on Node 24 as a compatibility shim — withdrawn when Node 20 leaves the runner images on
2026-09-23. A brand-new mandatory gate would have broken repo-wide about three weeks
after landing. TestPRGateContract_ActionPinVersionCommentsAreAtOrAboveTheNode24Floor enforces a floor
on each pin's trailing version comment — deliberately named and worded for what it can
actually prove, since a commit SHA cannot be resolved to a version or a declared runtime
without network access a unit test must not have. Immutability itself is proven
separately and completely by TestPRGateContract_ExternalActionsArePinnedToFullCommitSHA. The two existing publisher workflows are deliberately left on floating tags —
separate hardening, out of scope here.

Trigger activity types are explicit, and edited is load-bearing. A bare
pull_request: defaults to opened/synchronize/reopened only. A base-branch retarget
arrives as edited and keeps the same head SHA, and the check-run is recorded against
that head SHA — so a PR proven green against another base could be retargeted onto
protected main and carry its green Required PR gate across without the new merge
result ever being tested. types: [opened, synchronize, reopened, edited] closes that,
and the contract test now requires exactly that list.

No path filters, no allowlists. Future tests added under internal/securitytest
(including the planned TestDependencyContract_ProductionClosureHasNoGeneralLDAP) enroll
in the gate with no workflow edit.

internal/securitytest/pr_gate_contract_test.go

The plan's invariant map originally left three invariants — "all five commands control
one merge result", "no skipped aggregator can report misleading success", and
"required-check name cannot silently drift" — enforced only by human static review. In a
repository that already keeps machine-checked contracts in internal/securitytest, that
was the weakest link, so this PR converts them into an executable contract over the
workflow YAML: trigger shape, absence of pull_request_target/path filters/if/needs/
continue-on-error, exactly one job, the literal job name, the five commands exactly once
each in order, and full-SHA action pinning.

Untagged, matching convention: release_gate_test.go is the only phase5release-tagged
file in that package. So it runs in plain go test ./... and in this gate's own
go test -race ./... step.

Two deliberate design constraints:

  • Every expectation is a literal in the test source. A test that reads the job name
    out of the file under test and compares it to itself detects nothing.
  • Action pins are asserted for immutability, not exact value (known identity +
    ^[0-9a-f]{40}$). Hardcoding today's SHAs would turn a legitimate action upgrade into
    unrelated test churn for no security gain.

Known limitation, stated plainly

This gate is not tamper-resistant against an authorized pull-request author. Because
the trigger is pull_request (correctly — not pull_request_target), a PR runs its own
copy
of the workflow. A PR that keeps the job name but neuters the steps, or adds a
job-level if: false, produces a green required check that verified nothing. The
non-obvious part is that a skipped job satisfies a required status context, so reading
the steps during review does not catch it. The new contract test does not close this
either — it lives in the same PR-controlled tree and runs only because that same workflow
runs it.

So the honest claim is: this enforces process for trusted maintainers and catches ordinary
regressions. Closing the deliberate case would require an organization-level
required-workflow ruleset sourced from outside the repository being modified — org-owner
authority, a different enforcement architecture, and out of scope for #23. The full
reasoning and the decision to accept this is recorded in the issue's ship log.

Documentation reconciliation

Shipping this gate made several committed statements false, so they are fixed in the same
change (per CLAUDE.md's working discipline):

  • CLAUDE.md — new repo-map rows; the gate described in Build & test; the
    Working discipline claim "There is no PR-time CI gate" replaced; the future-linter
    guidance now points at pr-gate.yml; Unlike the Go gate (was Like, which
    asserted the opposite of this commit); the integration/clickhouse/ row no longer
    implies nothing there is CI-gated.
  • README.md — the new non-publishing workflow described alongside the two publishers.
  • integration/clickhouse/README.md — tests/lib-tests.sh is in CI; only the Docker
    suite is manual.
  • skills/ship/references/repo-footguns.md, skills/ship/SKILL.md — the stale "there is
    no PR-time CI to wait on" guidance replaced with head-SHA-keyed waiting on
    Required PR gate.

Two accuracy notes now recorded in CLAUDE.md: lib-tests.sh is Docker-daemon-free but
needs the docker CLI on PATH (lib/common.sh calls detect_compose_cmd at source
time and dies — reproduced locally), which makes the issue body's "needs only
bash/coreutils" imprecise; and the gate never compiles third_party/goldap /
third_party/ldapserver, which is where the parser #33 replaces currently lives.

Verification

All coordinator-run at head f743963, not taken from a self-report:

Check Result
go build ./... / go vet ./... exit 0
go test -race -count=1 ./... exit 0
go test -tags phase5release ./internal/securitytest -count=1 exit 0
bash integration/clickhouse/tests/lib-tests.sh exit 0
go vet -tags phase5release ./internal/securitytest exit 0
bash helm/ch-oauth-ldap/test.sh GATE RESULT: PASS (0 failures)
pinned actionlint v1.7.7 on pr-gate.yml clean
workflow structure independently re-derived by parsing the committed YAML

Sabotage cases for the new contract test — each must fail, and each failure message
must name the security consequence:

Mutation Caught by
job renamed Required PR Gate ExactlyOneJobNamedForBranchProtection
paths: ["**.go"] added under pull_request TriggersAreUnfilteredPRAndMainPush + NoPathFiltersAnywhere
continue-on-error: true on the security step JobIsUnconditionalAndFailsClosed
permissions: contents: write TopLevelPermissionsAreReadOnly
actions/checkout@v4 (unpinned) ExternalActionsArePinnedToFullCommitSHA
if: false on the job (the neutering shape above) JobIsUnconditionalAndFailsClosed

All six failed as required; pr-gate.yml was restored byte-identical after each.

Hosted CI

Reviewed head: f743963. Required PR gate succeeded at that exact SHA in run
33283550959
(pull_request, all five steps: Build, Vet, Race tests, Phase 5 security tests, ClickHouse
shell library tests), as of 2026-08-30.

This section deliberately no longer tries to enumerate every run. It can't have a fixed
point: because the trigger now includes edited, editing this description starts another
gate run, which immediately makes any exhaustive table stale again. The complete
run-by-run ledger — every run, each keyed to its head SHA — lives in the
ship log,
whose edits do not retrigger the gate and which is current as of this head. Every wait in this PR was keyed to an exact head
SHA, never to gh run list ordering.

The two runs worth calling out from that ledger:

  • c8538af — 33276319268
    ❌ at step 6. The gate found two real defects on its first ever run, neither
    reproducible on the darwin dev platform.
  • e37e24d — 33277581488
    ❌ at step 7 only, with steps 4-6 passing: the deliberate phase5release red proof
    for Definition-of-Done item 11, reverted by a normal inverse commit so it stays in
    history.

What run 1 found. TestAdversarial_WriteDeadlineClosesStalledConnection… created
backpressure with a 20 MiB response and a 300ms WriteTimeout; on the runner all
~40 MiB was delivered, so the deadline never fired and the mechanism under test never
engaged (darwin buffers 128 KiB; the runner buffered everything). And a DATA RACE on
ldapserver.Logger turned out to be a consequence of that failure — the test
t.Fatalfs before its own teardown, leaking a live server into the next test's New(),
violating a constraint server.go already documents. Both fixes are test-only;
git diff origin/main..HEAD -- internal/ldap/server.go is empty. The stall now runs over
an unbuffered net.Pipe (deterministic on every host, no payload-size heuristic), and a
shared startServing helper gives seven server-start sites fail-safe teardown registered
before any assertion can abort.

DoD item 11 — the deliberate red proof (run 33277581488). Steps 4 Build, 5 Vet and
6 Race tests all passed; step 7 Phase 5 security tests failed; job Required PR gate failed. Two choices make it a proof rather than a coincidence: steps 4-6
passing means the red result is attributable to the security step alone, and the test
carried //go:build phase5release so it was invisible to go test -race ./... and fatal
only to the security command. Run 1's accidental failure could not serve this purpose —
it failed at step 6, which skipped steps 7-8, so the security step's wiring to the
required status had never been exercised. Removed by a normal inverse commit
(a11eac0), never a force-push, so the red run and its cause stay in this PR's history
as the audit trail.

Known limitation of the enforcement model, restated for the record

Branch protection is enabled after this merges and a successful push-event run is
observed on main: required context exactly Required PR gate, strict: true
(require branches up to date), enforce_admins: true, and no review requirement.
strict: true is load-bearing, not incidental — with loose required checks a PR green
against main = B0 can merge unchanged after another PR moves main to B1, since H
itself never gets a synchronize event, so a green gate would authorize an untested
B1+H merge result. That is the same fail-open class as the retarget case above, for the
far more common trigger. Read-back verifies all four fields plus the GitHub Actions app
binding, not just the context string. #33 stays blocked until all three hold: merged, green run on main,
and protection reading back the exact context. Merging this PR alone does not satisfy
that. And per the limitation above, the gate enforces process for trusted maintainers
rather than resisting deliberate tampering — closing that would need an organization-level
required-workflow ruleset sourced outside this repository.

🤖 Generated with Claude Code

https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS

BorisTyshkevich and others added 6 commits August 29, 2026 22:58
Adds `.github/workflows/pr-gate.yml` — workflow `PR gate`, one job
`Required PR gate` — as this repo's first PR-time verification gate. It
triggers unfiltered on every `pull_request` and on `push` to `main`, and
runs the five contractual commands sequentially in one job:

    go build ./...
    go vet ./...
    go test -race ./...
    go test -tags phase5release ./internal/securitytest -count=1
    bash integration/clickhouse/tests/lib-tests.sh

Why one job with five steps rather than five jobs plus an aggregator: it
gives branch protection exactly one merge-relevant result. Any command
failing fails the job; there is no `needs` graph, no job-level `if`, and
no aggregator that could report success after a skipped dependency
(GitHub reports a condition-skipped job as successful).

Security posture: no `pull_request_target`, top-level `permissions:
contents: read` only, no `secrets.*`, no registry login, no Actions
environment, no publication, and `persist-credentials: false` on
checkout, so an untrusted PR's code never runs with repository write
authority. The two third-party actions are pinned by full commit SHA and
annotated with their exact release tags (`actions/checkout@v4.4.0`,
`actions/setup-go@v5.6.0`); `cache: false` keeps setup-go cache
restore/save off the untrusted-PR execution surface. `timeout-minutes:
15` bounds a hang with headroom for cold Linux setup, module download,
race instrumentation, and the socket-binding tests.

The real Docker ClickHouse compatibility matrix stays deliberately
outside this gate: `run-all-builds.sh` is neither invoked nor changed,
and the shell-library step (`tests/lib-tests.sh`) is Docker-daemon-free.

Shipping the gate makes existing prose false, so the same commit
reconciles it:

- `CLAUDE.md` — repo-map row for the new workflow; the "Build & test"
  section now names `Required PR gate` as the mandatory PR/push gate and
  keeps the two image publishers explicitly separate from verification
  (a green publish is not a green gate); the future-facing linter
  guidance now points at `pr-gate.yml` as the wiring target, without
  adding `golangci-lint` here; "Working discipline" no longer claims
  there is no PR-time CI, while keeping local verification mandatory.
- `README.md` — the "Building images" intro now describes the separate,
  non-publishing third workflow.
- `skills/ship/references/repo-footguns.md` — replaces the "no PR-time
  CI to wait on" guidance with head-SHA-keyed waiting on `Required PR
  gate`, and states that the two publishers are not substitutes for it.
- `skills/ship/SKILL.md` step 2.5 — the coordinator must now wait for
  `Required PR gate` at the exact current head SHA and treat missing,
  pending, cancelled, or failed state as not ready.

Locally verified: all five commands green, plus `go vet -tags
phase5release ./internal/securitytest`; pinned actionlint v1.7.7 reports
no findings on the new workflow; `helm/ch-oauth-ldap/test.sh` passes
(0 failures), including its literal README/CLAUDE/footguns docs
assertions and the untouched-path proof. A temporary tagged fatal test
under `internal/securitytest` was proven to turn the `phase5release`
command red — showing future tagged security tests enroll automatically
with no workflow-side allowlist — then removed; that file is
deliberately not part of this commit.

Part of #23. Enabling classic branch protection to require exactly
`Required PR gate` is a follow-on rollout step, not part of this diff.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
…ed gate contract

Internal review of ca79a9e found two factually inverted docs claims that the
same commit introduced, plus three plan invariants that were enforced only by
human static review.

Two blocking inversions:

- `CLAUDE.md`'s `helm/ch-oauth-ldap/test.sh` bullet said the chart gate is "like
  the Go gate, not run by any workflow". The Go gate *is* now run by a workflow
  — that is what ca79a9e added — so the comparison read backwards. It now says
  "unlike".
- `CLAUDE.md`'s `integration/clickhouse/` repo-map row still ended
  "Manual/local gate, not CI", never mentioning `tests/lib-tests.sh`, which is
  the fifth command of the required gate. Someone editing
  `integration/clickhouse/lib/*.sh` would have concluded nothing there is
  CI-gated while in fact breaking `lib/leakscan.sh` turns `Required PR gate`
  red. The row now inventories `tests/lib-tests.sh` and distinguishes the
  manual Docker suite from the CI-run shell-library tests.
  `integration/clickhouse/README.md` gains the same one-sentence distinction.

New contract test, per the plan author's explicit amendment:

`internal/securitytest/pr_gate_contract_test.go` parses
`.github/workflows/pr-gate.yml` and asserts its structure against expectations
written as literals in the test source — unfiltered `pull_request` plus
`push.branches: [main]` and nothing else, no `pull_request_target`, no
`paths`/`paths-ignore` at any depth, top-level `permissions` exactly
`{contents: read}`, exactly one job with no `if`/`needs`/`continue-on-error`,
that job named exactly `Required PR gate`, the five contractual commands
exactly once each in order, and every external `uses:` pinned to a full 40-hex
commit SHA. No expectation is read out of the file under test, or the test
would compare the file to itself and detect nothing. The action pins are
checked for *immutability* (a 40-hex ref for a known action identity), not for
today's specific SHA values, so a legitimate action upgrade is not test churn.

It is deliberately untagged, matching `docs_contract_test.go`,
`redaction_inventory_test.go` and `sdk_contract_test.go` (`release_gate_test.go`
is the only `phase5release`-tagged file in the package). Untagged means it runs
in plain `go test ./...` and in the gate's own `go test -race ./...` step.

Its doc comment states plainly what it is not: an accidental-drift detector,
not a tamper-resistance mechanism. It lives in the same pull-request-controlled
tree as the workflow it inspects and runs only because that workflow runs it, so
a PR that neuters or skips the job also neuters or skips this test. Real tamper
resistance needs an enforcement authority outside the PR-controlled workflow
(an organization-level required-workflow ruleset). What it buys is that an
honest edit which quietly weakens the gate fails with a message naming the
consequence. All five sabotage cases (job rename, added path filter, added
continue-on-error, contents: write, unpinned checkout tag) were verified to fail
with such a message.

Remaining accuracy fixes:

- `CLAUDE.md` (two places) hedged the tense on branch protection: `main` is
  still unprotected, and protection is enabled as part of #23's rollout after
  this merges. The substance — the job name is an externally consumed interface
  — is unchanged.
- `CLAUDE.md` and `skills/ship/references/repo-footguns.md`: the two image
  publishers also carry `workflow_dispatch`, as `README.md` already stated.
- `repo-footguns.md` and `skills/ship/SKILL.md` disambiguated "the gate": the
  local gate is `go build && go vet && go test`; hosted CI's `Required PR gate`
  runs a superset, cross-referenced rather than duplicated.
- `CLAUDE.md` records two real limits of the gate's reach: `lib-tests.sh` is
  Docker-*daemon*-free but still needs the `docker` CLI on `PATH` (`lib/common.sh`
  calls `detect_compose_cmd` at source time and dies otherwise), and the gate
  never compiles `third_party/goldap`/`third_party/ldapserver`, which the root
  `go.mod` only `replace`s in.
- `pr-gate.yml` gains one comment: if a merge queue is ever enabled on `main`, a
  `merge_group:` trigger must be added or queue entries stall. The trigger
  itself is deliberately not added.

Verified: go build, go vet, go test -race ./... (includes the new untagged
test), go test -tags phase5release ./internal/securitytest -count=1,
bash integration/clickhouse/tests/lib-tests.sh,
go vet -tags phase5release ./internal/securitytest,
bash helm/ch-oauth-ldap/test.sh, and actionlint v1.7.7 on pr-gate.yml.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
…sts fail-safe

The new PR gate's first hosted Linux run (ubuntu-latest, linux/amd64, Go
1.26.7) failed at `go test -race ./...` in internal/ldap with two failures,
neither reproducible on darwin/arm64.

1. TestAdversarial_WriteDeadlineClosesStalledConnectionAndUnblocksGracefulShutdown
   drained 41,943,235 bytes — the entire ~40 MiB response — and then timed
   out on its own client-side read deadline. The server's write never
   blocked, so WriteTimeout never fired and the mechanism under test was
   never engaged. Real-socket backpressure cannot prove this property
   portably: how much a non-reading peer absorbs before the server blocks is
   host TCP buffer sizing (net.inet.tcp.sendspace/recvspace is 128 KiB on
   darwin, where a 20 MiB write blocks at once; the hosted runner buffered
   the whole thing). Any payload constant is a magic number racing the next
   host's tuning — and an expensive one under -race.

   The test now drives the real production server over an unbuffered
   net.Pipe behind a minimal one-connection net.Listener. net.Pipe is
   synchronous and completely unbuffered by construction, so a Write blocks
   until the peer reads: the instant the client stops reading, the server's
   flush is genuinely blocked with zero dependence on host tuning, and
   net.Pipe implements the deadline setters the vendored write path needs.
   Server.Serve takes a net.Listener and the vendored serve loop applies
   deadlines through net.Conn, so only the transport is substituted. A
   normal-sized Search response now suffices; the 20 MiB role,
   fullResponseFloor and truncatedCeiling are gone. What this gives up is
   end-to-end real-TCP backpressure, which was never this test's
   distinctive job — protocol_test.go and friends cover real TCP
   extensively, while writeMessage's SetWriteDeadline, the writer
   goroutine's close-on-write-error path and Stop()'s drain all live above
   the transport. The graceful-shutdown half is unchanged.

2. The ldapserver.Logger data race was a consequence of (1), not a
   production defect. The reader goroutine belonged to the failing
   write-deadline test and the writer was the next test's New(): a t.Fatalf
   skipped that test's own ln.Close/serveErr/Stop/cancel teardown, leaking a
   live server whose vendored client close() path then read the
   package-level Logger global concurrently with the next New() writing it —
   exactly the "at most one Server per process at a time" precondition
   server.go's type doc already states. So internal/ldap/server.go is
   deliberately unchanged: no init(), no sync.Once around the Logger
   assignment. New() re-asserting the logging hardening on every
   construction is intended, and that line is security-relevant and
   machine-checked by internal/securitytest/redaction_inventory_test.go.

   Instead, every hand-rolled listener + Serve goroutine in the package's
   tests now goes through a shared startServing helper that registers a
   sync.Once-guarded, bounded teardown with t.Cleanup before any assertion
   can abort — mirroring protocol_test.go's startTestServer, preserving its
   load-bearing ordering (close our own listener, synchronize on serveErr,
   only then Stop()). serveErr is closed after its single send so repeated
   receives are safe, which lets the tests whose subject IS graceful
   shutdown keep their explicit assertions and bounds while the cleanup
   becomes a no-op.

Verified: go build ./..., go vet ./..., go test -race -count=1 ./...,
go test -race -count=5 ./internal/ldap (no DATA RACE),
go test -tags phase5release ./internal/securitytest -count=1,
go vet -tags phase5release ./internal/securitytest, and
bash integration/clickhouse/tests/lib-tests.sh — all green. Sabotaging
srv.ldapSrv.WriteTimeout to 60s makes the rewritten test fail as intended
("server delivered 163 bytes of the stalled response after 2.5s of the
client reading nothing"), and that same failing run no longer produces the
Logger race in the test that follows it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
fb2f6e5 replaced this test's real-TCP, buffer-size-dependent stall with an
unbuffered net.Pipe, but PATCHES.md still described the old mechanism ("a
shrunk receive buffer plus a volume of ordinary, fast-completing Binds") —
which was already stale before that commit and is now doubly so.

Records the reason the substrate changed, since that reason is the load-bearing
part: whether a real socket write blocks at all depends on host TCP buffer
sizing, and the 20 MiB response that stalled immediately on macOS was
delivered in full on a hosted Linux runner without ever engaging WriteTimeout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
Temporary, removed by the immediately following revert commit in this same
PR. Issue #23 Definition-of-Done item 11 requires proving on real hosted CI
that a deliberately failing security test makes the required check red, so
the gate cannot be mistaken for advisory.

Tagged `phase5release` on purpose. An earlier accidental red run on this
branch already showed a `go test -race ./...` failure turning `Required PR
gate` red, but that says nothing about the security step, which never even
executed (the job is sequential, so later steps were skipped). This failure
is invisible to `go test -race ./...` and fails only
`go test -tags phase5release ./internal/securitytest -count=1`, isolating
that step's own wiring to the required status.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
Reverts e37e24d, whose only purpose was to prove on real hosted CI that a
failing security contract makes the required check red.

Proof captured before this revert (run 33277581488, head e37e24d):
  4. Build                        success
  5. Vet                          success
  6. Race tests                   success
  7. Phase 5 security tests       FAILURE   <- TestPRGateDeliberateFailureProof
  8. ClickHouse shell library     skipped
  job "Required PR gate"          FAILURE
  check-run context "Required PR gate" conclusion=failure

Steps 4-6 passing is what makes it a proof rather than a coincidence: the red
result is attributable to the security step alone. Issue #23 Definition-of-Done
item 11 satisfied.

Reverted with a normal inverse commit rather than a force-push, so the red run
and the commit that caused it stay in this PR's history as the audit trail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS

Copy link
Copy Markdown
Collaborator Author

ChatGPT review pass 1

Reviewed head: a11eac0b21468a77ef909170f202ca71ba9bd509

Findings

  1. Major — the new startServing fail-safe can still leak a server and recreate the races it was introduced to prevent.

    internal/ldap/protocol_test.go:startServing silently ignores both 15s timeout branches. If waiting for serveErr times out, it still calls srv.Stop() without the channel receive that this same helper documents as the load-bearing synchronization between Serve()'s unsynchronized Server.Listener write and Stop()'s read. If the subsequent srv.Stop() wait times out, the helper cancels and returns while that goroutine may still be live, allowing the next test to construct another server and potentially reproducing the cross-test ldapserver.Logger race from the first hosted run.

    There is also a concrete way for the first timeout path to be reached in the new test listener. internal/ldap/adversarial_test.go:singleConnListener says that after Close, every Accept returns an error, but Accept selects between a preloaded handout channel and the closed channel. If Close() happens before the first Accept(), both cases are ready and Go may return the connection after the listener is closed. I reproduced that exact Close-before-Accept sequence in isolation: 5,007 of 10,000 calls accepted the queued connection after Close returned.

    Fix: make singleConnListener deterministically honor Close-before-Accept (e.g. check closed state before consuming the handout, with appropriate synchronization), and make cleanup timeouts fail closed rather than silently return. In particular, do not call srv.Stop() until the Serve synchronization point has actually been observed, and do not let cleanup finish while the Stop goroutine is still unjoined. Mark the test failed on a bound breach, then preserve lifecycle ordering while completing teardown (the package-level test timeout can remain the ultimate hard bound).

  2. Medium — the newly mandatory gate is born on already-deprecated Action runtimes.

    The exact-head hosted run is green, but its log warns that both pinned actions target Node 20 and are being forcibly run on Node 24. The pinned manifests confirm actions/checkout@11d596... and actions/setup-go@40f158... both declare node20. Current upstream releases already provide native Node 24 variants: actions/checkout v7.0.1 (3d3c42e5aac5ba805825da76410c181273ba90b1) and actions/setup-go v7.0.0 (b7ad1dad31e06c5925ef5d2fc7ad053ef454303e). For an ordinary optional workflow this could be follow-up work; for the status that will become mandatory branch protection, avoidable runtime deprecation is an eventual repo-wide merge-outage risk.

    Fix: verify the current Node-24 major releases against this workflow, pin their exact SHAs, keep human-readable release comments, and rerun the exact-head gate. This does not require touching the existing publisher workflows.

  3. Minor — the canonical PR body no longer reflects the reviewed head or completed hosted proof.

    The PR is now at a11eac0..., but its Verification section still says the checks were coordinator-run at c8538af, and the deliberate hosted red run is still described in future tense. The red proof has in fact already happened (run 33277581488: build/vet/race passed, phase5 security failed, shell step skipped) and the restored head run 33277682239 is green through all five commands.

    Fix: refresh the PR description before merge so its audit trail names the current reviewed/restored head and the completed red/green run IDs rather than an obsolete intermediate state.

Focused checks

  • Canonical PR head re-read immediately before this review: a11eac0b21468a77ef909170f202ca71ba9bd509; 12 changed files, mergeable.
  • Run 33277682239 at this head reports one Required PR gate job and all five contractual commands succeeded; token permissions in the job log are contents: read plus GitHub's implicit metadata read.
  • Run 33277581488 verifies the deliberate phase5release failure propagates to the single required job.
  • I found no separate defect in the net.Pipe substitution itself: it still traverses the production Server.Serve/handler/write-deadline path while making backpressure deterministic. The blocking issue is the surrounding listener/cleanup lifecycle above.
  • The workflow contract test's core expectations are source literals rather than values derived from the YAML under test; I found no self-reference defect in that mechanism.

VERDICT: REVISE

BorisTyshkevich and others added 2 commits August 30, 2026 00:23
ChatGPT review of PR #34 flagged that pr-gate.yml pinned actions/checkout@v4.4.0
and actions/setup-go@v5.6.0, both still declared `using: node20` in their
action.yml even though the hosted run only succeeds today because GitHub
runners re-execute node20-declared actions on Node 24 as a temporary
compatibility shim (Node 20 itself leaves runner images on 2026-09-23).
Native node24 releases were already available for both actions.

Re-pinned to actions/checkout@v7.0.1 (3d3c42e5aac5ba805825da76410c181273ba90b1)
and actions/setup-go@v7.0.0 (b7ad1dad31e06c5925ef5d2fc7ad053ef454303e) — both
SHAs verified against the real actions/checkout and actions/setup-go tag refs,
and both actions' action.yml confirmed to declare `using: node24` at those
commits. Neither upgrade changes this workflow's actual usage (checkout's
`persist-credentials: false`, setup-go's `go-version-file`/`cache: false`).

Added TestPRGateContract_ActionPinsAreAtOrAboveTheNode24Floor to
internal/securitytest/pr_gate_contract_test.go so a future re-pin back below
the node24 floor fails the gate's own contract test instead of silently
riding the runner's compatibility shim again. Since the invariant this file
enforces elsewhere is SHA immutability (no version readable from the pin
itself), the new test reads each pin's trailing `# actions/<name>@vX.Y.Z`
comment and compares it against a literal per-action floor
(checkout >= v6.0.0, setup-go >= v7.0.0). Verified with a sabotage case:
downgrading the checkout pin's comment to v4.4.0 makes the new test fail with
the expected message; restored and re-verified green afterward.

The review's second finding (PR #34's body is stale relative to its actual
head) is out of scope for this change — it is a GitHub PR description edit,
not a repository file, and outside this change's mutation boundary (no `gh`
mutations). Left for the coordinator to update directly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
Review pass 1 raised both of these and the verification pass rejected them
as unreachable. That is correct about today's call sites and beside the point
now that `startServing` and `singleConnListener` are shared infrastructure
behind seven server-start sites: both were silent fail-opens whose contract
held only because of how their one caller happened to be sequenced.

singleConnListener.Accept selected over `handout` and `closed` in a single
select. When several cases are ready Go picks one uniformly at random, so
Close() followed by Accept() returned the queued connection roughly half the
time — measured 5007/10000 before this change, 0/10000 after — violating
net.Listener's contract that Accept fails once Close has been called.
Accept now checks closure first in its own non-blocking select.

startServing's two 15s teardown timeouts had empty bodies. A timeout on the
first meant proceeding to srv.Stop() without the serveErr synchronization
point the helper's own doc comment calls load-bearing — committing the exact
Stop-racing-Serve data race that ordering exists to prevent — and a timeout
on the second meant returning with a live Stop goroutine still able to touch
the vendored server during a later test's New(). Both now t.Errorf, so a
wedged Serve or Stop fails the test that caused it instead of surfacing later
as an unexplained race somewhere unrelated.

Neither branch executes unless something has already gone wrong, so no
passing test changes behavior.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS

Copy link
Copy Markdown
Collaborator Author

ChatGPT review pass 2

Previously reviewed head: a11eac0b21468a77ef909170f202ca71ba9bd509

Reviewed head: 30c403056488fd67dd089d8ec8e9d37e8961bae1

Reassessment of pass 1

  • Node 20 action pins: resolved in the actual workflow. The new pins are native-Node-24 releases (actions/checkout@v7.0.1, actions/setup-go@v7.0.0), and exact-head run 33278654950 completed all five contractual steps successfully with no old Node-20 compatibility warning.
  • Stale PR body: resolved for the substantive audit trail. It now names 30c4030 and records the deliberate phase5release red proof as completed. The hosted-run table could name run 33278654950 explicitly, but that is polish, not a merge blocker.
  • Shared LDAP teardown/listener finding: only partially resolved. The sequential Close(); Accept() defect is fixed, but the timeout fix still executes the unsafe path after reporting it.

Findings

  1. Major — startServing still proceeds unsafely after either teardown timeout.

    internal/ldap/protocol_test.go:startServing now calls t.Errorf on the two 15s bound breaches, but Errorf marks the test failed and continues execution. Therefore a serveErr timeout is immediately followed by launching srv.Stop() even though the helper's own documented synchronization point was not observed — exactly the Stop-racing-Serve path the helper says is a data race. Likewise, if the Stop wait times out, the helper reports the error, cancels, and returns while the Stop goroutine can still be live; that preserves the cross-test contamination mechanism from pass 1.

    Calling t.Errorf from cleanup is fine; the problem is continuation after it. The fix should fail the test and preserve the lifecycle invariant: after the first bound breach, cancel if useful but do not call Stop() until serveErr has actually been received; after the Stop bound breach, cancel and then join the Stop goroutine before cleanup returns. Let the package/test-binary timeout be the ultimate hard bound if teardown is truly unrecoverable. That prevents a broken cleanup from advancing to the next test with live server state.

  2. Medium — the new Node-24 floor test does not bind the claimed version/runtime to the immutable SHA.

    TestPRGateContract_ActionPinsAreAtOrAboveTheNode24Floor derives the action identity from uses: but derives the version only from uses.LineComment. An old Node-20 SHA can therefore pass both contract tests if its trailing comment is left or changed to a high version, e.g. the old checkout SHA 11d5960a326750d5838078e36cf38b85af677262 with # actions/checkout@v7.0.1: it is still a full 40-hex pin, and the floor test sees only v7.0.1. The regex also does not bind the comment's action name to the uses: identity.

    So the actual v7 upgrade is good, but this new test does not machine-enforce the runtime property it claims. For an offline test, the sound options are either (a) keep a small literal allowlist/map of approved action identity -> SHA -> version/runtime and accept intentional test churn on action upgrades, or (b) remove/rename this as comment-hygiene rather than runtime enforcement. Also, if retaining a semantic floor, actions/checkout v5.0.0 already declares using: node24; v6.0.0 is not the first Node-24 release. actions/setup-go v6.0.0 is still Node 20, so its v7 floor is correct.

  3. Medium — the workflow contract still has two concrete false negatives against invariants it says it enforces.

    • TestPRGateContract_TopLevelPermissionsAreReadOnly checks only the top-level permissions: map. GitHub also permits jobs.<job_id>.permissions; adding permissions: {contents: write} to required-pr-gate would give that job a different token permission set while this test still passes. If the invariant is "PR code receives only contents: read", reject job-level permissions entirely (or validate it as exactly read-only too).
    • TestPRGateContract_FiveCommandsRunExactlyOnceInOrder uses strings.Count over each run: scalar. A step such as run: echo 'go build ./...' or a comment containing the command satisfies the substring check without executing the command. The current workflow uses one-line commands, so the simpler and stronger contract is to require the normalized run: values to equal the five literal commands in order.

    These do not make the current pr-gate.yml unsafe; the current workflow is correctly read-only and executes the five real commands. They do mean the new accidental-drift detector is weaker than its stated security contract.

Focused checks

  • Canonical head was re-read immediately before this review: 30c403056488fd67dd089d8ec8e9d37e8961bae1, 8 commits / 12 changed files, mergeable.
  • The delta from a11eac0... is exactly two commits: 8e6e1be... and 30c4030....
  • Exact-head run 33278654950 is green through Build, Vet, Race tests, Phase 5 security tests, and ClickHouse shell-library tests; its job token is contents: read plus GitHub's implicit metadata read.
  • The upgraded action SHAs resolve to the stated v7 tags, and both pinned action.yml files declare node24.
  • The extra closure pre-check in singleConnListener.Accept fixes the pass-1 sequential Close-before-Accept case. I found no current-call-site regression from that extra select.
  • t.Errorf in a t.Cleanup-registered closure is valid; the defect is that Errorf deliberately continues, so the unsafe cleanup state transition still occurs.
  • main remains unprotected and Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 remains blocked, which is consistent with ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23's post-merge rollout: DoD 10 and 12 are not satisfied merely by this PR being green. The deliberate-red DoD item is already evidenced in PR history.

VERDICT: REVISE

BorisTyshkevich and others added 2 commits August 30, 2026 00:51
Close four ChatGPT pass-2 findings against ci/pr-gate:

- internal/ldap/protocol_test.go: startServing's stop() fell through to
  srv.Stop() after a serveErr fail-safe timeout because t.Errorf (Logf+Fail)
  does not halt execution — exactly the unsynchronized Stop-racing-Serve
  data race the ordering exists to prevent. Extracted the body into
  runShutdownSequence, which now returns immediately after reporting a
  serveErr timeout instead of proceeding to Stop().

- internal/securitytest/pr_gate_contract_test.go:
  - TestPRGateContract_ActionPinsAreAtOrAboveTheNode24Floor now cross-checks
    a pin's trailing version comment identity against its real `uses:`
    identity (a stale/mismatched comment no longer parses a passing
    version out of the wrong action's text), and corrects the
    actions/checkout node24 floor to v5.0.0 (the actual first-node24
    release; v6.0.0 was never it).
  - TestPRGateContract_TopLevelPermissionsAreReadOnly now also checks the
    job's own `permissions:` block, since GitHub Actions job-level
    permissions replace rather than merge with the workflow-level grant.
  - TestPRGateContract_FiveCommandsRunExactlyOnceInOrder now reduces each
    run: step to the single command it actually executes (prGateScriptCommand)
    and matches exactly, rather than substring-counting — a step like
    `run: echo 'go build ./...'` no longer satisfies the contract.

Extracted the changed logic into small pure functions (runShutdownSequence,
pinVersionComment, checkActionNode24Floor, permissionsMapping/Match,
prGateScriptCommand) so each fix has a direct unit test exercising the
exact bypass it closes, in addition to the existing contract tests still
passing against the real files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
Pass 2 corrected the actions/checkout floor from v6.0.0 to v5.0.0 after
finding the cited "first node24 release" was wrong. The same class of error
remained for the other action: the comment claimed "setup-go v6.x is still
node20" and set the floor at v7.0.0.

Verified by fetching each action.yml at the tag boundary:
  actions/checkout  v4.3.1 node20 -> v5.0.0 node24   (floor v5.0.0 correct)
  actions/setup-go  v6.1.0 node20 -> v6.2.0 node24   (floor was v7.0.0)

setup-go's node24 line starts mid-v6, not at the major bump. The old floor
was safe (stricter than necessary) but its stated justification was false,
and it would have rejected a legitimate node24 pin at v6.2.x while claiming
that release was node20. Floor is now the true boundary and the comment
records how it was determined, so the next person to touch a pin can check
it rather than trust it.

The workflow's own pin stays at setup-go v7.0.0, comfortably above.

Sabotage-verified at this head, each restoring the workflow byte-identical:
  job-level `permissions: contents: write`  -> job-level permissions check fails
  comment identity swapped to another action -> identity cross-check fails
  setup-go comment set to v6.1.0            -> node24 floor fails, naming v6.2.0

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS

Copy link
Copy Markdown
Collaborator Author

ChatGPT review pass 3

Previously reviewed head: 30c403056488fd67dd089d8ec8e9d37e8961bae1

Reviewed head: e82c74988930d8c73fdfb0d2ef128a031dca7cfe

Reassessment of earlier findings

  • Native Node 24 action pins: resolved in the actual workflow. The current checkout/setup-go SHAs both declare node24, and exact-head run 33279680085 is green through all five required commands.
  • Job-level permission override: resolved. The contract now checks a job-level permissions: block independently and accepts only absence or the same exact {contents: read} mapping.
  • Substring command matching: resolved. run: scripts are reduced and exact-matched against the five literal commands; the old echo 'go build ./...' bypass is closed.
  • Action floor boundaries: the literal boundaries are now factually correct (checkout v5.0.0, setup-go v6.2.0), but the SHA-binding defect from pass 2 remains.
  • Shared teardown: improved, but the pass-2 lifecycle finding is not fully closed; timeout paths can still return while the prior server lifecycle is live.

Findings

  1. Major — a PR base-branch retarget can reuse a green Required PR gate that tested the old merge result.

    .github/workflows/pr-gate.yml uses bare pull_request:. GitHub's default pull_request activity types are only opened, synchronize, and reopened; changing the PR base is an edited activity. Therefore retargeting a PR does not start this workflow unless the head also changes.

    This matters for the exact enforcement model here. The hosted run checks out and tests the synthetic merge ref (df02cfe... for this head/base), but GitHub records Required PR gate on the PR head SHA e82c749...; there is no check run on that synthetic merge SHA. Required-status evaluation falls back to the latest head commit when the test merge commit has no status. A PR can therefore be tested against base A, retain the same head SHA, be retargeted to protected main, and still carry the successful required check even though the merge result against main was never exercised.

    Fix: explicitly include the base-change activity: pull_request.types: [opened, synchronize, reopened, edited], and update the contract test to require exactly that activity set while still forbidding path/base-branch filters. edited also fires for title/body edits, but this gate is deliberately cheap; an extra full run is safer than introducing a conditional job/step that can create another skipped-success path.

  2. Major — runShutdownSequence can still return from cleanup with live old-server state.

    The first timeout now correctly avoids the immediate unsynchronized Stop() call, but it reports, cancels, and returns while the Serve goroutine is still live. cancel() only cancels the LDAP handler root context; it is not a substitute for observing Serve terminate. The vendored accept loop can still touch package-global ldapserver.Logger when it eventually returns, while the next test's New() unconditionally reassigns that same global.

    The second timeout is even more direct: after Stop() exceeds the bound, the helper reports, cancels, and returns while the Stop goroutine may still be live. Stop() itself touches ldapserver.Logger after closing the listener and while/after waiting on clients. The new unit test demonstrates this lifecycle explicitly: after runShutdownSequence returns from the simulated Stop timeout, it has to close(block) itself to let the leaked Stop goroutine finish before the test exits. A focused standalone reproduction of the exact helper behavior likewise confirms the helper returns while the fake Stop is still blocked.

    This will already make the test red, so it is not a gate-green bypass; the problem is cross-test isolation and race attribution. It preserves the mechanism that originally made one failed test surface as a ldapserver.Logger race in the next test.

    Fix: a timeout may mark the test failed, but cleanup must not allow the next test to start until this server lifecycle has actually quiesced. On Serve timeout, cancel if useful and then wait for the real serveErr synchronization before any Stop() call. On Stop timeout, cancel and then join the stopped channel before returning. If teardown is irrecoverably wedged, let the package/test-binary timeout be the ultimate fail-stop rather than continuing with a second Server in the same process.

  3. Medium — the Node-24 floor test still does not bind its version/runtime claim to the immutable SHA.

    The new regex correctly binds the trailing comment's identity to the uses: identity, and both corrected version floors are accurate. But checkActionNode24Floor still receives only (identity, lineComment); it never receives the SHA. Thus this still passes the two contract checks:

    uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # actions/checkout@v7.0.1

    That SHA is the old Node-20 pin. It is a valid 40-hex immutable ref, and the floor test sees a same-identity v7.0.1 comment, so it passes. The current workflow itself is safe — its actual v7 SHAs are Node 24 — but the test still does not machine-enforce the runtime property its name/documentation claim.

    Fix: simplest is to remove/rename this extra floor test as comment hygiene; issue ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23 requires immutable pins, not an offline runtime oracle. If machine enforcement is worth keeping, use a small literal identity -> approved SHA -> known version/runtime mapping and accept intentional test churn when these security-sensitive actions are upgraded. Also reconcile the file-header comment, which still states the obsolete checkout >= v6.0.0 / setup-go >= v7.0.0 floors while the actual map now correctly uses v5.0.0 / v6.2.0.

  4. Minor — the canonical PR description is stale again.

    It still says verification was run at 30c4030 and its hosted-CI table labels 30c4030 as (current), while the canonical head is now e82c749... and run 33279680085 is the current exact-head green proof. Refresh that audit trail before merge.

Focused checks

  • Canonical head re-read immediately before this comment: e82c74988930d8c73fdfb0d2ef128a031dca7cfe; previous reviewed head 30c403056488fd67dd089d8ec8e9d37e8961bae1; the delta is exactly two commits and only internal/ldap/protocol_test.go plus internal/securitytest/pr_gate_contract_test.go changed.
  • Exact-head hosted run 33279680085 has one Required PR gate job and Build, Vet, Race tests, Phase 5 security tests, and ClickHouse shell-library tests all succeeded.
  • I independently rechecked the runtime boundaries: checkout v4.3.1 is Node 20 and v5.0.0 is Node 24; setup-go v6.1.0 is Node 20 and v6.2.0 is Node 24. The current v7 pinned SHAs both declare Node 24.
  • I found no regression in the net.Pipe write-deadline substitution or the fixed singleConnListener Close-before-Accept behavior.
  • main remains unprotected, which is still consistent with the stated post-merge rollout for DoD 10; it is not the reason for this REVISE verdict.

VERDICT: REVISE

BorisTyshkevich and others added 2 commits August 30, 2026 01:13
Close the base-branch-retarget fail-open path in the PR gate: a bare
`pull_request:` trigger's default activity set (opened/synchronize/
reopened) never re-fires on a retarget, which GitHub delivers as
`edited` on the same head SHA — so a PR proven green against base A
could be retargeted onto protected `main` and keep that same green
required check with the new merge result never tested. Add
`pull_request.types: [opened, synchronize, reopened, edited]` to
pr-gate.yml and update pr_gate_contract_test.go's trigger contract to
require that exact `types` list (previously it forbade any
`pull_request` configuration on principle).

Also fix stale node24-floor version numbers in the contract test's
header comment (v6.0.0/v7.0.0) to match the real, already-corrected
prGateNode24FloorByAction map (v5.0.0/v6.2.0) it sits above.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
… node24 check honestly

Post-cap architecture consultation returned REVISE on two test-quality points
beyond its main branch-protection finding. Both are addressed here.

1. startServing's teardown no longer bounds each step with a 15s fail-safe.

That machinery was introduced during THIS PR and is what created the problem:
on timeout it reported via t.Errorf and returned, releasing the next test
while the previous server was still live — which is precisely how a wedged
Serve becomes an inexplicable ldapserver.Logger race in some unrelated test
later in the run, since that global is reassigned by every New(). It traded a
bounded hang for silent cross-test contamination.

Teardown is now `ln.Close(); <-serveErr; srv.Stop(); cancel()` — unbounded,
identical to what startTestServer used unchanged throughout the #19 series and
what origin/main still has. A teardown bug must fail-stop this process rather
than hand the next test a contaminated one; if Serve or Stop genuinely wedges,
`go test`'s own timeout panics with a full goroutine dump, which is strictly
better diagnostics than an Errorf and lands well inside pr-gate.yml's
15-minute job bound.

Removes failSafeShutdownTimeout, the errorfer seam, runShutdownSequence,
fakeErrorfT, and the three TestStartServing_* tests whose only subject was the
timeout branches that no longer exist. Net -180 lines.

2. TestPRGateContract_ActionPinsAreAtOrAboveTheNode24Floor now states its
scope limit instead of implying more than it proves.

It verifies the trailing version COMMENT, not the pin: a commit SHA cannot be
resolved to a version, let alone to the runtime its action.yml declares,
without network access a unit test must not have. So a regressed node20 SHA
carrying a plausible above-floor comment for the same action still passes.
That gap is accepted deliberately — an approved identity/SHA/runtime table
would make the binding real but converts every legitimate upgrade into
unrelated churn and re-couples to exact SHAs, which is what pinning for
immutability deliberately avoids (immutability itself is proven independently
and completely by ExternalActionsArePinnedToFullCommitSHA). The check is
version-comment hygiene with a floor: useful against the realistic accident,
honestly not a runtime oracle.

The consultation's main finding — branch protection must be strict/up-to-date
rather than loose — is a rollout-configuration change, recorded in the ship
log and applied when protection is enabled after merge.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS

Copy link
Copy Markdown
Collaborator Author

ChatGPT review pass 1

Reviewed head: 67e1fb7cc6bda3c3ca93372da35072bfb5e7730d

Findings

  1. Major — the strict: true rollout correction is not actually present in the canonical ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23 rollout record.

    The re-certification handoff and the new 67e1fb7 commit both say the post-cap branch-protection finding was accepted and recorded, with protection to be configured strict/up-to-date. But the current canonical ship-log comment on issue ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23 still says:

    enforce_admins: true, strict/up-to-date false, no review-count requirement.

    Its current GitHub updated_at also predates this correction. This is not merely stale prose: that ship log is the recorded post-merge procedure. If followed literally, it recreates the base-advancement fail-open from the architecture review. GitHub's strict mode is the control that requires a PR to be tested with the latest protected-branch code; loose mode explicitly permits merging without bringing the branch up to date.

    The post-merge bootstrap ordering itself is acceptable: the workflow has to land and emit its stable check before classic protection can require it, and Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 remains blocked until enforcement is active. The problem is that the authoritative rollout instruction still specifies the unsafe setting.

    Fix: before certification/merge, correct the canonical ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23 rollout record to require strict: true, enforce_admins: true, the exact Required PR gate context, and the expected GitHub Actions App source. After merge, observe the push-event run on the resulting main SHA, apply protection, and read back all of those fields before treating Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 as unblocked.

  2. Medium — the Node-24 check is now honestly scoped in prose, but its public/test surface still overclaims what it proves.

    The latest code explicitly states the right limitation: TestPRGateContract_ActionPinsAreAtOrAboveTheNode24Floor checks the trailing human-readable version comment, not the pinned SHA's actual release/runtime, and a Node-20 SHA paired with a plausible same-action above-floor comment still passes. I agree with the simpler resolution: issue ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23 requires immutable action pins, and an approved SHA/runtime table is not necessary just to satisfy that contract.

    However, the test is still named ActionPinsAreAtOrAboveTheNode24Floor, its diagnostics describe the comment-derived version as the action being "pinned to v...", and the canonical PR body still says the test "now enforces the floor." Those surfaces continue to communicate SHA/runtime enforcement that the implementation deliberately does not provide.

    Fix: rename the test/helper/diagnostics around what is actually checked (for example, ActionVersionCommentsAreAtOrAboveNode24Floor, with messages saying the version comment is below the floor), and update the PR description to call this comment hygiene rather than runtime enforcement. Keep the independent 40-hex immutability check as the actual pin contract.

  3. Minor — the canonical PR verification/audit section is stale at 22b9aad.

    The PR is currently at 67e1fb7cc6bda3c3ca93372da35072bfb5e7730d, and hosted run 33281026583 is green through Build, Vet, Race tests, Phase 5 security tests, and ClickHouse shell-library tests. The PR body still says coordinator verification was at 22b9aad and labels that older head as current.

    Fix: refresh the PR description before merge so the audit trail names the exact reviewed head and current hosted run.

Focused checks

  • Re-read canonical head immediately before posting: 67e1fb7cc6bda3c3ca93372da35072bfb5e7730d; PR is open and mergeable, 12 changed files.
  • Delta from 22b9aadce354a6c94b7cd5487a5b7f6a1993187b is exactly one commit touching only internal/ldap/protocol_test.go and internal/securitytest/pr_gate_contract_test.go.
  • The teardown correction is sound: startServing now preserves the load-bearing ln.Close() -> <-serveErr -> srv.Stop() -> cancel() ordering under sync.Once/t.Cleanup, and no timeout branch can release the next test with old server state still live. If shutdown truly wedges, the test process/job timeout is the fail-stop boundary.
  • Exact-head run 33281026583 has one Required PR gate job and all five contractual commands succeeded; token permissions in the log are contents: read plus GitHub's implicit metadata read.
  • I inspected the complete current 12-file PR, not just the latest two-file delta, and found no additional confirmed workflow fail-open beyond the still-uncorrected strict:false rollout instruction.
  • A local clone/package rerun was not feasible in this sandbox because outbound DNS could not resolve github.com; the canonical GitHub fetch path and exact-head hosted Linux gate were used instead.

VERDICT: REVISE

Rename the node24-floor pin check's test, helper, and diagnostics away
from "ActionPins"/"checkActionNode24Floor" to
"ActionPinVersionComments"/"checkVersionCommentNode24Floor" naming, and
reword the below-floor t.Errorf to say the version *comment* claims a
version rather than describing the pin itself as "pinned to" that
version. The check only ever validated the human-readable trailing
comment (a commit SHA cannot be resolved to a version or declared
runtime offline, per its own documented scope limit) — the prior
names and message overstated that as verifying the pin. The separate
TestPRGateContract_ExternalActionsArePinnedToFullCommitSHA remains the
real immutable-pin guarantee, unchanged.

The other two accepted pass-1 findings (issue #23's ship-log rollout
record still specifying strict:false; PR #34's description citing a
stale head/verification audit trail) are ship-log/PR-body edits, not
code changes, and are out of scope for this branch's mutation boundary
(no issue or ship-log edits, no gh mutations) — left for the
coordinator to apply directly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS

Copy link
Copy Markdown
Collaborator Author

ChatGPT review pass 2

Previously reviewed head: 67e1fb7cc6bda3c3ca93372da35072bfb5e7730d

Reviewed head: 56b253279de89bfe9cf583c7673558d94891ab7c

Reassessment of pass 1

  • Major — canonical rollout still specified strict: false: resolved. Issue ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23's canonical ship log now contains an explicit authoritative rollout table requiring exact context Required PR gate (app github-actions), strict: true, enforce_admins: true, and no review-count requirement, with read-back instructions. main is still unprotected today, but that is the documented bootstrap state: Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 remains blocked until this PR is merged, a green push-event gate is observed on main, and protection is applied/read back. I do not consider the post-merge ordering itself a PR defect.
  • Medium — Node-24 check name/diagnostic overclaimed SHA/runtime enforcement: resolved. The only code delta since 67e1fb7 renames the test/helper around version comments and changes the below-floor diagnostic from "pinned to v..." to "version comment claims v...". The PR body now states the same limitation and keeps immutable-SHA validation separate. I do not require an identity→SHA→runtime table for ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23: the issue requires immutable action pins; the comment-floor check is optional hygiene as long as it is honestly scoped. The documented floors are factually consistent with the action manifests (checkout v5.0.0 is node24; setup-go v6.1.0 is node20 and v6.2.0 is node24).
  • Minor — stale PR verification/audit head: resolved. The canonical PR body now records verification at 56b2532 and marks that head current.

Finding

  1. Minor, non-blocking — one stale sentence survived the Node-24 rename.

    In internal/securitytest/pr_gate_contract_test.go, the scope-limit comment still says:

    SCOPE LIMIT, stated plainly because the test's name promises more than it can deliver

    That was true before the rename; it is no longer true for TestPRGateContract_ActionPinVersionCommentsAreAtOrAboveTheNode24Floor, whose name now accurately describes a comment check. Reword this to state the scope limit without claiming the current test name overpromises. Separately, the PR body's heading Hosted CI — five runs now precedes more than five historical runs; that is similarly harmless audit-text drift.

    Neither item weakens enforcement or changes the ship decision.

Focused checks

  • Delta 67e1fb7cc6bda3c3ca93372da35072bfb5e7730d...56b253279de89bfe9cf583c7673558d94891ab7c is exactly one commit and one file: internal/securitytest/pr_gate_contract_test.go, 30 insertions / 30 deletions. The changes are naming/diagnostic wording; gate logic is unchanged.
  • I reused the complete pass-1 review of the 12-file PR and rechecked the current complete scope. The other 11 changed files are unchanged since the previous reviewed head; I found no new regression in them.
  • The workflow still has one unconditional Required PR gate job, read-only permissions, immutable action SHAs, explicit edited PR activity, and the five contractual commands sequentially.
  • Exact-head runs 33281621345 and the later same-head rerun 33281711455 both completed successfully; each Required PR gate job passed Build, Vet, Race tests, Phase 5 security tests, and ClickHouse shell-library tests.
  • main is still unprotected pre-merge, and Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 is still open with the explicit precondition that ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23/equivalent be active and enforced before production parser replacement code merges. Those are post-merge rollout obligations, not evidence that this PR's gate can currently fail open.
  • I found no remaining confirmed merge-gate fail-open in the updated PR.

VERDICT: SHIP

Reword the stale SCOPE LIMIT comment on
TestPRGateContract_ActionPinVersionCommentsAreAtOrAboveTheNode24Floor: it
still framed the limitation as "the test's name promises more than it
can deliver", leftover from before the same commit's rename from
TestPRGateContract_ActionPinsAreAtOrAboveTheNode24Floor. The current
name already scopes itself to version comments, so state the scope
limit directly without referencing the old name.

(PR #34's stale "five runs" body heading is a text-only PR-description
fix outside this commit's code-only mutation boundary — left for a
follow-up gh pr edit, as in the prior pass.)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS

Copy link
Copy Markdown
Collaborator Author

ChatGPT review pass 3

Previously reviewed head: 56b253279de89bfe9cf583c7673558d94891ab7c

Reviewed head: 8a0dc6ae1f4e13216f5f41be2db975085dcc7b21

Reassessment of pass 2

  • Minor — stale Node-24 scope sentence: resolved. The only code delta since 56b2532 rewrites the scope-limit comment to state directly that the check verifies the version COMMENT, not the SHA/runtime. The current test/helper/diagnostic names remain accurately scoped to version-comment hygiene.
  • Minor — stale Hosted CI — five runs heading: resolved. The canonical PR body now uses Hosted CI — every run, each keyed to its exact head and records coordinator verification at 8a0dc6a.
  • All earlier substantive findings remain resolved: the authoritative ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23 rollout requires strict: true; startServing preserves ordered fail-stop teardown; the immutable-action check is separate from comment-floor hygiene; the workflow includes edited; permissions and five commands remain fail-closed and exact.

Findings

  1. Minor, non-blocking — the current-head hosted-run row is already stale again.

    The PR body currently ends its hosted-run table with:

    (pending) | 8a0dc6a | comment/heading wording — current head

    but exact-head runs 33282032417 and 33282035937 have both completed successfully. This does not affect enforcement, but if the table is intended as the audit record, replace the pending row with the completed run(s) before merge.

  2. Minor, non-blocking — the contract-test header uses past tense for a future Node-20 removal date.

    internal/securitytest/pr_gate_contract_test.go says GitHub Actions runners stopped shipping Node 20 on 2026-09-23. As of August 30, 2026, GitHub's current changelog schedules runner Node-20 removal for September 23, 2026; the date is correct, but stopped is premature. Use future tense such as will remove Node 20 on 2026-09-23 / will stop including Node 20....

    This is documentation only and does not change the Node-24 pins or the ship decision.

Focused checks

VERDICT: SHIP

BorisTyshkevich and others added 4 commits August 30, 2026 02:05
The contract test's header comment read "GitHub Actions runners stopped
shipping Node 20 on 2026-09-23" — past tense for a date roughly three weeks
ahead. A reader before that date could reasonably conclude the removal had
already happened, and therefore that the node24 floor was either moot or
already breached; the opposite is true, which is exactly why the floor exists.
The two other references to the same date (lines ~165 and in the diagnostic
itself) already read "is withdrawn ... on", so only this one was wrong.

Review declined this as a purely cosmetic nit on non-machine-checked prose,
which is fair on impact. Fixed anyway: several rounds of review on this PR
turned on comments and records claiming something that was not true — a
version floor justified by a wrong first-node24 release, a scope limit the
test's own name contradicted, a rollout table asserting a setting nobody had
recorded. Leaving a knowingly-wrong tense in a deadline comment would be
inconsistent with all of that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
…UB_TOKEN retarget limit

Second post-cap consultation found no code-level bypass at 7006c60 but two
real accuracy defects. Both are the same class this PR keeps producing: a
comment asserting something that is not true.

1. The contract test claimed a `paths:` filter lets a pull request "merge
unverified". That is backwards. GitHub documents that when a workflow is
skipped by path or branch filtering, its associated required check stays
*Pending* and blocks merging — so a path filter fails CLOSED. The genuinely
fail-open case is a skipped JOB, which GitHub reports as *Success* and which
therefore satisfies a required check; that is a different mechanism, already
covered by TestPRGateContract_JobIsUnconditionalAndFailsClosed. Verified
against GitHub's required-status-checks troubleshooting docs.

The no-path-filters policy is unchanged and still correct, but for the honest
reasons: universal coverage, no deadlocked PRs whose stuck required check
invites someone to drop the requirement, and no judgement calls about which
paths "matter" when a shared file, a dependency bump or a generated fixture
moves the LDAP/security boundary in a non-obvious way. The header comment now
separates all three skip modes — filtered workflow (pending, blocks), skipped
job (success, satisfies), narrowed `types` (no run, stale check still
satisfies) — because conflating them is what produced the wrong claim.

2. `edited` closes the retarget gap only for base changes whose events
actually start Actions. Events produced with the repository's own
GITHUB_TOKEN do not, so automation retargeting a PR base that way would
bypass the new trigger at the event-delivery layer. Recorded as a constraint
in the workflow itself, deliberately without a contract test: a YAML check
cannot observe it, and pretending otherwise would be another false assurance.
`strict: true` narrows the reachable cases but is not a complete substitute,
since an unchanged head can already be up to date with the new base.

Issue #23's normative "Required workflow" snippet still showed a bare
`pull_request:`; it now carries the explicit four activity types with the
rationale, since a future implementation working from that section would
otherwise faithfully reproduce the unsafe trigger.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
…iagnostics

Third post-cap consultation again found no code-level bypass, and again found
this file claiming things that are not true. Both are refinements of the
previous round's own fixes.

1. The GITHUB_TOKEN statement was too broad. "Events produced with
GITHUB_TOKEN do not start new workflow runs" is the general rule, but it has
documented exceptions: workflow_dispatch and repository_dispatch always create
runs, and pull_request with opened/synchronize/reopened does create runs — in
an APPROVAL-REQUIRED state. `edited` is not an exception, which is the part
that actually matters here.

The previous wording also implied automation could retarget safely if it "also
produces a real synchronize". A GITHUB_TOKEN-generated synchronize yields an
approval-gated run, not silent revalidation, so an operator reading that could
expect automatic re-verification that does not happen. Both cases fail closed;
neither revalidates automatically. Narrowed in the workflow and in issue #23.

The deliberate absence of a contract test for this was confirmed correct: a
structural YAML check cannot observe an event-delivery property.

2. The path-filter correction was applied only to the flagged instance. Two
sibling diagnostics kept the identical fail-open misconception:

  - missing workflow: "nothing verifies a pull request before it merges to main"
  - unparseable YAML: "does not run, so PRs would merge unverified"

Neither is true once `Required PR gate` is required: an unreported required
check blocks the merge, and GitHub generates a FAILED run for an invalid
workflow file. Both now say the gate becomes unavailable and merges stall or
fail closed, and name the real hazard — dropping the requirement to unblock a
stuck queue is the separate action that permits unverified merging.

I swept every remaining diagnostic in the file rather than only the two named.
The other three sites are already accurate and deliberately unchanged: the job
rename comment and diagnostic both say merges "stall (or, if protection is
dropped instead, merge unverified)", and the five-commands diagnostic's
"no longer gates merges" is correct — a dropped command leaves the job running
and reporting green, which genuinely does fail open.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
Fourth post-cap consultation caught a material misclassification — and a
self-contradiction in the previous commit's own comment.

That comment said the GITHUB_TOKEN `edited` case and the GITHUB_TOKEN
`synchronize` case "both fail closed", then two sentences later correctly
noted that `strict: true` is "not a complete substitute, since an unchanged
head can already be up to date with the newly selected base". Both cannot be
true, and the second one is right.

The retarget case is FAIL-OPEN, and it is precisely the path the `types` list
was added to close:

  1. head H already has a successful `Required PR gate`
  2. automation retargets the base to `main` using GITHUB_TOKEN
  3. `pull_request.edited` is produced but creates no workflow run
  4. head SHA is still H — nothing invalidates the existing check
  5. H already contains the current `main` tip, so `strict: true` is already
     satisfied and demands no fresh run
  6. the old green check on H satisfies branch protection, and the untested
     new merge result merges

So "automation must not retarget with GITHUB_TOKEN" is a real prohibition
rather than a stylistic preference, and saying otherwise would mislead an
operator into treating an unsafe automation as safe. The `synchronize` case
genuinely does fail closed — it creates an approval-required run — but it is
not silent revalidation either.

Corrected in both the workflow and issue #23, with the two cases split
explicitly rather than lumped together. The deliberate absence of a contract
test remains correct and confirmed: this is an event-delivery and
branch-protection property that a structural YAML check cannot establish.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KxntXZk6BFoz3uT3wFtiS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant