Skip to content

ci: open a fix PR when the nightly audit fails; fold duplicate changelog headings in the back-merge - #158

Merged
daften merged 2 commits into
developfrom
claude/confident-bardeen-vzptx9
Oct 30, 2026
Merged

daften merged 2 commits into
developfrom
claude/confident-bardeen-vzptx9

Conversation

@daften

@daften daften commented Oct 30, 2026

Copy link
Copy Markdown
Owner

Summary

Two independent fixes for the two things that kept the nightly red and the releases manual. They are separate commits so they can be reviewed one at a time.

1. audit-remediate.yml: the nightly audit fixes itself where it can

strict-audit was red on 22 of the 29 scheduled nightlies in October. Dependabot often opened no security PR for these advisories, and each time the same by-hand hotfix followed (AGENTS.md: update only the vulnerable lockfile entries, add a Security bullet, open a PR). Now a workflow does it.

  • Runs after a failed Nightly Integration Tests (workflow_run) or by hand. A manual run is a dry run by default (prints the diff and the PR body, pushes nothing).
  • Picks packages with a moderate+ advisory and a non-major fix from npm audit --omit=dev --json (scripts/audit-remediate.mjs), runs npm update --package-lock-only for only those (never npm audit fix), and fails if package.json changes.
  • Adds the ### Security bullets under [Unreleased] (without creating a second heading) and opens one PR: into main first (what ships; backmerge.yml carries it to develop), then develop for an advisory that only exists there.
  • Never merges, tags or releases. auto-release.yml only fires for Dependabot merges and this is not one, so a human merges (as a merge commit) and cuts the patch release by hand.
  • Does nothing while its own PR or a Dependabot security PR (dependabot/npm_and_yarn/security-fixes-*) is open; ordinary Dependabot version-update PRs do not block it.
  • A failed npm audit (registry unreachable) fails the run; it is never read as all-clear. Majors and unpatched advisories are reported in the log and left alone.
  • Uses RELEASE_TOKEN (a PR made with GITHUB_TOKEN would not trigger CI), the same secret backmerge.yml already uses to push and run gh pr create.

2. backmerge.yml: fold duplicate ### headings in [Unreleased] before the guard

A hotfix on main that adds ### Security meets develop's own ### Security there; merge=union keeps both, changelog-guard rejects it, and the back-merge ends in a PR for a human to move the bullets (19 of the last 30 back-merges failed; #153 was one). scripts/changelog-fold-headings.mjs does that mechanical step: it merges same-named subsections inside [Unreleased] only (shipped sections are immutable), returns a clean file byte for byte, and the workflow commits the result separately. A failure of the fold is a warning; the guard and the PR fallback still apply.

Verification (real tools, not only mocks)

  • Fold, real incident: the script run on the bad changelog from the chore: back-merge main into develop (resolves #153) #154 back-merge reproduces my hand-made fix byte for byte; the guard's real decision check(v0.5.5, …) goes from 1 problem to 0.
  • Fold, end to end: the genuine back-merge step script run with real git, the real merge=union driver, the real guard and a local origin. Previous workflow: conflict=true, nothing pushed. New workflow: fold commit, guard passes, pushed, one ### Security heading holding both bullets.
  • Remediation, real data: the workflow's actual run: blocks executed under bash -e in a throwaway repo seeded with the lockfile main had before today's hand-made hotfixes, the real npm registry and real npm audit (only gh faked). It found the same six packages, npm audit went from 6 vulnerabilities to 0, lockfile entries changed: 8 (the six plus body-parser's own content-type and iconv-lite, as in fix(deps): bump zod, body-parser, fast-uri, ip-address, qs to patched versions #152), package.json untouched, one bot-authored commit, PR created against main.
  • Scenarios run the same way: PR already open (skipped); nothing vulnerable (no PR); dry run (nothing pushed); develop-only advisory (PR into develop); registry unreachable (step fails, no PR).
  • tests/automation/fixtures/npm-audit-vulnerable.json is that real audit output. 89 automation tests pass (was 60), npm run check passes (598 tests), the changelog guard passes against the real v1.0.0-alpha.1 baseline tag, npm audit --omit=dev reports 0.

Not verified / needs your attention

  • No real GitHub Actions run yet. workflow_run workflows run from the default branch, so audit-remediate.yml does nothing until this ships to main. After the next release, run it once by hand from the Actions tab (it is a dry run by default) before relying on it.
  • gh pr create and push with RELEASE_TOKEN from this workflow are untested; only evidence is that backmerge.yml already does both with that token.
  • npm version: my runs used npm 10.9.4; the runner's Node 24 ships a newer npm. npm update --package-lock-only should behave the same but I did not run it there.
  • If you close a remediation PR without merging, the next failed nightly opens a new one.

Type of change

  • feat (new tool or feature)
  • fix (bug fix)
  • refactor (code cleanup, no behavior change)
  • test (add or update tests)
  • chore (dependencies, config, scaffolding)
  • docs (documentation only)

Checklist

  • Tests added or updated
  • npm test passes locally (598 passed, 29 skipped); node --test tests/automation/*.node-test.mjs 89 passed
  • npx tsc --noEmit passes locally
  • README tool table updated (no tools changed)
  • CLAUDE.md updated (AGENTS.md updated instead)
  • Verified against the Firefly III OpenAPI spec (not API-touching)

Merge as a merge commit, never squash or rebase.


Generated by Claude Code

…log guard

A hotfix on main that adds a '### Security' bullet under [Unreleased] meets
develop's own '### Security' heading there. merge=union keeps both, git calls
the merge clean, and changelog-guard rightly rejects two headings with the
same name, so the back-merge ended in a PR for a human to move the bullets
under one heading. That is mechanical, with one right answer.

scripts/changelog-fold-headings.mjs merges same-named ### subsections inside
[Unreleased] only (dated sections are immutable) and returns a clean file
byte for byte. backmerge.yml runs it after a clean merge and commits the
result separately; a failure of the fold is a warning, and the guard and the
PR fallback still apply. Run against the real bad changelog from the #154
back-merge it reproduces the hand-made fix exactly.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ESxuEjCeVsRAAxAb29Yjr
The nightly strict-audit was red on 22 of 29 scheduled runs in October because
Dependabot often opened no security PR, and the fix (update only the vulnerable
lockfile entries, add a Security bullet, open a PR) was done by hand each time.

audit-remediate.yml runs after a failed nightly (or by hand; manual runs are
dry runs by default). scripts/audit-remediate.mjs reads npm audit --json, picks
the packages with a moderate+ advisory and a non-major fix, and the workflow
runs npm update --package-lock-only for only those, refuses to touch
package.json, adds the changelog bullets and opens one PR, into main first,
then develop. It never merges, tags or releases, does nothing while its own PR
or a Dependabot security PR is open, and fails when npm audit itself fails
instead of reading it as all-clear.

Run against the lockfile main had before the hand-made hotfixes it finds the
same six packages and brings the audit from 6 vulnerabilities to 0.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ESxuEjCeVsRAAxAb29Yjr
@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 95.63% 1227 / 1283
🔵 Statements 95.28% 1393 / 1462
🔵 Functions 96.73% 355 / 367
🔵 Branches 93.07% 712 / 765
File CoverageNo changed files found.
Generated in workflow #372 for commit cefd629 by the Vitest Coverage Report Action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants