Skip to content

Flag left-out packages, known-range conversions and version spelling - #3

Merged
zuhairhalawa merged 1 commit into
mainfrom
checks/real-advisory-edits
Nov 5, 2026
Merged

zuhairhalawa merged 1 commit into
mainfrom
checks/real-advisory-edits

Conversation

@zuhairhalawa

@zuhairhalawa zuhairhalawa commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

Adds three checks for edit shapes found by running AdvisoryGuard on real advisory edits. There is no new command or option, no version bump and no release in this PR.

What changes

Finding When it appears Level
sibling-package-unchanged Version ranges changed for one package but not for another in the same advisory that had the same version data review
known-range-converted last_known_affected_version_range was replaced by a range's one new end. It says whether the two name the same end, comparing strings only review
version-spelling-changed A range value changed only in leading zeros, such as 2025.08.14 to 2025.8.14 review
version-leading-zero validate found a leading zero in a range event of an npm package or a SEMVER range. The file is still valid OSV, so it still exits 0 warning

Behaviour changes to check

Change Who it affects
--allow range-event-added,field-removed no longer covers a known range replaced by an end. Use known-range-converted. Scripts with an allow list
--allow range-event-changed no longer covers a change that is only in leading zeros. Use version-spelling-changed. Scripts with an allow list
An edit that passed with an allow list now also needs sibling-package-unchanged if it leaves out a package that had the same version data. Scripts with an allow list

Without an allow list, exit codes did not change on any real edit tested below.

How it was checked

Locally, on Windows with Node 24.16.0: typecheck, lint, build, 508 tests (447 existing plus 61 new), verify:pack 46 of 46, verify:schema, and npm audit with no vulnerabilities.

One existing test changed. The real-data test in test/real.test.ts keeps its original check, and now also requires sibling-package-unchanged exactly when another package shares the data.

Deliberate breaks. The new code was broken in 23 ways in a throwaway copy, and the tests caught all 23.

Real edits:

Edits New findings Notes
61 open advisory PRs of mine 2 left-out packages, 6 known-range, 3 spelling These PRs shaped the checks, so this isn't independent evidence
45 merged advisory PRs of mine None A weak test: few had these shapes
37 public edits from October, not used for tuning, on advisories I have no PR for 1 left-out package, 2 known-range The left-out finding may well be a correct edit, so it may be noise

An earlier sample of 10 public edits exposed a false alarm, which is fixed and has its own test: only a change to the ranges now counts, not a known-range note leaving on its own.

Limits

  • The left-out check can only ask the question, because packages can really differ. It has confirmed defects only in my own PRs.
  • The spelling checks have not met a misspelled version outside my own PRs.
  • The tool still does not compare versions, so it cannot tell a harmless last_affected to fixed conversion from one that changes status.
  • A known-range note removed from a range that already has an end still gets field-removed with its "dropped by accident" wording. That looks common, and this PR does not change it.
  • No outside review.

…lling

Three shapes from real advisory edits were missed or described poorly.

- sibling-package-unchanged: version ranges changed for one package but not
  for another in the same advisory that had the same version data. Needs review.
- known-range-converted: last_known_affected_version_range replaced by a range's
  one new end. This was an added event plus a removed field that suggested data
  loss. The finding says whether the two name the same end, comparing strings only.
- version-spelling-changed: a range value that changes only in leading zeros.
  validate also warns with version-leading-zero for npm packages and SEMVER ranges.

Allow lists naming the old codes no longer cover these cases. The version is not
bumped; the changelog entry is under Unreleased.
@zuhairhalawa
zuhairhalawa merged commit 5d3201d into main Nov 5, 2026
11 checks passed
@zuhairhalawa zuhairhalawa mentioned this pull request Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant