Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
357 changes: 60 additions & 297 deletions .agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md

Large diffs are not rendered by default.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ The team follows a daily ship cycle. All maintainer skills operate within this r
2. **During the day** (`/nemoclaw-maintainer-day`) — land PRs using the maintainer loop. Version labels make progress visible on dashboards.
3. **Evening** (`/nemoclaw-maintainer-evening`) — Check shipped work and the pre-tag changelog PR.
Confirm that the pre-tag changelog PR contains `docs/changelog/YYYY-MM-DD.mdx` for the release.
Identify open items and prepare the QA summary. Record the release SHA and required E2E evidence.
Identify open items and prepare the QA summary. Record the release SHA and asynchronous E2E runs that still need overnight triage; E2E does not block the tag or merging.
Cut the tag after confirmation. Move open items to the next patch label and delete the released label.
Prepare the Announcement.
4. **Overnight** — A QA team in another time zone validates the tag.
Expand Down
12 changes: 5 additions & 7 deletions .agents/skills/nemoclaw-maintainer-e2e/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ gh api --paginate --slurp \
>"$EVIDENCE_DIR/jobs-$RUN_ID.json"
```

Reuse `run-$RUN_ID.json` and `jobs-$RUN_ID.json` as the `nemoclaw-maintainer-cut-release-tag` manifest inputs and as the full-mode validator inputs. Do not fetch the same run again. `jobs-latest-$RUN_ID.json` is only for ordinary and Launchable modes.
Reuse `run-$RUN_ID.json` and `jobs-$RUN_ID.json` as the full-mode validator inputs. Do not fetch the same run again. `jobs-latest-$RUN_ID.json` is only for ordinary and Launchable modes.

For ordinary and Launchable modes, require `run-$RUN_ID.json` to report:

Expand Down Expand Up @@ -337,7 +337,7 @@ A missing, mismatched, or failed cleanup receipt is not evidence.

## Bind Release Evidence

If no release plan exists, label a successful full run against `origin/main` as provisional release evidence.
Label a successful full run against `origin/main` as SHA-bound diagnostic evidence.
Return:

- candidate SHA;
Expand All @@ -347,13 +347,11 @@ Return:
- Launchable E2E identity; and
- cleanup result.

If the release candidate SHA changes, discard the earlier full run and dispatch full mode for the new SHA.
No release-note-only delta exception is currently defined.
When validating another SHA, dispatch or locate a run bound to that exact SHA rather than reusing an earlier result.

When `nemoclaw-maintainer-cut-release-tag` invokes this skill, return the validated fields for its pre-tag E2E evidence ledger.
Return the validated fields for post-merge triage or any explicitly requested full-run audit.
The trusted `dispatch.json` receipt proves that full mode used empty selectors and included `Exact staging Brev Launchable`.
The release evidence ledger proves the result of each workflow E2E.
Do not ask for the release confirmation phrase in this skill.
The evidence proves the result of each workflow E2E; it does not gate merging or a release tag.

## Access Failures

Expand Down
28 changes: 9 additions & 19 deletions .agents/skills/nemoclaw-maintainer-evening/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ user_invocable: true

# NemoClaw Maintainer Evening

Wrap up the day: check progress, identify stragglers, summarize for QA, cut the tag, automatically carry stragglers to the next patch, retire the released label, and prepare release notes for posting.
Wrap up the day: prepare the dated changelog, check progress, identify stragglers, summarize for QA, cut the semi-automatic signed tag, and hand asynchronous E2E stabilization to the overnight loop.

See [PR-REVIEW-PRIORITIES.md](../nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md) for the daily cadence.

Expand All @@ -30,7 +30,7 @@ gh issue list --repo NVIDIA/NemoClaw --state open --label <version> --limit 100

List open labeled PRs and issues as the post-tag housekeeping plan. Tell the maintainer that, after the tag and workflow-managed `latest` are verified, `cut-release-tag` will automatically move all of them to the next patch label and delete the released label.

If an item should leave the daily release flow instead of moving forward, remove it from the released-version label before asking for the release confirmation phrase.
If an item should leave the daily release flow instead of moving forward, remove it from the released-version label before the 4 PM tag.

## Step 3: Generate Handoff Summary

Expand All @@ -46,32 +46,22 @@ Run `/nemoclaw-contributor-update-docs for <version>` before loading `cut-releas
Confirm that the release-prep docs PR creates or updates one direct child of `docs/changelog/` for the planned date and contains the exact `## <version>` heading, a parser-safe MDX SPDX comment, the summary, and the detailed release bullets.
An ordinary docs refresh or a post-tag Discussion draft does not satisfy this step.
The release-prep docs PR, including the dated changelog entry, must be merged, or explicitly waived with a reason that names the missing changelog entry, before `release:plan` captures the release commit.
If a docs PR or any other intended PR merges after `release:plan`, regenerate the plan before cutting the tag.
Finish and merge the changelog before generating the 4 PM release plan. If another PR merges after `release:plan`, regenerate the plan before cutting the tag.

## Step 4: Cut the Tag and Publish Release Notes
## Step 4: Cut the Tag and Start Overnight Stabilization

Load `cut-release-tag`.
The version is already known, so use a patch bump unless the maintainer selects another bump.
Show the commit, changelog, carry-forward plan, label-retirement plan, and release notes draft.
Load `cut-release-tag` at 4 PM America/Los_Angeles. Let the agent verify the merged changelog and housekeeping, generate the plan for current `origin/main`, exercise the maintainer's local signer, and show the exact confirmation phrase. After maintainer confirmation, cut the signed tag regardless of E2E state. Let the workflow move `latest`, automatically carry stragglers to the next patch, and retire the released label.

After the release plan captures the candidate SHA, load `nemoclaw-maintainer-e2e`.
Run full mode unless one existing full run for the candidate SHA contains complete workflow E2E and `Exact staging Brev Launchable` evidence.
Review the pre-tag E2E evidence ledger from `.github/workflows/e2e.yaml` at that commit.
Require the accepted workflow run to conclude with `success`. Require successful `Exact staging Brev Launchable` evidence with matching Launchable E2E identity and verified workspace absence, or record the permitted itemized exception described below.
Each missing or skipped execution in that successful run requires its own itemized maintainer exception.
Missing or invalid Launchable E2E evidence in that successful run requires a separate itemized exception with run and job URLs, the missing or invalid receipt, and rationale.
Do not ask for the release confirmation phrase until the run succeeds and each required execution has successful evidence or a permitted exception.
Every push to `main` already runs the complete E2E workflow. From 4 PM through 8 AM, keep merging normally while agents consolidate failures, remove redundant coverage, and fix broken or flaky E2Es. Do not delay or retry the tag because of those results.

Tag the confirmed release commit with `vX.Y.Z`.
Let the workflow move `latest`, carry open work forward, and delete the released label.
Prepare the Announcement draft for the maintainer to post.
Prepare the Announcement draft for the maintainer to post. Keep the signing key on the maintainer's workstation; do not add it to GitHub Actions.

## Step 5: Confirm and Share

After the tag is cut and release notes are drafted or posted by the maintainer, present the final summary:

- **Tag**: `v0.0.8` at commit `abc1234`
- **Pre-tag E2E evidence**: 12/13 tests and exact Brev Launchable E2E passing for the candidate SHA; 1 itemized maintainer exception
- **E2E stabilization**: links to remaining failed or flaky post-merge runs for overnight follow-up
- **Release notes draft**: `../nemoclaw-release-v0.0.8/release-note-draft.md`
- **Shipped**: 4 items (#1234, #1235, #1236, #1237)
- **Moved to v0.0.9**: 1 item (#1238 — still needs CI fix)
Expand All @@ -88,7 +78,7 @@ node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer

## Notes

- Never cut a tag or hand off release notes without user confirmation.
- Never cut a tag without the maintainer's exact confirmation phrase. E2E does not block that confirmation or the tag.
- If nothing was labeled or nothing shipped, ask whether to skip the tag today.
- A PR version label activates release work; it is not a readiness claim.
- If an open item misses the tag, post-tag housekeeping moves its target to the next patch version.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Agents may recommend labels, assignments, Project field changes, comments, merge
- A PR daily version label activates daily release work; it is not a readiness claim.
- Release inclusion requires a PR to be both merged and carrying the relevant daily version label at release cutoff.
- Issue daily version labels are tracking or coordination signals only.
- Before tag confirmation, capture the candidate SHA and review every E2E test declared by `.github/workflows/e2e.yaml` at that commit. Require the accepted workflow run's `head_sha` and all associated evidence to match the candidate SHA, and require the run to conclude with `success`. Each declared execution must have successful evidence from that run or an explicit itemized maintainer exception. Permit exceptions only for missing or skipped executions, not failed executions. If a late merge changes the candidate SHA, invalidate the prior run and evidence and repeat the complete workflow-wide validation.
- At 4 PM America/Los_Angeles, let the release agent verify the merged changelog and housekeeping, then have an authorized maintainer confirm and locally sign the current `main` tag regardless of E2E state. Keep merging while the 4 PM-8 AM loop triages each SHA-bound post-merge E2E run.
- Open PRs and issues that miss a tagged release carry forward by automatically moving from the released version label to the next patch label after the tag and `latest` are verified.
- After carry-forward leaves no open item on the released label, delete that repository label. Never rename or reuse it.
- Durable release history belongs in releases, release notes, or manifests, not in long-lived labels.
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ If any merge lands after `release:plan`, generate a fresh plan before cutting th

## Cutoff

The daily cutoff is the maintainer-defined point where the release tag is prepared.
The daily cutoff is 4 PM America/Los_Angeles, when the release agent prepares the current `main` commit for an authorized maintainer to confirm and sign locally. Merging does not stop for cutoff, E2E, or overnight stabilization.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

At cutoff:

Expand All @@ -40,37 +40,18 @@ At cutoff:
3. List open PRs and issues still carrying the target label as post-tag stragglers.
4. Confirm the merged release-note docs PR contains the dated changelog entry for the target version, or record an explicit waiver that names the missing entry.
5. Generate QA handoff from merged PRs.
6. Generate the release plan to capture the candidate commit. Merges may continue; a late drift check advances the candidate and invalidates evidence for the older SHA.
7. Review the candidate commit's pre-tag E2E evidence.
8. Cut the release tag only with explicit maintainer confirmation.
9. After the tag and workflow-managed `latest` are verified, automatically move every open straggler to the next patch label, verify none remain, and delete the released version label.

## Pre-Tag E2E Evidence

The release candidate is the full `origin/main` commit SHA captured by the generated release plan. At that commit, `.github/workflows/e2e.yaml` is the sole source of truth for the release E2E test set. Do not maintain a separate release-gating test list.

Before asking for the release confirmation phrase, build and show an evidence ledger for that SHA:

- Preflight the candidate workflow and existing candidate evidence before dispatching new work.
- Derive the denominator from the candidate workflow. Do not copy it into a second release test list.
- Require every declared `RELEASE_E2E_ACTIVATION_PATH` to exist at the candidate SHA. A missing path is a preflight failure.
- Require the workflow-produced trusted dispatch receipt to bind the accepted run candidate SHA, run ID, attempt, and selector inputs.
- Run `nemoclaw-maintainer-e2e` in full mode when the ledger lacks complete evidence for the candidate SHA.
- Require one completed, successful full workflow run that selects every workflow E2E, including `Exact staging Brev Launchable`.
- Require the trusted dispatch receipt to bind the workflow run and an attempt no later than the run's latest attempt. The receipt must record empty selectors and `include_staging_brev_launchable=true`.
- Require the Launchable E2E receipt to identify the candidate SHA in the repository and provision records.
- Require the cleanup receipt to identify the qualified workspace and report `ABSENT`.
- Every E2E execution declared by the workflow must have at least one completed, successful execution for the candidate SHA.
- Treat each expanded matrix execution as a separate ledger entry. Use its matrix `id`, or all distinguishing matrix dimensions when no single ID exists, in the test identifier so results for distinct expansions are never collapsed under the parent job.
- Successful evidence may accumulate across rerun attempts of that workflow run. Evidence from another workflow run does not satisfy the ledger. A later failure does not erase an earlier successful execution for the same test and SHA.
- Skipped, unexecuted, queued, in-progress, cancelled, and failing results do not count as successful evidence.
- Map each test with successful evidence to its successful run or job URL and attempt number.
- Each missing or skipped execution in the accepted successful workflow run requires its own itemized maintainer exception. Record the test identifier, relevant run links or available evidence, the current result, and the rationale.
- Missing or invalid exact Brev Launchable E2E evidence in the accepted successful workflow run requires a separate itemized maintainer exception. Record the run and job URLs, the missing or invalid receipt, and the rationale.

The accepted workflow run must be completed and have a `success` conclusion. A failed workflow run cannot supply the release ledger. Rerun its failed jobs until the workflow concludes with `success`. An itemized test exception applies only to a missing or skipped execution in that otherwise successful run.

Each test and the exact Brev Launchable E2E job in the accepted successful workflow run must have successful evidence or its own permitted itemized exception before release confirmation. Immediately before confirmation, compare `origin/main` with the planned SHA. If the candidate SHA changes, discard the ledger and its exceptions, including Launchable E2E evidence. Regenerate the release plan and repeat the review for the new SHA. This does not freeze `main` or prevent merges. No release-note-only delta exception is currently defined.
6. Generate the release plan for current `origin/main`, exercise the maintainer's local signer, and show the exact confirmation phrase. If `main` moves before confirmation, regenerate the plan rather than stopping merges.
7. After explicit maintainer confirmation, cut the locally signed tag regardless of E2E state. Never put the release signing key in GitHub Actions or use a release bot to sign it.
8. After the tag and workflow-managed `latest` are verified, automatically move every open straggler to the next patch label, verify none remain, and delete the released version label.
9. From 4 PM through 8 AM, continue merging while agents consolidate failures, remove redundant coverage, and fix broken or flaky E2Es.

## Asynchronous E2E Stabilization

Every push to `main` starts the complete E2E workflow. Each run is bound to that push SHA, so a later merge does not cancel or replace the earlier result.

E2E results are advisory release-health signals. They never block merging, select the release candidate, delay the 4 PM tag, or require a maintainer exception. Keep failed results attached to their workflow runs for asynchronous triage.

From 4 PM through 8 AM, agents work the accumulated results methodically: group duplicate failures, remove redundant tests, repair broken or flaky tests, and merge fixes normally. At 8 AM, hand the remaining state to the next release doula. The daytime merge window continues from 8 AM through the next 4 PM tag.

## Carry Forward

Expand Down
2 changes: 1 addition & 1 deletion .agents/skills/nemoclaw-skills-guide/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ automated review feedback, drafting documentation updates, and onboarding new me
| `nemoclaw-maintainer-cross-issue-sweep` | Find open issues that a PR can also fix or conflict with. Report file and line evidence. |
| `nemoclaw-maintainer-day` | Run one daytime maintainer pass for the release version. Select a merge, salvage, security, test, conflict, or sequencing workflow. Designed for `/loop`. |
| `nemoclaw-maintainer-evening` | End-of-day handoff: require the pre-tag dated changelog PR, check version progress, identify stragglers, generate a QA handoff summary, cut the release tag, carry stragglers forward, retire the released label, and hand off the Announcement. |
| `nemoclaw-maintainer-cut-release-tag` | Verify the dated changelog entry, cut an annotated semver tag on a maintainer-confirmed `origin/main` commit, wait for workflow-managed `latest`, carry remaining open items forward, and delete the released label; `lkg` stays manual. |
| `nemoclaw-maintainer-cut-release-tag` | At 4 PM, verify the dated changelog and housekeeping, prepare a maintainer-confirmed tag for local signing independent of E2E, wait for workflow-managed `latest`, carry remaining open items forward, and delete the released label; `lkg` stays manual. |
| `nemoclaw-maintainer-e2e` | Describe default E2E triggered by pushes to `main`, dispatch exact-revision manual PR E2E, and verify applicable workflow evidence. |
| `nemoclaw-maintainer-release-notes` | Draft the post-tag Announcement from live tag/compare data, with the three-paragraph narrative, categorized change list, and external-only contributor thanks. |
| `nemoclaw-maintainer-find-review-pr` | Find open security PRs with Urgent or High Project Priority. Link each PR to its issue and identify competing PRs. |
Expand Down
1 change: 0 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,6 @@
"test-conditionals:scan": "tsx scripts/find-test-conditionals.mts",
"bump:version": "tsx scripts/bump-version.mts",
"release:plan": "tsx scripts/release-plan.mts",
"release:e2e-evidence": "node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts",
"release:cut": "bash scripts/release-cut-tag.sh",
"release:wait-latest": "bash scripts/release-wait-latest.sh",
"release:notes-data": "tsx scripts/release-notes-data.mts",
Expand Down
8 changes: 4 additions & 4 deletions test/maintainer-e2e-skill.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ describe("nemoclaw-maintainer-e2e workflow routing", () => {
expect(skill).not.toMatch(/variable (?:set|delete) NEMOCLAW_BREV_LAUNCHABLE_E2E_ENABLED/u);
});

it("binds dispatch, evidence, invalidation, and release handoff to one SHA (#7487)", () => {
it("binds dispatch and diagnostic evidence to one SHA without gating merges or tags (#7487)", () => {
expect(skill).toContain("git rev-parse origin/main");
expect(skill).toContain("correlation_id=${CORRELATION_ID}");
expect(skill).toContain("head_sha");
Expand All @@ -219,8 +219,8 @@ describe("nemoclaw-maintainer-e2e workflow routing", () => {
expect(skill).toContain("cleanup.json");
expect(skill).toContain("dispatch.json");
expect(skill).toContain("validate-full-e2e-evidence.mts");
expect(skill).toContain("provisional release evidence");
expect(skill).toContain("If the release candidate SHA changes");
expect(skill).toContain("nemoclaw-maintainer-cut-release-tag");
expect(skill).toContain("SHA-bound diagnostic evidence");
expect(skill).toContain("When validating another SHA");
expect(skill).toContain("does not gate merging or a release tag");
});
});
Loading
Loading