From e311851c7be72fa37954dc38b0a00642919c9156 Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Thu, 20 Aug 2026 12:22:15 +0530 Subject: [PATCH 01/19] chore(vercel): gate preview builds on pull request readiness Orbit ran 700 deployments in the 22 days after the project was created on 2026-07-28, peaking at 131 in a single day, and 77% of them were previews. Builds were $110 of the $506.93 August Vercel invoice. The Ignored Build Step now runs scripts/vercel-build-gate.sh. Production always builds; previews build once the pull request leaves draft. Work in a draft and commits stop triggering builds, then Ready for review starts them. A preview label forces builds while still drafting, a no-preview label suppresses them. Every failure path builds. A missing token, an unreachable GitHub API, a malformed response, a diff base outside the shallow clone, or system environment variables that were never exposed all fall through to a build, so the gate cannot silently withhold a deployment. Only apps/web deploys here, so the ignore command defaults BUILD_GATE_WATCH_PATHS to apps/web, packages and the root manifests: a push that only touches apps/realtime has nothing to preview. Setting the variable in project settings overrides the default. Watch paths resolve against the repository root rather than the working directory. Vercel runs the Ignored Build Step from the Root Directory, so a pathspec of apps/web evaluated from apps/web would look for apps/web/apps/web and skip everything. Verified with a stubbed curl over twelve cases, and the path filter separately from a subdirectory to match how Vercel invokes it. Refs AM-125 --- apps/web/vercel.json | 1 + docs/VERCEL_BUILD_GATE.md | 93 ++++++++++++++++++++++++++++++++++++ scripts/vercel-build-gate.sh | 93 ++++++++++++++++++++++++++++++++++++ 3 files changed, 187 insertions(+) create mode 100644 docs/VERCEL_BUILD_GATE.md create mode 100755 scripts/vercel-build-gate.sh diff --git a/apps/web/vercel.json b/apps/web/vercel.json index 21497a7ba..ae20180f0 100644 --- a/apps/web/vercel.json +++ b/apps/web/vercel.json @@ -1,6 +1,7 @@ { "$schema": "https://openapi.vercel.sh/vercel.json", "installCommand": "rm -rf ../../node_modules node_modules && bun install --frozen-lockfile", + "ignoreCommand": "BUILD_GATE_WATCH_PATHS=\"${BUILD_GATE_WATCH_PATHS:-apps/web packages package.json bun.lock}\" bash ../../scripts/vercel-build-gate.sh", "regions": ["hnd1"], "crons": [ { "path": "/api/cron/analytics-snapshots", "schedule": "0 */6 * * *" }, diff --git a/docs/VERCEL_BUILD_GATE.md b/docs/VERCEL_BUILD_GATE.md new file mode 100644 index 000000000..5298f8bf1 --- /dev/null +++ b/docs/VERCEL_BUILD_GATE.md @@ -0,0 +1,93 @@ +# Vercel build gate + +Preview deployments only build once a pull request is marked **Ready for review**. +Production always builds. + +## Why the gate exists + +Orbit ran **700 deployments in the 22 days** after the project was created on +2026-07-28, peaking at 131 in a single day, and 77% of them were previews. +Across the team, 81% of 4,393 deployments in the 90 days to 2026-08-19 were +previews, and builds were $110 of the $506.93 August invoice. + +## How it works + +`apps/web/vercel.json` points Vercel's Ignored Build Step at +`scripts/vercel-build-gate.sh`. + +**Exit codes are inverted from intuition: `exit 0` skips the build, `exit 1` runs it.** + +Decision order: + +| Condition | Result | +|---|---| +| `VERCEL_ENV=production` | build | +| system environment variables not exposed | build | +| branch has no open PR | skip | +| PR labelled `no-preview` | skip | +| PR labelled `preview` | build (even while draft) | +| PR is a draft | skip | +| PR is ready for review | build, subject to the path filter | +| nothing changed under `BUILD_GATE_WATCH_PATHS` | skip | + +Every failure path - missing token, GitHub API error, unparseable response, +unreachable diff base - **builds**. The gate never silently withholds a +deployment because something broke. + +The metadata check has to come before the pull request check, and the order is +load-bearing. `VERCEL_GIT_PULL_REQUEST_ID` is empty both when a branch genuinely +has no pull request *and* when system environment variables are not exposed at +all. Testing the PR id first would read the second case as the first and skip +every preview in the project, silently, which is the one behaviour this gate +must never have. `VERCEL_GIT_REPO_OWNER` and `VERCEL_GIT_REPO_SLUG` are set +whenever the variables are exposed, regardless of pull request state, so they +are what distinguishes the two. + +## The button + +Open the PR as a **draft** while you work. Commits accumulate with zero builds. +When you want a preview, click **Ready for review** - that is the button. Adding +the `preview` label also works if you want previews while staying in draft. + +## Setup per project + +1. Project Settings → Environment Variables → tick **Enable access to System + Environment Variables**. The gate needs `VERCEL_GIT_PULL_REQUEST_ID`, + `VERCEL_GIT_REPO_OWNER`, `VERCEL_GIT_REPO_SLUG` and `VERCEL_GIT_PREVIOUS_SHA`. + Note that `VERCEL_GIT_PREVIOUS_SHA` is *only* exposed when an Ignored Build + Step is configured. +2. Add `BUILD_GATE_GITHUB_TOKEN` - a fine-grained token with **Pull requests: + read** on the repo. Without it the gate fails open and every push builds. +3. Optionally add `BUILD_GATE_WATCH_PATHS` (space separated, repo-relative) to + skip builds when nothing under those paths changed. + +## Monorepo path filtering + +This repo holds two apps. Only `apps/web` is deployed to Vercel, so a push that +only touches `apps/realtime` has nothing to preview. `apps/web/vercel.json` +therefore supplies a default: + +``` +BUILD_GATE_WATCH_PATHS="apps/web packages package.json bun.lock" +``` + +Setting the variable in project settings overrides that default. + +Watch paths are **repo-relative**, and the script resolves them against +`git rev-parse --show-toplevel` rather than the working directory. This matters: +Vercel runs the Ignored Build Step from the project's **Root Directory**, so for +a project rooted at `apps/web` a plain `git diff -- apps/web` looks for +`apps/web/apps/web`, finds nothing, and skips every build. Test any change to +this script from a subdirectory, not just from the repo root. + +The diff base is `VERCEL_GIT_PREVIOUS_SHA`, the last **successfully deployed** +commit - not `HEAD^`. `HEAD^` is wrong whenever more than one commit lands at +once, which is the normal case for a squash merge or a batch of pushes. If that +SHA is missing from Vercel's shallow clone the gate builds rather than guessing. + +## Testing changes to the gate + +The script shells out to `curl` and `git`, so it is testable by putting a stub +`curl` earlier on `PATH`. See the harness used when this landed - it covers +production, draft, ready, both labels, a missing token, API failure, malformed +JSON, and the path filter against real git history. diff --git a/scripts/vercel-build-gate.sh b/scripts/vercel-build-gate.sh new file mode 100755 index 000000000..1cd180a81 --- /dev/null +++ b/scripts/vercel-build-gate.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +set -uo pipefail + +WATCH_PATHS="${BUILD_GATE_WATCH_PATHS:-.}" +READY_LABEL="${BUILD_GATE_READY_LABEL:-preview}" +BLOCK_LABEL="${BUILD_GATE_BLOCK_LABEL:-no-preview}" + +announce() { echo "[build-gate] $*" >&2; } +build() { announce "BUILD - $1"; exit 1; } +skip() { announce "SKIP - $1"; exit 0; } + +if [ "${VERCEL_ENV:-}" = "production" ]; then + build "production deployment" +fi + +REPO_OWNER="${VERCEL_GIT_REPO_OWNER:-}" +REPO_SLUG="${VERCEL_GIT_REPO_SLUG:-}" +if [ -z "$REPO_OWNER" ] || [ -z "$REPO_SLUG" ]; then + build "system environment variables are not exposed to this build so pull request state is unreadable, failing open" +fi + +PR_ID="${VERCEL_GIT_PULL_REQUEST_ID:-}" +if [ -z "$PR_ID" ]; then + skip "branch has no open pull request, nothing to preview yet" +fi + +if [ -z "${BUILD_GATE_GITHUB_TOKEN:-}" ]; then + build "BUILD_GATE_GITHUB_TOKEN is unset so pull request state cannot be read, failing open" +fi + +if ! PR_JSON="$(curl -sS --max-time 15 \ + -H "Authorization: Bearer ${BUILD_GATE_GITHUB_TOKEN}" \ + -H "Accept: application/vnd.github+json" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + "https://api.github.com/repos/${REPO_OWNER}/${REPO_SLUG}/pulls/${PR_ID}")"; then + build "GitHub API unreachable, failing open" +fi + +if [ -z "$PR_JSON" ]; then + build "GitHub API returned an empty response, failing open" +fi + +VERDICT="$( + PR_JSON="$PR_JSON" READY_LABEL="$READY_LABEL" BLOCK_LABEL="$BLOCK_LABEL" node -e ' + try { + const pr = JSON.parse(process.env.PR_JSON); + if (!pr || typeof pr.draft !== "boolean") { + console.log("unknown:pull request payload had no draft flag"); + process.exit(0); + } + const labels = (pr.labels || []).map((label) => String(label.name).toLowerCase()); + if (labels.includes(process.env.BLOCK_LABEL.toLowerCase())) { + console.log(`skip:pull request ${pr.number} carries the ${process.env.BLOCK_LABEL} label`); + } else if (labels.includes(process.env.READY_LABEL.toLowerCase())) { + console.log(`build:pull request ${pr.number} carries the ${process.env.READY_LABEL} label`); + } else if (pr.draft) { + console.log(`skip:pull request ${pr.number} is still a draft, mark it ready for review to start previews`); + } else { + console.log(`build:pull request ${pr.number} is ready for review`); + } + } catch (error) { + console.log(`unknown:${error.message}`); + } + ' +)" + +REASON="$(printf '%s' "$VERDICT" | cut -d: -f2-)" +case "$(printf '%s' "$VERDICT" | cut -d: -f1)" in +skip) skip "$REASON" ;; +build) announce "gate passed, $REASON" ;; +*) build "pull request state could not be evaluated, failing open" ;; +esac + +BASE_SHA="${VERCEL_GIT_PREVIOUS_SHA:-}" +if [ -z "$BASE_SHA" ]; then + build "no diff base supplied, cannot tell which paths changed" +fi + +REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null)" +if [ -z "$REPO_ROOT" ]; then + build "not inside a git work tree, cannot tell which paths changed" +fi + +if ! git -C "$REPO_ROOT" cat-file -e "${BASE_SHA}^{commit}" 2>/dev/null; then + build "diff base ${BASE_SHA} is not in this clone, cannot tell which paths changed" +fi + +read -r -a WATCH_ARRAY <<<"$WATCH_PATHS" +if git -C "$REPO_ROOT" diff --quiet "$BASE_SHA" HEAD -- "${WATCH_ARRAY[@]}"; then + skip "no changes under '${WATCH_PATHS}' since ${BASE_SHA}" +fi + +build "changes under '${WATCH_PATHS}' since ${BASE_SHA}" From 8620b54579ba0d9f1033087ec581d82765f7da64 Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Thu, 20 Aug 2026 12:33:54 +0530 Subject: [PATCH 02/19] test(vercel): cover the build gate, and watch the root tsconfig Two findings from review. The default path filter did not include tsconfig.base.json, which apps/web and every package extends. A ready pull request changing only compiler settings would have reported nothing relevant and skipped the preview, so the change would never have been exercised on Vercel. The gate also had no committed regression coverage. It decides whether a deployment happens, its exit codes are inverted, and it has already needed two corrections: failing open when system environment variables are absent, and resolving watch paths from the repository root rather than the working directory. Both were the kind of fault that silently suppresses every preview. scripts/vercel-build-gate.test.ts drives the real script with a stubbed curl on PATH and a throwaway git repository, covering production, absent metadata, no pull request, missing token, draft, ready, both labels, unreadable and empty responses, transport failure, the path filter in both directions, an unreachable and a missing diff base, and the root directory case that hid the cwd bug. The root test script now runs it, so CI does too. Refs AM-125 --- apps/web/vercel.json | 2 +- package.json | 2 +- scripts/vercel-build-gate.test.ts | 225 ++++++++++++++++++++++++++++++ 3 files changed, 227 insertions(+), 2 deletions(-) create mode 100644 scripts/vercel-build-gate.test.ts diff --git a/apps/web/vercel.json b/apps/web/vercel.json index ae20180f0..2a44f7e97 100644 --- a/apps/web/vercel.json +++ b/apps/web/vercel.json @@ -1,7 +1,7 @@ { "$schema": "https://openapi.vercel.sh/vercel.json", "installCommand": "rm -rf ../../node_modules node_modules && bun install --frozen-lockfile", - "ignoreCommand": "BUILD_GATE_WATCH_PATHS=\"${BUILD_GATE_WATCH_PATHS:-apps/web packages package.json bun.lock}\" bash ../../scripts/vercel-build-gate.sh", + "ignoreCommand": "BUILD_GATE_WATCH_PATHS=\"${BUILD_GATE_WATCH_PATHS:-apps/web packages package.json bun.lock tsconfig.base.json}\" bash ../../scripts/vercel-build-gate.sh", "regions": ["hnd1"], "crons": [ { "path": "/api/cron/analytics-snapshots", "schedule": "0 */6 * * *" }, diff --git a/package.json b/package.json index 80c68945d..6d595a849 100644 --- a/package.json +++ b/package.json @@ -28,7 +28,7 @@ "build": "bun run --filter '*' build", "dev": "bun run --filter '*' dev", "typecheck": "tsc -p scripts/tsconfig.json --noEmit && bun run --filter '*' typecheck", - "test": "bun run --filter '*' test", + "test": "bun test scripts && bun run --filter '*' test", "test:e2e": "bun run --filter '@orbit/web' test:e2e", "lint": "biome check .", "lint:fix": "biome check --write .", diff --git a/scripts/vercel-build-gate.test.ts b/scripts/vercel-build-gate.test.ts new file mode 100644 index 000000000..fc0a6dd97 --- /dev/null +++ b/scripts/vercel-build-gate.test.ts @@ -0,0 +1,225 @@ +import { afterAll, beforeAll, describe, expect, test } from 'bun:test'; +import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +const GATE = join(import.meta.dir, 'vercel-build-gate.sh'); + +const SKIP = 0; +const BUILD = 1; + +const READY = JSON.stringify({ number: 7, draft: false, labels: [] }); +const DRAFT = JSON.stringify({ number: 7, draft: true, labels: [] }); +const DRAFT_LABELLED = JSON.stringify({ + number: 7, + draft: true, + labels: [{ name: 'preview' }], +}); +const READY_BLOCKED = JSON.stringify({ + number: 7, + draft: false, + labels: [{ name: 'no-preview' }], +}); + +let sandbox: string; +let stubBin: string; +let repo: string; +let firstCommit: string; + +function git(cwd: string, ...args: string[]): string { + const result = Bun.spawnSync(['git', ...args], { cwd }); + if (result.exitCode !== 0) { + throw new Error(`git ${args.join(' ')}: ${result.stderr.toString()}`); + } + return result.stdout.toString().trim(); +} + +function runGate(env: Record, cwd: string = repo): { code: number; log: string } { + const result = Bun.spawnSync(['bash', GATE], { + cwd, + env: { + PATH: `${stubBin}:${process.env['PATH'] ?? ''}`, + HOME: process.env['HOME'] ?? '', + ...env, + }, + }); + return { + code: result.exitCode ?? -1, + log: result.stderr.toString(), + }; +} + +const withPullRequest = (json: string, extra: Record = {}) => ({ + VERCEL_ENV: 'preview', + VERCEL_GIT_PULL_REQUEST_ID: '7', + VERCEL_GIT_REPO_OWNER: 'Noveum', + VERCEL_GIT_REPO_SLUG: 'orbit', + BUILD_GATE_GITHUB_TOKEN: 'token', + MOCK_PR_JSON: json, + ...extra, +}); + +beforeAll(() => { + sandbox = mkdtempSync(join(tmpdir(), 'build-gate-')); + + stubBin = join(sandbox, 'bin'); + mkdirSync(stubBin); + const stub = join(stubBin, 'curl'); + writeFileSync( + stub, + [ + '#!/usr/bin/env bash', + 'if [ -n "$MOCK_CURL_FAIL" ]; then exit 7; fi', + 'printf "%s" "$MOCK_PR_JSON"', + '', + ].join('\n'), + ); + chmodSync(stub, 0o755); + + repo = join(sandbox, 'repo'); + mkdirSync(join(repo, 'apps', 'web'), { recursive: true }); + mkdirSync(join(repo, 'apps', 'realtime'), { recursive: true }); + mkdirSync(join(repo, 'packages'), { recursive: true }); + writeFileSync(join(repo, 'apps', 'web', 'page.tsx'), 'web\n'); + writeFileSync(join(repo, 'apps', 'realtime', 'server.ts'), 'realtime\n'); + writeFileSync(join(repo, 'tsconfig.base.json'), '{}\n'); + + git(repo, 'init', '-q', '.'); + git(repo, 'config', 'user.email', 'gate@test.invalid'); + git(repo, 'config', 'user.name', 'gate'); + git(repo, 'add', '-A'); + git(repo, 'commit', '-qm', 'base'); + firstCommit = git(repo, 'rev-parse', 'HEAD'); + + writeFileSync(join(repo, 'apps', 'web', 'page.tsx'), 'web changed\n'); + git(repo, 'commit', '-qam', 'touch apps/web'); +}); + +afterAll(() => { + rmSync(sandbox, { recursive: true, force: true }); +}); + +describe('vercel build gate', () => { + test('production always builds', () => { + expect(runGate({ VERCEL_ENV: 'production' }).code).toBe(BUILD); + }); + + test('absent system environment variables fail open', () => { + const { code, log } = runGate({ VERCEL_ENV: 'preview' }); + expect(code).toBe(BUILD); + expect(log).toContain('system environment variables'); + }); + + test('a branch with no pull request skips', () => { + expect( + runGate({ + VERCEL_ENV: 'preview', + VERCEL_GIT_REPO_OWNER: 'Noveum', + VERCEL_GIT_REPO_SLUG: 'orbit', + VERCEL_GIT_PULL_REQUEST_ID: '', + }).code, + ).toBe(SKIP); + }); + + test('a missing token fails open', () => { + expect( + runGate({ + VERCEL_ENV: 'preview', + VERCEL_GIT_REPO_OWNER: 'Noveum', + VERCEL_GIT_REPO_SLUG: 'orbit', + VERCEL_GIT_PULL_REQUEST_ID: '7', + }).code, + ).toBe(BUILD); + }); + + test('a draft pull request skips', () => { + expect(runGate(withPullRequest(DRAFT)).code).toBe(SKIP); + }); + + test('a ready pull request builds', () => { + expect(runGate(withPullRequest(READY, { BUILD_GATE_WATCH_PATHS: '.' })).code).toBe(BUILD); + }); + + test('the preview label builds a draft', () => { + expect(runGate(withPullRequest(DRAFT_LABELLED, { BUILD_GATE_WATCH_PATHS: '.' })).code).toBe( + BUILD, + ); + }); + + test('the no-preview label skips a ready pull request', () => { + expect(runGate(withPullRequest(READY_BLOCKED)).code).toBe(SKIP); + }); + + test('an unreadable API response fails open', () => { + expect(runGate(withPullRequest('not json')).code).toBe(BUILD); + expect(runGate(withPullRequest('')).code).toBe(BUILD); + expect(runGate(withPullRequest(READY, { MOCK_CURL_FAIL: '1' })).code).toBe(BUILD); + }); + + describe('path filter', () => { + const base = () => ({ VERCEL_GIT_PREVIOUS_SHA: firstCommit }); + + test('builds when a watched path changed', () => { + expect( + runGate( + withPullRequest(READY, { + ...base(), + BUILD_GATE_WATCH_PATHS: 'apps/web', + }), + ).code, + ).toBe(BUILD); + }); + + test('skips when nothing under the watched paths changed', () => { + expect( + runGate( + withPullRequest(READY, { + ...base(), + BUILD_GATE_WATCH_PATHS: 'packages tsconfig.base.json', + }), + ).code, + ).toBe(SKIP); + }); + + test('resolves watch paths from the repository root, so a pathspec of apps/web run from apps/web does not look for apps/web/apps/web and skip every build', () => { + const fromRootDirectory = join(repo, 'apps', 'web'); + + expect( + runGate( + withPullRequest(READY, { + ...base(), + BUILD_GATE_WATCH_PATHS: 'apps/web', + }), + fromRootDirectory, + ).code, + ).toBe(BUILD); + + expect( + runGate( + withPullRequest(READY, { + ...base(), + BUILD_GATE_WATCH_PATHS: 'packages', + }), + fromRootDirectory, + ).code, + ).toBe(SKIP); + }); + + test('an unreachable diff base fails open', () => { + expect( + runGate( + withPullRequest(READY, { + VERCEL_GIT_PREVIOUS_SHA: '0'.repeat(40), + BUILD_GATE_WATCH_PATHS: 'apps/web', + }), + ).code, + ).toBe(BUILD); + }); + + test('a missing diff base fails open', () => { + expect(runGate(withPullRequest(READY, { BUILD_GATE_WATCH_PATHS: 'apps/web' })).code).toBe( + BUILD, + ); + }); + }); +}); From 2da9f8904d18ecb4fdcbfd021ce41d81c0841dba Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Thu, 20 Aug 2026 12:46:33 +0530 Subject: [PATCH 03/19] fix(vercel): require a complete pull request payload before skipping A payload of {"draft":true} passed the old check and skipped, so a truncated or unexpected response could silently withhold a preview. Skipping is the only direction that hides a deployment, so it now requires a well formed pull request: a boolean draft, a positive integer number, and a labels array whose entries all carry a string name. Anything else is unknown and builds. Node itself needs no guarding. If it were missing the command substitution yields an empty verdict, which the default case already treats as unevaluable and builds. Docs now state both label overrides in the opening rule, since preview builds a draft and no-preview suppresses a ready pull request, and the remaining fence carries a language. Refs AM-125 --- docs/VERCEL_BUILD_GATE.md | 7 ++++--- scripts/vercel-build-gate.sh | 13 ++++++++++--- scripts/vercel-build-gate.test.ts | 15 +++++++++++++++ 3 files changed, 29 insertions(+), 6 deletions(-) diff --git a/docs/VERCEL_BUILD_GATE.md b/docs/VERCEL_BUILD_GATE.md index 5298f8bf1..189f2ddd4 100644 --- a/docs/VERCEL_BUILD_GATE.md +++ b/docs/VERCEL_BUILD_GATE.md @@ -1,7 +1,8 @@ # Vercel build gate Preview deployments only build once a pull request is marked **Ready for review**. -Production always builds. +Production always builds. Two labels override the rule: `preview` builds a draft, +and `no-preview` suppresses a pull request that is ready. ## Why the gate exists @@ -67,8 +68,8 @@ This repo holds two apps. Only `apps/web` is deployed to Vercel, so a push that only touches `apps/realtime` has nothing to preview. `apps/web/vercel.json` therefore supplies a default: -``` -BUILD_GATE_WATCH_PATHS="apps/web packages package.json bun.lock" +```sh +BUILD_GATE_WATCH_PATHS="apps/web packages package.json bun.lock tsconfig.base.json" ``` Setting the variable in project settings overrides that default. diff --git a/scripts/vercel-build-gate.sh b/scripts/vercel-build-gate.sh index 1cd180a81..8cbe677dc 100755 --- a/scripts/vercel-build-gate.sh +++ b/scripts/vercel-build-gate.sh @@ -44,11 +44,18 @@ VERDICT="$( PR_JSON="$PR_JSON" READY_LABEL="$READY_LABEL" BLOCK_LABEL="$BLOCK_LABEL" node -e ' try { const pr = JSON.parse(process.env.PR_JSON); - if (!pr || typeof pr.draft !== "boolean") { - console.log("unknown:pull request payload had no draft flag"); + const wellFormed = + pr && + typeof pr.draft === "boolean" && + Number.isInteger(pr.number) && + pr.number > 0 && + Array.isArray(pr.labels) && + pr.labels.every((label) => label && typeof label.name === "string"); + if (!wellFormed) { + console.log("unknown:pull request payload was not a complete pull request"); process.exit(0); } - const labels = (pr.labels || []).map((label) => String(label.name).toLowerCase()); + const labels = pr.labels.map((label) => label.name.toLowerCase()); if (labels.includes(process.env.BLOCK_LABEL.toLowerCase())) { console.log(`skip:pull request ${pr.number} carries the ${process.env.BLOCK_LABEL} label`); } else if (labels.includes(process.env.READY_LABEL.toLowerCase())) { diff --git a/scripts/vercel-build-gate.test.ts b/scripts/vercel-build-gate.test.ts index fc0a6dd97..ec3d96e67 100644 --- a/scripts/vercel-build-gate.test.ts +++ b/scripts/vercel-build-gate.test.ts @@ -150,6 +150,21 @@ describe('vercel build gate', () => { expect(runGate(withPullRequest(READY_BLOCKED)).code).toBe(SKIP); }); + test('an incomplete pull request payload fails open rather than skipping', () => { + const partials = [ + JSON.stringify({ draft: true }), + JSON.stringify({ draft: true, number: 7 }), + JSON.stringify({ draft: true, number: 0, labels: [] }), + JSON.stringify({ draft: true, number: 7, labels: [{}] }), + JSON.stringify({ draft: true, number: 7, labels: 'preview' }), + JSON.stringify({ message: 'Not Found' }), + ]; + + for (const payload of partials) { + expect(runGate(withPullRequest(payload)).code).toBe(BUILD); + } + }); + test('an unreadable API response fails open', () => { expect(runGate(withPullRequest('not json')).code).toBe(BUILD); expect(runGate(withPullRequest('')).code).toBe(BUILD); From 2a78c65ca427f479e59e8114e1e52dcf06ffd5ff Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 14:36:14 +0530 Subject: [PATCH 04/19] docs(vercel): design the preview deployment gate --- ...26-08-21-vercel-preview-deployment-gate.md | 466 ++++++++++++++++++ ...1-vercel-preview-deployment-gate-design.md | 192 ++++++++ 2 files changed, 658 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md create mode 100644 docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md diff --git a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md new file mode 100644 index 000000000..0b6666fe8 --- /dev/null +++ b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md @@ -0,0 +1,466 @@ +# Vercel Preview Deployment Gate Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace the token-bearing Vercel Ignored Build Step with a trusted, CI-green GitHub dispatcher that creates only eligible same-repository Preview deployments. + +**Architecture:** Vercel keeps automatic production deployment for `main` and disables automatic feature-branch deployment with a minimatch branch map. A default-branch GitHub workflow reconciles pull request state after CI success or an eligibility transition, and a trusted Bun controller validates GitHub and Vercel data with shared Zod schemas before creating or canceling exact Preview deployments. + +**Tech Stack:** Bun 1.3.14, TypeScript 5.9, Zod 4, GitHub Actions, GitHub REST API, Vercel REST API, Bun test. + +**Spec:** `docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md` + +## Global Constraints + +- Use Bun only. No npm, pnpm, yarn, turbo. +- Shipped server code must not import Bun built-ins. +- No code comments except functional directives accepted by `bun run check-comments`. +- No em-dash characters in code, docs, commits, branch names, or pull request text. +- No AI attribution anywhere. +- Strict types: no `any`, no non-null assertions, and every external payload is parsed with a Zod schema exported from `@orbit/shared/validators`. +- Tests import from `bun:test`. Shared validator tests mirror `packages/shared/src/validators`; repository scripts use the existing `scripts/*.test.ts` convention. +- No new runtime or development dependency is added. +- The trusted workflow never checks out, fetches, installs, builds, caches, or executes pull request code and never downloads an untrusted artifact. +- Automatic token-backed deployment is limited to open pull requests whose head repository ID equals the base repository ID. +- A Preview requires successful `CI` for the exact current head SHA; `no-preview` wins over `preview`; active ineligible builds are canceled. +- Web-impacting paths are `apps/web/**`, `packages/**`, `package.json`, `bun.lock`, and `tsconfig.base.json`. +- Vercel branch suppression uses `"**": false`, not `"*": false`, because unspecified slash-containing branches default to enabled. +- Git Fork Protection stays enabled. Fork Preview automation is out of scope. +- Branch: `chore/gate-preview-builds`. Merge current `origin/main` before implementation and retain both the root script-test command and main's dependency overrides. + +--- + +## File structure map + +- Create `packages/shared/src/validators/vercel-preview.ts`: schemas and inferred types for GitHub events, GitHub pull request/files/workflow responses, Vercel deployment pages and mutations, and controller environment. +- Modify `packages/shared/src/validators/index.ts`: export the new validator module. +- Create `packages/shared/tests/validators/vercel-preview.test.ts`: accepted and rejected external payload shapes. +- Create `scripts/vercel-preview-policy.ts`: pure eligibility, repository, path, state, and deployment-matching rules. +- Create `scripts/vercel-preview-policy.test.ts`: policy truth tables, label precedence, slash-path coverage, and deployment identity. +- Create `scripts/vercel-preview-deploy.ts`: event resolution, bounded HTTP client, GitHub reconciliation, Vercel creation/cancellation, and CLI entry point. +- Create `scripts/vercel-preview-deploy.test.ts`: injected-fetch orchestration tests with no real network calls. +- Create `scripts/vercel-preview-config.test.ts`: repository configuration, managed labels, workflow trust invariants, and removal of the old token gate. +- Create `.github/workflows/vercel-preview.yml`: default-branch metadata-only dispatcher. +- Modify `apps/web/vercel.json`: remove `ignoreCommand`; disable automatic feature branches and retain `main`. +- Modify `scripts/labels.ts`: manage `preview` and `no-preview`. +- Delete `scripts/vercel-build-gate.sh` and `scripts/vercel-build-gate.test.ts`. +- Rewrite `docs/VERCEL_BUILD_GATE.md`: CI-green behavior, setup, security, billing scope, transitions, and canary procedure. +- Modify `docs/README.md`: link the deployment gate guide. +- Modify the pull request body after push: describe the final architecture and tests, remove stale counts and prohibited attribution. + +--- + +### Task 1: Shared schemas and pure Preview policy + +**Files:** +- Create: `packages/shared/src/validators/vercel-preview.ts` +- Modify: `packages/shared/src/validators/index.ts` +- Test: `packages/shared/tests/validators/vercel-preview.test.ts` +- Create: `scripts/vercel-preview-policy.ts` +- Test: `scripts/vercel-preview-policy.test.ts` + +**Interfaces:** +- Produces `githubPreviewPullRequestSchema`, `githubPreviewPullRequestTargetEventSchema`, `githubPreviewWorkflowRunEventSchema`, `githubPreviewWorkflowDispatchEventSchema`, `githubPreviewFilesSchema`, `githubPreviewWorkflowRunsSchema`, `githubPreviewCommitPullsSchema`, `vercelDeploymentSchema`, `vercelDeploymentsPageSchema`, `vercelCreatedDeploymentSchema`, and `vercelPreviewEnvironmentSchema`. +- Produces inferred `GithubPreviewPullRequest`, `VercelDeployment`, and `VercelPreviewEnvironment` types. +- Produces `PREVIEW_LABEL`, `NO_PREVIEW_LABEL`, `isPreviewEligible`, `isSameRepositoryPullRequest`, `isWebPreviewFile`, `isActiveVercelDeployment`, `isReadyVercelDeployment`, and `matchesVercelPullRequest`. +- The controller in Task 2 consumes every interface above and does not define a second copy of validation or policy. + +- [ ] **Step 1: Write failing validator tests** + +Create fixtures with a 40-character lowercase SHA and assert the schemas accept a complete open pull request, state event, successful CI workflow event, manual input, GitHub file page, workflow-run page, Vercel deployment page, create response, and complete environment. Add rejection cases for a short SHA, missing repository ID, missing draft state, unknown Vercel ready state, nonnumeric manual pull request input, pagination without a finite cursor, and missing secrets. + +```ts +expect(githubPreviewPullRequestSchema.parse(pullRequest).head.sha).toBe(SHA); +expect(() => + githubPreviewPullRequestSchema.parse({ + ...pullRequest, + head: { ...pullRequest.head, sha: 'short' }, + }), +).toThrow(); +expect(() => + vercelPreviewEnvironmentSchema.parse({ + GITHUB_EVENT_NAME: 'pull_request_target', + GITHUB_EVENT_PATH: '/tmp/event.json', + }), +).toThrow(); +``` + +- [ ] **Step 2: Run the validator test and confirm failure** + +Run: `bun test packages/shared/tests/validators/vercel-preview.test.ts` + +Expected: FAIL because `../../src/validators/vercel-preview.ts` does not exist. + +- [ ] **Step 3: Implement the external schemas** + +Use strict discriminants for known event and deployment states, a reusable 40-character lowercase SHA, positive integer identifiers, bounded nonempty strings, and `.passthrough()` only where GitHub or Vercel legitimately returns unrelated fields. The pull request schema must include this complete decision surface: + +```ts +export const githubPreviewPullRequestSchema = z.object({ + number: z.number().int().positive(), + state: z.enum(['open', 'closed']), + draft: z.boolean(), + labels: z.array(z.object({ name: z.string().min(1).max(100) })), + head: z.object({ + sha: gitShaSchema, + ref: z.string().min(1).max(255), + repo: githubPreviewRepositorySchema, + }), + base: z.object({ + ref: z.string().min(1).max(255), + repo: githubPreviewRepositorySchema, + }), +}); +``` + +The workflow-run schema must retain `name`, `event`, `head_sha`, `conclusion`, and pull request numbers. Vercel metadata accepts string, number, boolean, or null values. Vercel pagination accepts finite nonnegative `next` and `prev` cursors or null. Export all inferred types and add `export * from './vercel-preview.ts';` to the validator index. + +- [ ] **Step 4: Run the validator tests** + +Run: `bun test packages/shared/tests/validators/vercel-preview.test.ts` + +Expected: PASS. + +- [ ] **Step 5: Write failing pure-policy tests** + +Cover this truth table and identity behavior: + +```ts +expect(isPreviewEligible(readyPullRequest)).toBe(true); +expect(isPreviewEligible(draftPullRequest)).toBe(false); +expect(isPreviewEligible(withLabels(draftPullRequest, ['preview']))).toBe(true); +expect(isPreviewEligible(withLabels(readyPullRequest, ['preview', 'no-preview']))).toBe(false); +expect(isSameRepositoryPullRequest(readyPullRequest)).toBe(true); +expect(isSameRepositoryPullRequest(forkPullRequest)).toBe(false); +expect(isWebPreviewFile('apps/web/src/app/page.tsx')).toBe(true); +expect(isWebPreviewFile('packages/shared/src/index.ts')).toBe(true); +expect(isWebPreviewFile('apps/realtime/src/index.ts')).toBe(false); +expect(isWebPreviewFile('docs/README.md')).toBe(false); +``` + +Add deployment fixtures proving that a Preview must match repository ID, pull request number, ref, and SHA; a production deployment never matches; `QUEUED`, `INITIALIZING`, and `BUILDING` are active; only `READY` is ready. + +- [ ] **Step 6: Run the policy tests and confirm failure** + +Run: `bun test scripts/vercel-preview-policy.test.ts` + +Expected: FAIL because `./vercel-preview-policy.ts` does not exist. + +- [ ] **Step 7: Implement the pure policy** + +Use fixed label and path values: + +```ts +export const PREVIEW_LABEL = 'preview'; +export const NO_PREVIEW_LABEL = 'no-preview'; + +export function isPreviewEligible(pullRequest: GithubPreviewPullRequest): boolean { + const labels = new Set(pullRequest.labels.map(({ name }) => name.toLowerCase())); + if (labels.has(NO_PREVIEW_LABEL)) return false; + return !pullRequest.draft || labels.has(PREVIEW_LABEL); +} + +export function isWebPreviewFile(filename: string): boolean { + return ( + filename.startsWith('apps/web/') || + filename.startsWith('packages/') || + filename === 'package.json' || + filename === 'bun.lock' || + filename === 'tsconfig.base.json' + ); +} +``` + +`matchesVercelPullRequest` must reject `target === 'production'` and compare normalized metadata values for `githubRepoId`, `githubPrId`, `githubCommitRef`, and, when supplied, `githubCommitSha`. + +- [ ] **Step 8: Run Task 1 checks and commit** + +Run: `bun test packages/shared/tests/validators/vercel-preview.test.ts scripts/vercel-preview-policy.test.ts` + +Run: `bun run --filter '@orbit/shared' typecheck && bun x tsc -p scripts/tsconfig.json --noEmit` + +Expected: all commands PASS. + +Commit: `feat(ci): define preview deployment policy` + +--- + +### Task 2: Trusted GitHub and Vercel reconciler + +**Files:** +- Create: `scripts/vercel-preview-deploy.ts` +- Test: `scripts/vercel-preview-deploy.test.ts` + +**Interfaces:** +- Consumes all Task 1 schemas, types, constants, and pure policy functions. +- Produces `PreviewRuntime`, `PreviewResult`, and `reconcileVercelPreviews(runtime): Promise`. +- `PreviewRuntime` supplies `env`, `readText`, `fetch`, `sleep`, and `log` so tests never access the network, process secrets, or real event files. +- `PreviewResult` is a discriminated union with `kind: 'skipped' | 'created' | 'canceled'`, a pull request number, a stable reason, and an optional deployment ID and URL only for a mutation. + +- [ ] **Step 1: Write failing event and eligibility tests** + +Use an injected fetch router that records method, URL, headers, and parsed body. Cover: + +- A successful `workflow_run` for the current ready same-repository head creates one deployment. +- `pull_request_target` ready and `preview` transitions create only when the exact SHA already has a successful CI run. +- Draft without `preview`, either control label combination, closed PR, stale event SHA, fork head, failed CI, in-progress CI, and unrelated files create zero deployments. +- `workflow_dispatch` parses its pull request input and follows the same live-state and CI checks. +- An empty workflow-run pull request list falls back to the commit-pulls endpoint. +- GitHub files and commit pulls paginate until a short page, with a hard page limit. + +The creation assertion must inspect the exact request: + +```ts +expect(createRequest.method).toBe('POST'); +expect(createRequest.url).toContain('/v13/deployments'); +expect(createRequest.url).toContain('forceNew=1'); +expect(createRequest.body).toEqual({ + name: 'orbit', + project: 'prj_orbit', + gitSource: { + type: 'github', + repoId: 123, + ref: 'feature/preview', + sha: SHA, + }, + meta: { + githubCommitOrg: 'Noveum', + githubCommitRef: 'feature/preview', + githubCommitRepo: 'orbit', + githubCommitSha: SHA, + githubPrId: '341', + githubRepoId: '123', + }, +}); +expect(createRequest.body).not.toHaveProperty('target'); +``` + +- [ ] **Step 2: Run the focused tests and confirm failure** + +Run: `bun test scripts/vercel-preview-deploy.test.ts --test-name-pattern 'event|eligibility|creates'` + +Expected: FAIL because `./vercel-preview-deploy.ts` does not exist. + +- [ ] **Step 3: Implement event resolution and GitHub reconciliation** + +Parse `GITHUB_EVENT_NAME`, read `GITHUB_EVENT_PATH`, and select the matching event schema. Resolve candidates as follows: + +```ts +type PreviewCandidate = { + readonly number: number; + readonly expectedHeadSha: string | null; + readonly ciProven: boolean; +}; +``` + +- `pull_request_target`: one candidate from the event PR number and event head SHA, with `ciProven: false`. +- `workflow_run`: no candidates unless the workflow is `CI`, source event is `pull_request`, conclusion is `success`, and action is `completed`; candidates use the workflow head SHA and `ciProven: true`; use the commit-pulls endpoint when the payload list is empty. +- `workflow_dispatch`: one candidate from the positive integer input, no expected SHA, with `ciProven: false`. + +For every candidate, refetch `/repos/{owner}/{repo}/pulls/{number}` and require an open PR targeting the event repository and `main`, a same-repository head, and an exact expected SHA when one exists. If `ciProven` is false, query `/repos/{owner}/{repo}/actions/workflows/ci.yml/runs?event=pull_request&head_sha={sha}&status=completed&per_page=100` and require a successful run with the same head SHA. Query pull request files with `per_page=100&page=N` and require at least one `isWebPreviewFile` match before a create. + +- [ ] **Step 4: Write failing idempotency and cancellation tests** + +Cover Vercel pages with an exact deployment on page 2 and prove: + +- Exact `QUEUED`, `INITIALIZING`, `BUILDING`, or `READY` produces no create call. +- Exact `CANCELED` or `ERROR` allows exactly one create call. +- Same SHA in another project, production target, repository, pull request, or ref does not suppress creation. +- An ineligible state event cancels every matching active Preview for that PR and does not cancel ready, canceled, errored, production, another ref, or another repository. +- A second reconciliation after creation sees the created metadata and is a no-op. + +- [ ] **Step 5: Run the idempotency tests and confirm failure** + +Run: `bun test scripts/vercel-preview-deploy.test.ts --test-name-pattern 'existing|cancel|duplicate|pagination'` + +Expected: FAIL because deployment listing, matching, and cancellation are not implemented. + +- [ ] **Step 6: Implement bounded Vercel reconciliation** + +List `/v6/deployments` with `teamId`, `projectId`, `limit=100`, and a metadata filter. Follow the validated `pagination.next` cursor with a finite page cap. Filter again in trusted code with `matchesVercelPullRequest` before using any result. + +For eligible PRs, return a skipped result when an exact active or ready deployment exists. Otherwise create one deployment with `POST /v13/deployments?teamId={teamId}&forceNew=1`, omitted `target`, exact Git source, and the metadata shown in Step 1. + +For ineligible state events, list by pull request metadata and call `PATCH /v12/deployments/{id}/cancel?teamId={teamId}` only for matching active Preview deployments. A CI failure does not cancel a previously ready Preview; cancellation is driven by current PR state. + +- [ ] **Step 7: Write failing transport and secret-safety tests** + +Cover missing configuration, 401, 403, 429 with `Retry-After`, 500, timeout abort, invalid JSON, invalid schema, exhausted pagination, and malformed create/cancel responses. Assert 429 and 5xx use at most three total attempts, 401 and 403 do not retry, and no thrown error, log line, URL, or serialized result contains either token. + +- [ ] **Step 8: Implement the bounded JSON client and CLI entry point** + +The JSON client must apply a 15 second abort timeout per request, retry only 429 and 5xx responses, cap attempts at three, parse response text as JSON, validate with the supplied schema, and throw a redacted error on failure. Inject `sleep` for tests. Never include request headers in an error. + +The executable path uses `Bun.file(path).text()`, global `fetch`, a timer-backed sleep, and `console.log`. Guard it with `if (import.meta.main)`, set `process.exitCode = 1` on error, and print only the redacted error message. + +- [ ] **Step 9: Run Task 2 checks and commit** + +Run: `bun test scripts/vercel-preview-deploy.test.ts` + +Run: `bun x tsc -p scripts/tsconfig.json --noEmit && bun run lint -- scripts/vercel-preview-deploy.ts scripts/vercel-preview-deploy.test.ts` + +Expected: all commands PASS. + +Commit: `feat(ci): deploy previews after successful checks` + +--- + +### Task 3: Trusted workflow, Vercel configuration, labels, and operations guide + +**Files:** +- Create: `.github/workflows/vercel-preview.yml` +- Modify: `apps/web/vercel.json` +- Modify: `scripts/labels.ts` +- Delete: `scripts/vercel-build-gate.sh` +- Delete: `scripts/vercel-build-gate.test.ts` +- Create: `scripts/vercel-preview-config.test.ts` +- Rewrite: `docs/VERCEL_BUILD_GATE.md` +- Modify: `docs/README.md` + +**Interfaces:** +- Consumes `scripts/vercel-preview-deploy.ts` as the only workflow command. +- Requires secret `VERCEL_TOKEN` and variables `VERCEL_TEAM_ID`, `VERCEL_PROJECT_ID`, and `VERCEL_PROJECT_NAME`. +- Uses immutable `actions/checkout` SHA `3d3c42e5aac5ba805825da76410c181273ba90b1` and immutable `oven-sh/setup-bun` SHA `0c5077e51419868618aeaa5fe8019c62421857d6`. +- Produces managed labels named exactly `preview` and `no-preview`. + +- [ ] **Step 1: Write failing repository configuration tests** + +Parse `apps/web/vercel.json`, read the workflow and docs as text, and import `LABELS`. Assert: + +```ts +expect(vercel.git.deploymentEnabled).toEqual({ '**': false, main: true }); +expect(vercel).not.toHaveProperty('ignoreCommand'); +expect(LABELS.filter(({ name }) => name === 'preview')).toHaveLength(1); +expect(LABELS.filter(({ name }) => name === 'no-preview')).toHaveLength(1); +expect(workflow).toContain('pull_request_target:'); +expect(workflow).toContain('workflow_run:'); +expect(workflow).toContain('workflow_dispatch:'); +expect(workflow).toContain('persist-credentials: false'); +expect(workflow).not.toContain('github.event.pull_request.head.ref'); +expect(allGateFiles).not.toContain('BUILD_GATE_GITHUB_TOKEN'); +``` + +Also test the `**` rule against `feature`, `feature/preview`, and `codex/review/pr341` using the same `minimatch` semantics documented by Vercel. Do not add a dependency: implement the narrow expected assertion by checking that the configured key is exactly `**` and enumerate the branch examples in the test name. + +- [ ] **Step 2: Run the configuration test and confirm failure** + +Run: `bun test scripts/vercel-preview-config.test.ts` + +Expected: FAIL because the workflow and managed labels do not exist and `ignoreCommand` remains. + +- [ ] **Step 3: Replace the Vercel gate and manage labels** + +Remove `ignoreCommand` from `apps/web/vercel.json` and add: + +```json +"git": { + "deploymentEnabled": { + "**": false, + "main": true + } +} +``` + +Add `preview` and `no-preview` beside the status labels in `scripts/labels.ts`, with descriptions matching the design. Delete the old shell gate and its tests. Keep the root `bun test scripts` command so all new script tests remain part of CI. + +- [ ] **Step 4: Add the default-branch workflow** + +Create a workflow named `Vercel Preview` with this trigger and trust boundary: + +```yaml +on: + pull_request_target: + types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled] + workflow_run: + workflows: [CI] + types: [completed] + workflow_dispatch: + inputs: + pull_request: + description: Pull request number + required: true + type: number + +permissions: + actions: read + contents: read + pull-requests: read + +concurrency: + group: vercel-preview-${{ github.event.pull_request.number || github.event.workflow_run.pull_requests[0].number || inputs.pull_request || github.event.workflow_run.head_sha }} + cancel-in-progress: false +``` + +The job condition allows state events and manual dispatch, and allows a workflow run only when `github.event.workflow_run.event == 'pull_request'`. Checkout the trusted default-branch workflow commit using the pinned checkout action, `repository: ${{ github.repository }}`, `ref: ${{ github.sha }}`, `persist-credentials: false`, `submodules: false`, and `lfs: false`. Set up Bun 1.3.14 with the pinned setup action, run `bun install --frozen-lockfile`, then run `bun scripts/vercel-preview-deploy.ts` with tokens and settings scoped to that step through `env`. + +- [ ] **Step 5: Rewrite the operations guide and documentation index** + +Document the exact eligibility table, CI-green timing, same-repository restriction, active cancellation, web path list, Vercel API behavior, GitHub secret and variables, label synchronization, Git Fork Protection, the deployment-count caveat, the repository-controlled cost-policy limitation, removal of all four old `BUILD_GATE_*` Vercel values, manual recovery, and the post-merge canary. Link the guide from `docs/README.md` under the contributor/operations entries. + +Do not claim that ignored builds are free, that Ready alone is trust, that fork previews are automatic, or that the privileged workflow can be exercised before it exists on `main`. + +- [ ] **Step 6: Run Task 3 checks and commit** + +Run: `bun test scripts/vercel-preview-config.test.ts scripts/vercel-preview-policy.test.ts scripts/vercel-preview-deploy.test.ts packages/shared/tests/validators/vercel-preview.test.ts` + +Run: `bun run lint && bun run check-comments && bun run check-bytes && bun run check-bun-imports && bun run typecheck` + +Expected: all commands PASS. + +Run: `rg -n 'BUILD_GATE_|vercel-build-gate|Generated with|Claude|—' apps/web/vercel.json scripts docs/VERCEL_BUILD_GATE.md .github/workflows/vercel-preview.yml` + +Expected: no old gate setting, prohibited attribution, or em-dash match. A link or historical plan outside this task's changed files is not edited. + +Commit: `chore(vercel): gate previews after CI` + +--- + +### Task 4: Full verification and existing pull request handoff + +**Files:** +- Modify only files required by current-main conflict resolution or a verification failure caused by this branch. +- Update remote pull request 341 body and review-thread replies after the branch is verified and pushed. + +**Interfaces:** +- Consumes the complete branch from Tasks 1 through 3. +- Produces a branch merged with current `origin/main`, a green local verification record, an updated remote branch, an accurate PR body, and resolved addressed threads. + +- [ ] **Step 1: Merge the latest main and rerun focused tests** + +Run: `git fetch origin main && git merge --no-edit origin/main` + +Resolve `package.json` by retaining `"test": "bun test scripts && bun run --filter '*' test"` and every current-main override. Do not touch the dirty ordinary checkout. + +Run: `bun install --frozen-lockfile` + +Run: `bun test scripts/vercel-preview-config.test.ts scripts/vercel-preview-policy.test.ts scripts/vercel-preview-deploy.test.ts packages/shared/tests/validators/vercel-preview.test.ts` + +Expected: PASS. + +- [ ] **Step 2: Run the complete repository verification** + +Run: `ORBIT_TEST_LANE=preview-gate bun run verify` + +Expected: lint, comment policy, source-byte check, Bun-import check, dependency dedupe, all typechecks, and all tests PASS. If a database service is unavailable, start the repository's existing infrastructure and initialize only the isolated `preview-gate` test lane before rerunning. + +- [ ] **Step 3: Review the final diff and operational text** + +Run: `git diff --check origin/main...HEAD` + +Run: `git diff --stat origin/main...HEAD && git log --oneline origin/main..HEAD` + +Run: `rg -n 'Generated with|Claude|—|BUILD_GATE_GITHUB_TOKEN|BUILD_GATE_WATCH_PATHS|BUILD_GATE_READY_LABEL|BUILD_GATE_BLOCK_LABEL' $(git diff --name-only --diff-filter=ACMR origin/main...HEAD)` + +Expected: no whitespace error, prohibited attribution, em dash, or old token/config reference in changed files except the operations guide's explicit removal instructions for the four legacy setting names. + +- [ ] **Step 4: Push and update pull request 341** + +Push `chore/gate-preview-builds` after all local checks pass. Replace the PR body with the final motivation, architecture, security boundary, setup requirements, test evidence, and the honest post-merge canary limitation. Remove the stale test count and all attribution. + +Reply to the unresolved event-trigger review thread with the trusted workflow and exact event paths. Reply to the old Greptile thread with the final test coverage. Resolve a thread only when its cited issue is demonstrably addressed by the pushed diff. + +- [ ] **Step 5: Inspect hosted checks and merge readiness** + +Wait for CodeRabbit, Greptile, GitHub Actions, and Vercel/GitHub deployment checks to complete. Reconcile every current head comment and check. Do not merge while a required check is red, a current thread is unresolved, the branch is behind `main`, or a human approval is still required. + +The PR is ready to merge when current-main ancestry, hosted checks, review threads, required approval, labels, GitHub secret/variables, and the documented Vercel project settings are all confirmed. The privileged deployment behavior receives its real canary only after the workflow exists on `main`. diff --git a/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md new file mode 100644 index 000000000..d776e7269 --- /dev/null +++ b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md @@ -0,0 +1,192 @@ +# Vercel Preview Deployment Gate Design + +## Problem + +Orbit currently lets Vercel create a Preview deployment for every pushed commit. Most pull +request commits do not need a live preview, and failed commits should not consume Vercel build +minutes. Pull request 341 attempted to reduce that cost with a repository-provided Ignored Build +Step that reads pull request state using a GitHub token. + +That design has three structural problems. Vercel runs the ignored command from the pull request +checkout with environment access, so the pull request controls code that receives the GitHub +token. Ready-for-review and label changes do not create a Vercel deployment, so they cannot act as +the promised preview button. Ignored builds also still count as deployments and occupy concurrent +build capacity, even when they avoid the application build. + +## Outcome + +Vercel automatically deploys `main`, but does not automatically deploy feature branches. A +trusted GitHub Actions workflow creates a Preview deployment only when all of these conditions are +true for the current pull request head: + +- The pull request comes from the Orbit repository rather than a fork. +- The pull request is ready for review, or carries the managed `preview` label. +- The pull request does not carry the managed `no-preview` label. +- The `CI` workflow completed successfully for the exact head SHA. +- At least one changed file affects the web deployment. +- No active or ready Preview deployment already exists for the same pull request and SHA. + +The `no-preview` label wins when both control labels are present. Converting a pull request back to +draft also makes it ineligible unless `preview` remains present. When a state change makes a pull +request ineligible, the workflow cancels active Preview deployments for that pull request but does +not delete completed previews. + +This design optimizes for avoided Vercel application builds. The small GitHub metadata job still +uses GitHub Actions time, and an API-created eligible preview is still a normal Vercel deployment. + +## Deployment ownership + +`apps/web/vercel.json` replaces the Ignored Build Step with a Git deployment rule: + +```json +{ + "git": { + "deploymentEnabled": { + "**": false, + "main": true + } + } +} +``` + +The minimatch catch-all prevents automatic Preview deployments, including branch names containing +slashes. The explicit `main` rule preserves the existing production path. Vercel documents that a +true match wins when more than one branch rule matches, so `main` remains enabled even though it +also matches `**`. Unspecified branches default to enabled, which is why a single-star pattern is +not sufficient. + +This repository setting is an operational cost policy, not a security boundary. A same-repository +author can change `vercel.json` in a branch, and Vercel does not document that the deployment rule +is always read from the default branch. Git Fork Protection remains the security boundary for +forks. A future move to a disconnected or dedicated Preview project can make automatic-deployment +suppression independent of pull request contents; that external migration is not required to +remove the token exposure or to stop normal feature-branch builds in this change. + +The repository removes `scripts/vercel-build-gate.sh` and its tests. No secret is made available to +code from a pull request checkout, and the gate no longer depends on Vercel system environment +variables or an ignored-build exit-code convention. + +## Trusted workflow + +`.github/workflows/vercel-preview.yml` runs from the default branch through three trusted event +paths: + +- `pull_request_target` handles `opened`, `reopened`, `ready_for_review`, + `converted_to_draft`, `labeled`, and `unlabeled` state transitions. +- `workflow_run` handles a completed `CI` workflow and proceeds only when the source event was a + pull request and the conclusion was success. +- `workflow_dispatch` accepts a pull request number for maintainer recovery when an event was + missed or a deployment needs to be retried. + +The workflow checks out only the trusted workflow SHA from the default branch with persisted Git +credentials disabled. It never +fetches, checks out, installs, builds, caches, or executes the pull request head, and it never +downloads an artifact from the untrusted CI run. Pull request data appears only in environment +values and API request bodies, never as shell source. + +Permissions are limited to `actions: read`, `contents: read`, and `pull-requests: read`. The Vercel +token is a GitHub Actions secret. Team ID, project ID, and project name are Actions variables. +Concurrency is keyed by pull request number when the event supplies one and does not cancel an +in-progress reconciler. Vercel metadata checks provide a second idempotency boundary. A rare +workflow-run payload without a pull request number uses its exact head SHA until the controller +resolves the pull request through GitHub. + +## Controller and validation + +`scripts/vercel-preview-deploy.ts` is the only executable controller. It imports Zod schemas from +`@orbit/shared/validators` and validates the GitHub event payload, every GitHub response, every +Vercel response, and configuration before using them. + +The controller resolves one or more pull request numbers from the triggering event, then refetches +each pull request from GitHub. An event is stale when its recorded head SHA no longer equals the +live pull request head. Stale events are no-ops. Closed pull requests and fork heads are also +no-ops. + +For a pull request state event or manual dispatch, the controller queries completed runs of +`.github/workflows/ci.yml` and requires a successful `pull_request` run for the exact head SHA. A +successful `workflow_run` event already supplies that fact, but the controller still verifies +that its head SHA matches the live pull request. + +Changed files are read from the paginated pull request files endpoint. The web deployment is +affected when a filename is below `apps/web/` or `packages/`, or is exactly `package.json`, +`bun.lock`, or `tsconfig.base.json`. Configuration is fixed in trusted code rather than split +between Vercel and GitHub settings. + +GitHub and Vercel requests have a finite timeout. Rate-limit and server failures use bounded +retries. Authentication failures, malformed payloads, exhausted pagination, and unsuccessful +mutations fail the workflow visibly. Logs never contain either token. + +## Vercel API contract + +Before mutation, the controller lists deployments for the configured project and filters them by +Preview target and metadata for repository ID, pull request number, branch ref, and commit SHA. +Pagination is bounded and every page is validated. + +If an exact deployment is queued, initializing, building, or ready, deployment is a no-op. If no +such deployment exists, the controller calls Vercel's Create Deployment endpoint with the linked +GitHub repository ID, exact branch ref, and exact SHA. `target` is omitted so Vercel selects the +Preview environment. Metadata repeats the repository, pull request, branch, and SHA so later runs +can identify the deployment without guessing. + +When current pull request state is ineligible, active deployments associated with that pull +request are canceled through Vercel's cancel endpoint. The filter requires the configured project, +Preview target, repository ID, pull request number, and branch ref before cancellation. Ready, +failed, and canceled deployments are left unchanged. + +## Fork policy + +The token-backed workflow never creates a deployment for a fork. This preserves the security +boundary Vercel documents for Git Fork Protection: unreviewed fork code must not automatically +receive Preview environment variables or OIDC authority. + +Fork previews remain a manual maintainer decision. A maintainer can use Vercel's explicit Git +reference deployment flow after reviewing the code, or deploy into a separate Preview project +whose environment has no sensitive values. The `preview` label does not authorize a fork in this +change. Automating fork previews requires a separate design for a credential-free project and an +auditable maintainer approval signal. + +## Labels + +`preview` and `no-preview` become fixed entries in `scripts/labels.ts`. This makes the documented +controls available in GitHub and prevents `scripts/sync-labels.ts --prune` from deleting them. + +- `preview`: build a Preview for an otherwise eligible draft after CI succeeds. +- `no-preview`: suppress Preview creation and cancel an active Preview build. + +## Setup and operations + +The repository requires these GitHub Actions settings: + +- Secret `VERCEL_TOKEN`, scoped to the team that owns the Orbit project. +- Variable `VERCEL_TEAM_ID`. +- Variable `VERCEL_PROJECT_ID`. +- Variable `VERCEL_PROJECT_NAME`, currently `orbit`. + +After the new flow is merged, the old `BUILD_GATE_GITHUB_TOKEN`, +`BUILD_GATE_WATCH_PATHS`, `BUILD_GATE_READY_LABEL`, and `BUILD_GATE_BLOCK_LABEL` values should be +removed from Vercel. Git Fork Protection stays enabled. + +The workflow is defined by the default branch, so pull request 341 can prove its controller and +workflow structure locally but cannot exercise the new privileged event path until the workflow +has landed on `main`. The first same-repository test pull request after merge is the production +canary for Vercel Git metadata, the Preview URL, and label transitions. + +## Tests + +Shared validator tests cover accepted payloads and rejection of missing identifiers, invalid SHAs, +unknown states, and malformed pagination. + +Controller tests use injected fetch and delay functions. They cover ready and draft policy, +control-label precedence, state transitions, CI success for the exact SHA, stale events, closed +pull requests, fork refusal, relevant and irrelevant paths, GitHub pagination, Vercel pagination, +existing active and ready deployments, one exact create, active cancellation, bounded retries, +timeouts, authentication failures, invalid JSON, invalid response shapes, and missing settings. + +Repository checks cover the branch deployment map, the managed labels, removal of the old ignored +command and token references, and discovery of the script tests by the root test command. + +## Out of scope + +Automatic fork previews, a new credential-free Vercel project, deletion of completed previews, +promotion of a Preview to production, application database migrations, and changes to the main +production deployment path are outside this change. From e25561650f1929f44801f7cecdae2e9e7af813ce Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 14:37:46 +0530 Subject: [PATCH 05/19] docs(vercel): tighten the preview gate plan --- .../plans/2026-08-21-vercel-preview-deployment-gate.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md index 0b6666fe8..5c243ec38 100644 --- a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md +++ b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md @@ -257,7 +257,7 @@ type PreviewCandidate = { - `workflow_run`: no candidates unless the workflow is `CI`, source event is `pull_request`, conclusion is `success`, and action is `completed`; candidates use the workflow head SHA and `ciProven: true`; use the commit-pulls endpoint when the payload list is empty. - `workflow_dispatch`: one candidate from the positive integer input, no expected SHA, with `ciProven: false`. -For every candidate, refetch `/repos/{owner}/{repo}/pulls/{number}` and require an open PR targeting the event repository and `main`, a same-repository head, and an exact expected SHA when one exists. If `ciProven` is false, query `/repos/{owner}/{repo}/actions/workflows/ci.yml/runs?event=pull_request&head_sha={sha}&status=completed&per_page=100` and require a successful run with the same head SHA. Query pull request files with `per_page=100&page=N` and require at least one `isWebPreviewFile` match before a create. +For every candidate, refetch `/repos/{owner}/{repo}/pulls/{number}` and require an open PR targeting the event repository and `main`, a same-repository head, and an exact expected SHA when one exists. Evaluate current labels and draft state next; an ineligible state event follows the cancellation path without requiring successful CI. For an eligible candidate whose `ciProven` is false, query `/repos/{owner}/{repo}/actions/workflows/ci.yml/runs?event=pull_request&head_sha={sha}&status=completed&per_page=100` and require a successful run with the same head SHA. Query pull request files with `per_page=100&page=N` and require at least one `isWebPreviewFile` match before a create. - [ ] **Step 4: Write failing idempotency and cancellation tests** @@ -407,7 +407,9 @@ Run: `bun run lint && bun run check-comments && bun run check-bytes && bun run c Expected: all commands PASS. -Run: `rg -n 'BUILD_GATE_|vercel-build-gate|Generated with|Claude|—' apps/web/vercel.json scripts docs/VERCEL_BUILD_GATE.md .github/workflows/vercel-preview.yml` +Run: `rg -n 'BUILD_GATE_|vercel-build-gate|Generated with|Claude|—' apps/web/vercel.json scripts .github/workflows/vercel-preview.yml` + +Run: `rg -n 'Generated with|Claude|—' docs/VERCEL_BUILD_GATE.md docs/README.md` Expected: no old gate setting, prohibited attribution, or em-dash match. A link or historical plan outside this task's changed files is not edited. @@ -449,7 +451,7 @@ Run: `git diff --check origin/main...HEAD` Run: `git diff --stat origin/main...HEAD && git log --oneline origin/main..HEAD` -Run: `rg -n 'Generated with|Claude|—|BUILD_GATE_GITHUB_TOKEN|BUILD_GATE_WATCH_PATHS|BUILD_GATE_READY_LABEL|BUILD_GATE_BLOCK_LABEL' $(git diff --name-only --diff-filter=ACMR origin/main...HEAD)` +Run the attribution and em-dash scan against every changed text file. Run the four legacy-setting scan against changed code and workflow files, excluding `docs/VERCEL_BUILD_GATE.md`, where the removal instructions intentionally name them. Expected: no whitespace error, prohibited attribution, em dash, or old token/config reference in changed files except the operations guide's explicit removal instructions for the four legacy setting names. From ae5fefc50dd47c6db960588d6fc72dacb1c3a404 Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 14:48:52 +0530 Subject: [PATCH 06/19] feat(ci): define preview deployment policy --- packages/shared/src/validators/index.ts | 1 + .../shared/src/validators/vercel-preview.ts | 235 ++++++++++++++++++ .../tests/validators/vercel-preview.test.ts | 226 +++++++++++++++++ scripts/vercel-preview-policy.test.ts | 139 +++++++++++ scripts/vercel-preview-policy.ts | 56 +++++ 5 files changed, 657 insertions(+) create mode 100644 packages/shared/src/validators/vercel-preview.ts create mode 100644 packages/shared/tests/validators/vercel-preview.test.ts create mode 100644 scripts/vercel-preview-policy.test.ts create mode 100644 scripts/vercel-preview-policy.ts diff --git a/packages/shared/src/validators/index.ts b/packages/shared/src/validators/index.ts index e4c6149e9..b46724273 100644 --- a/packages/shared/src/validators/index.ts +++ b/packages/shared/src/validators/index.ts @@ -17,6 +17,7 @@ export * from './profile.ts'; export * from './project.ts'; export * from './team.ts'; export * from './upload.ts'; +export * from './vercel-preview.ts'; export * from './view.ts'; export * from './view-preference.ts'; export * from './vitals.ts'; diff --git a/packages/shared/src/validators/vercel-preview.ts b/packages/shared/src/validators/vercel-preview.ts new file mode 100644 index 000000000..29ef28858 --- /dev/null +++ b/packages/shared/src/validators/vercel-preview.ts @@ -0,0 +1,235 @@ +import { z } from 'zod'; + +const boundedString = (maximum: number) => z.string().trim().min(1).max(maximum); +const positiveIntegerSchema = z.number().int().positive(); +const nonnegativeCursorSchema = z.number().finite().int().nonnegative().nullable(); +const githubWorkflowConclusionSchema = z + .enum([ + 'success', + 'failure', + 'neutral', + 'cancelled', + 'skipped', + 'timed_out', + 'action_required', + 'startup_failure', + 'stale', + ]) + .nullable(); +const githubWorkflowRunStatusSchema = z.enum([ + 'queued', + 'in_progress', + 'completed', + 'waiting', + 'requested', + 'pending', +]); +const vercelMetadataValueSchema = z.union([z.string(), z.number().finite(), z.boolean(), z.null()]); + +export const gitShaSchema = z.string().regex(/^[a-f0-9]{40}$/); + +export const githubPreviewRepositorySchema = z + .object({ + id: positiveIntegerSchema, + name: boundedString(100), + owner: z.object({ login: boundedString(100) }), + }) + .passthrough(); +export type GithubPreviewRepository = z.infer; + +export const githubPreviewPullRequestSchema = z.object({ + number: positiveIntegerSchema, + state: z.enum(['open', 'closed']), + draft: z.boolean(), + labels: z.array(z.object({ name: boundedString(100) })).max(100), + head: z.object({ + sha: gitShaSchema, + ref: boundedString(255), + repo: githubPreviewRepositorySchema, + }), + base: z.object({ + ref: boundedString(255), + repo: githubPreviewRepositorySchema, + }), +}); +export type GithubPreviewPullRequest = z.infer; + +export const githubPreviewRefSchema = z + .object({ object: z.object({ sha: gitShaSchema }).passthrough() }) + .passthrough(); +export type GithubPreviewRef = z.infer; + +export const githubPreviewWorkflowSchema = z + .object({ + id: positiveIntegerSchema, + name: boundedString(255), + path: z + .string() + .regex(/^\.github\/workflows\/.+\.ya?ml$/) + .max(1024), + state: z.enum([ + 'active', + 'deleted', + 'disabled_fork', + 'disabled_inactivity', + 'disabled_manually', + ]), + }) + .passthrough(); +export type GithubPreviewWorkflow = z.infer; + +export const githubPreviewPullRequestTargetEventSchema = z + .object({ + action: z.enum([ + 'opened', + 'reopened', + 'ready_for_review', + 'converted_to_draft', + 'labeled', + 'unlabeled', + ]), + number: positiveIntegerSchema, + pull_request: githubPreviewPullRequestSchema, + repository: githubPreviewRepositorySchema, + }) + .passthrough(); +export type GithubPreviewPullRequestTargetEvent = z.infer< + typeof githubPreviewPullRequestTargetEventSchema +>; + +const githubPreviewWorkflowPullRequestSchema = z + .object({ + number: positiveIntegerSchema, + head: z.object({ + sha: gitShaSchema, + ref: boundedString(255), + repo: githubPreviewRepositorySchema, + }), + base: z.object({ + sha: gitShaSchema, + ref: boundedString(255), + repo: githubPreviewRepositorySchema, + }), + }) + .passthrough(); + +const githubPreviewWorkflowRunSchema = z + .object({ + id: positiveIntegerSchema, + workflow_id: positiveIntegerSchema, + name: boundedString(255), + event: boundedString(100), + head_sha: gitShaSchema, + status: githubWorkflowRunStatusSchema, + conclusion: githubWorkflowConclusionSchema, + created_at: z.string().datetime({ offset: true }), + pull_requests: z.array(githubPreviewWorkflowPullRequestSchema).max(100), + }) + .passthrough(); +export type GithubPreviewWorkflowRun = z.infer; + +export const githubPreviewWorkflowRunEventSchema = z + .object({ + action: z.enum(['completed', 'requested', 'in_progress']), + repository: githubPreviewRepositorySchema, + workflow_run: githubPreviewWorkflowRunSchema, + }) + .passthrough(); +export type GithubPreviewWorkflowRunEvent = z.infer; + +export const githubPreviewWorkflowDispatchEventSchema = z + .object({ + inputs: z.object({ + pull_request: z + .string() + .regex(/^[1-9]\d*$/) + .max(10), + }), + repository: githubPreviewRepositorySchema, + }) + .passthrough(); +export type GithubPreviewWorkflowDispatchEvent = z.infer< + typeof githubPreviewWorkflowDispatchEventSchema +>; + +export const githubPreviewFilesSchema = z.array( + z.object({ filename: boundedString(1024) }).passthrough(), +); +export type GithubPreviewFiles = z.infer; + +export const githubPreviewWorkflowRunsSchema = z + .object({ workflow_runs: z.array(githubPreviewWorkflowRunSchema).max(100) }) + .passthrough(); +export type GithubPreviewWorkflowRuns = z.infer; + +export const githubPreviewCommitPullsSchema = z.array( + z + .object({ + number: positiveIntegerSchema, + }) + .passthrough(), +); +export type GithubPreviewCommitPulls = z.infer; + +export const vercelDeploymentSchema = z + .object({ + uid: boundedString(100), + url: boundedString(255).nullable(), + target: z.enum(['production', 'staging']).nullable().optional(), + readyState: z.enum([ + 'QUEUED', + 'INITIALIZING', + 'BUILDING', + 'READY', + 'ERROR', + 'CANCELED', + 'BLOCKED', + 'DELETED', + ]), + meta: z.record(z.string(), vercelMetadataValueSchema), + }) + .passthrough(); +export type VercelDeployment = z.infer; + +export const vercelDeploymentsPageSchema = z + .object({ + deployments: z.array(vercelDeploymentSchema).max(100), + pagination: z.object({ next: nonnegativeCursorSchema, prev: nonnegativeCursorSchema }), + }) + .passthrough(); +export type VercelDeploymentsPage = z.infer; + +export const vercelCreatedDeploymentSchema = z + .object({ + id: boundedString(100), + url: boundedString(255).nullable(), + target: z.enum(['production', 'staging']).nullable().optional(), + readyState: z.enum([ + 'QUEUED', + 'INITIALIZING', + 'BUILDING', + 'READY', + 'ERROR', + 'CANCELED', + 'BLOCKED', + 'DELETED', + ]), + meta: z.record(z.string(), vercelMetadataValueSchema), + }) + .passthrough(); +export type VercelCreatedDeployment = z.infer; + +export const vercelPreviewEnvironmentSchema = z.object({ + GITHUB_EVENT_NAME: z.enum(['pull_request_target', 'workflow_run', 'workflow_dispatch']), + GITHUB_EVENT_PATH: boundedString(4096), + GITHUB_REPOSITORY: z + .string() + .regex(/^[^/\s]+\/[^/\s]+$/) + .max(255), + GITHUB_TOKEN: boundedString(2048), + VERCEL_TOKEN: boundedString(2048), + VERCEL_TEAM_ID: boundedString(255), + VERCEL_PROJECT_ID: boundedString(255), + VERCEL_PROJECT_NAME: boundedString(100), +}); +export type VercelPreviewEnvironment = z.infer; diff --git a/packages/shared/tests/validators/vercel-preview.test.ts b/packages/shared/tests/validators/vercel-preview.test.ts new file mode 100644 index 000000000..c7b0267d6 --- /dev/null +++ b/packages/shared/tests/validators/vercel-preview.test.ts @@ -0,0 +1,226 @@ +import { describe, expect, test } from 'bun:test'; +import { + githubPreviewCommitPullsSchema, + githubPreviewFilesSchema, + githubPreviewPullRequestSchema, + githubPreviewPullRequestTargetEventSchema, + githubPreviewRefSchema, + githubPreviewWorkflowDispatchEventSchema, + githubPreviewWorkflowRunEventSchema, + githubPreviewWorkflowRunsSchema, + githubPreviewWorkflowSchema, + vercelCreatedDeploymentSchema, + vercelDeploymentSchema, + vercelDeploymentsPageSchema, + vercelPreviewEnvironmentSchema, +} from '../../src/validators/vercel-preview.ts'; + +const SHA = 'a'.repeat(40); + +const repository = { + id: 123, + name: 'orbit', + owner: { login: 'Noveum' }, +}; + +const pullRequest = { + number: 341, + state: 'open' as const, + draft: false, + labels: [{ name: 'preview' }], + head: { + sha: SHA, + ref: 'feature/preview', + repo: repository, + }, + base: { + ref: 'main', + repo: repository, + }, +}; + +const deployment = { + uid: 'dpl_preview', + url: null, + target: null, + readyState: 'BLOCKED' as const, + meta: { + orbitGithubHeadRef: 'feature/preview', + orbitGithubHeadSha: SHA, + orbitGithubPrNumber: '341', + orbitGithubRepositoryId: 123, + orbitGithubWorkflowRunId: 987, + orbitDeploymentReason: 'ci-green', + retried: false, + note: null, + }, +}; + +const workflowRun = { + id: 987, + workflow_id: 456, + name: 'CI', + event: 'pull_request', + head_sha: SHA, + status: 'completed' as const, + conclusion: 'success', + created_at: '2026-08-21T00:00:00Z', + pull_requests: [ + { + number: pullRequest.number, + head: { sha: SHA, ref: 'feature/preview', repo: repository }, + base: { sha: 'b'.repeat(40), ref: 'main', repo: repository }, + }, + ], +}; + +const environment = { + GITHUB_EVENT_NAME: 'pull_request_target', + GITHUB_EVENT_PATH: '/tmp/event.json', + GITHUB_REPOSITORY: 'Noveum/orbit', + GITHUB_TOKEN: 'github-token', + VERCEL_TOKEN: 'vercel-token', + VERCEL_TEAM_ID: 'team_orbit', + VERCEL_PROJECT_ID: 'prj_orbit', + VERCEL_PROJECT_NAME: 'orbit', +}; + +describe('Vercel Preview GitHub schemas', () => { + test('accept a complete open pull request', () => { + expect(githubPreviewPullRequestSchema.parse(pullRequest).head.sha).toBe(SHA); + }); + + test('accept a Git reference with an immutable object SHA', () => { + expect(githubPreviewRefSchema.parse({ object: { sha: SHA } }).object.sha).toBe(SHA); + }); + + test('accept the canonical CI workflow identity', () => { + expect( + githubPreviewWorkflowSchema.parse({ + id: 456, + name: 'CI', + path: '.github/workflows/ci.yml', + state: 'active', + }).path, + ).toBe('.github/workflows/ci.yml'); + }); + + test('accept a pull request state event', () => { + expect( + githubPreviewPullRequestTargetEventSchema.parse({ + action: 'ready_for_review', + number: pullRequest.number, + pull_request: pullRequest, + repository, + }), + ).toMatchObject({ action: 'ready_for_review', number: pullRequest.number }); + }); + + test('accept a successful CI workflow event', () => { + expect( + githubPreviewWorkflowRunEventSchema.parse({ + action: 'completed', + repository, + workflow_run: workflowRun, + }), + ).toMatchObject({ workflow_run: { head_sha: SHA } }); + }); + + test('accept a manual pull request input', () => { + expect( + githubPreviewWorkflowDispatchEventSchema.parse({ + inputs: { pull_request: '341' }, + repository, + }).inputs.pull_request, + ).toBe('341'); + }); + + test('accept GitHub files, workflow runs, and commit pull pages', () => { + expect( + githubPreviewFilesSchema.parse([{ filename: 'apps/web/src/app/page.tsx' }]), + ).toHaveLength(1); + expect( + githubPreviewWorkflowRunsSchema.parse({ + workflow_runs: [workflowRun], + }).workflow_runs, + ).toHaveLength(1); + expect(githubPreviewCommitPullsSchema.parse([{ number: pullRequest.number }])).toHaveLength(1); + }); + + test('accept Vercel deployment pages and create responses', () => { + expect(vercelDeploymentSchema.parse(deployment).readyState).toBe('BLOCKED'); + expect( + vercelDeploymentsPageSchema.parse({ + deployments: [deployment], + pagination: { next: 123, prev: null }, + }).pagination.next, + ).toBe(123); + expect( + vercelCreatedDeploymentSchema.parse({ + id: 'dpl_preview', + url: null, + target: null, + readyState: 'INITIALIZING', + meta: deployment.meta, + }).id, + ).toBe('dpl_preview'); + }); + + test('accept a complete controller environment', () => { + expect(vercelPreviewEnvironmentSchema.parse(environment).VERCEL_PROJECT_ID).toBe('prj_orbit'); + }); + + test('reject a short pull request head SHA', () => { + expect(() => + githubPreviewPullRequestSchema.parse({ + ...pullRequest, + head: { ...pullRequest.head, sha: 'short' }, + }), + ).toThrow(); + }); + + test('reject a pull request without a repository ID', () => { + expect(() => + githubPreviewPullRequestSchema.parse({ + ...pullRequest, + head: { ...pullRequest.head, repo: { ...repository, id: undefined } }, + }), + ).toThrow(); + }); + + test('reject a pull request without draft state', () => { + const { draft: _draft, ...withoutDraft } = pullRequest; + expect(() => githubPreviewPullRequestSchema.parse(withoutDraft)).toThrow(); + }); + + test('reject an unknown Vercel ready state', () => { + expect(() => vercelDeploymentSchema.parse({ ...deployment, readyState: 'UNKNOWN' })).toThrow(); + }); + + test('reject a nonnumeric manual pull request input', () => { + expect(() => + githubPreviewWorkflowDispatchEventSchema.parse({ + inputs: { pull_request: '341a' }, + repository, + }), + ).toThrow(); + }); + + test('reject pagination without a finite cursor', () => { + expect(() => + vercelDeploymentsPageSchema.parse({ + deployments: [], + pagination: { next: Number.POSITIVE_INFINITY, prev: null }, + }), + ).toThrow(); + }); + + test('reject an environment with missing secrets', () => { + expect(() => + vercelPreviewEnvironmentSchema.parse({ + GITHUB_EVENT_NAME: 'pull_request_target', + GITHUB_EVENT_PATH: '/tmp/event.json', + }), + ).toThrow(); + }); +}); diff --git a/scripts/vercel-preview-policy.test.ts b/scripts/vercel-preview-policy.test.ts new file mode 100644 index 000000000..ce00625e0 --- /dev/null +++ b/scripts/vercel-preview-policy.test.ts @@ -0,0 +1,139 @@ +import { describe, expect, test } from 'bun:test'; +import type { + GithubPreviewPullRequest, + VercelDeployment, +} from '../packages/shared/src/validators/index.ts'; +import { + isActiveVercelDeployment, + isPreviewEligible, + isReadyVercelDeployment, + isSameRepositoryPullRequest, + isWebPreviewFile, + matchesVercelPullRequest, +} from './vercel-preview-policy.ts'; + +const SHA = 'a'.repeat(40); + +const repository = { + id: 123, + name: 'orbit', + owner: { login: 'Noveum' }, +}; + +const readyPullRequest: GithubPreviewPullRequest = { + number: 341, + state: 'open', + draft: false, + labels: [], + head: { sha: SHA, ref: 'feature/preview', repo: repository }, + base: { ref: 'main', repo: repository }, +}; + +const draftPullRequest: GithubPreviewPullRequest = { ...readyPullRequest, draft: true }; +const forkPullRequest: GithubPreviewPullRequest = { + ...readyPullRequest, + head: { + ...readyPullRequest.head, + repo: { ...repository, id: 456, owner: { login: 'contributor' } }, + }, +}; + +const deployment: VercelDeployment = { + uid: 'dpl_preview', + url: null, + target: null, + readyState: 'READY', + meta: { + orbitGithubRepositoryId: '123', + orbitGithubPrNumber: 341, + orbitGithubHeadRef: 'feature/preview', + orbitGithubHeadSha: SHA, + orbitGithubWorkflowRunId: 987, + orbitDeploymentReason: 'ci-green', + }, +}; + +function withLabels( + pullRequest: GithubPreviewPullRequest, + labels: readonly string[], +): GithubPreviewPullRequest { + return { ...pullRequest, labels: labels.map((name) => ({ name })) }; +} + +describe('Preview eligibility', () => { + test('uses the preview labels and draft state as the eligibility truth table', () => { + expect(isPreviewEligible(readyPullRequest)).toBe(true); + expect(isPreviewEligible(draftPullRequest)).toBe(false); + expect(isPreviewEligible(withLabels(draftPullRequest, ['preview']))).toBe(true); + expect(isPreviewEligible(withLabels(readyPullRequest, ['preview', 'no-preview']))).toBe(false); + }); + + test('requires a pull request head from the base repository', () => { + expect(isSameRepositoryPullRequest(readyPullRequest)).toBe(true); + expect(isSameRepositoryPullRequest(forkPullRequest)).toBe(false); + }); +}); + +describe('Web Preview path policy', () => { + test('includes the application, packages, and deployment configuration', () => { + expect(isWebPreviewFile('apps/web/src/app/page.tsx')).toBe(true); + expect(isWebPreviewFile('packages/shared/src/index.ts')).toBe(true); + expect(isWebPreviewFile('package.json')).toBe(true); + expect(isWebPreviewFile('bun.lock')).toBe(true); + expect(isWebPreviewFile('tsconfig.base.json')).toBe(true); + }); + + test('excludes unrelated applications and documentation', () => { + expect(isWebPreviewFile('apps/realtime/src/index.ts')).toBe(false); + expect(isWebPreviewFile('docs/README.md')).toBe(false); + }); +}); + +describe('Vercel deployment policy', () => { + test('matches Preview metadata after normalizing Vercel metadata values', () => { + expect(matchesVercelPullRequest(deployment, readyPullRequest, SHA)).toBe(true); + }); + + test('requires repository, pull request, ref, and supplied SHA to match', () => { + expect( + matchesVercelPullRequest( + { ...deployment, meta: { ...deployment.meta, orbitGithubRepositoryId: '456' } }, + readyPullRequest, + SHA, + ), + ).toBe(false); + expect( + matchesVercelPullRequest( + { ...deployment, meta: { ...deployment.meta, orbitGithubPrNumber: '342' } }, + readyPullRequest, + SHA, + ), + ).toBe(false); + expect( + matchesVercelPullRequest( + { ...deployment, meta: { ...deployment.meta, orbitGithubHeadRef: 'other-ref' } }, + readyPullRequest, + SHA, + ), + ).toBe(false); + expect(matchesVercelPullRequest(deployment, readyPullRequest, 'b'.repeat(40))).toBe(false); + }); + + test('never matches a production deployment', () => { + expect( + matchesVercelPullRequest({ ...deployment, target: 'production' }, readyPullRequest, SHA), + ).toBe(false); + }); + + test('recognizes only queued, initializing, and building deployments as active', () => { + for (const readyState of ['QUEUED', 'INITIALIZING', 'BUILDING'] as const) { + expect(isActiveVercelDeployment({ ...deployment, readyState })).toBe(true); + } + expect(isActiveVercelDeployment(deployment)).toBe(false); + }); + + test('recognizes only READY deployments as ready', () => { + expect(isReadyVercelDeployment(deployment)).toBe(true); + expect(isReadyVercelDeployment({ ...deployment, readyState: 'BLOCKED' })).toBe(false); + }); +}); diff --git a/scripts/vercel-preview-policy.ts b/scripts/vercel-preview-policy.ts new file mode 100644 index 000000000..1ce7d1c24 --- /dev/null +++ b/scripts/vercel-preview-policy.ts @@ -0,0 +1,56 @@ +import type { + GithubPreviewPullRequest, + VercelDeployment, +} from '../packages/shared/src/validators/index.ts'; + +export const PREVIEW_LABEL = 'preview'; +export const NO_PREVIEW_LABEL = 'no-preview'; + +export function isPreviewEligible(pullRequest: GithubPreviewPullRequest): boolean { + const labels = new Set(pullRequest.labels.map(({ name }) => name.toLowerCase())); + if (labels.has(NO_PREVIEW_LABEL)) return false; + return !pullRequest.draft || labels.has(PREVIEW_LABEL); +} + +export function isSameRepositoryPullRequest(pullRequest: GithubPreviewPullRequest): boolean { + return pullRequest.head.repo.id === pullRequest.base.repo.id; +} + +export function isWebPreviewFile(filename: string): boolean { + return ( + filename.startsWith('apps/web/') || + filename.startsWith('packages/') || + filename === 'package.json' || + filename === 'bun.lock' || + filename === 'tsconfig.base.json' + ); +} + +export function isActiveVercelDeployment(deployment: VercelDeployment): boolean { + return ['QUEUED', 'INITIALIZING', 'BUILDING'].includes(deployment.readyState); +} + +export function isReadyVercelDeployment(deployment: VercelDeployment): boolean { + return deployment.readyState === 'READY'; +} + +function metadataValue(deployment: VercelDeployment, key: string): string | null { + const value = deployment.meta[key]; + return value === undefined || value === null ? null : String(value); +} + +export function matchesVercelPullRequest( + deployment: VercelDeployment, + pullRequest: GithubPreviewPullRequest, + headSha?: string, +): boolean { + if (deployment.target === 'production') return false; + + const metadataMatches = + metadataValue(deployment, 'orbitGithubRepositoryId') === String(pullRequest.base.repo.id) && + metadataValue(deployment, 'orbitGithubPrNumber') === String(pullRequest.number) && + metadataValue(deployment, 'orbitGithubHeadRef') === pullRequest.head.ref; + + if (!metadataMatches) return false; + return headSha === undefined || metadataValue(deployment, 'orbitGithubHeadSha') === headSha; +} From 2e359cdee19929b347f50995dfeffa012e1edd12 Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 14:59:11 +0530 Subject: [PATCH 07/19] fix(ci): harden preview event schemas --- .../shared/src/validators/vercel-preview.ts | 52 ++++++------ .../tests/validators/vercel-preview.test.ts | 85 +++++++++++++++---- 2 files changed, 94 insertions(+), 43 deletions(-) diff --git a/packages/shared/src/validators/vercel-preview.ts b/packages/shared/src/validators/vercel-preview.ts index 29ef28858..6054ea818 100644 --- a/packages/shared/src/validators/vercel-preview.ts +++ b/packages/shared/src/validators/vercel-preview.ts @@ -2,6 +2,7 @@ import { z } from 'zod'; const boundedString = (maximum: number) => z.string().trim().min(1).max(maximum); const positiveIntegerSchema = z.number().int().positive(); +const githubPullRequestNumberSchema = positiveIntegerSchema.max(2_147_483_647); const nonnegativeCursorSchema = z.number().finite().int().nonnegative().nullable(); const githubWorkflowConclusionSchema = z .enum([ @@ -97,18 +98,39 @@ export type GithubPreviewPullRequestTargetEvent = z.infer< typeof githubPreviewPullRequestTargetEventSchema >; +export const githubPreviewRepositoryDispatchEventSchema = z + .object({ + action: z.literal('vercel-preview-reconcile'), + client_payload: z.object({ pull_request: githubPullRequestNumberSchema }), + repository: githubPreviewRepositorySchema, + }) + .passthrough(); +export type GithubPreviewRepositoryDispatchEvent = z.infer< + typeof githubPreviewRepositoryDispatchEventSchema +>; + +export const githubPreviewWorkflowLinkedRepositorySchema = z + .object({ + id: positiveIntegerSchema, + name: boundedString(100), + }) + .passthrough(); +export type GithubPreviewWorkflowLinkedRepository = z.infer< + typeof githubPreviewWorkflowLinkedRepositorySchema +>; + const githubPreviewWorkflowPullRequestSchema = z .object({ number: positiveIntegerSchema, head: z.object({ sha: gitShaSchema, ref: boundedString(255), - repo: githubPreviewRepositorySchema, + repo: githubPreviewWorkflowLinkedRepositorySchema, }), base: z.object({ sha: gitShaSchema, ref: boundedString(255), - repo: githubPreviewRepositorySchema, + repo: githubPreviewWorkflowLinkedRepositorySchema, }), }) .passthrough(); @@ -137,21 +159,6 @@ export const githubPreviewWorkflowRunEventSchema = z .passthrough(); export type GithubPreviewWorkflowRunEvent = z.infer; -export const githubPreviewWorkflowDispatchEventSchema = z - .object({ - inputs: z.object({ - pull_request: z - .string() - .regex(/^[1-9]\d*$/) - .max(10), - }), - repository: githubPreviewRepositorySchema, - }) - .passthrough(); -export type GithubPreviewWorkflowDispatchEvent = z.infer< - typeof githubPreviewWorkflowDispatchEventSchema ->; - export const githubPreviewFilesSchema = z.array( z.object({ filename: boundedString(1024) }).passthrough(), ); @@ -162,15 +169,6 @@ export const githubPreviewWorkflowRunsSchema = z .passthrough(); export type GithubPreviewWorkflowRuns = z.infer; -export const githubPreviewCommitPullsSchema = z.array( - z - .object({ - number: positiveIntegerSchema, - }) - .passthrough(), -); -export type GithubPreviewCommitPulls = z.infer; - export const vercelDeploymentSchema = z .object({ uid: boundedString(100), @@ -220,7 +218,7 @@ export const vercelCreatedDeploymentSchema = z export type VercelCreatedDeployment = z.infer; export const vercelPreviewEnvironmentSchema = z.object({ - GITHUB_EVENT_NAME: z.enum(['pull_request_target', 'workflow_run', 'workflow_dispatch']), + GITHUB_EVENT_NAME: z.enum(['pull_request_target', 'workflow_run', 'repository_dispatch']), GITHUB_EVENT_PATH: boundedString(4096), GITHUB_REPOSITORY: z .string() diff --git a/packages/shared/tests/validators/vercel-preview.test.ts b/packages/shared/tests/validators/vercel-preview.test.ts index c7b0267d6..d48b75690 100644 --- a/packages/shared/tests/validators/vercel-preview.test.ts +++ b/packages/shared/tests/validators/vercel-preview.test.ts @@ -1,11 +1,10 @@ import { describe, expect, test } from 'bun:test'; import { - githubPreviewCommitPullsSchema, githubPreviewFilesSchema, githubPreviewPullRequestSchema, githubPreviewPullRequestTargetEventSchema, githubPreviewRefSchema, - githubPreviewWorkflowDispatchEventSchema, + githubPreviewRepositoryDispatchEventSchema, githubPreviewWorkflowRunEventSchema, githubPreviewWorkflowRunsSchema, githubPreviewWorkflowSchema, @@ -68,14 +67,22 @@ const workflowRun = { pull_requests: [ { number: pullRequest.number, - head: { sha: SHA, ref: 'feature/preview', repo: repository }, - base: { sha: 'b'.repeat(40), ref: 'main', repo: repository }, + head: { + sha: SHA, + ref: 'feature/preview', + repo: { id: 123, name: 'orbit', url: 'https://api.github.com/repos/Noveum/orbit' }, + }, + base: { + sha: 'b'.repeat(40), + ref: 'main', + repo: { id: 123, name: 'orbit', url: 'https://api.github.com/repos/Noveum/orbit' }, + }, }, ], }; const environment = { - GITHUB_EVENT_NAME: 'pull_request_target', + GITHUB_EVENT_NAME: 'repository_dispatch', GITHUB_EVENT_PATH: '/tmp/event.json', GITHUB_REPOSITORY: 'Noveum/orbit', GITHUB_TOKEN: 'github-token', @@ -126,16 +133,17 @@ describe('Vercel Preview GitHub schemas', () => { ).toMatchObject({ workflow_run: { head_sha: SHA } }); }); - test('accept a manual pull request input', () => { + test('accept a repository dispatch pull request input', () => { expect( - githubPreviewWorkflowDispatchEventSchema.parse({ - inputs: { pull_request: '341' }, + githubPreviewRepositoryDispatchEventSchema.parse({ + action: 'vercel-preview-reconcile', + client_payload: { pull_request: 341 }, repository, - }).inputs.pull_request, - ).toBe('341'); + }).client_payload.pull_request, + ).toBe(341); }); - test('accept GitHub files, workflow runs, and commit pull pages', () => { + test('accept GitHub files and workflow runs with minimal linked repositories', () => { expect( githubPreviewFilesSchema.parse([{ filename: 'apps/web/src/app/page.tsx' }]), ).toHaveLength(1); @@ -144,7 +152,6 @@ describe('Vercel Preview GitHub schemas', () => { workflow_runs: [workflowRun], }).workflow_runs, ).toHaveLength(1); - expect(githubPreviewCommitPullsSchema.parse([{ number: pullRequest.number }])).toHaveLength(1); }); test('accept Vercel deployment pages and create responses', () => { @@ -170,6 +177,15 @@ describe('Vercel Preview GitHub schemas', () => { expect(vercelPreviewEnvironmentSchema.parse(environment).VERCEL_PROJECT_ID).toBe('prj_orbit'); }); + test('reject a manual workflow dispatch environment', () => { + expect(() => + vercelPreviewEnvironmentSchema.parse({ + ...environment, + GITHUB_EVENT_NAME: 'workflow_dispatch', + }), + ).toThrow(); + }); + test('reject a short pull request head SHA', () => { expect(() => githubPreviewPullRequestSchema.parse({ @@ -197,11 +213,48 @@ describe('Vercel Preview GitHub schemas', () => { expect(() => vercelDeploymentSchema.parse({ ...deployment, readyState: 'UNKNOWN' })).toThrow(); }); - test('reject a nonnumeric manual pull request input', () => { + test('reject malformed repository dispatch pull request inputs', () => { + const event = { + action: 'vercel-preview-reconcile', + repository, + client_payload: { pull_request: 341 }, + }; + expect(() => - githubPreviewWorkflowDispatchEventSchema.parse({ - inputs: { pull_request: '341a' }, - repository, + githubPreviewRepositoryDispatchEventSchema.parse({ + action: event.action, + repository: event.repository, + client_payload: {}, + }), + ).toThrow(); + expect(() => + githubPreviewRepositoryDispatchEventSchema.parse({ + ...event, + client_payload: { pull_request: '341' }, + }), + ).toThrow(); + expect(() => + githubPreviewRepositoryDispatchEventSchema.parse({ + ...event, + client_payload: { pull_request: 341.5 }, + }), + ).toThrow(); + expect(() => + githubPreviewRepositoryDispatchEventSchema.parse({ + ...event, + client_payload: { pull_request: 0 }, + }), + ).toThrow(); + expect(() => + githubPreviewRepositoryDispatchEventSchema.parse({ + ...event, + client_payload: { pull_request: -1 }, + }), + ).toThrow(); + expect(() => + githubPreviewRepositoryDispatchEventSchema.parse({ + ...event, + client_payload: { pull_request: 2_147_483_648 }, }), ).toThrow(); }); From 756ea8807e4f0de377930949cbcc6e7cb7e61a03 Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 15:13:08 +0530 Subject: [PATCH 08/19] fix(ci): validate preview deployment identity --- .../shared/src/validators/vercel-preview.ts | 66 +++++++++------ .../tests/validators/vercel-preview.test.ts | 83 +++++++++++++++---- scripts/vercel-preview-policy.test.ts | 44 +++++++++- scripts/vercel-preview-policy.ts | 3 +- 4 files changed, 147 insertions(+), 49 deletions(-) diff --git a/packages/shared/src/validators/vercel-preview.ts b/packages/shared/src/validators/vercel-preview.ts index 6054ea818..04a054ce0 100644 --- a/packages/shared/src/validators/vercel-preview.ts +++ b/packages/shared/src/validators/vercel-preview.ts @@ -4,6 +4,7 @@ const boundedString = (maximum: number) => z.string().trim().min(1).max(maximum) const positiveIntegerSchema = z.number().int().positive(); const githubPullRequestNumberSchema = positiveIntegerSchema.max(2_147_483_647); const nonnegativeCursorSchema = z.number().finite().int().nonnegative().nullable(); +const nonnegativeIntegerSchema = z.number().finite().int().nonnegative(); const githubWorkflowConclusionSchema = z .enum([ 'success', @@ -26,6 +27,17 @@ const githubWorkflowRunStatusSchema = z.enum([ 'pending', ]); const vercelMetadataValueSchema = z.union([z.string(), z.number().finite(), z.boolean(), z.null()]); +const vercelTargetSchema = z.enum(['production', 'staging']).nullable(); +const vercelReadyStateSchema = z.enum([ + 'QUEUED', + 'INITIALIZING', + 'BUILDING', + 'READY', + 'ERROR', + 'CANCELED', + 'BLOCKED', + 'DELETED', +]); export const gitShaSchema = z.string().regex(/^[a-f0-9]{40}$/); @@ -88,6 +100,7 @@ export const githubPreviewPullRequestTargetEventSchema = z 'converted_to_draft', 'labeled', 'unlabeled', + 'closed', ]), number: positiveIntegerSchema, pull_request: githubPreviewPullRequestSchema, @@ -165,26 +178,21 @@ export const githubPreviewFilesSchema = z.array( export type GithubPreviewFiles = z.infer; export const githubPreviewWorkflowRunsSchema = z - .object({ workflow_runs: z.array(githubPreviewWorkflowRunSchema).max(100) }) + .object({ + total_count: nonnegativeIntegerSchema, + workflow_runs: z.array(githubPreviewWorkflowRunSchema).max(100), + }) .passthrough(); export type GithubPreviewWorkflowRuns = z.infer; export const vercelDeploymentSchema = z .object({ uid: boundedString(100), + projectId: boundedString(255), url: boundedString(255).nullable(), - target: z.enum(['production', 'staging']).nullable().optional(), - readyState: z.enum([ - 'QUEUED', - 'INITIALIZING', - 'BUILDING', - 'READY', - 'ERROR', - 'CANCELED', - 'BLOCKED', - 'DELETED', - ]), - meta: z.record(z.string(), vercelMetadataValueSchema), + target: vercelTargetSchema.optional(), + readyState: vercelReadyStateSchema, + meta: z.record(z.string(), vercelMetadataValueSchema).optional().default({}), }) .passthrough(); export type VercelDeployment = z.infer; @@ -192,31 +200,35 @@ export type VercelDeployment = z.infer; export const vercelDeploymentsPageSchema = z .object({ deployments: z.array(vercelDeploymentSchema).max(100), - pagination: z.object({ next: nonnegativeCursorSchema, prev: nonnegativeCursorSchema }), + pagination: z.object({ + count: nonnegativeIntegerSchema, + next: nonnegativeCursorSchema, + prev: nonnegativeCursorSchema, + }), }) .passthrough(); export type VercelDeploymentsPage = z.infer; -export const vercelCreatedDeploymentSchema = z +const vercelDeploymentMutationSchema = z .object({ id: boundedString(100), - url: boundedString(255).nullable(), - target: z.enum(['production', 'staging']).nullable().optional(), - readyState: z.enum([ - 'QUEUED', - 'INITIALIZING', - 'BUILDING', - 'READY', - 'ERROR', - 'CANCELED', - 'BLOCKED', - 'DELETED', - ]), + projectId: boundedString(255), + url: boundedString(255), + target: vercelTargetSchema, + readyState: vercelReadyStateSchema, meta: z.record(z.string(), vercelMetadataValueSchema), }) .passthrough(); + +export const vercelCreatedDeploymentSchema = vercelDeploymentMutationSchema; export type VercelCreatedDeployment = z.infer; +export const vercelDeploymentDetailSchema = vercelDeploymentMutationSchema; +export type VercelDeploymentDetail = z.infer; + +export const vercelCanceledDeploymentSchema = vercelDeploymentMutationSchema; +export type VercelCanceledDeployment = z.infer; + export const vercelPreviewEnvironmentSchema = z.object({ GITHUB_EVENT_NAME: z.enum(['pull_request_target', 'workflow_run', 'repository_dispatch']), GITHUB_EVENT_PATH: boundedString(4096), diff --git a/packages/shared/tests/validators/vercel-preview.test.ts b/packages/shared/tests/validators/vercel-preview.test.ts index d48b75690..ab7ec9976 100644 --- a/packages/shared/tests/validators/vercel-preview.test.ts +++ b/packages/shared/tests/validators/vercel-preview.test.ts @@ -8,7 +8,9 @@ import { githubPreviewWorkflowRunEventSchema, githubPreviewWorkflowRunsSchema, githubPreviewWorkflowSchema, + vercelCanceledDeploymentSchema, vercelCreatedDeploymentSchema, + vercelDeploymentDetailSchema, vercelDeploymentSchema, vercelDeploymentsPageSchema, vercelPreviewEnvironmentSchema, @@ -40,6 +42,7 @@ const pullRequest = { const deployment = { uid: 'dpl_preview', + projectId: 'prj_orbit', url: null, target: null, readyState: 'BLOCKED' as const, @@ -55,6 +58,15 @@ const deployment = { }, }; +const mutationDeployment = { + id: 'dpl_preview', + projectId: 'prj_orbit', + url: 'orbit-preview.vercel.app', + target: null, + readyState: 'INITIALIZING' as const, + meta: deployment.meta, +}; + const workflowRun = { id: 987, workflow_id: 456, @@ -115,12 +127,12 @@ describe('Vercel Preview GitHub schemas', () => { test('accept a pull request state event', () => { expect( githubPreviewPullRequestTargetEventSchema.parse({ - action: 'ready_for_review', + action: 'closed', number: pullRequest.number, pull_request: pullRequest, repository, }), - ).toMatchObject({ action: 'ready_for_review', number: pullRequest.number }); + ).toMatchObject({ action: 'closed', number: pullRequest.number }); }); test('accept a successful CI workflow event', () => { @@ -149,27 +161,39 @@ describe('Vercel Preview GitHub schemas', () => { ).toHaveLength(1); expect( githubPreviewWorkflowRunsSchema.parse({ + total_count: 1, workflow_runs: [workflowRun], }).workflow_runs, ).toHaveLength(1); }); - test('accept Vercel deployment pages and create responses', () => { + test('requires a workflow run total count', () => { + expect(() => githubPreviewWorkflowRunsSchema.parse({ workflow_runs: [workflowRun] })).toThrow(); + }); + + test('accept Vercel deployment pages and mutation responses', () => { expect(vercelDeploymentSchema.parse(deployment).readyState).toBe('BLOCKED'); + const page = vercelDeploymentsPageSchema.parse({ + deployments: [ + deployment, + { + uid: 'dpl_unrelated', + projectId: 'prj_orbit', + url: null, + target: null, + readyState: 'READY', + }, + ], + pagination: { count: 2, next: 123, prev: null }, + }); + expect(page.pagination.next).toBe(123); + expect(page.deployments[1]?.meta).toEqual({}); + expect(vercelCreatedDeploymentSchema.parse(mutationDeployment).id).toBe('dpl_preview'); + expect(vercelDeploymentDetailSchema.parse(mutationDeployment).url).toBe( + 'orbit-preview.vercel.app', + ); expect( - vercelDeploymentsPageSchema.parse({ - deployments: [deployment], - pagination: { next: 123, prev: null }, - }).pagination.next, - ).toBe(123); - expect( - vercelCreatedDeploymentSchema.parse({ - id: 'dpl_preview', - url: null, - target: null, - readyState: 'INITIALIZING', - meta: deployment.meta, - }).id, + vercelCanceledDeploymentSchema.parse({ ...mutationDeployment, readyState: 'CANCELED' }).id, ).toBe('dpl_preview'); }); @@ -213,6 +237,25 @@ describe('Vercel Preview GitHub schemas', () => { expect(() => vercelDeploymentSchema.parse({ ...deployment, readyState: 'UNKNOWN' })).toThrow(); }); + test('reject mutation responses without complete deployment identity', () => { + expect(() => vercelCreatedDeploymentSchema.parse({ ...mutationDeployment, id: '' })).toThrow(); + expect(() => { + const { projectId: _projectId, ...withoutProjectId } = mutationDeployment; + return vercelDeploymentDetailSchema.parse(withoutProjectId); + }).toThrow(); + expect(() => + vercelCanceledDeploymentSchema.parse({ ...mutationDeployment, url: null }), + ).toThrow(); + expect(() => { + const { meta: _meta, ...withoutMetadata } = mutationDeployment; + return vercelCreatedDeploymentSchema.parse(withoutMetadata); + }).toThrow(); + expect(() => { + const { target: _target, ...withoutTarget } = mutationDeployment; + return vercelDeploymentDetailSchema.parse(withoutTarget); + }).toThrow(); + }); + test('reject malformed repository dispatch pull request inputs', () => { const event = { action: 'vercel-preview-reconcile', @@ -263,7 +306,13 @@ describe('Vercel Preview GitHub schemas', () => { expect(() => vercelDeploymentsPageSchema.parse({ deployments: [], - pagination: { next: Number.POSITIVE_INFINITY, prev: null }, + pagination: { next: null, prev: null }, + }), + ).toThrow(); + expect(() => + vercelDeploymentsPageSchema.parse({ + deployments: [], + pagination: { count: 0, next: Number.POSITIVE_INFINITY, prev: null }, }), ).toThrow(); }); diff --git a/scripts/vercel-preview-policy.test.ts b/scripts/vercel-preview-policy.test.ts index ce00625e0..f5c6cef2c 100644 --- a/scripts/vercel-preview-policy.test.ts +++ b/scripts/vercel-preview-policy.test.ts @@ -3,6 +3,7 @@ import type { GithubPreviewPullRequest, VercelDeployment, } from '../packages/shared/src/validators/index.ts'; +import { vercelDeploymentSchema } from '../packages/shared/src/validators/index.ts'; import { isActiveVercelDeployment, isPreviewEligible, @@ -40,6 +41,7 @@ const forkPullRequest: GithubPreviewPullRequest = { const deployment: VercelDeployment = { uid: 'dpl_preview', + projectId: 'prj_orbit', url: null, target: null, readyState: 'READY', @@ -91,7 +93,7 @@ describe('Web Preview path policy', () => { describe('Vercel deployment policy', () => { test('matches Preview metadata after normalizing Vercel metadata values', () => { - expect(matchesVercelPullRequest(deployment, readyPullRequest, SHA)).toBe(true); + expect(matchesVercelPullRequest(deployment, readyPullRequest, 'prj_orbit', SHA)).toBe(true); }); test('requires repository, pull request, ref, and supplied SHA to match', () => { @@ -99,6 +101,7 @@ describe('Vercel deployment policy', () => { matchesVercelPullRequest( { ...deployment, meta: { ...deployment.meta, orbitGithubRepositoryId: '456' } }, readyPullRequest, + 'prj_orbit', SHA, ), ).toBe(false); @@ -106,6 +109,7 @@ describe('Vercel deployment policy', () => { matchesVercelPullRequest( { ...deployment, meta: { ...deployment.meta, orbitGithubPrNumber: '342' } }, readyPullRequest, + 'prj_orbit', SHA, ), ).toBe(false); @@ -113,16 +117,48 @@ describe('Vercel deployment policy', () => { matchesVercelPullRequest( { ...deployment, meta: { ...deployment.meta, orbitGithubHeadRef: 'other-ref' } }, readyPullRequest, + 'prj_orbit', SHA, ), ).toBe(false); - expect(matchesVercelPullRequest(deployment, readyPullRequest, 'b'.repeat(40))).toBe(false); + expect( + matchesVercelPullRequest(deployment, readyPullRequest, 'prj_orbit', 'b'.repeat(40)), + ).toBe(false); }); - test('never matches a production deployment', () => { + test('requires the configured project Preview environment', () => { + expect(matchesVercelPullRequest(deployment, readyPullRequest, 'prj_other', SHA)).toBe(false); + expect( + matchesVercelPullRequest( + { ...deployment, target: 'staging' }, + readyPullRequest, + 'prj_orbit', + SHA, + ), + ).toBe(false); expect( - matchesVercelPullRequest({ ...deployment, target: 'production' }, readyPullRequest, SHA), + matchesVercelPullRequest( + { ...deployment, target: 'production' }, + readyPullRequest, + 'prj_orbit', + SHA, + ), ).toBe(false); + const { target: _target, ...withoutTarget } = deployment; + expect(matchesVercelPullRequest(withoutTarget, readyPullRequest, 'prj_orbit', SHA)).toBe(false); + }); + + test('does not match an unrelated deployment without metadata', () => { + const withoutMetadata = vercelDeploymentSchema.parse({ + uid: 'dpl_unrelated', + projectId: 'prj_orbit', + url: null, + target: null, + readyState: 'READY', + }); + expect(matchesVercelPullRequest(withoutMetadata, readyPullRequest, 'prj_orbit', SHA)).toBe( + false, + ); }); test('recognizes only queued, initializing, and building deployments as active', () => { diff --git a/scripts/vercel-preview-policy.ts b/scripts/vercel-preview-policy.ts index 1ce7d1c24..31413eb7d 100644 --- a/scripts/vercel-preview-policy.ts +++ b/scripts/vercel-preview-policy.ts @@ -42,9 +42,10 @@ function metadataValue(deployment: VercelDeployment, key: string): string | null export function matchesVercelPullRequest( deployment: VercelDeployment, pullRequest: GithubPreviewPullRequest, + projectId: string, headSha?: string, ): boolean { - if (deployment.target === 'production') return false; + if (deployment.projectId !== projectId || deployment.target !== null) return false; const metadataMatches = metadataValue(deployment, 'orbitGithubRepositoryId') === String(pullRequest.base.repo.id) && From a861e3819d461affade2555a7cd0d87d52fccd7c Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 15:16:34 +0530 Subject: [PATCH 09/19] docs(vercel): finalize the preview controller contract --- ...26-08-21-vercel-preview-deployment-gate.md | 111 +++++++++++------- ...1-vercel-preview-deployment-gate-design.md | 85 +++++++++----- 2 files changed, 123 insertions(+), 73 deletions(-) diff --git a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md index 5c243ec38..5359b1386 100644 --- a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md +++ b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md @@ -41,6 +41,7 @@ - Create `scripts/vercel-preview-deploy.test.ts`: injected-fetch orchestration tests with no real network calls. - Create `scripts/vercel-preview-config.test.ts`: repository configuration, managed labels, workflow trust invariants, and removal of the old token gate. - Create `.github/workflows/vercel-preview.yml`: default-branch metadata-only dispatcher. +- Modify `.github/workflows/ci.yml`: make hosted CI include the two repository verification checks currently omitted from its static job. - Modify `apps/web/vercel.json`: remove `ignoreCommand`; disable automatic feature branches and retain `main`. - Modify `scripts/labels.ts`: manage `preview` and `no-preview`. - Delete `scripts/vercel-build-gate.sh` and `scripts/vercel-build-gate.test.ts`. @@ -60,14 +61,14 @@ - Test: `scripts/vercel-preview-policy.test.ts` **Interfaces:** -- Produces `githubPreviewPullRequestSchema`, `githubPreviewPullRequestTargetEventSchema`, `githubPreviewWorkflowRunEventSchema`, `githubPreviewWorkflowDispatchEventSchema`, `githubPreviewFilesSchema`, `githubPreviewWorkflowRunsSchema`, `githubPreviewCommitPullsSchema`, `vercelDeploymentSchema`, `vercelDeploymentsPageSchema`, `vercelCreatedDeploymentSchema`, and `vercelPreviewEnvironmentSchema`. +- Produces `githubPreviewPullRequestSchema`, `githubPreviewPullRequestTargetEventSchema`, `githubPreviewWorkflowRunEventSchema`, `githubPreviewRepositoryDispatchEventSchema`, `githubPreviewFilesSchema`, `githubPreviewWorkflowSchema`, `githubPreviewWorkflowRunsSchema`, `githubPreviewRefSchema`, `vercelDeploymentSchema`, `vercelDeploymentsPageSchema`, `vercelCreatedDeploymentSchema`, `vercelDeploymentDetailSchema`, `vercelCanceledDeploymentSchema`, and `vercelPreviewEnvironmentSchema`. - Produces inferred `GithubPreviewPullRequest`, `VercelDeployment`, and `VercelPreviewEnvironment` types. - Produces `PREVIEW_LABEL`, `NO_PREVIEW_LABEL`, `isPreviewEligible`, `isSameRepositoryPullRequest`, `isWebPreviewFile`, `isActiveVercelDeployment`, `isReadyVercelDeployment`, and `matchesVercelPullRequest`. - The controller in Task 2 consumes every interface above and does not define a second copy of validation or policy. - [ ] **Step 1: Write failing validator tests** -Create fixtures with a 40-character lowercase SHA and assert the schemas accept a complete open pull request, state event, successful CI workflow event, manual input, GitHub file page, workflow-run page, Vercel deployment page, create response, and complete environment. Add rejection cases for a short SHA, missing repository ID, missing draft state, unknown Vercel ready state, nonnumeric manual pull request input, pagination without a finite cursor, and missing secrets. +Create fixtures with a 40-character lowercase SHA and assert the schemas accept a complete open pull request, state event including `closed`, successful CI workflow event, repository-dispatch recovery input, GitHub file page, canonical workflow response, workflow-run page with `total_count`, live Git ref response, Vercel deployment page with `pagination.count`, create response, detail response, cancel response, and complete environment. Add rejection cases for a short SHA, missing repository ID, missing draft state, unknown Vercel ready state, nonnumeric recovery input, invalid pagination, and missing secrets. Workflow-run linked pull requests use GitHub's minimal repository shape with ID and name rather than the full repository shape with `owner.login`. Vercel list items require `projectId` but accept missing metadata as an empty record; mutation and detail responses require `id`, `projectId`, a nonempty URL, target, state, and metadata. ```ts expect(githubPreviewPullRequestSchema.parse(pullRequest).head.sha).toBe(SHA); @@ -113,7 +114,7 @@ export const githubPreviewPullRequestSchema = z.object({ }); ``` -The workflow-run schema must retain `name`, `event`, `head_sha`, `conclusion`, and pull request numbers. Vercel metadata accepts string, number, boolean, or null values. Vercel pagination accepts finite nonnegative `next` and `prev` cursors or null. Export all inferred types and add `export * from './vercel-preview.ts';` to the validator index. +The workflow-run schema must retain run `id`, `workflow_id`, `name`, `event`, `head_sha`, `status`, `conclusion`, creation time, and linked pull request head and base identity; the page retains nonnegative `total_count`. The live Git ref schema retains `object.sha`. Vercel metadata accepts string, number, boolean, or null values. Vercel pagination retains nonnegative `count` and finite nonnegative `next` and `prev` cursors or null. Export all inferred types and add `export * from './vercel-preview.ts';` to the validator index. - [ ] **Step 4: Run the validator tests** @@ -171,7 +172,7 @@ export function isWebPreviewFile(filename: string): boolean { } ``` -`matchesVercelPullRequest` must reject `target === 'production'` and compare normalized metadata values for `githubRepoId`, `githubPrId`, `githubCommitRef`, and, when supplied, `githubCommitSha`. +`matchesVercelPullRequest` receives the configured project ID, requires exact `projectId` and `target === null`, and compares normalized metadata values for `orbitGithubRepositoryId`, `orbitGithubPrNumber`, `orbitGithubHeadRef`, and, when supplied, `orbitGithubHeadSha`. Production, staging, missing target, another project, and absent metadata never match. - [ ] **Step 8: Run Task 1 checks and commit** @@ -194,8 +195,10 @@ Commit: `feat(ci): define preview deployment policy` **Interfaces:** - Consumes all Task 1 schemas, types, constants, and pure policy functions. - Produces `PreviewRuntime`, `PreviewResult`, and `reconcileVercelPreviews(runtime): Promise`. -- `PreviewRuntime` supplies `env`, `readText`, `fetch`, `sleep`, and `log` so tests never access the network, process secrets, or real event files. -- `PreviewResult` is a discriminated union with `kind: 'skipped' | 'created' | 'canceled'`, a pull request number, a stable reason, and an optional deployment ID and URL only for a mutation. +- `PreviewRuntime` supplies `env`, `readText`, `fetch`, `sleep`, `now`, and `log` so tests never access the network, process secrets, clocks, or real event files. +- `PreviewResult` is a closed discriminated union with `kind: 'skipped' | 'created' | 'canceled'`, a pull request number, and a stable reason. `created` and each `canceled` result include deployment ID and URL; `skipped` results do not. Candidates and canceled results are sorted for deterministic output. + +Use this closed reason vocabulary: `event-not-actionable`, `workflow-run-unassociated`, `stale-event`, `repository-mismatch`, `fork-pull-request`, `base-mismatch`, `preview-ineligible`, `no-active-deployment`, `web-unaffected`, `ci-unavailable`, `ci-not-current`, `ci-not-green`, `ready-deployment-reused`, `active-deployment-reused`, `created-ready`, and `canceled-active`. Configuration, malformed external data, incomplete pagination, transport failure, identity drift, and terminal build failure throw redacted errors rather than returning a skipped result. - [ ] **Step 1: Write failing event and eligibility tests** @@ -203,17 +206,20 @@ Use an injected fetch router that records method, URL, headers, and parsed body. - A successful `workflow_run` for the current ready same-repository head creates one deployment. - `pull_request_target` ready and `preview` transitions create only when the exact SHA already has a successful CI run. -- Draft without `preview`, either control label combination, closed PR, stale event SHA, fork head, failed CI, in-progress CI, and unrelated files create zero deployments. -- `workflow_dispatch` parses its pull request input and follows the same live-state and CI checks. -- An empty workflow-run pull request list falls back to the commit-pulls endpoint. -- GitHub files and commit pulls paginate until a short page, with a hard page limit. +- Draft without `preview`, either control label combination, stale event SHA, fork head, wrong base, failed CI, in-progress CI, and unrelated files create zero deployments. +- `closed`, converted-to-draft, and `no-preview` transitions cancel matching active deployments without requiring CI; a stale state event cannot cancel a different live head. +- `repository_dispatch` parses its pull request input and follows the same live-state and CI checks. +- An empty workflow-run pull request list fails closed because it cannot prove the run's base SHA and repository association. +- Duplicate workflow-run associations resolve a pull request once, event and embedded PR numbers must agree, and event repository identity must match `GITHUB_REPOSITORY` plus the live base repository. +- GitHub files paginate until a short page, with a 30-page cap and early relevant-file exit. Workflow runs paginate against `total_count` with a 10-page cap. A full final files page or an unexhausted run count fails closed. +- A newer queued, failed, canceled, or stale-base run blocks an older success. Equal creation times use the larger run ID. Wrong canonical workflow path, name, ID, or state fails closed. The creation assertion must inspect the exact request: ```ts expect(createRequest.method).toBe('POST'); expect(createRequest.url).toContain('/v13/deployments'); -expect(createRequest.url).toContain('forceNew=1'); +expect(createRequest.url).not.toContain('forceNew=1'); expect(createRequest.body).toEqual({ name: 'orbit', project: 'prj_orbit', @@ -224,12 +230,12 @@ expect(createRequest.body).toEqual({ sha: SHA, }, meta: { - githubCommitOrg: 'Noveum', - githubCommitRef: 'feature/preview', - githubCommitRepo: 'orbit', - githubCommitSha: SHA, - githubPrId: '341', - githubRepoId: '123', + orbitDeploymentReason: 'ci-green-pr-preview', + orbitGithubHeadRef: 'feature/preview', + orbitGithubHeadSha: SHA, + orbitGithubPrNumber: '341', + orbitGithubRepositoryId: '123', + orbitGithubWorkflowRunId: '987654321', }, }); expect(createRequest.body).not.toHaveProperty('target'); @@ -249,24 +255,32 @@ Parse `GITHUB_EVENT_NAME`, read `GITHUB_EVENT_PATH`, and select the matching eve type PreviewCandidate = { readonly number: number; readonly expectedHeadSha: string | null; - readonly ciProven: boolean; }; ``` -- `pull_request_target`: one candidate from the event PR number and event head SHA, with `ciProven: false`. -- `workflow_run`: no candidates unless the workflow is `CI`, source event is `pull_request`, conclusion is `success`, and action is `completed`; candidates use the workflow head SHA and `ciProven: true`; use the commit-pulls endpoint when the payload list is empty. -- `workflow_dispatch`: one candidate from the positive integer input, no expected SHA, with `ciProven: false`. +- `pull_request_target`: one candidate from the matching event and embedded PR number plus event head SHA; `closed` is an accepted cancellation transition. +- `workflow_run`: no candidates unless the workflow is `CI`, source event is `pull_request`, conclusion is `success`, action is `completed`, and the payload links at least one pull request; candidates use the linked pull request number and workflow head SHA. An empty linked list fails closed. +- `repository_dispatch`: one candidate from the validated positive integer `client_payload.pull_request` and no expected SHA. + +For every candidate, refetch `/repos/{owner}/{repo}/pulls/{number}` and prove event repository ID and slug, base repository ID, base ref `main`, same-repository head, and exact expected SHA when one exists. A fork or unrelated repository is a no-op. Current closed, draft, or label-ineligible state follows the cancellation path without requiring successful CI. -For every candidate, refetch `/repos/{owner}/{repo}/pulls/{number}` and require an open PR targeting the event repository and `main`, a same-repository head, and an exact expected SHA when one exists. Evaluate current labels and draft state next; an ineligible state event follows the cancellation path without requiring successful CI. For an eligible candidate whose `ciProven` is false, query `/repos/{owner}/{repo}/actions/workflows/ci.yml/runs?event=pull_request&head_sha={sha}&status=completed&per_page=100` and require a successful run with the same head SHA. Query pull request files with `per_page=100&page=N` and require at least one `isWebPreviewFile` match before a create. +For every eligible candidate, resolve `/repos/{owner}/{repo}/actions/workflows/ci.yml` and require exact path `.github/workflows/ci.yml`, name `CI`, and active state. Fetch `/repos/{owner}/{repo}/git/ref/heads/main`, then paginate `/repos/{owner}/{repo}/actions/workflows/{workflowId}/runs?event=pull_request&head_sha={sha}&per_page=100&page=N`. Require a stable `total_count`, stop only when that count is exhausted, and fail after ten pages if results remain. Select the maximum `(created_at, id)` pair before checking conclusion. Require its workflow ID, event, status, conclusion, linked pull request number, head repository ID, ref, and SHA, base repository ID and ref, and linked base SHA equal to the separately fetched live `main` SHA. This prevents an older green run from winning over newer queued, failed, canceled, or stale-base work. + +Query pull request files with `per_page=100&page=N`. Return true as soon as an `isWebPreviewFile` match appears. A short page proves no relevant path; a full page at the 30-page cap fails closed. Immediately before a create or cancel mutation, refetch the live pull request. Before create, also repeat the live-main and latest-CI proof. Abort the mutation when head, state, labels, base, or CI changed during reconciliation. - [ ] **Step 4: Write failing idempotency and cancellation tests** Cover Vercel pages with an exact deployment on page 2 and prove: - Exact `QUEUED`, `INITIALIZING`, `BUILDING`, or `READY` produces no create call. -- Exact `CANCELED` or `ERROR` allows exactly one create call. -- Same SHA in another project, production target, repository, pull request, or ref does not suppress creation. -- An ineligible state event cancels every matching active Preview for that PR and does not cancel ready, canceled, errored, production, another ref, or another repository. +- Exact `CANCELED`, `ERROR`, `BLOCKED`, or `DELETED` allows exactly one forced create call. +- Same SHA in another project, staging or production target, missing target, repository, pull request, or ref does not suppress creation. +- A list item without metadata is ignored without rejecting its page. +- An ineligible current state cancels every matching active Preview for that PR and does not cancel ready, canceled, errored, staging, production, another project, ref, or repository. +- An existing active deployment is polled instead of duplicated; an existing ready deployment is reused. +- Exact READY wins over duplicate active and terminal items; exact active wins over terminal items. +- Successful create, detail, and cancel responses must retain requested ID, project, null target, and Orbit metadata. Cancel must return `CANCELED`. +- A cancel 400 or ambiguous response reads detail once and accepts a now-terminal state without retrying PATCH; an active or identity-drifted detail fails. - A second reconciliation after creation sees the created metadata and is a no-op. - [ ] **Step 5: Run the idempotency tests and confirm failure** @@ -277,19 +291,27 @@ Expected: FAIL because deployment listing, matching, and cancellation are not im - [ ] **Step 6: Implement bounded Vercel reconciliation** -List `/v6/deployments` with `teamId`, `projectId`, `limit=100`, and a metadata filter. Follow the validated `pagination.next` cursor with a finite page cap. Filter again in trusted code with `matchesVercelPullRequest` before using any result. +List `/v7/deployments` with `teamId`, `projectId`, `branch`, `sha` where appropriate, and `limit=100`. The current endpoint has no documented metadata query. Follow validated `pagination.next` through `until`, preserve every original filter, reject repeated cursors including zero, and fail when a non-null cursor remains at the finite page cap. Filter again in trusted code with exact project, null target, and `matchesVercelPullRequest`. Parse list identifiers from `uid`; create, detail, and cancel responses use `id`. Accept `url: null` only on list items. + +For eligible PRs, prefer any exact ready deployment, then any exact active deployment, then terminal history. Fetch detail when a READY list item has no URL. Poll an active deployment immediately, then allow at most 240 five-second sleeps followed by a final GET. Require ID, project, null target, and Orbit metadata on every detail. `READY` with a nonempty URL succeeds; `ERROR`, `CANCELED`, `BLOCKED`, or `DELETED` fails; an active final response times out. -For eligible PRs, return a skipped result when an exact active or ready deployment exists. Otherwise create one deployment with `POST /v13/deployments?teamId={teamId}&forceNew=1`, omitted `target`, exact Git source, and the metadata shown in Step 1. +When no ready or active exact deployment exists, create one deployment with `POST /v13/deployments?teamId={teamId}`, omitted `target`, exact Git source, and the metadata shown in Step 1. Add `forceNew=1` only when the complete pre-create list already contained an exact terminal deployment. Record all pre-create deployment IDs and enforce one POST per reconciliation. -For ineligible state events, list by pull request metadata and call `PATCH /v12/deployments/{id}/cancel?teamId={teamId}` only for matching active Preview deployments. A CI failure does not cancel a previously ready Preview; cancellation is driven by current PR state. +An ambiguous create outcome is a network error, timeout, 429, 5xx, 409, or successful response that cannot be parsed, validated, or matched to the requested identity. Ordinary 4xx responses are definitive. After ambiguity, set `createAttempted` and make a second POST impossible. Run three exact-list observation attempts separated by two seconds. Reuse only a newly visible ready or active ID that was not in the pre-create set. A new terminal deployment or no new exact ID fails visibly. A later event starts a new reconciliation from a complete list and may decide independently. + +For current ineligible state, list with `teamId`, `projectId`, `branch`, and `limit=100` without SHA, then call `PATCH /v12/deployments/{id}/cancel?teamId={teamId}` only for locally matched active Preview deployments. Never retry PATCH. Validate a successful cancel as the requested ID, project, null target, Orbit metadata, and `CANCELED`, then emit one `canceled-active` result. After a 400 or ambiguous PATCH, read v13 detail once; accept `CANCELED`, `READY`, or another terminal state as a completed race with no active spend, and fail if the deployment remains active or its identity drifted. Race-only reconciliation emits `no-active-deployment`; mixed reconciliation emits results only for deployments actually canceled. A CI failure does not cancel a previously ready Preview; cancellation is driven by current pull request state. - [ ] **Step 7: Write failing transport and secret-safety tests** -Cover missing configuration, 401, 403, 429 with `Retry-After`, 500, timeout abort, invalid JSON, invalid schema, exhausted pagination, and malformed create/cancel responses. Assert 429 and 5xx use at most three total attempts, 401 and 403 do not retry, and no thrown error, log line, URL, or serialized result contains either token. +Cover missing configuration, 401, ordinary 403, rate-limited GitHub 403, 429 with delta-seconds and HTTP-date `Retry-After`, 500, network failure, timeout abort, redirects, invalid JSON, invalid schema, endpoint-specific exhausted or repeated pagination, polling timeout, every terminal build state, and malformed create/detail/cancel responses. Assert safe GET requests use at most three total attempts; ordinary 401/403 do not retry; explicit GitHub rate-limit 403, 429, 5xx, network failure, and timeout may retry within the same cap; and excessive waits fail rather than sleeping without bound. + +Assert timeout, 500, invalid-success body, and 409 create outcomes each perform one POST total and enter the three-attempt exact-list observation. Active or ready visibility is reused; terminal or absent visibility fails. Assert definitive 400, 401, 403, and 422 create responses send no reconciliation retry and no second POST. Cover cancel 400 and ambiguous-PATCH detail reconciliation without another PATCH. No error body, thrown error, log line, URL, request summary, or serialized result may contain either token. - [ ] **Step 8: Implement the bounded JSON client and CLI entry point** -The JSON client must apply a 15 second abort timeout per request, retry only 429 and 5xx responses, cap attempts at three, parse response text as JSON, validate with the supplied schema, and throw a redacted error on failure. Inject `sleep` for tests. Never include request headers in an error. +The JSON client applies a fresh 15-second AbortController per request and clears its timer after body consumption. It rejects redirects and sends a fixed `User-Agent`; GitHub requests also send `Accept: application/vnd.github+json` and `X-GitHub-Api-Version: 2022-11-28`. Parse response text as JSON, validate with the supplied shared schema, and throw only a token-redacted bounded error. Never include headers or raw external bodies in errors. + +Safe GET reads retry network errors, per-attempt timeout, 429, 5xx, and GitHub 403 only with explicit rate-limit evidence. Cap total read attempts at three. Parse delta-seconds and HTTP-date `Retry-After` using injected `now`, bound any sleep to 30 seconds, and treat invalid or excessive waits as failure. Mutations are single-attempt and use the explicit create-observation or cancel-detail rules from Step 6 instead of transport retries. The executable path uses `Bun.file(path).text()`, global `fetch`, a timer-backed sleep, and `console.log`. Guard it with `if (import.meta.main)`, set `process.exitCode = 1` on error, and print only the redacted error message. @@ -309,6 +331,7 @@ Commit: `feat(ci): deploy previews after successful checks` **Files:** - Create: `.github/workflows/vercel-preview.yml` +- Modify: `.github/workflows/ci.yml` - Modify: `apps/web/vercel.json` - Modify: `scripts/labels.ts` - Delete: `scripts/vercel-build-gate.sh` @@ -334,10 +357,13 @@ expect(LABELS.filter(({ name }) => name === 'preview')).toHaveLength(1); expect(LABELS.filter(({ name }) => name === 'no-preview')).toHaveLength(1); expect(workflow).toContain('pull_request_target:'); expect(workflow).toContain('workflow_run:'); -expect(workflow).toContain('workflow_dispatch:'); +expect(workflow).toContain('repository_dispatch:'); +expect(workflow).not.toContain('workflow_dispatch:'); expect(workflow).toContain('persist-credentials: false'); expect(workflow).not.toContain('github.event.pull_request.head.ref'); expect(allGateFiles).not.toContain('BUILD_GATE_GITHUB_TOKEN'); +expect(ciWorkflow).toContain('bun run check-bytes'); +expect(ciWorkflow).toContain('bun run check-bun-imports'); ``` Also test the `**` rule against `feature`, `feature/preview`, and `codex/review/pr341` using the same `minimatch` semantics documented by Vercel. Do not add a dependency: implement the narrow expected assertion by checking that the configured key is exactly `**` and enumerate the branch examples in the test name. @@ -363,6 +389,8 @@ Remove `ignoreCommand` from `apps/web/vercel.json` and add: Add `preview` and `no-preview` beside the status labels in `scripts/labels.ts`, with descriptions matching the design. Delete the old shell gate and its tests. Keep the root `bun test scripts` command so all new script tests remain part of CI. +Add named `Source byte policy` and `No Bun built-ins in shipped server code` steps to the `static` job in `.github/workflows/ci.yml`, invoking `bun run check-bytes` and `bun run check-bun-imports`. This makes the canonical `CI` success used by the dispatcher cover every non-test verification command from `bun run verify`. + - [ ] **Step 4: Add the default-branch workflow** Create a workflow named `Vercel Preview` with this trigger and trust boundary: @@ -370,16 +398,13 @@ Create a workflow named `Vercel Preview` with this trigger and trust boundary: ```yaml on: pull_request_target: - types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled] + branches: [main] + types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled, closed] workflow_run: workflows: [CI] types: [completed] - workflow_dispatch: - inputs: - pull_request: - description: Pull request number - required: true - type: number + repository_dispatch: + types: [vercel-preview-reconcile] permissions: actions: read @@ -387,15 +412,15 @@ permissions: pull-requests: read concurrency: - group: vercel-preview-${{ github.event.pull_request.number || github.event.workflow_run.pull_requests[0].number || inputs.pull_request || github.event.workflow_run.head_sha }} + group: vercel-preview-${{ github.event.pull_request.number || github.event.workflow_run.pull_requests[0].number || github.event.client_payload.pull_request || github.event.workflow_run.head_sha || github.run_id }} cancel-in-progress: false ``` -The job condition allows state events and manual dispatch, and allows a workflow run only when `github.event.workflow_run.event == 'pull_request'`. Checkout the trusted default-branch workflow commit using the pinned checkout action, `repository: ${{ github.repository }}`, `ref: ${{ github.sha }}`, `persist-credentials: false`, `submodules: false`, and `lfs: false`. Set up Bun 1.3.14 with the pinned setup action, run `bun install --frozen-lockfile`, then run `bun scripts/vercel-preview-deploy.ts` with tokens and settings scoped to that step through `env`. +The job condition allows state events and repository dispatch, and allows a workflow run only when `github.event.workflow_run.event == 'pull_request'` and its conclusion is success. Set `timeout-minutes: 25`. GitHub documents that `repository_dispatch` uses the last commit on the default branch, unlike `workflow_dispatch`, which can run a workflow version from a selected non-default ref. Checkout the trusted default-branch workflow commit using the pinned checkout action, `repository: ${{ github.repository }}`, `ref: ${{ github.sha }}`, `persist-credentials: false`, `submodules: false`, and `lfs: false`. Set up Bun 1.3.14 with the pinned setup action, run `bun install --frozen-lockfile --ignore-scripts`, then run `bun scripts/vercel-preview-deploy.ts` with tokens and settings scoped to that step through `env`. - [ ] **Step 5: Rewrite the operations guide and documentation index** -Document the exact eligibility table, CI-green timing, same-repository restriction, active cancellation, web path list, Vercel API behavior, GitHub secret and variables, label synchronization, Git Fork Protection, the deployment-count caveat, the repository-controlled cost-policy limitation, removal of all four old `BUILD_GATE_*` Vercel values, manual recovery, and the post-merge canary. Link the guide from `docs/README.md` under the contributor/operations entries. +Document the exact eligibility table, CI-green timing, same-repository restriction, active cancellation including closed pull requests, web path list, Vercel API behavior, GitHub secret and variables, label synchronization, Git Fork Protection, the deployment-count caveat, the repository-controlled cost-policy limitation, removal of all four old `BUILD_GATE_*` Vercel values, and the post-merge canary. Manual recovery uses a maintainer-authenticated `repository_dispatch` named `vercel-preview-reconcile` with numeric `client_payload.pull_request`; explicitly forbid `workflow_dispatch` because a caller can select a non-default ref. Link the guide from `docs/README.md` under the contributor/operations entries. Do not claim that ignored builds are free, that Ready alone is trust, that fork previews are automatic, or that the privileged workflow can be exercised before it exists on `main`. @@ -407,9 +432,9 @@ Run: `bun run lint && bun run check-comments && bun run check-bytes && bun run c Expected: all commands PASS. -Run: `rg -n 'BUILD_GATE_|vercel-build-gate|Generated with|Claude|—' apps/web/vercel.json scripts .github/workflows/vercel-preview.yml` +Run: `rg -n 'BUILD_GATE_|vercel-build-gate|Generated with' apps/web/vercel.json scripts .github/workflows/vercel-preview.yml` -Run: `rg -n 'Generated with|Claude|—' docs/VERCEL_BUILD_GATE.md docs/README.md` +Run: `rg -n 'Generated with' docs/VERCEL_BUILD_GATE.md docs/README.md` Expected: no old gate setting, prohibited attribution, or em-dash match. A link or historical plan outside this task's changed files is not edited. diff --git a/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md index d776e7269..fa779610a 100644 --- a/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md +++ b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md @@ -72,11 +72,11 @@ variables or an ignored-build exit-code convention. paths: - `pull_request_target` handles `opened`, `reopened`, `ready_for_review`, - `converted_to_draft`, `labeled`, and `unlabeled` state transitions. + `converted_to_draft`, `labeled`, `unlabeled`, and `closed` state transitions. - `workflow_run` handles a completed `CI` workflow and proceeds only when the source event was a pull request and the conclusion was success. -- `workflow_dispatch` accepts a pull request number for maintainer recovery when an event was - missed or a deployment needs to be retried. +- `repository_dispatch` accepts the custom `vercel-preview-reconcile` event with a pull request + number for maintainer recovery when an event was missed or a deployment needs to be retried. The workflow checks out only the trusted workflow SHA from the default branch with persisted Git credentials disabled. It never @@ -87,9 +87,9 @@ values and API request bodies, never as shell source. Permissions are limited to `actions: read`, `contents: read`, and `pull-requests: read`. The Vercel token is a GitHub Actions secret. Team ID, project ID, and project name are Actions variables. Concurrency is keyed by pull request number when the event supplies one and does not cancel an -in-progress reconciler. Vercel metadata checks provide a second idempotency boundary. A rare -workflow-run payload without a pull request number uses its exact head SHA until the controller -resolves the pull request through GitHub. +in-progress reconciler. Vercel metadata checks provide a second idempotency boundary. A +workflow-run payload without a linked pull request is rejected because it cannot prove the run's +base SHA and repository association. ## Controller and validation @@ -99,39 +99,62 @@ Vercel response, and configuration before using them. The controller resolves one or more pull request numbers from the triggering event, then refetches each pull request from GitHub. An event is stale when its recorded head SHA no longer equals the -live pull request head. Stale events are no-ops. Closed pull requests and fork heads are also -no-ops. - -For a pull request state event or manual dispatch, the controller queries completed runs of -`.github/workflows/ci.yml` and requires a successful `pull_request` run for the exact head SHA. A -successful `workflow_run` event already supplies that fact, but the controller still verifies -that its head SHA matches the live pull request. +live pull request head. Stale events are no-ops. Fork heads and pull requests targeting another +repository or branch are also no-ops. A proven closed pull request follows the active-cancellation +path without requiring CI. + +For every eligible event, the controller resolves the canonical `.github/workflows/ci.yml`, fetches +the live `main` ref, and queries every bounded page of runs for the exact head SHA. It selects the +maximum creation time and run ID rather than filtering to successful runs. The run must be +completed successfully and its linked pull request must match the current number, repository IDs, +head ref and SHA, base ref, and live base SHA. A newer queued, failed, canceled, or stale-base run +blocks deployment even when an older green run exists. The triggering `workflow_run` is only a +wake-up signal and is not trusted as proof. Immediately before mutation, the controller refetches +live pull request state and repeats the current CI proof so a concurrent push, label change, or +newer run cannot authorize stale work. Changed files are read from the paginated pull request files endpoint. The web deployment is affected when a filename is below `apps/web/` or `packages/`, or is exactly `package.json`, `bun.lock`, or `tsconfig.base.json`. Configuration is fixed in trusted code rather than split between Vercel and GitHub settings. -GitHub and Vercel requests have a finite timeout. Rate-limit and server failures use bounded -retries. Authentication failures, malformed payloads, exhausted pagination, and unsuccessful -mutations fail the workflow visibly. Logs never contain either token. +GitHub and Vercel requests have a finite timeout. Safe reads use bounded retries for network +failures, server failures, 429 responses, and 403 responses carrying explicit GitHub rate-limit +evidence. Mutations are never retried blindly. Authentication failures, malformed payloads, +exhausted pagination, and unsuccessful mutations fail the workflow visibly. Redirects are rejected +for token-bearing requests, and logs never contain either token. ## Vercel API contract -Before mutation, the controller lists deployments for the configured project and filters them by -Preview target and metadata for repository ID, pull request number, branch ref, and commit SHA. -Pagination is bounded and every page is validated. - -If an exact deployment is queued, initializing, building, or ready, deployment is a no-op. If no -such deployment exists, the controller calls Vercel's Create Deployment endpoint with the linked -GitHub repository ID, exact branch ref, and exact SHA. `target` is omitted so Vercel selects the -Preview environment. Metadata repeats the repository, pull request, branch, and SHA so later runs -can identify the deployment without guessing. +Before mutation, the controller lists deployments through Vercel's current `/v7/deployments` +endpoint, using the configured project plus branch and SHA filters where applicable. It filters +again by exact project ID, null Preview target, and namespaced Orbit metadata for repository ID, +pull request number, branch ref, and commit SHA. List metadata may be absent on unrelated historical +deployments. Pagination is bounded, repeated cursors are rejected, and every page is validated. + +If an exact deployment is ready, it is reused. An exact queued, initializing, or building +deployment is polled rather than duplicated. If no such deployment exists, the controller calls +Vercel's Create Deployment endpoint with the linked GitHub repository ID, exact branch ref, and +exact SHA. `target` is omitted so Vercel selects the Preview environment. Namespaced metadata +repeats the repository, pull request, branch, SHA, workflow run ID, and reason so later runs can +identify the deployment without guessing. `forceNew=1` is used only when a new reconciliation has +already observed an exact terminal failed deployment. Create, detail, and cancel responses must +prove deployment ID, project ID, null Preview target, state, and Orbit metadata. The workflow polls +the created deployment immediately and then through at most 240 five-second sleeps to a ready or +terminal state, requiring a nonempty final URL. + +Create Deployment has no idempotency key. After a network error, timeout, 429, 5xx, 409, or an +unparseable success response, the controller marks the one POST as attempted and performs only a +short bounded exact-list observation. A newly visible ready or active deployment is reused; a +terminal deployment or no visible new deployment fails the workflow. The same reconciliation never +sends a second create request. When current pull request state is ineligible, active deployments associated with that pull request are canceled through Vercel's cancel endpoint. The filter requires the configured project, -Preview target, repository ID, pull request number, and branch ref before cancellation. Ready, -failed, and canceled deployments are left unchanged. +null Preview target, repository ID, pull request number, and branch ref before cancellation. Ready, +failed, and canceled deployments are left unchanged. A 400 or ambiguous cancel response is followed +by one validated detail read so a normal transition to a terminal state is not mistaken for an +unsafe retry. ## Fork policy @@ -179,11 +202,13 @@ unknown states, and malformed pagination. Controller tests use injected fetch and delay functions. They cover ready and draft policy, control-label precedence, state transitions, CI success for the exact SHA, stale events, closed pull requests, fork refusal, relevant and irrelevant paths, GitHub pagination, Vercel pagination, -existing active and ready deployments, one exact create, active cancellation, bounded retries, -timeouts, authentication failures, invalid JSON, invalid response shapes, and missing settings. +existing active and ready deployments, exact project and Preview identity, one exact create, +ambiguous create observation, active cancellation races, bounded retries, timeouts, authentication +failures, invalid JSON, invalid response shapes, and missing settings. Repository checks cover the branch deployment map, the managed labels, removal of the old ignored -command and token references, and discovery of the script tests by the root test command. +command and token references, discovery of the script tests by the root test command, and the two +source-policy checks that hosted CI previously omitted from `bun run verify`. ## Out of scope From 89201b1d5a6cec9f1f44403c58a32836c7c8cd07 Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 15:52:56 +0530 Subject: [PATCH 10/19] feat(ci): deploy previews after successful checks --- .../task-2-report.md | 33 + scripts/vercel-preview-deploy.test.ts | 1221 +++++++++++++++++ scripts/vercel-preview-deploy.ts | 1070 +++++++++++++++ 3 files changed, 2324 insertions(+) create mode 100644 .superpowers/sdd/2026-08-21-vercel-preview-deployment-gate/task-2-report.md create mode 100644 scripts/vercel-preview-deploy.test.ts create mode 100644 scripts/vercel-preview-deploy.ts diff --git a/.superpowers/sdd/2026-08-21-vercel-preview-deployment-gate/task-2-report.md b/.superpowers/sdd/2026-08-21-vercel-preview-deployment-gate/task-2-report.md new file mode 100644 index 000000000..60cd0ffcf --- /dev/null +++ b/.superpowers/sdd/2026-08-21-vercel-preview-deployment-gate/task-2-report.md @@ -0,0 +1,33 @@ +# Task 2 report + +## Outcome + +Implemented the Vercel Preview deployment controller and its injected-runtime test suite. The controller reconciles eligible pull requests against exact live-main CI proof, changed-file relevance, and exact Vercel deployment identity before creating, polling, reusing, or canceling a Preview deployment. + +The implementation includes endpoint-specific bounded pagination, stable workflow-run totals and unique IDs, final freshness checks, a one-POST ambiguity state machine, null-safe Preview and project identity, mutation response validation, cancellation-race handling, redirect rejection, token redaction, and a controller-wide 23-minute monotonic deadline. + +## TDD evidence + +The implementation was developed in focused red-green slices: + +1. Event and eligibility tests were written before the controller existed. `bun test scripts/vercel-preview-deploy.test.ts --test-name-pattern 'event|eligibility|creates'` failed with the expected module-not-found error, with 0 passing tests and 1 loader failure. The completed slice then passed 19 tests. +2. Existing-deployment, idempotency, cancellation, and Vercel pagination cases were added next. The focused run failed 7 tests before the behavior was implemented, then passed. +3. Workflow-run and changed-file pagination, freshness, ambiguity recovery, polling, and deadline cases were added next. The focused run failed 14 tests before the behavior was implemented, then passed. +4. Mutation and detail identity validation plus secret-bearing URL rejection were added next. The focused run failed 4 tests before the behavior was implemented, then passed. +5. The final focused suite passes 81 tests with 182 assertions. + +## Verification + +- `bun test scripts/vercel-preview-deploy.test.ts`: 81 pass, 0 fail, 182 assertions. +- `bun x biome check scripts/vercel-preview-deploy.ts scripts/vercel-preview-deploy.test.ts`: passed. +- `bun x tsc -p scripts/tsconfig.json --noEmit`: passed. +- `bun run lint -- scripts/vercel-preview-deploy.ts scripts/vercel-preview-deploy.test.ts`: exited 0 with only pre-existing repository notices. +- `bun run check-comments`: passed. +- `bun run check-bytes`: passed. +- `bun run check-bun-imports`: passed. +- `git diff --check`: passed. +- `ORBIT_TEST_LANE=preview-gate-task2 bun run verify`: the lint, policy, dependency, typecheck, script-test, and core-test phases passed. Root script tests reported 106 pass and 0 fail. Core reported 974 pass and 0 fail. The realtime phase failed because the worktree has no `BETTER_AUTH_SECRET`. The unrelated web phase later stopped producing output and the already non-green run was interrupted after five minutes without progress. + +## Concerns + +There are no known Task 2 test, type, lint, policy, or byte-check failures. A fully green repository-wide verify requires the local realtime test secret and a non-stalling web suite. Live Vercel and GitHub API behavior remains a production canary concern for the workflow integration task. diff --git a/scripts/vercel-preview-deploy.test.ts b/scripts/vercel-preview-deploy.test.ts new file mode 100644 index 000000000..dd7c5b8bc --- /dev/null +++ b/scripts/vercel-preview-deploy.test.ts @@ -0,0 +1,1221 @@ +import { describe, expect, test } from 'bun:test'; +import type { PreviewRuntime } from './vercel-preview-deploy.ts'; +import { reconcileVercelPreviews } from './vercel-preview-deploy.ts'; + +const SHA = 'a'.repeat(40); +const MAIN_SHA = 'b'.repeat(40); +const GITHUB_TOKEN = 'github-secret-token'; +const VERCEL_TOKEN = 'vercel-secret-token'; + +type RecordedRequest = { + readonly method: string; + readonly url: string; + readonly headers: Headers; + readonly body: unknown; +}; + +type Scenario = { + eventName?: string; + event?: unknown; + pullRequest?: Record; + files?: readonly string[]; + workflowRuns?: readonly Record[]; + deployments?: readonly Record[]; + detailStates?: readonly string[]; + respond?: (request: RecordedRequest, requestNumber: number) => Response | undefined; + now?: () => number; + sleep?: (milliseconds: number) => Promise; +}; + +const repository = { + id: 123, + name: 'orbit', + owner: { login: 'Noveum' }, +}; + +function pullRequest(overrides: Record = {}) { + return { + number: 341, + state: 'open', + draft: false, + labels: [], + head: { sha: SHA, ref: 'feature/preview', repo: repository }, + base: { ref: 'main', repo: repository }, + ...overrides, + }; +} + +function workflowRun(overrides: Record = {}) { + return { + id: 987654321, + workflow_id: 456, + name: 'CI', + event: 'pull_request', + head_sha: SHA, + status: 'completed', + conclusion: 'success', + created_at: '2026-08-21T00:00:00Z', + pull_requests: [ + { + number: 341, + head: { sha: SHA, ref: 'feature/preview', repo: { id: 123, name: 'orbit' } }, + base: { sha: MAIN_SHA, ref: 'main', repo: { id: 123, name: 'orbit' } }, + }, + ], + ...overrides, + }; +} + +function workflowRunEvent(overrides: Record = {}) { + return { + action: 'completed', + repository, + workflow_run: workflowRun(), + ...overrides, + }; +} + +function deployment( + readyState: string, + overrides: Record = {}, +): Record { + return { + uid: `dpl_${readyState.toLowerCase()}`, + projectId: 'prj_orbit', + url: readyState === 'READY' ? 'orbit-preview.vercel.app' : null, + target: null, + readyState, + meta: { + orbitDeploymentReason: 'ci-green-pr-preview', + orbitGithubHeadRef: 'feature/preview', + orbitGithubHeadSha: SHA, + orbitGithubPrNumber: '341', + orbitGithubRepositoryId: '123', + orbitGithubWorkflowRunId: '987654321', + }, + ...overrides, + }; +} + +function metadata(overrides: Record = {}) { + return { + orbitDeploymentReason: 'ci-green-pr-preview', + orbitGithubHeadRef: 'feature/preview', + orbitGithubHeadSha: SHA, + orbitGithubPrNumber: '341', + orbitGithubRepositoryId: '123', + orbitGithubWorkflowRunId: '987654321', + ...overrides, + }; +} + +function mutationDeployment( + id: string, + readyState: string, + overrides: Record = {}, +) { + const listed = deployment(readyState, { uid: id, ...overrides }); + const { uid: _uid, ...rest } = listed; + return { id, ...rest, url: 'orbit-preview.vercel.app' }; +} + +function json(value: unknown, status = 200, headers?: Record): Response { + return new Response(JSON.stringify(value), { + status, + headers: { 'content-type': 'application/json', ...headers }, + }); +} + +function createHarness(scenario: Scenario = {}) { + const requests: RecordedRequest[] = []; + const sleeps: number[] = []; + const logs: string[] = []; + const currentPullRequest = scenario.pullRequest ?? pullRequest(); + const event = + scenario.event ?? + (scenario.eventName === 'repository_dispatch' + ? { + action: 'vercel-preview-reconcile', + client_payload: { pull_request: 341 }, + repository, + } + : workflowRunEvent()); + const detailStates = [...(scenario.detailStates ?? ['READY'])]; + + const fetch = async (input: string | URL | Request, init?: RequestInit): Promise => { + await Promise.resolve(); + const url = String(input); + const method = init?.method ?? 'GET'; + const headers = new Headers(init?.headers); + const body = typeof init?.body === 'string' ? JSON.parse(init.body) : null; + const request = { method, url, headers, body }; + requests.push(request); + const customResponse = scenario.respond?.(request, requests.length); + if (customResponse) return customResponse; + + if (url.includes('/pulls/341/files')) { + return json( + (scenario.files ?? ['apps/web/src/app/page.tsx']).map((filename) => ({ filename })), + ); + } + if (url.endsWith('/pulls/341')) return json(currentPullRequest); + if (url.endsWith('/actions/workflows/ci.yml')) { + return json({ id: 456, name: 'CI', path: '.github/workflows/ci.yml', state: 'active' }); + } + if (url.endsWith('/git/ref/heads/main')) return json({ object: { sha: MAIN_SHA } }); + if (url.includes('/actions/workflows/456/runs')) { + const runs = scenario.workflowRuns ?? [workflowRun()]; + return json({ total_count: runs.length, workflow_runs: runs }); + } + if (url.includes('/v7/deployments')) { + const deployments = scenario.deployments ?? []; + return json({ + deployments, + pagination: { count: deployments.length, next: null, prev: null }, + }); + } + if (url.includes('/v13/deployments/') && method === 'GET') { + const id = url.split('/').at(-1)?.split('?')[0] ?? 'dpl_created'; + return json(mutationDeployment(id, detailStates.shift() ?? 'READY')); + } + if (url.includes('/v13/deployments') && method === 'POST') { + return json(mutationDeployment('dpl_created', 'QUEUED')); + } + if (url.includes('/cancel') && method === 'PATCH') { + const id = url.split('/').at(-2) ?? 'dpl_building'; + return json(mutationDeployment(id, 'CANCELED')); + } + return json({ message: 'unexpected request' }, 500); + }; + + const runtime: PreviewRuntime = { + env: { + GITHUB_EVENT_NAME: scenario.eventName ?? 'workflow_run', + GITHUB_EVENT_PATH: '/event.json', + GITHUB_REPOSITORY: 'Noveum/orbit', + GITHUB_TOKEN, + VERCEL_TOKEN, + VERCEL_TEAM_ID: 'team_orbit', + VERCEL_PROJECT_ID: 'prj_orbit', + VERCEL_PROJECT_NAME: 'orbit', + }, + readText: async () => JSON.stringify(event), + fetch, + sleep: async (milliseconds) => { + sleeps.push(milliseconds); + await scenario.sleep?.(milliseconds); + }, + now: scenario.now ?? (() => Date.parse('2026-08-21T00:00:00Z')), + log: (message) => { + logs.push(message); + }, + }; + + return { runtime, requests, sleeps, logs }; +} + +describe('event and eligibility reconciliation', () => { + test('successful workflow_run for the current ready head creates one deployment', async () => { + const harness = createHarness(); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'created', + pullRequestNumber: 341, + reason: 'created-ready', + deploymentId: 'dpl_created', + url: 'orbit-preview.vercel.app', + }, + ]); + const creates = harness.requests.filter( + ({ method, url }) => method === 'POST' && url.includes('/v13/deployments'), + ); + expect(creates).toHaveLength(1); + const createRequest = creates[0]; + expect(createRequest?.method).toBe('POST'); + expect(createRequest?.url).toContain('/v13/deployments'); + expect(createRequest?.url).not.toContain('forceNew=1'); + expect(createRequest?.body).toEqual({ + name: 'orbit', + project: 'prj_orbit', + gitSource: { type: 'github', repoId: 123, ref: 'feature/preview', sha: SHA }, + meta: { + orbitDeploymentReason: 'ci-green-pr-preview', + orbitGithubHeadRef: 'feature/preview', + orbitGithubHeadSha: SHA, + orbitGithubPrNumber: '341', + orbitGithubRepositoryId: '123', + orbitGithubWorkflowRunId: '987654321', + }, + }); + expect(createRequest?.body).not.toHaveProperty('target'); + }); + + test('repository_dispatch follows current eligibility and exact CI proof', async () => { + const harness = createHarness({ eventName: 'repository_dispatch' }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ kind: 'created', reason: 'created-ready' }); + }); + + test.each([ + ['ready pull request', false, []], + ['preview-labeled draft', true, [{ name: 'preview' }]], + ])('%s state event creates after exact CI success', async (_name, draft, labels) => { + const embeddedPullRequest = pullRequest({ draft, labels }); + const harness = createHarness({ + eventName: 'pull_request_target', + event: { + action: draft ? 'labeled' : 'ready_for_review', + number: 341, + pull_request: embeddedPullRequest, + repository, + }, + pullRequest: embeddedPullRequest, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ kind: 'created', reason: 'created-ready' }); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + }); + + test.each(['closed', 'converted_to_draft', 'labeled'])( + '%s state transition cancels active deployments without CI', + async (action) => { + const livePullRequest = pullRequest({ + state: action === 'closed' ? 'closed' : 'open', + draft: action === 'converted_to_draft', + labels: action === 'labeled' ? [{ name: 'no-preview' }] : [], + }); + const harness = createHarness({ + eventName: 'pull_request_target', + event: { action, number: 341, pull_request: livePullRequest, repository }, + pullRequest: livePullRequest, + deployments: [deployment('BUILDING', { uid: 'dpl_active' })], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'canceled', + pullRequestNumber: 341, + reason: 'canceled-active', + deploymentId: 'dpl_active', + url: 'orbit-preview.vercel.app', + }, + ]); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(1); + expect(harness.requests.some(({ url }) => url.includes('/actions/workflows/'))).toBe(false); + }, + ); + + test.each([ + ['draft without preview', { draft: true }, 'no-active-deployment'], + [ + 'mixed case no-preview wins', + { labels: [{ name: 'PrEvIeW' }, { name: 'No-PrEvIeW' }] }, + 'no-active-deployment', + ], + ['wrong base branch', { base: { ref: 'release', repo: repository } }, 'base-mismatch'], + ] as const)('%s eligibility creates no deployment', async (_name, overrides, reason) => { + const harness = createHarness({ pullRequest: pullRequest(overrides) }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason }]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }); + + test.each([ + ['failed', 'completed', 'failure', 'ci-not-green'], + ['in-progress', 'in_progress', null, 'ci-not-green'], + ] as const)('%s CI creates no deployment', async (_name, status, conclusion, reason) => { + const harness = createHarness({ workflowRuns: [workflowRun({ status, conclusion })] }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason }]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }); + + test('unrelated files create no deployment', async () => { + const harness = createHarness({ files: ['docs/README.md'] }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { kind: 'skipped', pullRequestNumber: 341, reason: 'web-unaffected' }, + ]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }); + + test('stale event SHA cannot create or cancel the current head', async () => { + const event = workflowRunEvent({ workflow_run: workflowRun({ head_sha: 'c'.repeat(40) }) }); + const harness = createHarness({ event }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason: 'stale-event' }]); + expect(harness.requests.some(({ method }) => method === 'POST' || method === 'PATCH')).toBe( + false, + ); + }); + + test('fork heads create no deployment', async () => { + const harness = createHarness({ + pullRequest: pullRequest({ + head: { + sha: SHA, + ref: 'feature/preview', + repo: { id: 999, name: 'orbit', owner: { login: 'contributor' } }, + }, + }), + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { kind: 'skipped', pullRequestNumber: 341, reason: 'fork-pull-request' }, + ]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }); + + test('a newer non-green run blocks an older success', async () => { + const harness = createHarness({ + workflowRuns: [ + workflowRun(), + workflowRun({ + id: 987654322, + created_at: '2026-08-21T00:01:00Z', + status: 'queued', + conclusion: null, + }), + ], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason: 'ci-not-green' }]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }); + + test('event repository identity must match the configured and live repository', async () => { + const harness = createHarness({ + event: workflowRunEvent({ + repository: { ...repository, id: 999, name: 'other' }, + }), + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { kind: 'skipped', pullRequestNumber: 341, reason: 'repository-mismatch' }, + ]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }); + + test('a pull request target event rejects disagreement between event numbers', async () => { + const harness = createHarness({ + eventName: 'pull_request_target', + event: { + action: 'opened', + number: 342, + pull_request: pullRequest(), + repository, + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('event pull request'); + }); + + test('an empty workflow run association logs a closed reason without inventing a candidate', async () => { + const event = workflowRunEvent({ workflow_run: workflowRun({ pull_requests: [] }) }); + const harness = createHarness({ event }); + + await expect(reconcileVercelPreviews(harness.runtime)).resolves.toEqual([]); + expect(harness.logs).toContain('workflow-run-unassociated'); + }); +}); + +describe('existing deployments, cancellation, and pagination', () => { + test.each(['QUEUED', 'INITIALIZING', 'BUILDING'])( + 'existing exact %s deployment is polled without a duplicate create', + async (readyState) => { + const harness = createHarness({ + deployments: [deployment(readyState, { uid: 'dpl_active' })], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'created', + pullRequestNumber: 341, + reason: 'active-deployment-reused', + deploymentId: 'dpl_active', + url: 'orbit-preview.vercel.app', + }, + ]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }, + ); + + test('existing exact ready deployment wins over active and terminal duplicates', async () => { + const harness = createHarness({ + deployments: [ + deployment('ERROR', { uid: 'dpl_error' }), + deployment('BUILDING', { uid: 'dpl_active' }), + deployment('READY', { uid: 'dpl_ready' }), + ], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'created', + pullRequestNumber: 341, + reason: 'ready-deployment-reused', + deploymentId: 'dpl_ready', + url: 'orbit-preview.vercel.app', + }, + ]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + expect(harness.requests.some(({ url }) => url.includes('/v13/deployments/'))).toBe(false); + }); + + test.each(['CANCELED', 'ERROR', 'BLOCKED', 'DELETED'])( + 'existing terminal %s deployment allows one forced create', + async (readyState) => { + const harness = createHarness({ + deployments: [deployment(readyState, { uid: 'dpl_terminal' })], + }); + + await reconcileVercelPreviews(harness.runtime); + + const creates = harness.requests.filter(({ method }) => method === 'POST'); + expect(creates).toHaveLength(1); + expect(creates[0]?.url).toContain('forceNew=1'); + }, + ); + + test.each([ + ['another project', { projectId: 'prj_other' }], + ['staging target', { target: 'staging' }], + ['production target', { target: 'production' }], + ['missing target', { target: undefined }], + ['another repository', { meta: metadata({ orbitGithubRepositoryId: '999' }) }], + ['another pull request', { meta: metadata({ orbitGithubPrNumber: '342' }) }], + ['another ref', { meta: metadata({ orbitGithubHeadRef: 'other' }) }], + ])('%s does not suppress an exact-SHA create', async (_name, overrides) => { + const harness = createHarness({ deployments: [deployment('READY', overrides)] }); + + await reconcileVercelPreviews(harness.runtime); + + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + }); + + test('an unrelated list item without metadata is ignored', async () => { + const harness = createHarness({ + deployments: [ + { + uid: 'dpl_unrelated', + projectId: 'prj_orbit', + url: null, + target: null, + readyState: 'READY', + }, + ], + }); + + await reconcileVercelPreviews(harness.runtime); + + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + }); + + test('Vercel pagination preserves filters and finds exact deployment on page two', async () => { + let listPage = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (!url.includes('/v7/deployments')) return undefined; + listPage += 1; + if (listPage === 1) { + return json({ deployments: [], pagination: { count: 0, next: 123, prev: null } }); + } + return json({ + deployments: [deployment('READY', { uid: 'dpl_page_two' })], + pagination: { count: 1, next: null, prev: 123 }, + }); + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ + reason: 'ready-deployment-reused', + deploymentId: 'dpl_page_two', + }); + const pages = harness.requests.filter(({ url }) => url.includes('/v7/deployments')); + expect(pages).toHaveLength(2); + for (const request of pages) { + const url = new URL(request.url); + expect(url.searchParams.get('teamId')).toBe('team_orbit'); + expect(url.searchParams.get('projectId')).toBe('prj_orbit'); + expect(url.searchParams.get('branch')).toBe('feature/preview'); + expect(url.searchParams.get('sha')).toBe(SHA); + expect(url.searchParams.get('limit')).toBe('100'); + } + expect(new URL(pages[1]?.url ?? '').searchParams.get('until')).toBe('123'); + }); + + test('an exact active deployment wins over terminal history', async () => { + const harness = createHarness({ + deployments: [ + deployment('ERROR', { uid: 'dpl_error' }), + deployment('BUILDING', { uid: 'dpl_active' }), + ], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ + reason: 'active-deployment-reused', + deploymentId: 'dpl_active', + }); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }); + + test('a ready list item with a null URL reads validated detail', async () => { + const harness = createHarness({ + deployments: [deployment('READY', { uid: 'dpl_ready', url: null })], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ + reason: 'ready-deployment-reused', + deploymentId: 'dpl_ready', + url: 'orbit-preview.vercel.app', + }); + expect( + harness.requests.filter(({ url }) => url.includes('/v13/deployments/dpl_ready')), + ).toHaveLength(1); + }); + + test('ineligible state cancels only matching active Preview deployments in ID order', async () => { + const livePullRequest = pullRequest({ draft: true }); + const matching = deployment('BUILDING', { uid: 'dpl_b' }); + const alsoMatching = deployment('QUEUED', { uid: 'dpl_a' }); + const harness = createHarness({ + pullRequest: livePullRequest, + deployments: [ + matching, + deployment('READY', { uid: 'dpl_ready' }), + deployment('ERROR', { uid: 'dpl_error' }), + deployment('BUILDING', { uid: 'dpl_stage', target: 'staging' }), + deployment('BUILDING', { uid: 'dpl_project', projectId: 'prj_other' }), + deployment('BUILDING', { + uid: 'dpl_repo', + meta: metadata({ orbitGithubRepositoryId: '999' }), + }), + alsoMatching, + ], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results.map((result) => result.kind === 'canceled' && result.deploymentId)).toEqual([ + 'dpl_a', + 'dpl_b', + ]); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(2); + }); + + test('cancel 400 accepts a terminal detail race without retrying PATCH', async () => { + const livePullRequest = pullRequest({ draft: true }); + const harness = createHarness({ + pullRequest: livePullRequest, + deployments: [deployment('BUILDING', { uid: 'dpl_race' })], + respond: ({ method, url }) => { + if (method === 'PATCH') return json({ message: 'not cancelable' }, 400); + if (url.includes('/v13/deployments/dpl_race')) { + return json(mutationDeployment('dpl_race', 'READY')); + } + return undefined; + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { kind: 'skipped', pullRequestNumber: 341, reason: 'no-active-deployment' }, + ]); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(1); + expect( + harness.requests.filter(({ url }) => url.includes('/v13/deployments/dpl_race')), + ).toHaveLength(1); + }); + + test('a second reconciliation reuses the deployment created by the first', async () => { + const stored: Record[] = []; + const harness = createHarness({ + respond: ({ method, url }) => { + if (url.includes('/v7/deployments')) { + return json({ + deployments: stored, + pagination: { count: stored.length, next: null, prev: null }, + }); + } + if (method === 'POST') { + stored.push(deployment('READY', { uid: 'dpl_created' })); + return json(mutationDeployment('dpl_created', 'READY')); + } + return undefined; + }, + }); + + await reconcileVercelPreviews(harness.runtime); + const second = await reconcileVercelPreviews(harness.runtime); + + expect(second[0]).toMatchObject({ reason: 'ready-deployment-reused' }); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + }); +}); + +describe('endpoint pagination and final freshness', () => { + test('workflow runs paginate to total_count and a newer page-two run blocks creation', async () => { + const firstPage = Array.from({ length: 100 }, (_, index) => + workflowRun({ id: index + 1, created_at: '2026-08-20T00:00:00Z' }), + ); + const harness = createHarness({ + respond: ({ url }) => { + if (!url.includes('/actions/workflows/456/runs')) return undefined; + const page = new URL(url).searchParams.get('page'); + return page === '1' + ? json({ total_count: 101, workflow_runs: firstPage }) + : json({ + total_count: 101, + workflow_runs: [ + workflowRun({ + id: 999, + created_at: '2026-08-22T00:00:00Z', + status: 'queued', + conclusion: null, + }), + ], + }); + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason: 'ci-not-green' }]); + expect( + harness.requests.filter(({ url }) => url.includes('/actions/workflows/456/runs')), + ).toHaveLength(2); + }); + + test('workflow run pages reject duplicate IDs across pages', async () => { + const page = Array.from({ length: 100 }, (_, index) => workflowRun({ id: index + 1 })); + const harness = createHarness({ + respond: ({ url }) => { + if (!url.includes('/actions/workflows/456/runs')) return undefined; + const pageNumber = new URL(url).searchParams.get('page'); + return pageNumber === '1' + ? json({ total_count: 101, workflow_runs: page }) + : json({ total_count: 101, workflow_runs: [workflowRun({ id: 1 })] }); + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('duplicate'); + }); + + test('workflow run pages reject inconsistent total_count', async () => { + const page = Array.from({ length: 100 }, (_, index) => workflowRun({ id: index + 1 })); + const harness = createHarness({ + respond: ({ url }) => { + if (!url.includes('/actions/workflows/456/runs')) return undefined; + const pageNumber = new URL(url).searchParams.get('page'); + return pageNumber === '1' + ? json({ total_count: 101, workflow_runs: page }) + : json({ total_count: 102, workflow_runs: [workflowRun({ id: 101 })] }); + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('total_count'); + }); + + test('pull request files paginate until a relevant file appears', async () => { + const fullIrrelevantPage = Array.from({ length: 100 }, (_, index) => ({ + filename: `docs/page-${index}.md`, + })); + const harness = createHarness({ + respond: ({ url }) => { + if (!url.includes('/pulls/341/files')) return undefined; + return new URL(url).searchParams.get('page') === '1' + ? json(fullIrrelevantPage) + : json([{ filename: 'packages/shared/src/index.ts' }]); + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ kind: 'created', reason: 'created-ready' }); + expect(harness.requests.filter(({ url }) => url.includes('/pulls/341/files'))).toHaveLength(2); + }); + + test('a full final files page fails closed at the 30-page cap', async () => { + const fullPage = Array.from({ length: 100 }, (_, index) => ({ + filename: `docs/page-${index}.md`, + })); + const harness = createHarness({ + respond: ({ url }) => (url.includes('/pulls/341/files') ? json(fullPage) : undefined), + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('files pagination'); + expect(harness.requests.filter(({ url }) => url.includes('/pulls/341/files'))).toHaveLength(30); + }); + + test('final label freshness prevents mutation after a concurrent state change', async () => { + let pullRequestReads = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (!url.endsWith('/pulls/341')) return undefined; + pullRequestReads += 1; + return json( + pullRequestReads === 1 + ? pullRequest() + : pullRequest({ labels: [{ name: 'no-preview' }] }), + ); + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason: 'stale-event' }]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }); + + test('workflow_run with multiple distinct PR associations logs ambiguity and does no work', async () => { + const event = workflowRunEvent({ + workflow_run: workflowRun({ + pull_requests: [ + ...workflowRun().pull_requests, + { + number: 342, + head: { sha: SHA, ref: 'feature/other', repo: { id: 123, name: 'orbit' } }, + base: { sha: MAIN_SHA, ref: 'main', repo: { id: 123, name: 'orbit' } }, + }, + ], + }), + }); + const harness = createHarness({ event }); + + await expect(reconcileVercelPreviews(harness.runtime)).resolves.toEqual([]); + expect(harness.logs).toContain('workflow-run-ambiguous-associations'); + expect(harness.requests).toHaveLength(0); + }); +}); + +describe('bounded transport, polling, and ambiguity recovery', () => { + test('a safe read uses at most three attempts before the later freshness read', async () => { + let pullReads = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (!url.endsWith('/pulls/341')) return undefined; + pullReads += 1; + return pullReads < 3 ? json({ message: 'temporary' }, 500) : json(pullRequest()); + }, + }); + + await reconcileVercelPreviews(harness.runtime); + + expect(pullReads).toBe(4); + expect(harness.sleeps.slice(0, 2)).toEqual([1000, 1000]); + }); + + test.each([401, 403])('ordinary %s read failures do not retry', async (status) => { + let pullReads = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (!url.endsWith('/pulls/341')) return undefined; + pullReads += 1; + return json({ message: 'denied' }, status); + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow(String(status)); + expect(pullReads).toBe(1); + }); + + test('rate-limited GitHub 403 retries with bounded Retry-After', async () => { + let pullReads = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (!url.endsWith('/pulls/341')) return undefined; + pullReads += 1; + return pullReads === 1 + ? json({ message: 'limited' }, 403, { 'Retry-After': '2' }) + : json(pullRequest()); + }, + }); + + await reconcileVercelPreviews(harness.runtime); + + expect(pullReads).toBeGreaterThan(1); + expect(harness.sleeps[0]).toBe(2000); + }); + + test('redirects are rejected without following token-bearing requests', async () => { + const harness = createHarness({ + respond: ({ url }) => + url.endsWith('/pulls/341') + ? new Response('', { status: 302, headers: { location: 'https://example.com' } }) + : undefined, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('redirect rejected'); + expect(harness.requests).toHaveLength(1); + }); + + test('active polling sleeps through transitions and returns READY', async () => { + const harness = createHarness({ + deployments: [deployment('QUEUED', { uid: 'dpl_active' })], + detailStates: ['INITIALIZING', 'BUILDING', 'READY'], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ reason: 'active-deployment-reused' }); + expect(harness.sleeps.filter((milliseconds) => milliseconds === 5000)).toHaveLength(2); + expect( + harness.requests.filter(({ url }) => url.includes('/v13/deployments/dpl_active')), + ).toHaveLength(3); + }); + + test.each(['ERROR', 'CANCELED', 'BLOCKED', 'DELETED'])( + 'terminal polling state %s fails visibly', + async (readyState) => { + const harness = createHarness({ + deployments: [deployment('BUILDING', { uid: 'dpl_active' })], + detailStates: [readyState], + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow(readyState); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }, + ); + + test.each(['network', '500', '409', 'invalid-success'])( + 'ambiguous create outcome %s sends one POST and observes a newly visible ready deployment', + async (outcome) => { + let listReads = 0; + const harness = createHarness({ + respond: ({ method, url }) => { + if (url.includes('/v7/deployments')) { + listReads += 1; + const items = listReads === 1 ? [] : [deployment('READY', { uid: 'dpl_observed' })]; + return json({ + deployments: items, + pagination: { count: items.length, next: null, prev: null }, + }); + } + if (method === 'POST') { + if (outcome === 'network') throw new Error(`${GITHUB_TOKEN} ${VERCEL_TOKEN}`); + if (outcome === '500') return json({ message: VERCEL_TOKEN }, 500); + if (outcome === '409') return json({ message: GITHUB_TOKEN }, 409); + return new Response('{', { status: 200 }); + } + return undefined; + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ deploymentId: 'dpl_observed' }); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + }, + ); + + test('ambiguous create ignores a pre-existing exact ID and fails without a second POST', async () => { + const existing = deployment('ERROR', { uid: 'dpl_existing' }); + const harness = createHarness({ + deployments: [existing], + respond: ({ method, url }) => { + if (method === 'POST') return json({ message: 'temporary' }, 500); + return url.includes('/v7/deployments') + ? json({ deployments: [existing], pagination: { count: 1, next: null, prev: null } }) + : undefined; + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('ambiguous'); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + }); + + test.each([400, 401, 403, 422])( + 'definitive create %s sends no retry or observation', + async (status) => { + let listReads = 0; + const harness = createHarness({ + respond: ({ method, url }) => { + if (url.includes('/v7/deployments')) listReads += 1; + if (method === 'POST') return json({ message: 'definitive' }, status); + return undefined; + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow(String(status)); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + expect(listReads).toBe(1); + }, + ); + + test('controller deadline stops before the next request or sleep', async () => { + let currentTime = 0; + const harness = createHarness({ + now: () => currentTime, + respond: ({ url }) => { + if (url.endsWith('/pulls/341')) currentTime = 23 * 60 * 1000; + return undefined; + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('controller-timeout'); + expect(harness.requests).toHaveLength(1); + expect(harness.sleeps).toHaveLength(0); + }); + + test('errors and logs redact both tokens', async () => { + const harness = createHarness({ + respond: ({ url }) => { + if (url.endsWith('/pulls/341')) throw new Error(`${GITHUB_TOKEN} ${VERCEL_TOKEN}`); + return undefined; + }, + }); + + let message = ''; + try { + await reconcileVercelPreviews(harness.runtime); + } catch (error) { + message = String(error); + } + expect(message).not.toContain(GITHUB_TOKEN); + expect(message).not.toContain(VERCEL_TOKEN); + expect(JSON.stringify(harness.logs)).not.toContain(GITHUB_TOKEN); + expect(JSON.stringify(harness.logs)).not.toContain(VERCEL_TOKEN); + }); +}); + +describe('identity invariants and bounded edge cases', () => { + test('created response must retain every requested Orbit metadata field', async () => { + const harness = createHarness({ + respond: ({ method }) => + method === 'POST' + ? json( + mutationDeployment('dpl_created', 'QUEUED', { + meta: metadata({ orbitGithubWorkflowRunId: 'wrong' }), + }), + ) + : undefined, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('ambiguous create'); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + }); + + test('detail response must retain metadata from the exact list item', async () => { + const harness = createHarness({ + deployments: [deployment('BUILDING', { uid: 'dpl_active' })], + respond: ({ method, url }) => + method === 'GET' && url.includes('/v13/deployments/dpl_active') + ? json( + mutationDeployment('dpl_active', 'READY', { + meta: metadata({ orbitDeploymentReason: 'wrong' }), + }), + ) + : undefined, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('identity drift'); + }); + + test('cancel response must retain metadata from the listed deployment', async () => { + const livePullRequest = pullRequest({ draft: true }); + const harness = createHarness({ + pullRequest: livePullRequest, + deployments: [deployment('BUILDING', { uid: 'dpl_active' })], + respond: ({ method }) => + method === 'PATCH' + ? json( + mutationDeployment('dpl_active', 'CANCELED', { + meta: metadata({ orbitGithubWorkflowRunId: 'wrong' }), + }), + ) + : undefined, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('identity drift'); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(1); + }); + + test('a deployment URL containing either token is rejected before serialization', async () => { + const harness = createHarness({ + deployments: [ + deployment('READY', { uid: 'dpl_ready', url: `${GITHUB_TOKEN}-${VERCEL_TOKEN}` }), + ], + }); + + let message = ''; + try { + await reconcileVercelPreviews(harness.runtime); + } catch (error) { + message = String(error); + } + expect(message).toContain('unsafe'); + expect(message).not.toContain(GITHUB_TOKEN); + expect(message).not.toContain(VERCEL_TOKEN); + }); + + test('Vercel pagination rejects a repeated zero cursor', async () => { + const harness = createHarness({ + respond: ({ url }) => + url.includes('/v7/deployments') + ? json({ deployments: [], pagination: { count: 0, next: 0, prev: null } }) + : undefined, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('repeated'); + expect(harness.requests.filter(({ url }) => url.includes('/v7/deployments'))).toHaveLength(2); + }); + + test('Vercel pagination fails with a cursor remaining at its finite cap', async () => { + let cursor = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (!url.includes('/v7/deployments')) return undefined; + cursor += 1; + return json({ deployments: [], pagination: { count: 0, next: cursor, prev: null } }); + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('incomplete'); + expect(harness.requests.filter(({ url }) => url.includes('/v7/deployments'))).toHaveLength(20); + }); + + test('equal workflow creation times choose the larger run ID', async () => { + const harness = createHarness({ + workflowRuns: [ + workflowRun({ id: 10 }), + workflowRun({ id: 11, status: 'completed', conclusion: 'failure' }), + ], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason: 'ci-not-green' }]); + }); + + test('final CI proof blocks a run that turns non-green before create', async () => { + let workflowReads = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (!url.includes('/actions/workflows/456/runs')) return undefined; + workflowReads += 1; + const run = + workflowReads === 1 + ? workflowRun() + : workflowRun({ id: 987654322, status: 'completed', conclusion: 'failure' }); + return json({ total_count: 1, workflow_runs: [run] }); + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason: 'ci-not-green' }]); + expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); + }); + + test('an HTTP-date Retry-After uses injected monotonic time', async () => { + let pullReads = 0; + const now = Date.parse('2026-08-21T00:00:00Z'); + const harness = createHarness({ + now: () => now, + respond: ({ url }) => { + if (!url.endsWith('/pulls/341')) return undefined; + pullReads += 1; + return pullReads === 1 + ? json({ message: 'limited' }, 429, { + 'Retry-After': new Date(now + 3000).toUTCString(), + }) + : json(pullRequest()); + }, + }); + + await reconcileVercelPreviews(harness.runtime); + + expect(harness.sleeps[0]).toBe(3000); + }); + + test('an excessive Retry-After fails without sleeping or retrying', async () => { + let pullReads = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (!url.endsWith('/pulls/341')) return undefined; + pullReads += 1; + return json({ message: 'limited' }, 429, { 'Retry-After': '31' }); + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('excessive'); + expect(pullReads).toBe(1); + expect(harness.sleeps).toHaveLength(0); + }); + + test('each retry attempt uses a fresh abort signal', async () => { + const signals: AbortSignal[] = []; + let pullReads = 0; + const harness = createHarness({ + respond: (request) => { + if (!request.url.endsWith('/pulls/341')) return undefined; + pullReads += 1; + const matching = harness.requests.at(-1); + const signalRequest = matching; + if (signalRequest) { + const requestSignal = request.headers; + expect(requestSignal.get('authorization')).toBe(`Bearer ${GITHUB_TOKEN}`); + } + return pullReads === 1 ? json({ message: 'temporary' }, 500) : json(pullRequest()); + }, + }); + const originalFetch = harness.runtime.fetch; + const runtime: PreviewRuntime = { + ...harness.runtime, + fetch: async (input, init) => { + if (String(input).endsWith('/pulls/341') && init?.signal) signals.push(init.signal); + return await originalFetch(input, init); + }, + }; + + await reconcileVercelPreviews(runtime); + + expect(signals).toHaveLength(3); + expect(new Set(signals).size).toBe(3); + }); + + test('missing configuration fails before reading the event', async () => { + const harness = createHarness(); + const runtime: PreviewRuntime = { + ...harness.runtime, + env: { ...harness.runtime.env, VERCEL_PROJECT_ID: undefined }, + }; + + await expect(reconcileVercelPreviews(runtime)).rejects.toThrow(); + expect(harness.requests).toHaveLength(0); + }); +}); diff --git a/scripts/vercel-preview-deploy.ts b/scripts/vercel-preview-deploy.ts new file mode 100644 index 000000000..718a44a3a --- /dev/null +++ b/scripts/vercel-preview-deploy.ts @@ -0,0 +1,1070 @@ +import { + type GithubPreviewPullRequest, + type GithubPreviewRepository, + type GithubPreviewWorkflowRun, + githubPreviewFilesSchema, + githubPreviewPullRequestSchema, + githubPreviewPullRequestTargetEventSchema, + githubPreviewRefSchema, + githubPreviewRepositoryDispatchEventSchema, + githubPreviewWorkflowRunEventSchema, + githubPreviewWorkflowRunsSchema, + githubPreviewWorkflowSchema, + type VercelCanceledDeployment, + type VercelCreatedDeployment, + type VercelDeployment, + type VercelDeploymentDetail, + type VercelPreviewEnvironment, + vercelCanceledDeploymentSchema, + vercelCreatedDeploymentSchema, + vercelDeploymentDetailSchema, + vercelDeploymentsPageSchema, + vercelPreviewEnvironmentSchema, +} from '../packages/shared/src/validators/index.ts'; +import { + isActiveVercelDeployment, + isPreviewEligible, + isReadyVercelDeployment, + isSameRepositoryPullRequest, + isWebPreviewFile, + matchesVercelPullRequest, +} from './vercel-preview-policy.ts'; + +const GITHUB_API = 'https://api.github.com'; +const VERCEL_API = 'https://api.vercel.com'; +const REQUEST_TIMEOUT_MS = 15_000; +const CONTROLLER_TIMEOUT_MS = 23 * 60 * 1000; +const MAX_READ_ATTEMPTS = 3; +const MAX_RETRY_SLEEP_MS = 30_000; +const MAX_VERCEL_PAGES = 20; +const USER_AGENT = 'orbit-vercel-preview-reconciler'; +const ORBIT_METADATA_KEYS = [ + 'orbitDeploymentReason', + 'orbitGithubHeadRef', + 'orbitGithubHeadSha', + 'orbitGithubPrNumber', + 'orbitGithubRepositoryId', + 'orbitGithubWorkflowRunId', +] as const; + +type PreviewReason = + | 'event-not-actionable' + | 'workflow-run-unassociated' + | 'stale-event' + | 'repository-mismatch' + | 'fork-pull-request' + | 'base-mismatch' + | 'preview-ineligible' + | 'no-active-deployment' + | 'web-unaffected' + | 'ci-unavailable' + | 'ci-not-current' + | 'ci-not-green' + | 'ready-deployment-reused' + | 'active-deployment-reused' + | 'created-ready' + | 'canceled-active'; + +type SkippedReason = Exclude< + PreviewReason, + 'ready-deployment-reused' | 'active-deployment-reused' | 'created-ready' | 'canceled-active' +>; + +export type PreviewResult = + | { + readonly kind: 'skipped'; + readonly pullRequestNumber: number; + readonly reason: SkippedReason; + } + | { + readonly kind: 'created'; + readonly pullRequestNumber: number; + readonly reason: 'ready-deployment-reused' | 'active-deployment-reused' | 'created-ready'; + readonly deploymentId: string; + readonly url: string; + } + | { + readonly kind: 'canceled'; + readonly pullRequestNumber: number; + readonly reason: 'canceled-active'; + readonly deploymentId: string; + readonly url: string; + }; + +export type PreviewRuntime = { + readonly env: Readonly>; + readonly readText: (path: string) => Promise; + readonly fetch: (input: string | URL | Request, init?: RequestInit) => Promise; + readonly sleep: (milliseconds: number) => Promise; + readonly now: () => number; + readonly log: (message: string) => void; +}; + +type ControllerRuntime = PreviewRuntime & { + readonly assertBudget: (duration: number) => void; +}; + +type PreviewCandidate = { + readonly number: number; + readonly expectedHeadSha: string | null; +}; + +type RequestKind = 'github' | 'vercel'; + +class RequestFailure extends Error { + readonly status: number | null; + readonly ambiguousMutation: boolean; + readonly retryDelay: number | null; + + constructor( + message: string, + status: number | null, + ambiguousMutation = false, + retryDelay: number | null = null, + ) { + super(message); + this.status = status; + this.ambiguousMutation = ambiguousMutation; + this.retryDelay = retryDelay; + } +} + +type ResponseSchema = { + readonly safeParse: ( + value: unknown, + ) => { readonly success: true; readonly data: T } | { readonly success: false }; +}; + +function safeMessage(value: unknown, environment?: VercelPreviewEnvironment): string { + const initial = value instanceof Error ? value.message : String(value); + const secrets = environment ? [environment.GITHUB_TOKEN, environment.VERCEL_TOKEN] : []; + return secrets + .reduce((message, secret) => message.split(secret).join('[redacted]'), initial) + .slice(0, 512); +} + +function fail(message: string): never { + throw new Error(message); +} + +function withControllerDeadline(runtime: PreviewRuntime): ControllerRuntime { + let previousTime = runtime.now(); + const deadline = previousTime + CONTROLLER_TIMEOUT_MS; + const assertBudget = (duration: number) => { + const currentTime = runtime.now(); + if (currentTime < previousTime || duration < 0 || currentTime > deadline - duration) { + fail('controller-timeout'); + } + previousTime = currentTime; + }; + return { + ...runtime, + assertBudget, + readText: async (path) => { + assertBudget(0); + const text = await runtime.readText(path); + assertBudget(0); + return text; + }, + fetch: async (input, init) => { + assertBudget(REQUEST_TIMEOUT_MS); + const response = await runtime.fetch(input, init); + assertBudget(0); + return response; + }, + sleep: async (milliseconds) => { + assertBudget(milliseconds); + await runtime.sleep(milliseconds); + assertBudget(0); + }, + }; +} + +function parseJsonText(text: string, label: string): unknown { + try { + return JSON.parse(text); + } catch { + return fail(`${label} returned invalid JSON`); + } +} + +function retryAfterMilliseconds(response: Response, runtime: PreviewRuntime): number | null { + const retryAfter = response.headers.get('retry-after'); + if (retryAfter !== null) { + const seconds = Number(retryAfter); + const milliseconds = Number.isFinite(seconds) + ? seconds * 1000 + : Date.parse(retryAfter) - runtime.now(); + if (!Number.isFinite(milliseconds) || milliseconds < 0 || milliseconds > MAX_RETRY_SLEEP_MS) { + return fail('retry wait is invalid or excessive'); + } + return milliseconds; + } + const remaining = response.headers.get('x-ratelimit-remaining'); + const reset = response.headers.get('x-ratelimit-reset'); + if (remaining === '0' && reset !== null) { + const milliseconds = Number(reset) * 1000 - runtime.now(); + if (!Number.isFinite(milliseconds) || milliseconds < 0 || milliseconds > MAX_RETRY_SLEEP_MS) { + return fail('retry wait is invalid or excessive'); + } + return milliseconds; + } + return null; +} + +function shouldRetryRead(response: Response, kind: RequestKind): boolean { + if (response.status === 429 || response.status >= 500) return true; + if (kind !== 'github' || response.status !== 403) return false; + return ( + response.headers.has('retry-after') || + (response.headers.get('x-ratelimit-remaining') === '0' && + response.headers.has('x-ratelimit-reset')) + ); +} + +function requestHeaders( + environment: VercelPreviewEnvironment, + kind: RequestKind, + includesBody: boolean, +): Headers { + const headers = new Headers({ + Authorization: `Bearer ${kind === 'github' ? environment.GITHUB_TOKEN : environment.VERCEL_TOKEN}`, + 'User-Agent': USER_AGENT, + }); + if (kind === 'github') { + headers.set('Accept', 'application/vnd.github+json'); + headers.set('X-GitHub-Api-Version', '2022-11-28'); + } + if (includesBody) headers.set('Content-Type', 'application/json'); + return headers; +} + +function parseSuccessfulResponse( + response: Response, + text: string, + schema: ResponseSchema, + safeRead: boolean, +): T { + let parsed: unknown; + try { + parsed = JSON.parse(text); + } catch { + throw new RequestFailure('response returned invalid JSON', response.status, !safeRead); + } + const result = schema.safeParse(parsed); + if (!result.success) { + throw new RequestFailure('response did not match its schema', response.status, !safeRead); + } + return result.data; +} + +function interpretResponse( + runtime: ControllerRuntime, + kind: RequestKind, + response: Response, + text: string, + schema: ResponseSchema, + safeRead: boolean, +): T { + if (response.status >= 300 && response.status < 400) { + throw new RequestFailure('redirect rejected', response.status); + } + if (response.ok) return parseSuccessfulResponse(response, text, schema, safeRead); + const retryDelay = + safeRead && shouldRetryRead(response, kind) + ? (retryAfterMilliseconds(response, runtime) ?? 1000) + : null; + const ambiguousMutation = + !safeRead && (response.status === 409 || response.status === 429 || response.status >= 500); + throw new RequestFailure( + `request failed with status ${response.status}`, + response.status, + ambiguousMutation, + retryDelay, + ); +} + +async function requestJsonAttempt( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + kind: RequestKind, + url: string, + schema: ResponseSchema, + method: 'GET' | 'POST' | 'PATCH', + body: unknown, +): Promise { + const safeRead = method === 'GET'; + const controller = new AbortController(); + const timeout = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS); + const request: RequestInit = { + method, + headers: requestHeaders(environment, kind, body !== undefined), + redirect: 'manual', + signal: controller.signal, + }; + if (body !== undefined) request.body = JSON.stringify(body); + try { + let response: Response; + try { + response = await runtime.fetch(url, request); + } catch (error) { + throw new RequestFailure( + safeMessage(error, environment), + null, + !safeRead, + safeRead ? 1000 : null, + ); + } + let text: string; + try { + text = await response.text(); + } catch (error) { + throw new RequestFailure( + safeMessage(error, environment), + response.status, + !safeRead, + safeRead ? 1000 : null, + ); + } + return interpretResponse(runtime, kind, response, text, schema, safeRead); + } finally { + clearTimeout(timeout); + } +} + +async function requestJson( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + kind: RequestKind, + url: string, + schema: ResponseSchema, + options: { readonly method?: 'GET' | 'POST' | 'PATCH'; readonly body?: unknown } = {}, +): Promise { + const method = options.method ?? 'GET'; + const safeRead = method === 'GET'; + const maximumAttempts = safeRead ? MAX_READ_ATTEMPTS : 1; + for (let attempt = 1; attempt <= maximumAttempts; attempt += 1) { + try { + return await requestJsonAttempt( + runtime, + environment, + kind, + url, + schema, + method, + options.body, + ); + } catch (error) { + const retryDelay = error instanceof RequestFailure ? error.retryDelay : null; + if (attempt === maximumAttempts || retryDelay === null) throw error; + await runtime.sleep(retryDelay); + } + } + return fail('read attempts exhausted'); +} + +function githubUrl(repositorySlug: string, path: string): string { + return `${GITHUB_API}/repos/${repositorySlug}${path}`; +} + +function vercelUrl(path: string, query: Readonly>): string { + const url = new URL(path, VERCEL_API); + for (const [key, value] of Object.entries(query)) url.searchParams.set(key, value); + return url.toString(); +} + +function repositorySlugMatches( + repository: GithubPreviewRepository, + configuredSlug: string, +): boolean { + return ( + `${repository.owner.login}/${repository.name}`.toLowerCase() === configuredSlug.toLowerCase() + ); +} + +function skipped(pullRequestNumber: number, reason: SkippedReason): PreviewResult { + return { kind: 'skipped', pullRequestNumber, reason }; +} + +function resolveCandidates( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + eventValue: unknown, +): { + readonly repository: GithubPreviewRepository; + readonly candidates: readonly PreviewCandidate[]; +} { + if (environment.GITHUB_EVENT_NAME === 'pull_request_target') { + const event = githubPreviewPullRequestTargetEventSchema.parse(eventValue); + if (event.number !== event.pull_request.number) fail('event pull request numbers disagree'); + return { + repository: event.repository, + candidates: [{ number: event.number, expectedHeadSha: event.pull_request.head.sha }], + }; + } + if (environment.GITHUB_EVENT_NAME === 'repository_dispatch') { + const event = githubPreviewRepositoryDispatchEventSchema.parse(eventValue); + return { + repository: event.repository, + candidates: [{ number: event.client_payload.pull_request, expectedHeadSha: null }], + }; + } + const event = githubPreviewWorkflowRunEventSchema.parse(eventValue); + const run = event.workflow_run; + if ( + event.action !== 'completed' || + run.name !== 'CI' || + run.event !== 'pull_request' || + run.status !== 'completed' || + run.conclusion !== 'success' + ) { + runtime.log('event-not-actionable'); + return { repository: event.repository, candidates: [] }; + } + if (run.pull_requests.length === 0) { + runtime.log('workflow-run-unassociated'); + return { repository: event.repository, candidates: [] }; + } + const numbers = [...new Set(run.pull_requests.map(({ number }) => number))].sort( + (left, right) => left - right, + ); + if (numbers.length > 1) { + runtime.log('workflow-run-ambiguous-associations'); + return { repository: event.repository, candidates: [] }; + } + return { + repository: event.repository, + candidates: numbers.map((number) => ({ number, expectedHeadSha: run.head_sha })), + }; +} + +function fetchPullRequest( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + number: number, +) { + return requestJson( + runtime, + environment, + 'github', + githubUrl(environment.GITHUB_REPOSITORY, `/pulls/${number}`), + githubPreviewPullRequestSchema, + ); +} + +type CiProof = + | { readonly ok: true; readonly workflowRunId: number } + | { readonly ok: false; readonly reason: 'ci-unavailable' | 'ci-not-current' | 'ci-not-green' }; + +function newestWorkflowRun( + runs: readonly GithubPreviewWorkflowRun[], +): GithubPreviewWorkflowRun | null { + let newest: GithubPreviewWorkflowRun | null = null; + for (const run of runs) { + if (newest === null) { + newest = run; + continue; + } + const timeDifference = Date.parse(run.created_at) - Date.parse(newest.created_at); + if (timeDifference > 0 || (timeDifference === 0 && run.id > newest.id)) newest = run; + } + return newest; +} + +function runAssociationMatches( + run: GithubPreviewWorkflowRun, + pullRequest: GithubPreviewPullRequest, + mainSha: string, +): boolean { + return run.pull_requests.some( + (association) => + association.number === pullRequest.number && + association.head.sha === pullRequest.head.sha && + association.head.ref === pullRequest.head.ref && + association.head.repo.id === pullRequest.head.repo.id && + association.base.sha === mainSha && + association.base.ref === pullRequest.base.ref && + association.base.repo.id === pullRequest.base.repo.id, + ); +} + +async function fetchWorkflowRuns( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + workflowId: number, + headSha: string, +): Promise { + const runsUrl = new URL( + githubUrl(environment.GITHUB_REPOSITORY, `/actions/workflows/${workflowId}/runs`), + ); + runsUrl.searchParams.set('event', 'pull_request'); + runsUrl.searchParams.set('head_sha', headSha); + runsUrl.searchParams.set('per_page', '100'); + const runs: GithubPreviewWorkflowRun[] = []; + const seenRunIds = new Set(); + let expectedTotal: number | null = null; + for (let pageNumber = 1; pageNumber <= 10; pageNumber += 1) { + runsUrl.searchParams.set('page', String(pageNumber)); + const page = await requestJson( + runtime, + environment, + 'github', + runsUrl.toString(), + githubPreviewWorkflowRunsSchema, + ); + if (expectedTotal === null) expectedTotal = page.total_count; + if (page.total_count !== expectedTotal) fail('workflow run total_count changed between pages'); + for (const run of page.workflow_runs) { + if (seenRunIds.has(run.id)) fail('workflow run pagination returned a duplicate ID'); + seenRunIds.add(run.id); + runs.push(run); + } + if (runs.length > expectedTotal) fail('workflow run total_count is inconsistent'); + if (runs.length === expectedTotal) return runs; + } + return fail('workflow run pagination is incomplete'); +} + +async function proveCurrentCi( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, +): Promise { + const workflow = await requestJson( + runtime, + environment, + 'github', + githubUrl(environment.GITHUB_REPOSITORY, '/actions/workflows/ci.yml'), + githubPreviewWorkflowSchema, + ); + if ( + workflow.name !== 'CI' || + workflow.path !== '.github/workflows/ci.yml' || + workflow.state !== 'active' + ) { + return { ok: false, reason: 'ci-not-current' }; + } + const main = await requestJson( + runtime, + environment, + 'github', + githubUrl(environment.GITHUB_REPOSITORY, '/git/ref/heads/main'), + githubPreviewRefSchema, + ); + const runs = await fetchWorkflowRuns(runtime, environment, workflow.id, pullRequest.head.sha); + const newest = newestWorkflowRun(runs); + if (newest === null) return { ok: false, reason: 'ci-unavailable' }; + if ( + newest.workflow_id !== workflow.id || + newest.event !== 'pull_request' || + newest.head_sha !== pullRequest.head.sha || + !runAssociationMatches(newest, pullRequest, main.object.sha) + ) { + return { ok: false, reason: 'ci-not-current' }; + } + if (newest.status !== 'completed' || newest.conclusion !== 'success') { + return { ok: false, reason: 'ci-not-green' }; + } + return { ok: true, workflowRunId: newest.id }; +} + +async function affectsWeb( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequestNumber: number, +): Promise { + const url = new URL( + githubUrl(environment.GITHUB_REPOSITORY, `/pulls/${pullRequestNumber}/files`), + ); + url.searchParams.set('per_page', '100'); + for (let pageNumber = 1; pageNumber <= 30; pageNumber += 1) { + url.searchParams.set('page', String(pageNumber)); + const files = await requestJson( + runtime, + environment, + 'github', + url.toString(), + githubPreviewFilesSchema, + ); + if (files.some(({ filename }) => isWebPreviewFile(filename))) return true; + if (files.length < 100) return false; + } + return fail('pull request files pagination is incomplete'); +} + +function deploymentMetadata(pullRequest: GithubPreviewPullRequest, workflowRunId: number) { + return { + orbitDeploymentReason: 'ci-green-pr-preview', + orbitGithubHeadRef: pullRequest.head.ref, + orbitGithubHeadSha: pullRequest.head.sha, + orbitGithubPrNumber: String(pullRequest.number), + orbitGithubRepositoryId: String(pullRequest.base.repo.id), + orbitGithubWorkflowRunId: String(workflowRunId), + }; +} + +async function listDeployments( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, + includeSha: boolean, +): Promise { + const query: Record = { + teamId: environment.VERCEL_TEAM_ID, + projectId: environment.VERCEL_PROJECT_ID, + branch: pullRequest.head.ref, + limit: '100', + }; + if (includeSha) query['sha'] = pullRequest.head.sha; + const deployments: VercelDeployment[] = []; + const seenCursors = new Set(); + let cursor: number | null = null; + for (let pageNumber = 1; pageNumber <= MAX_VERCEL_PAGES; pageNumber += 1) { + const pageQuery = { ...query }; + if (cursor !== null) pageQuery['until'] = String(cursor); + const page = await requestJson( + runtime, + environment, + 'vercel', + vercelUrl('/v7/deployments', pageQuery), + vercelDeploymentsPageSchema, + ); + deployments.push(...page.deployments); + const next = page.pagination.next; + if (next === null) return deployments; + if (seenCursors.has(next)) fail('Vercel deployment pagination repeated a cursor'); + seenCursors.add(next); + cursor = next; + } + return fail('Vercel deployment pagination is incomplete'); +} + +function detailIdentityMatches( + detail: VercelDeploymentDetail, + id: string, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, + expectedMetadata: Readonly>, + headSha?: string, +): boolean { + const comparable: VercelDeployment = { + uid: detail.id, + projectId: detail.projectId, + url: detail.url, + target: detail.target, + readyState: detail.readyState, + meta: detail.meta, + }; + return ( + detail.id === id && + ORBIT_METADATA_KEYS.every((key) => { + const expected = expectedMetadata[key]; + const actual = detail.meta[key]; + return expected !== undefined && expected !== null && String(actual) === String(expected); + }) && + matchesVercelPullRequest(comparable, pullRequest, environment.VERCEL_PROJECT_ID, headSha) + ); +} + +function assertSafeDeploymentUrl(url: string, environment: VercelPreviewEnvironment): string { + if (url.includes(environment.GITHUB_TOKEN) || url.includes(environment.VERCEL_TOKEN)) { + fail('deployment URL is unsafe'); + } + return url; +} + +async function pollDeployment( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, + deploymentId: string, + expectedMetadata: Readonly>, +): Promise { + while (true) { + const detail = await requestJson( + runtime, + environment, + 'vercel', + vercelUrl(`/v13/deployments/${deploymentId}`, { teamId: environment.VERCEL_TEAM_ID }), + vercelDeploymentDetailSchema, + ); + assertSafeDeploymentUrl(detail.url, environment); + if ( + !detailIdentityMatches( + detail, + deploymentId, + environment, + pullRequest, + expectedMetadata, + pullRequest.head.sha, + ) + ) { + return fail('deployment detail identity drift'); + } + if (detail.readyState === 'READY') return detail; + const comparable: VercelDeployment = { + uid: detail.id, + projectId: detail.projectId, + url: detail.url, + target: detail.target, + readyState: detail.readyState, + meta: detail.meta, + }; + if (!isActiveVercelDeployment(comparable)) + return fail(`deployment ended in ${detail.readyState}`); + await runtime.sleep(5000); + } +} + +function currentStateMatches( + first: GithubPreviewPullRequest, + current: GithubPreviewPullRequest, +): boolean { + const firstLabels = first.labels + .map(({ name }) => name) + .sort() + .join('\n'); + const currentLabels = current.labels + .map(({ name }) => name) + .sort() + .join('\n'); + return ( + first.number === current.number && + first.state === current.state && + first.draft === current.draft && + first.head.sha === current.head.sha && + first.head.ref === current.head.ref && + first.head.repo.id === current.head.repo.id && + first.base.ref === current.base.ref && + first.base.repo.id === current.base.repo.id && + firstLabels === currentLabels + ); +} + +async function cancelOneActiveDeployment( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, + item: VercelDeployment, +): Promise { + let canceled: VercelCanceledDeployment; + try { + canceled = await requestJson( + runtime, + environment, + 'vercel', + vercelUrl(`/v12/deployments/${item.uid}/cancel`, { teamId: environment.VERCEL_TEAM_ID }), + vercelCanceledDeploymentSchema, + { method: 'PATCH' }, + ); + } catch (error) { + if (!(error instanceof RequestFailure)) throw error; + if (error.status !== 400 && !error.ambiguousMutation) throw error; + const detail = await requestJson( + runtime, + environment, + 'vercel', + vercelUrl(`/v13/deployments/${item.uid}`, { teamId: environment.VERCEL_TEAM_ID }), + vercelDeploymentDetailSchema, + ); + assertSafeDeploymentUrl(detail.url, environment); + if (!detailIdentityMatches(detail, item.uid, environment, pullRequest, item.meta)) { + fail('canceled deployment identity drift'); + } + if (isActiveVercelDeployment({ ...item, readyState: detail.readyState })) { + fail('deployment remains active after cancel race'); + } + return null; + } + if ( + canceled.readyState !== 'CANCELED' || + !detailIdentityMatches(canceled, item.uid, environment, pullRequest, item.meta) + ) { + fail('canceled deployment identity drift'); + } + return { + kind: 'canceled', + pullRequestNumber: pullRequest.number, + reason: 'canceled-active', + deploymentId: canceled.id, + url: assertSafeDeploymentUrl(canceled.url, environment), + }; +} + +async function cancelActiveDeployments( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, +): Promise { + const listed = await listDeployments(runtime, environment, pullRequest, false); + const active = listed + .filter( + (item) => + isActiveVercelDeployment(item) && + matchesVercelPullRequest(item, pullRequest, environment.VERCEL_PROJECT_ID), + ) + .sort((left, right) => left.uid.localeCompare(right.uid)); + if (active.length === 0) { + return [skipped(pullRequest.number, 'no-active-deployment')]; + } + const finalPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); + if (!currentStateMatches(pullRequest, finalPullRequest)) { + return [skipped(pullRequest.number, 'stale-event')]; + } + if (finalPullRequest.state === 'open' && isPreviewEligible(finalPullRequest)) { + return [skipped(pullRequest.number, 'preview-ineligible')]; + } + const results: PreviewResult[] = []; + for (const item of active) { + const result = await cancelOneActiveDeployment(runtime, environment, finalPullRequest, item); + if (result) results.push(result); + } + return results.length > 0 ? results : [skipped(pullRequest.number, 'no-active-deployment')]; +} + +async function observedCreateResult( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, + deployments: readonly VercelDeployment[], +): Promise { + const ready = deployments.find(isReadyVercelDeployment); + if (ready) { + const final = ready.url + ? { id: ready.uid, url: assertSafeDeploymentUrl(ready.url, environment) } + : await pollDeployment(runtime, environment, pullRequest, ready.uid, ready.meta); + return { + kind: 'created', + pullRequestNumber: pullRequest.number, + reason: 'ready-deployment-reused', + deploymentId: final.id, + url: final.url, + }; + } + const active = deployments.find(isActiveVercelDeployment); + if (!active) return null; + const final = await pollDeployment(runtime, environment, pullRequest, active.uid, active.meta); + return { + kind: 'created', + pullRequestNumber: pullRequest.number, + reason: 'active-deployment-reused', + deploymentId: final.id, + url: final.url, + }; +} + +async function observeAmbiguousCreate( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, + preCreateDeploymentIds: ReadonlySet, +): Promise { + for (let observation = 1; observation <= 3; observation += 1) { + if (observation > 1) await runtime.sleep(2000); + const observed = await listDeployments(runtime, environment, pullRequest, true); + const newlyVisible = observed.filter( + (item) => + !preCreateDeploymentIds.has(item.uid) && + matchesVercelPullRequest( + item, + pullRequest, + environment.VERCEL_PROJECT_ID, + pullRequest.head.sha, + ), + ); + const result = await observedCreateResult(runtime, environment, pullRequest, newlyVisible); + if (result) return result; + if (newlyVisible.length > 0) fail('ambiguous create observed a terminal deployment'); + } + return fail('ambiguous create produced no new deployment'); +} + +async function createDeployment( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, + workflowRunId: number, + forceNew: boolean, + preCreateDeploymentIds: ReadonlySet, +): Promise { + const metadata = deploymentMetadata(pullRequest, workflowRunId); + let created: VercelCreatedDeployment; + try { + created = await requestJson( + runtime, + environment, + 'vercel', + vercelUrl('/v13/deployments', { + teamId: environment.VERCEL_TEAM_ID, + ...(forceNew ? { forceNew: '1' } : {}), + }), + vercelCreatedDeploymentSchema, + { + method: 'POST', + body: { + name: environment.VERCEL_PROJECT_NAME, + project: environment.VERCEL_PROJECT_ID, + gitSource: { + type: 'github', + repoId: pullRequest.base.repo.id, + ref: pullRequest.head.ref, + sha: pullRequest.head.sha, + }, + meta: metadata, + }, + }, + ); + assertSafeDeploymentUrl(created.url, environment); + if ( + !detailIdentityMatches( + created, + created.id, + environment, + pullRequest, + metadata, + pullRequest.head.sha, + ) + ) { + throw new RequestFailure('created deployment identity drift', 200, true); + } + } catch (error) { + if (!(error instanceof RequestFailure && error.ambiguousMutation)) throw error; + return observeAmbiguousCreate(runtime, environment, pullRequest, preCreateDeploymentIds); + } + const ready = await pollDeployment(runtime, environment, pullRequest, created.id, created.meta); + return { + kind: 'created', + pullRequestNumber: pullRequest.number, + reason: 'created-ready', + deploymentId: ready.id, + url: ready.url, + }; +} + +async function reconcileCandidate( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + eventRepository: GithubPreviewRepository, + candidate: PreviewCandidate, +): Promise { + const pullRequest = await fetchPullRequest(runtime, environment, candidate.number); + if ( + !repositorySlugMatches(eventRepository, environment.GITHUB_REPOSITORY) || + eventRepository.id !== pullRequest.base.repo.id || + !repositorySlugMatches(pullRequest.base.repo, environment.GITHUB_REPOSITORY) + ) { + return [skipped(candidate.number, 'repository-mismatch')]; + } + if (candidate.expectedHeadSha !== null && candidate.expectedHeadSha !== pullRequest.head.sha) { + return [skipped(candidate.number, 'stale-event')]; + } + if (!isSameRepositoryPullRequest(pullRequest)) { + return [skipped(candidate.number, 'fork-pull-request')]; + } + if (pullRequest.base.ref !== 'main') return [skipped(candidate.number, 'base-mismatch')]; + if (pullRequest.state !== 'open' || !isPreviewEligible(pullRequest)) { + return cancelActiveDeployments(runtime, environment, pullRequest); + } + const ci = await proveCurrentCi(runtime, environment, pullRequest); + if (!ci.ok) return [skipped(candidate.number, ci.reason)]; + if (!(await affectsWeb(runtime, environment, pullRequest.number))) { + return [skipped(candidate.number, 'web-unaffected')]; + } + const listed = await listDeployments(runtime, environment, pullRequest, true); + const exact = listed.filter((item) => + matchesVercelPullRequest( + item, + pullRequest, + environment.VERCEL_PROJECT_ID, + pullRequest.head.sha, + ), + ); + const ready = exact.find(isReadyVercelDeployment); + if (ready) { + const readyDeployment = ready.url + ? { id: ready.uid, url: assertSafeDeploymentUrl(ready.url, environment) } + : await pollDeployment(runtime, environment, pullRequest, ready.uid, ready.meta); + return [ + { + kind: 'created', + pullRequestNumber: pullRequest.number, + reason: 'ready-deployment-reused', + deploymentId: readyDeployment.id, + url: readyDeployment.url, + }, + ]; + } + const active = exact.find(isActiveVercelDeployment); + if (active) { + const final = await pollDeployment(runtime, environment, pullRequest, active.uid, active.meta); + return [ + { + kind: 'created', + pullRequestNumber: pullRequest.number, + reason: 'active-deployment-reused', + deploymentId: final.id, + url: final.url, + }, + ]; + } + const finalPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); + if (!currentStateMatches(pullRequest, finalPullRequest)) { + return [skipped(candidate.number, 'stale-event')]; + } + const finalCi = await proveCurrentCi(runtime, environment, finalPullRequest); + if (!finalCi.ok) return [skipped(candidate.number, finalCi.reason)]; + return [ + await createDeployment( + runtime, + environment, + finalPullRequest, + finalCi.workflowRunId, + exact.length > 0, + new Set(listed.map(({ uid }) => uid)), + ), + ]; +} + +export async function reconcileVercelPreviews( + runtime: PreviewRuntime, +): Promise { + const environment = vercelPreviewEnvironmentSchema.parse(runtime.env); + const controllerRuntime = withControllerDeadline(runtime); + try { + const eventText = await controllerRuntime.readText(environment.GITHUB_EVENT_PATH); + const eventValue = parseJsonText(eventText, 'event'); + const resolved = await resolveCandidates(controllerRuntime, environment, eventValue); + const results: PreviewResult[] = []; + for (const candidate of resolved.candidates) { + results.push( + ...(await reconcileCandidate( + controllerRuntime, + environment, + resolved.repository, + candidate, + )), + ); + } + return results; + } catch (error) { + throw new Error(safeMessage(error, environment)); + } +} + +if (import.meta.main) { + const runtime: PreviewRuntime = { + env: process.env, + readText: (path) => Bun.file(path).text(), + fetch: globalThis.fetch, + sleep: (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds)), + now: () => Date.now(), + log: (message) => console.log(message), + }; + try { + const results = await reconcileVercelPreviews(runtime); + for (const result of results) runtime.log(JSON.stringify(result)); + } catch (error) { + console.error(safeMessage(error)); + process.exitCode = 1; + } +} From cb219bfe7c943884b424607f391a3d302bfb1df7 Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 16:09:04 +0530 Subject: [PATCH 11/19] fix(ci): harden preview reconciliation --- .../task-2-report.md | 33 ------- scripts/vercel-preview-deploy.test.ts | 93 +++++++++++++++---- scripts/vercel-preview-deploy.ts | 22 +++-- 3 files changed, 87 insertions(+), 61 deletions(-) delete mode 100644 .superpowers/sdd/2026-08-21-vercel-preview-deployment-gate/task-2-report.md diff --git a/.superpowers/sdd/2026-08-21-vercel-preview-deployment-gate/task-2-report.md b/.superpowers/sdd/2026-08-21-vercel-preview-deployment-gate/task-2-report.md deleted file mode 100644 index 60cd0ffcf..000000000 --- a/.superpowers/sdd/2026-08-21-vercel-preview-deployment-gate/task-2-report.md +++ /dev/null @@ -1,33 +0,0 @@ -# Task 2 report - -## Outcome - -Implemented the Vercel Preview deployment controller and its injected-runtime test suite. The controller reconciles eligible pull requests against exact live-main CI proof, changed-file relevance, and exact Vercel deployment identity before creating, polling, reusing, or canceling a Preview deployment. - -The implementation includes endpoint-specific bounded pagination, stable workflow-run totals and unique IDs, final freshness checks, a one-POST ambiguity state machine, null-safe Preview and project identity, mutation response validation, cancellation-race handling, redirect rejection, token redaction, and a controller-wide 23-minute monotonic deadline. - -## TDD evidence - -The implementation was developed in focused red-green slices: - -1. Event and eligibility tests were written before the controller existed. `bun test scripts/vercel-preview-deploy.test.ts --test-name-pattern 'event|eligibility|creates'` failed with the expected module-not-found error, with 0 passing tests and 1 loader failure. The completed slice then passed 19 tests. -2. Existing-deployment, idempotency, cancellation, and Vercel pagination cases were added next. The focused run failed 7 tests before the behavior was implemented, then passed. -3. Workflow-run and changed-file pagination, freshness, ambiguity recovery, polling, and deadline cases were added next. The focused run failed 14 tests before the behavior was implemented, then passed. -4. Mutation and detail identity validation plus secret-bearing URL rejection were added next. The focused run failed 4 tests before the behavior was implemented, then passed. -5. The final focused suite passes 81 tests with 182 assertions. - -## Verification - -- `bun test scripts/vercel-preview-deploy.test.ts`: 81 pass, 0 fail, 182 assertions. -- `bun x biome check scripts/vercel-preview-deploy.ts scripts/vercel-preview-deploy.test.ts`: passed. -- `bun x tsc -p scripts/tsconfig.json --noEmit`: passed. -- `bun run lint -- scripts/vercel-preview-deploy.ts scripts/vercel-preview-deploy.test.ts`: exited 0 with only pre-existing repository notices. -- `bun run check-comments`: passed. -- `bun run check-bytes`: passed. -- `bun run check-bun-imports`: passed. -- `git diff --check`: passed. -- `ORBIT_TEST_LANE=preview-gate-task2 bun run verify`: the lint, policy, dependency, typecheck, script-test, and core-test phases passed. Root script tests reported 106 pass and 0 fail. Core reported 974 pass and 0 fail. The realtime phase failed because the worktree has no `BETTER_AUTH_SECRET`. The unrelated web phase later stopped producing output and the already non-green run was interrupted after five minutes without progress. - -## Concerns - -There are no known Task 2 test, type, lint, policy, or byte-check failures. A fully green repository-wide verify requires the local realtime test secret and a non-stalling web suite. Live Vercel and GitHub API behavior remains a production canary concern for the workflow integration task. diff --git a/scripts/vercel-preview-deploy.test.ts b/scripts/vercel-preview-deploy.test.ts index dd7c5b8bc..c23f8d111 100644 --- a/scripts/vercel-preview-deploy.test.ts +++ b/scripts/vercel-preview-deploy.test.ts @@ -636,6 +636,42 @@ describe('existing deployments, cancellation, and pagination', () => { expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(2); }); + test('eligibility changing after the first cancellation prevents a second PATCH', async () => { + let canceled = false; + const harness = createHarness({ + pullRequest: pullRequest({ draft: true }), + deployments: [ + deployment('BUILDING', { uid: 'dpl_b' }), + deployment('BUILDING', { uid: 'dpl_a' }), + ], + respond: ({ method, url }) => { + if (url.endsWith('/pulls/341')) { + return json(pullRequest({ draft: !canceled })); + } + if (method === 'PATCH') { + const id = url.split('/').at(-2) ?? 'missing'; + canceled = true; + return json(mutationDeployment(id, 'CANCELED')); + } + return undefined; + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'canceled', + pullRequestNumber: 341, + reason: 'canceled-active', + deploymentId: 'dpl_a', + url: 'orbit-preview.vercel.app', + }, + ]); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(1); + expect(harness.requests.filter(({ url }) => url.endsWith('/pulls/341'))).toHaveLength(3); + }); + test('cancel 400 accepts a terminal detail race without retrying PATCH', async () => { const livePullRequest = pullRequest({ draft: true }); const harness = createHarness({ @@ -898,6 +934,22 @@ describe('bounded transport, polling, and ambiguity recovery', () => { ).toHaveLength(3); }); + test('active through the final polling GET stops after 240 sleeps and 241 GETs', async () => { + let detailReads = 0; + const harness = createHarness({ + deployments: [deployment('BUILDING', { uid: 'dpl_active' })], + respond: ({ method, url }) => { + if (method !== 'GET' || !url.includes('/v13/deployments/dpl_active')) return undefined; + detailReads += 1; + return json(mutationDeployment('dpl_active', detailReads === 242 ? 'READY' : 'BUILDING')); + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('timed out'); + expect(detailReads).toBe(241); + expect(harness.sleeps.filter((milliseconds) => milliseconds === 5000)).toHaveLength(240); + }); + test.each(['ERROR', 'CANCELED', 'BLOCKED', 'DELETED'])( 'terminal polling state %s fails visibly', async (readyState) => { @@ -1013,15 +1065,15 @@ describe('bounded transport, polling, and ambiguity recovery', () => { }); describe('identity invariants and bounded edge cases', () => { - test('created response must retain every requested Orbit metadata field', async () => { + test.each([ + ['wrong project', { projectId: 'prj_other' }], + ['non-null target', { target: 'production' }], + ['wrong metadata', { meta: metadata({ orbitGithubWorkflowRunId: 'wrong' }) }], + ])('created response rejects %s', async (_name, responseOverrides) => { const harness = createHarness({ respond: ({ method }) => method === 'POST' - ? json( - mutationDeployment('dpl_created', 'QUEUED', { - meta: metadata({ orbitGithubWorkflowRunId: 'wrong' }), - }), - ) + ? json(mutationDeployment('dpl_created', 'QUEUED', responseOverrides)) : undefined, }); @@ -1029,34 +1081,35 @@ describe('identity invariants and bounded edge cases', () => { expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); }); - test('detail response must retain metadata from the exact list item', async () => { + test.each([ + ['wrong project', 'dpl_active', { projectId: 'prj_other' }], + ['non-null target', 'dpl_active', { target: 'production' }], + ['wrong ID', 'dpl_other', {}], + ['wrong metadata', 'dpl_active', { meta: metadata({ orbitDeploymentReason: 'wrong' }) }], + ])('detail response rejects %s', async (_name, responseId, responseOverrides) => { const harness = createHarness({ deployments: [deployment('BUILDING', { uid: 'dpl_active' })], respond: ({ method, url }) => method === 'GET' && url.includes('/v13/deployments/dpl_active') - ? json( - mutationDeployment('dpl_active', 'READY', { - meta: metadata({ orbitDeploymentReason: 'wrong' }), - }), - ) + ? json(mutationDeployment(responseId, 'READY', responseOverrides)) : undefined, }); await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('identity drift'); }); - test('cancel response must retain metadata from the listed deployment', async () => { - const livePullRequest = pullRequest({ draft: true }); + test.each([ + ['wrong project', 'dpl_active', { projectId: 'prj_other' }], + ['non-null target', 'dpl_active', { target: 'production' }], + ['wrong ID', 'dpl_other', {}], + ['wrong metadata', 'dpl_active', { meta: metadata({ orbitGithubWorkflowRunId: 'wrong' }) }], + ])('cancel response rejects %s', async (_name, responseId, responseOverrides) => { const harness = createHarness({ - pullRequest: livePullRequest, + pullRequest: pullRequest({ draft: true }), deployments: [deployment('BUILDING', { uid: 'dpl_active' })], respond: ({ method }) => method === 'PATCH' - ? json( - mutationDeployment('dpl_active', 'CANCELED', { - meta: metadata({ orbitGithubWorkflowRunId: 'wrong' }), - }), - ) + ? json(mutationDeployment(responseId, 'CANCELED', responseOverrides)) : undefined, }); diff --git a/scripts/vercel-preview-deploy.ts b/scripts/vercel-preview-deploy.ts index 718a44a3a..fc3eddb72 100644 --- a/scripts/vercel-preview-deploy.ts +++ b/scripts/vercel-preview-deploy.ts @@ -680,7 +680,7 @@ async function pollDeployment( deploymentId: string, expectedMetadata: Readonly>, ): Promise { - while (true) { + for (let detailRequest = 0; detailRequest <= 240; detailRequest += 1) { const detail = await requestJson( runtime, environment, @@ -712,8 +712,10 @@ async function pollDeployment( }; if (!isActiveVercelDeployment(comparable)) return fail(`deployment ended in ${detail.readyState}`); + if (detailRequest === 240) return fail('deployment polling timed out'); await runtime.sleep(5000); } + return fail('deployment polling timed out'); } function currentStateMatches( @@ -807,15 +809,19 @@ async function cancelActiveDeployments( if (active.length === 0) { return [skipped(pullRequest.number, 'no-active-deployment')]; } - const finalPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); - if (!currentStateMatches(pullRequest, finalPullRequest)) { - return [skipped(pullRequest.number, 'stale-event')]; - } - if (finalPullRequest.state === 'open' && isPreviewEligible(finalPullRequest)) { - return [skipped(pullRequest.number, 'preview-ineligible')]; - } const results: PreviewResult[] = []; for (const item of active) { + const finalPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); + const identityIsCurrent = + currentStateMatches(pullRequest, finalPullRequest) && + repositorySlugMatches(finalPullRequest.base.repo, environment.GITHUB_REPOSITORY) && + isSameRepositoryPullRequest(finalPullRequest); + if (!identityIsCurrent) { + return results.length > 0 ? results : [skipped(pullRequest.number, 'stale-event')]; + } + if (finalPullRequest.state === 'open' && isPreviewEligible(finalPullRequest)) { + return results.length > 0 ? results : [skipped(pullRequest.number, 'preview-ineligible')]; + } const result = await cancelOneActiveDeployment(runtime, environment, finalPullRequest, item); if (result) results.push(result); } From 73a42ccdd7a7b9840aa571946eb7bcf76e90fc4b Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 16:15:55 +0530 Subject: [PATCH 12/19] docs(vercel): harden preview workflow plan --- ...26-08-21-vercel-preview-deployment-gate.md | 29 +++++----- ...1-vercel-preview-deployment-gate-design.md | 56 +++++++++++-------- 2 files changed, 47 insertions(+), 38 deletions(-) diff --git a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md index 5359b1386..268e9c165 100644 --- a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md +++ b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md @@ -197,6 +197,7 @@ Commit: `feat(ci): define preview deployment policy` - Produces `PreviewRuntime`, `PreviewResult`, and `reconcileVercelPreviews(runtime): Promise`. - `PreviewRuntime` supplies `env`, `readText`, `fetch`, `sleep`, `now`, and `log` so tests never access the network, process secrets, clocks, or real event files. - `PreviewResult` is a closed discriminated union with `kind: 'skipped' | 'created' | 'canceled'`, a pull request number, and a stable reason. `created` and each `canceled` result include deployment ID and URL; `skipped` results do not. Candidates and canceled results are sorted for deterministic output. +- One monotonic 23-minute controller deadline bounds every request, retry, pagination loop, observation, poll, and sleep beneath the workflow's 25-minute timeout. Use this closed reason vocabulary: `event-not-actionable`, `workflow-run-unassociated`, `stale-event`, `repository-mismatch`, `fork-pull-request`, `base-mismatch`, `preview-ineligible`, `no-active-deployment`, `web-unaffected`, `ci-unavailable`, `ci-not-current`, `ci-not-green`, `ready-deployment-reused`, `active-deployment-reused`, `created-ready`, and `canceled-active`. Configuration, malformed external data, incomplete pagination, transport failure, identity drift, and terminal build failure throw redacted errors rather than returning a skipped result. @@ -209,7 +210,7 @@ Use an injected fetch router that records method, URL, headers, and parsed body. - Draft without `preview`, either control label combination, stale event SHA, fork head, wrong base, failed CI, in-progress CI, and unrelated files create zero deployments. - `closed`, converted-to-draft, and `no-preview` transitions cancel matching active deployments without requiring CI; a stale state event cannot cancel a different live head. - `repository_dispatch` parses its pull request input and follows the same live-state and CI checks. -- An empty workflow-run pull request list fails closed because it cannot prove the run's base SHA and repository association. +- An empty workflow-run pull request list fails closed because it cannot prove the run's base SHA and repository association. More than one distinct linked pull request also fails closed because the workflow cannot provide per-PR serialization for that event. - Duplicate workflow-run associations resolve a pull request once, event and embedded PR numbers must agree, and event repository identity must match `GITHUB_REPOSITORY` plus the live base repository. - GitHub files paginate until a short page, with a 30-page cap and early relevant-file exit. Workflow runs paginate against `total_count` with a 10-page cap. A full final files page or an unexhausted run count fails closed. - A newer queued, failed, canceled, or stale-base run blocks an older success. Equal creation times use the larger run ID. Wrong canonical workflow path, name, ID, or state fails closed. @@ -259,7 +260,7 @@ type PreviewCandidate = { ``` - `pull_request_target`: one candidate from the matching event and embedded PR number plus event head SHA; `closed` is an accepted cancellation transition. -- `workflow_run`: no candidates unless the workflow is `CI`, source event is `pull_request`, conclusion is `success`, action is `completed`, and the payload links at least one pull request; candidates use the linked pull request number and workflow head SHA. An empty linked list fails closed. +- `workflow_run`: no candidates unless the workflow is `CI`, source event is `pull_request`, conclusion is `success`, action is `completed`, and the payload links exactly one distinct pull request; duplicate links to that same number are deduplicated. Empty or ambiguous linked lists log a stable reason and return no candidates. - `repository_dispatch`: one candidate from the validated positive integer `client_payload.pull_request` and no expected SHA. For every candidate, refetch `/repos/{owner}/{repo}/pulls/{number}` and prove event repository ID and slug, base repository ID, base ref `main`, same-repository head, and exact expected SHA when one exists. A fork or unrelated repository is a no-op. Current closed, draft, or label-ineligible state follows the cancellation path without requiring successful CI. @@ -293,13 +294,13 @@ Expected: FAIL because deployment listing, matching, and cancellation are not im List `/v7/deployments` with `teamId`, `projectId`, `branch`, `sha` where appropriate, and `limit=100`. The current endpoint has no documented metadata query. Follow validated `pagination.next` through `until`, preserve every original filter, reject repeated cursors including zero, and fail when a non-null cursor remains at the finite page cap. Filter again in trusted code with exact project, null target, and `matchesVercelPullRequest`. Parse list identifiers from `uid`; create, detail, and cancel responses use `id`. Accept `url: null` only on list items. -For eligible PRs, prefer any exact ready deployment, then any exact active deployment, then terminal history. Fetch detail when a READY list item has no URL. Poll an active deployment immediately, then allow at most 240 five-second sleeps followed by a final GET. Require ID, project, null target, and Orbit metadata on every detail. `READY` with a nonempty URL succeeds; `ERROR`, `CANCELED`, `BLOCKED`, or `DELETED` fails; an active final response times out. +For eligible PRs, prefer any exact ready deployment, then any exact active deployment, then terminal history. Fetch detail when a READY list item has no URL. Poll an active deployment immediately, then allow at most 240 five-second sleeps followed by a final GET. The 23-minute controller deadline may stop the sequence earlier. Require ID, project, null target, and Orbit metadata on every detail. `READY` with a nonempty URL succeeds; `ERROR`, `CANCELED`, `BLOCKED`, or `DELETED` fails; an active final response times out. When no ready or active exact deployment exists, create one deployment with `POST /v13/deployments?teamId={teamId}`, omitted `target`, exact Git source, and the metadata shown in Step 1. Add `forceNew=1` only when the complete pre-create list already contained an exact terminal deployment. Record all pre-create deployment IDs and enforce one POST per reconciliation. An ambiguous create outcome is a network error, timeout, 429, 5xx, 409, or successful response that cannot be parsed, validated, or matched to the requested identity. Ordinary 4xx responses are definitive. After ambiguity, set `createAttempted` and make a second POST impossible. Run three exact-list observation attempts separated by two seconds. Reuse only a newly visible ready or active ID that was not in the pre-create set. A new terminal deployment or no new exact ID fails visibly. A later event starts a new reconciliation from a complete list and may decide independently. -For current ineligible state, list with `teamId`, `projectId`, `branch`, and `limit=100` without SHA, then call `PATCH /v12/deployments/{id}/cancel?teamId={teamId}` only for locally matched active Preview deployments. Never retry PATCH. Validate a successful cancel as the requested ID, project, null target, Orbit metadata, and `CANCELED`, then emit one `canceled-active` result. After a 400 or ambiguous PATCH, read v13 detail once; accept `CANCELED`, `READY`, or another terminal state as a completed race with no active spend, and fail if the deployment remains active or its identity drifted. Race-only reconciliation emits `no-active-deployment`; mixed reconciliation emits results only for deployments actually canceled. A CI failure does not cancel a previously ready Preview; cancellation is driven by current pull request state. +For current ineligible state, list with `teamId`, `projectId`, `branch`, and `limit=100` without SHA, then call `PATCH /v12/deployments/{id}/cancel?teamId={teamId}` only for locally matched active Preview deployments. Refetch and revalidate live pull request identity and ineligibility immediately before every individual PATCH. Never retry PATCH. Validate a successful cancel as the requested ID, project, null target, Orbit metadata, and `CANCELED`, then emit one `canceled-active` result. After a 400 or ambiguous PATCH, read v13 detail once; accept `CANCELED`, `READY`, or another terminal state as a completed race with no active spend, and fail if the deployment remains active or its identity drifted. Race-only reconciliation emits `no-active-deployment`; mixed reconciliation emits results only for deployments actually canceled. A CI failure does not cancel a previously ready Preview; cancellation is driven by current pull request state. - [ ] **Step 7: Write failing transport and secret-safety tests** @@ -348,25 +349,19 @@ Commit: `feat(ci): deploy previews after successful checks` - [ ] **Step 1: Write failing repository configuration tests** -Parse `apps/web/vercel.json`, read the workflow and docs as text, and import `LABELS`. Assert: +Parse `apps/web/vercel.json`, read the workflow and docs as text, and import `LABELS`. Assert the branch map and removal checks below, then use anchored assertions over the intentionally fixed workflow structure rather than global word searches: ```ts expect(vercel.git.deploymentEnabled).toEqual({ '**': false, main: true }); expect(vercel).not.toHaveProperty('ignoreCommand'); expect(LABELS.filter(({ name }) => name === 'preview')).toHaveLength(1); expect(LABELS.filter(({ name }) => name === 'no-preview')).toHaveLength(1); -expect(workflow).toContain('pull_request_target:'); -expect(workflow).toContain('workflow_run:'); -expect(workflow).toContain('repository_dispatch:'); -expect(workflow).not.toContain('workflow_dispatch:'); -expect(workflow).toContain('persist-credentials: false'); -expect(workflow).not.toContain('github.event.pull_request.head.ref'); expect(allGateFiles).not.toContain('BUILD_GATE_GITHUB_TOKEN'); -expect(ciWorkflow).toContain('bun run check-bytes'); -expect(ciWorkflow).toContain('bun run check-bun-imports'); ``` -Also test the `**` rule against `feature`, `feature/preview`, and `codex/review/pr341` using the same `minimatch` semantics documented by Vercel. Do not add a dependency: implement the narrow expected assertion by checking that the configured key is exactly `**` and enumerate the branch examples in the test name. +Require the exact top-level read-only permissions; only the three intended triggers and their exact activity types; no `workflow_dispatch`; the workflow-run job guard for a `pull_request` source and `success` conclusion; `timeout-minutes: 25`; and `cancel-in-progress: false`. Require the exact immutable checkout and setup-Bun SHAs, checkout of `${{ github.repository }}` at `${{ github.sha }}`, disabled persisted credentials, submodules, and LFS, Bun 1.3.14, and `bun install --frozen-lockfile --ignore-scripts`. Prove no cache or artifact-download action exists, no pull request head/ref expression can select checkout code, and the controller is the only run step receiving `VERCEL_TOKEN`. + +Assert the `static` job, rather than merely the whole CI file, contains named `bun run check-bytes` and `bun run check-bun-imports` steps. Assert each managed label description matches its executable semantics: `preview` enables an otherwise eligible draft after CI, while `no-preview` suppresses creation and cancels active Preview work. Also test the `**` rule against `feature`, `feature/preview`, and `codex/review/pr341` using the same minimatch semantics documented by Vercel. Do not add a dependency: implement the narrow expected assertion by checking that the configured key is exactly `**` and enumerate the branch examples in the test name. - [ ] **Step 2: Run the configuration test and confirm failure** @@ -416,11 +411,13 @@ concurrency: cancel-in-progress: false ``` -The job condition allows state events and repository dispatch, and allows a workflow run only when `github.event.workflow_run.event == 'pull_request'` and its conclusion is success. Set `timeout-minutes: 25`. GitHub documents that `repository_dispatch` uses the last commit on the default branch, unlike `workflow_dispatch`, which can run a workflow version from a selected non-default ref. Checkout the trusted default-branch workflow commit using the pinned checkout action, `repository: ${{ github.repository }}`, `ref: ${{ github.sha }}`, `persist-credentials: false`, `submodules: false`, and `lfs: false`. Set up Bun 1.3.14 with the pinned setup action, run `bun install --frozen-lockfile --ignore-scripts`, then run `bun scripts/vercel-preview-deploy.ts` with tokens and settings scoped to that step through `env`. +The controller rejects a workflow run linked to more than one distinct pull request, so the first linked number is used only for an event that resolves to one PR. Recovery requires a positive numeric `client_payload.pull_request`, which shares the same per-PR group as state and CI events; malformed recovery input may form an unused group but is rejected before any Vercel call. The job condition allows state events and repository dispatch, and allows a workflow run only when `github.event.workflow_run.event == 'pull_request'` and its conclusion is success. Set `timeout-minutes: 25`; the controller's own 23-minute deadline remains the primary bound. GitHub documents that `repository_dispatch` uses the last commit on the default branch, unlike `workflow_dispatch`, which can run a workflow version from a selected non-default ref. Checkout the trusted default-branch workflow commit using the pinned checkout action, `repository: ${{ github.repository }}`, `ref: ${{ github.sha }}`, `persist-credentials: false`, `submodules: false`, and `lfs: false`. Set up Bun 1.3.14 with the pinned setup action, run `bun install --frozen-lockfile --ignore-scripts`, then run `bun scripts/vercel-preview-deploy.ts` with tokens and settings scoped only to that controller step through `env`. - [ ] **Step 5: Rewrite the operations guide and documentation index** -Document the exact eligibility table, CI-green timing, same-repository restriction, active cancellation including closed pull requests, web path list, Vercel API behavior, GitHub secret and variables, label synchronization, Git Fork Protection, the deployment-count caveat, the repository-controlled cost-policy limitation, removal of all four old `BUILD_GATE_*` Vercel values, and the post-merge canary. Manual recovery uses a maintainer-authenticated `repository_dispatch` named `vercel-preview-reconcile` with numeric `client_payload.pull_request`; explicitly forbid `workflow_dispatch` because a caller can select a non-default ref. Link the guide from `docs/README.md` under the contributor/operations entries. +Document the exact eligibility table, CI-green timing, same-repository restriction, active cancellation including closed pull requests, web path list, Vercel API behavior, GitHub secret and variables, label synchronization, Git Fork Protection, the deployment-count caveat, the repository-controlled cost-policy limitation, and removal of all four old `BUILD_GATE_*` Vercel values. State precisely that only the trusted GitHub controller is isolated from pull request code: the API-created Vercel Preview still builds same-repository pull request code with the project's Preview environment scope. Manual recovery uses a maintainer-authenticated `repository_dispatch` named `vercel-preview-reconcile` with numeric `client_payload.pull_request`; explicitly forbid `workflow_dispatch` because a caller can select a non-default ref. Link the guide from `docs/README.md` under the contributor/operations entries. + +Make the post-merge canary an ordered procedure: confirm Git Fork Protection and configure the secret plus three variables; remove the four legacy values only after the workflow is on `main`; open a same-repository web-impacting draft and prove no Preview until `preview` is applied and exact-head CI succeeds; remove `preview` or add `no-preview` and prove active work is canceled while a ready URL remains; make the PR ready, push a new relevant head, and prove only that exact SHA deploys; then repeat with a docs-only change and a fork and prove neither gets an automatic Preview. Do not claim that ignored builds are free, that Ready alone is trust, that fork previews are automatic, or that the privileged workflow can be exercised before it exists on `main`. diff --git a/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md index fa779610a..009b809cc 100644 --- a/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md +++ b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md @@ -62,9 +62,12 @@ forks. A future move to a disconnected or dedicated Preview project can make aut suppression independent of pull request contents; that external migration is not required to remove the token exposure or to stop normal feature-branch builds in this change. -The repository removes `scripts/vercel-build-gate.sh` and its tests. No secret is made available to -code from a pull request checkout, and the gate no longer depends on Vercel system environment -variables or an ignored-build exit-code convention. +The repository removes `scripts/vercel-build-gate.sh` and its tests. The trusted GitHub controller +never exposes `VERCEL_TOKEN` to a pull request checkout, and the gate no longer depends on Vercel +system environment variables or an ignored-build exit-code convention. The Preview deployment +still builds same-repository pull request code inside Vercel and can receive that project's Preview +environment values. Same-repository authors therefore remain inside the Vercel project trust +boundary. ## Trusted workflow @@ -87,9 +90,11 @@ values and API request bodies, never as shell source. Permissions are limited to `actions: read`, `contents: read`, and `pull-requests: read`. The Vercel token is a GitHub Actions secret. Team ID, project ID, and project name are Actions variables. Concurrency is keyed by pull request number when the event supplies one and does not cancel an -in-progress reconciler. Vercel metadata checks provide a second idempotency boundary. A -workflow-run payload without a linked pull request is rejected because it cannot prove the run's -base SHA and repository association. +in-progress reconciler. Recovery dispatches require a bounded positive numeric pull request input +and share that pull request's group. Vercel metadata checks provide a second idempotency boundary. +A workflow-run payload with no linked pull request, or with more than one distinct linked pull +request, is rejected because one Actions concurrency group cannot serialize multiple pull requests +and the event cannot prove one unambiguous base association. ## Controller and validation @@ -97,11 +102,11 @@ base SHA and repository association. `@orbit/shared/validators` and validates the GitHub event payload, every GitHub response, every Vercel response, and configuration before using them. -The controller resolves one or more pull request numbers from the triggering event, then refetches -each pull request from GitHub. An event is stale when its recorded head SHA no longer equals the -live pull request head. Stale events are no-ops. Fork heads and pull requests targeting another -repository or branch are also no-ops. A proven closed pull request follows the active-cancellation -path without requiring CI. +The controller resolves one pull request number from an actionable event, then refetches that pull +request from GitHub. Duplicate links to the same pull request are deduplicated. An event is stale +when its recorded head SHA no longer equals the live pull request head. Stale events are no-ops. +Fork heads and pull requests targeting another repository or branch are also no-ops. A proven +closed pull request follows the active-cancellation path without requiring CI. For every eligible event, the controller resolves the canonical `.github/workflows/ci.yml`, fetches the live `main` ref, and queries every bounded page of runs for the exact head SHA. It selects the @@ -118,11 +123,13 @@ affected when a filename is below `apps/web/` or `packages/`, or is exactly `pac `bun.lock`, or `tsconfig.base.json`. Configuration is fixed in trusted code rather than split between Vercel and GitHub settings. -GitHub and Vercel requests have a finite timeout. Safe reads use bounded retries for network -failures, server failures, 429 responses, and 403 responses carrying explicit GitHub rate-limit -evidence. Mutations are never retried blindly. Authentication failures, malformed payloads, -exhausted pagination, and unsuccessful mutations fail the workflow visibly. Redirects are rejected -for token-bearing requests, and logs never contain either token. +GitHub and Vercel requests have a finite timeout. One monotonic 23-minute reconciliation deadline +also bounds every request, retry, pagination loop, observation, poll, and sleep beneath the +workflow's 25-minute job limit. Safe reads use bounded retries for network failures, server +failures, 429 responses, and 403 responses carrying explicit GitHub rate-limit evidence. Mutations +are never retried blindly. Authentication failures, malformed payloads, exhausted pagination, and +unsuccessful mutations fail the workflow visibly. Redirects are rejected for token-bearing +requests, and logs never contain either token. ## Vercel API contract @@ -140,8 +147,9 @@ repeats the repository, pull request, branch, SHA, workflow run ID, and reason s identify the deployment without guessing. `forceNew=1` is used only when a new reconciliation has already observed an exact terminal failed deployment. Create, detail, and cancel responses must prove deployment ID, project ID, null Preview target, state, and Orbit metadata. The workflow polls -the created deployment immediately and then through at most 240 five-second sleeps to a ready or -terminal state, requiring a nonempty final URL. +the created deployment immediately and then through at most 240 five-second sleeps plus one final +GET to a ready or terminal state, requiring a nonempty final URL. The 23-minute controller deadline +may stop this sequence earlier. Create Deployment has no idempotency key. After a network error, timeout, 429, 5xx, 409, or an unparseable success response, the controller marks the one POST as attempted and performs only a @@ -152,9 +160,10 @@ sends a second create request. When current pull request state is ineligible, active deployments associated with that pull request are canceled through Vercel's cancel endpoint. The filter requires the configured project, null Preview target, repository ID, pull request number, and branch ref before cancellation. Ready, -failed, and canceled deployments are left unchanged. A 400 or ambiguous cancel response is followed -by one validated detail read so a normal transition to a terminal state is not mistaken for an -unsafe retry. +failed, and canceled deployments are left unchanged. Live pull request identity and ineligibility +are refetched immediately before every individual cancellation mutation. A 400 or ambiguous cancel +response is followed by one validated detail read so a normal transition to a terminal state is not +mistaken for an unsafe retry. ## Fork policy @@ -192,7 +201,10 @@ removed from Vercel. Git Fork Protection stays enabled. The workflow is defined by the default branch, so pull request 341 can prove its controller and workflow structure locally but cannot exercise the new privileged event path until the workflow has landed on `main`. The first same-repository test pull request after merge is the production -canary for Vercel Git metadata, the Preview URL, and label transitions. +canary. It starts as a web-impacting draft with no Preview, applies `preview` and waits for exact-head +CI plus one Preview, then removes the label or applies `no-preview` to prove active cancellation +without deleting a ready URL. A new ready head proves exact-SHA behavior. Separate docs-only and +fork cases prove that neither receives an automatic Preview. ## Tests From d4fe89e6fb3793b926f9b92fffad266e50ba9091 Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 16:25:19 +0530 Subject: [PATCH 13/19] chore(vercel): gate previews after CI --- .github/workflows/ci.yml | 4 + .github/workflows/vercel-preview.yml | 54 ++++++ apps/web/vercel.json | 7 +- docs/README.md | 1 + docs/VERCEL_BUILD_GATE.md | 236 +++++++++++++++---------- scripts/labels.ts | 10 ++ scripts/vercel-build-gate.sh | 100 ----------- scripts/vercel-build-gate.test.ts | 240 -------------------------- scripts/vercel-preview-config.test.ts | 171 ++++++++++++++++++ 9 files changed, 394 insertions(+), 429 deletions(-) create mode 100644 .github/workflows/vercel-preview.yml delete mode 100755 scripts/vercel-build-gate.sh delete mode 100644 scripts/vercel-build-gate.test.ts create mode 100644 scripts/vercel-preview-config.test.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index daaafbab6..ff87f76a1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -27,6 +27,10 @@ jobs: run: bun run lint - name: No-comment policy run: bun run check-comments + - name: Source byte policy + run: bun run check-bytes + - name: No Bun built-ins in shipped server code + run: bun run check-bun-imports - name: One copy of every overridden dependency run: bun run check-deps - name: Typecheck diff --git a/.github/workflows/vercel-preview.yml b/.github/workflows/vercel-preview.yml new file mode 100644 index 000000000..a88518f87 --- /dev/null +++ b/.github/workflows/vercel-preview.yml @@ -0,0 +1,54 @@ +name: Vercel Preview + +on: + pull_request_target: + branches: [main] + types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled, closed] + workflow_run: + workflows: [CI] + types: [completed] + repository_dispatch: + types: [vercel-preview-reconcile] + +permissions: + actions: read + contents: read + pull-requests: read + +concurrency: + group: vercel-preview-${{ github.event.pull_request.number || github.event.workflow_run.pull_requests[0].number || github.event.client_payload.pull_request || github.event.workflow_run.head_sha || github.run_id }} + cancel-in-progress: false + +jobs: + reconcile: + if: >- + github.event_name == 'pull_request_target' || + github.event_name == 'repository_dispatch' || + (github.event_name == 'workflow_run' && + github.event.workflow_run.event == 'pull_request' && + github.event.workflow_run.conclusion == 'success') + runs-on: ubuntu-latest + timeout-minutes: 25 + steps: + - name: Check out trusted controller + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 + with: + repository: ${{ github.repository }} + ref: ${{ github.sha }} + persist-credentials: false + submodules: false + lfs: false + - name: Set up Bun + uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 + with: + bun-version: "1.3.14" + - name: Install trusted controller dependencies + run: bun install --frozen-lockfile --ignore-scripts + - name: Reconcile Vercel Preview + run: bun scripts/vercel-preview-deploy.ts + env: + GITHUB_TOKEN: ${{ github.token }} + VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }} + VERCEL_TEAM_ID: ${{ vars.VERCEL_TEAM_ID }} + VERCEL_PROJECT_ID: ${{ vars.VERCEL_PROJECT_ID }} + VERCEL_PROJECT_NAME: ${{ vars.VERCEL_PROJECT_NAME }} diff --git a/apps/web/vercel.json b/apps/web/vercel.json index 2a44f7e97..cf710026a 100644 --- a/apps/web/vercel.json +++ b/apps/web/vercel.json @@ -1,7 +1,12 @@ { "$schema": "https://openapi.vercel.sh/vercel.json", "installCommand": "rm -rf ../../node_modules node_modules && bun install --frozen-lockfile", - "ignoreCommand": "BUILD_GATE_WATCH_PATHS=\"${BUILD_GATE_WATCH_PATHS:-apps/web packages package.json bun.lock tsconfig.base.json}\" bash ../../scripts/vercel-build-gate.sh", + "git": { + "deploymentEnabled": { + "**": false, + "main": true + } + }, "regions": ["hnd1"], "crons": [ { "path": "/api/cron/analytics-snapshots", "schedule": "0 */6 * * *" }, diff --git a/docs/README.md b/docs/README.md index ac5ce7430..cdcb09aae 100644 --- a/docs/README.md +++ b/docs/README.md @@ -21,6 +21,7 @@ gated behind a plan. | Write or run the tests | [Testing](testing.md) | | Fix something that broke | [Troubleshooting](troubleshooting.md) | | See what is coming | [Roadmap](roadmap.md) | +| Operate gated Vercel Preview deployments | [Vercel Preview deployment gate](VERCEL_BUILD_GATE.md) | | Contribute | [CONTRIBUTING.md](../CONTRIBUTING.md) | ## The five minute version diff --git a/docs/VERCEL_BUILD_GATE.md b/docs/VERCEL_BUILD_GATE.md index 189f2ddd4..5a5d5f6f2 100644 --- a/docs/VERCEL_BUILD_GATE.md +++ b/docs/VERCEL_BUILD_GATE.md @@ -1,94 +1,154 @@ -# Vercel build gate - -Preview deployments only build once a pull request is marked **Ready for review**. -Production always builds. Two labels override the rule: `preview` builds a draft, -and `no-preview` suppresses a pull request that is ready. - -## Why the gate exists - -Orbit ran **700 deployments in the 22 days** after the project was created on -2026-07-28, peaking at 131 in a single day, and 77% of them were previews. -Across the team, 81% of 4,393 deployments in the 90 days to 2026-08-19 were -previews, and builds were $110 of the $506.93 August invoice. - -## How it works - -`apps/web/vercel.json` points Vercel's Ignored Build Step at -`scripts/vercel-build-gate.sh`. - -**Exit codes are inverted from intuition: `exit 0` skips the build, `exit 1` runs it.** - -Decision order: - -| Condition | Result | -|---|---| -| `VERCEL_ENV=production` | build | -| system environment variables not exposed | build | -| branch has no open PR | skip | -| PR labelled `no-preview` | skip | -| PR labelled `preview` | build (even while draft) | -| PR is a draft | skip | -| PR is ready for review | build, subject to the path filter | -| nothing changed under `BUILD_GATE_WATCH_PATHS` | skip | - -Every failure path - missing token, GitHub API error, unparseable response, -unreachable diff base - **builds**. The gate never silently withholds a -deployment because something broke. - -The metadata check has to come before the pull request check, and the order is -load-bearing. `VERCEL_GIT_PULL_REQUEST_ID` is empty both when a branch genuinely -has no pull request *and* when system environment variables are not exposed at -all. Testing the PR id first would read the second case as the first and skip -every preview in the project, silently, which is the one behaviour this gate -must never have. `VERCEL_GIT_REPO_OWNER` and `VERCEL_GIT_REPO_SLUG` are set -whenever the variables are exposed, regardless of pull request state, so they -are what distinguishes the two. - -## The button - -Open the PR as a **draft** while you work. Commits accumulate with zero builds. -When you want a preview, click **Ready for review** - that is the button. Adding -the `preview` label also works if you want previews while staying in draft. - -## Setup per project - -1. Project Settings → Environment Variables → tick **Enable access to System - Environment Variables**. The gate needs `VERCEL_GIT_PULL_REQUEST_ID`, - `VERCEL_GIT_REPO_OWNER`, `VERCEL_GIT_REPO_SLUG` and `VERCEL_GIT_PREVIOUS_SHA`. - Note that `VERCEL_GIT_PREVIOUS_SHA` is *only* exposed when an Ignored Build - Step is configured. -2. Add `BUILD_GATE_GITHUB_TOKEN` - a fine-grained token with **Pull requests: - read** on the repo. Without it the gate fails open and every push builds. -3. Optionally add `BUILD_GATE_WATCH_PATHS` (space separated, repo-relative) to - skip builds when nothing under those paths changed. - -## Monorepo path filtering - -This repo holds two apps. Only `apps/web` is deployed to Vercel, so a push that -only touches `apps/realtime` has nothing to preview. `apps/web/vercel.json` -therefore supplies a default: - -```sh -BUILD_GATE_WATCH_PATHS="apps/web packages package.json bun.lock tsconfig.base.json" +# Vercel Preview deployment gate + +Orbit creates Vercel Preview deployments only after the exact pull request head +has passed CI and still satisfies the repository policy. Production deployments +from `main` remain enabled through the Vercel Git integration. + +## Eligibility + +The controller evaluates the current pull request from GitHub on every event. +`no-preview` takes precedence over every other state. + +| Pull request state | Labels | Result after exact-head CI succeeds | +| --- | --- | --- | +| Ready for review | neither managed label | eligible | +| Ready for review | `preview` | eligible | +| Ready for review | `no-preview`, with or without `preview` | ineligible | +| Draft | `preview` without `no-preview` | eligible | +| Draft | neither managed label | ineligible | +| Draft | `no-preview`, with or without `preview` | ineligible | +| Closed | any labels | ineligible | +| Fork | any state or labels | never eligible for an automatic Preview | + +An eligible pull request must also target `main`, come from the same repository, +and change at least one web-impacting path: + +- `apps/web/**` +- `packages/**` +- `package.json` +- `bun.lock` +- `tsconfig.base.json` + +Ready status or the `preview` label does not establish trust. The controller +also proves that the newest `CI` run belongs to the current head SHA, is +associated with the same pull request and current `main`, and completed +successfully. A state event can create a Preview immediately when that proof +already exists. Otherwise the successful `workflow_run` event reconciles the +pull request after CI finishes. A later non-green run blocks an older success. + +## Trust boundary + +`Vercel Preview` is a privileged default-branch workflow. It checks out +`${{ github.sha }}`, which is the trusted base or default-branch commit for its +three event types. It never selects, fetches, installs, caches, downloads an +artifact from, builds, or executes pull request code. Dependency lifecycle +scripts are disabled. The only operational command is +`bun scripts/vercel-preview-deploy.ts`, and `VERCEL_TOKEN` exists only on that +step. + +Only the trusted GitHub controller is isolated from pull request code. The +API-created Vercel Preview still builds same-repository pull request code with +the project Preview environment scope. Git Fork Protection must remain enabled, +and forks are rejected by the controller, but maintainers must still treat the +Preview environment as available to same-repository pull request code. + +The `git.deploymentEnabled` map in `apps/web/vercel.json` disables automatic Git +deployments for `**` and enables them for `main`. This is a repository-controlled +cost policy, not a security boundary. A repository change can alter that policy, +so security depends on the trusted workflow and controller validation. + +Each API create remains a Vercel deployment. Canceled attempts and reused +deployments can remain visible in Vercel deployment history and counts. The gate +reduces unnecessary creation, but it does not promise that an ignored or +canceled attempt is free. + +## Reconciliation and Vercel API behavior + +The controller uses one deployment path: + +1. Vercel v7 lists Preview deployments by team, project, branch, and, when + creating, exact head SHA. +2. Vercel v13 creates or reads a deployment with the same-repository GitHub + repository ID, head ref, exact head SHA, and Orbit metadata. It omits a + target so Vercel uses the project's Preview environment. +3. Vercel v12 cancels matching active deployments. + +`QUEUED`, `INITIALIZING`, and `BUILDING` deployments are active. Making a pull +request ineligible by closing it, converting it to draft without `preview`, +removing `preview` from an otherwise ineligible draft, or adding `no-preview` +cancels matching active Preview work. A deployment that is already `READY` is +not canceled, so its ready URL remains available. + +Events for stale heads cannot create or cancel work for the current head. An +existing exact ready or active deployment is reused. Terminal deployment +history can cause one forced create for the exact head, using the same v13 +endpoint rather than an alternate build path. + +## Repository and Vercel setup + +The GitHub repository must provide: + +- Secret `VERCEL_TOKEN` +- Variable `VERCEL_TEAM_ID` +- Variable `VERCEL_PROJECT_ID` +- Variable `VERCEL_PROJECT_NAME` + +Keep Vercel Git Fork Protection enabled. Synchronize the managed `preview` and +`no-preview` labels with the rest of the repository labels: + +```bash +bun run labels:sync +bun run labels:sync --apply ``` -Setting the variable in project settings overrides that default. +The first command is a dry run. Review its plan before applying it. -Watch paths are **repo-relative**, and the script resolves them against -`git rev-parse --show-toplevel` rather than the working directory. This matters: -Vercel runs the Ignored Build Step from the project's **Root Directory**, so for -a project rooted at `apps/web` a plain `git diff -- apps/web` looks for -`apps/web/apps/web`, finds nothing, and skips every build. Test any change to -this script from a subdirectory, not just from the repo root. +After `.github/workflows/vercel-preview.yml` is present on `main`, remove these +legacy Vercel environment values: -The diff base is `VERCEL_GIT_PREVIOUS_SHA`, the last **successfully deployed** -commit - not `HEAD^`. `HEAD^` is wrong whenever more than one commit lands at -once, which is the normal case for a squash merge or a batch of pushes. If that -SHA is missing from Vercel's shallow clone the gate builds rather than guessing. +- `BUILD_GATE_GITHUB_TOKEN` +- `BUILD_GATE_WATCH_PATHS` +- `BUILD_GATE_READY_LABEL` +- `BUILD_GATE_BLOCK_LABEL` -## Testing changes to the gate +They belonged to the removed Ignored Build Step and are not read by the trusted +controller. -The script shells out to `curl` and `git`, so it is testable by putting a stub -`curl` earlier on `PATH`. See the harness used when this landed - it covers -production, draft, ready, both labels, a missing token, API failure, malformed -JSON, and the path filter against real git history. +## Manual recovery + +A maintainer can reconcile a positive numeric pull request number with an +authenticated repository dispatch: + +```bash +gh api repos/Noveum/orbit/dispatches \ + --method POST \ + -f event_type=vercel-preview-reconcile \ + -F 'client_payload[pull_request]=341' +``` + +The event type must be `vercel-preview-reconcile`, and +`client_payload.pull_request` must be a positive number. The event shares the +same per-pull-request concurrency group as state and CI events. Malformed input +can form an unused group but is rejected before any Vercel call. + +Do not add or use `workflow_dispatch` for recovery. A caller can select a +non-default ref for that trigger. GitHub runs `repository_dispatch` from the +last commit on the default branch, preserving the controller trust boundary. + +## Post-merge canary + +Run this procedure only after the workflow exists on `main`: + +1. Confirm Vercel Git Fork Protection is enabled. Configure `VERCEL_TOKEN` and + the `VERCEL_TEAM_ID`, `VERCEL_PROJECT_ID`, and `VERCEL_PROJECT_NAME` + repository variables. +2. Remove the four legacy values only after the workflow is on `main`. +3. Open a same-repository, web-impacting draft. Confirm that it gets no Preview, + apply `preview`, let CI succeed for that exact head, and confirm a Preview is + created only then. +4. Remove `preview` while it is still required, or add `no-preview`. Confirm + matching active work is canceled and an already ready URL remains. +5. Make the pull request ready, push a new relevant head, and confirm only that + exact SHA deploys after its CI succeeds. +6. Repeat with a docs-only change and with a fork. Confirm neither receives an + automatic Preview. diff --git a/scripts/labels.ts b/scripts/labels.ts index 4f66abe80..2e4ae4d12 100644 --- a/scripts/labels.ts +++ b/scripts/labels.ts @@ -44,6 +44,16 @@ export const LABELS: readonly LabelDefinition[] = [ { name: 'tests', color: 'c2e0c6', description: 'Test coverage and test infrastructure' }, { name: 'question', color: 'd876e3', description: 'Needs more information to act on' }, + { + name: 'preview', + color: '0e8a16', + description: 'Enables a Preview for an otherwise eligible draft after CI succeeds', + }, + { + name: 'no-preview', + color: BLOCKED_COLOR, + description: 'Suppresses Preview creation and cancels active Preview work', + }, { name: 'needs triage', color: STATUS_COLOR, diff --git a/scripts/vercel-build-gate.sh b/scripts/vercel-build-gate.sh deleted file mode 100755 index 8cbe677dc..000000000 --- a/scripts/vercel-build-gate.sh +++ /dev/null @@ -1,100 +0,0 @@ -#!/usr/bin/env bash -set -uo pipefail - -WATCH_PATHS="${BUILD_GATE_WATCH_PATHS:-.}" -READY_LABEL="${BUILD_GATE_READY_LABEL:-preview}" -BLOCK_LABEL="${BUILD_GATE_BLOCK_LABEL:-no-preview}" - -announce() { echo "[build-gate] $*" >&2; } -build() { announce "BUILD - $1"; exit 1; } -skip() { announce "SKIP - $1"; exit 0; } - -if [ "${VERCEL_ENV:-}" = "production" ]; then - build "production deployment" -fi - -REPO_OWNER="${VERCEL_GIT_REPO_OWNER:-}" -REPO_SLUG="${VERCEL_GIT_REPO_SLUG:-}" -if [ -z "$REPO_OWNER" ] || [ -z "$REPO_SLUG" ]; then - build "system environment variables are not exposed to this build so pull request state is unreadable, failing open" -fi - -PR_ID="${VERCEL_GIT_PULL_REQUEST_ID:-}" -if [ -z "$PR_ID" ]; then - skip "branch has no open pull request, nothing to preview yet" -fi - -if [ -z "${BUILD_GATE_GITHUB_TOKEN:-}" ]; then - build "BUILD_GATE_GITHUB_TOKEN is unset so pull request state cannot be read, failing open" -fi - -if ! PR_JSON="$(curl -sS --max-time 15 \ - -H "Authorization: Bearer ${BUILD_GATE_GITHUB_TOKEN}" \ - -H "Accept: application/vnd.github+json" \ - -H "X-GitHub-Api-Version: 2022-11-28" \ - "https://api.github.com/repos/${REPO_OWNER}/${REPO_SLUG}/pulls/${PR_ID}")"; then - build "GitHub API unreachable, failing open" -fi - -if [ -z "$PR_JSON" ]; then - build "GitHub API returned an empty response, failing open" -fi - -VERDICT="$( - PR_JSON="$PR_JSON" READY_LABEL="$READY_LABEL" BLOCK_LABEL="$BLOCK_LABEL" node -e ' - try { - const pr = JSON.parse(process.env.PR_JSON); - const wellFormed = - pr && - typeof pr.draft === "boolean" && - Number.isInteger(pr.number) && - pr.number > 0 && - Array.isArray(pr.labels) && - pr.labels.every((label) => label && typeof label.name === "string"); - if (!wellFormed) { - console.log("unknown:pull request payload was not a complete pull request"); - process.exit(0); - } - const labels = pr.labels.map((label) => label.name.toLowerCase()); - if (labels.includes(process.env.BLOCK_LABEL.toLowerCase())) { - console.log(`skip:pull request ${pr.number} carries the ${process.env.BLOCK_LABEL} label`); - } else if (labels.includes(process.env.READY_LABEL.toLowerCase())) { - console.log(`build:pull request ${pr.number} carries the ${process.env.READY_LABEL} label`); - } else if (pr.draft) { - console.log(`skip:pull request ${pr.number} is still a draft, mark it ready for review to start previews`); - } else { - console.log(`build:pull request ${pr.number} is ready for review`); - } - } catch (error) { - console.log(`unknown:${error.message}`); - } - ' -)" - -REASON="$(printf '%s' "$VERDICT" | cut -d: -f2-)" -case "$(printf '%s' "$VERDICT" | cut -d: -f1)" in -skip) skip "$REASON" ;; -build) announce "gate passed, $REASON" ;; -*) build "pull request state could not be evaluated, failing open" ;; -esac - -BASE_SHA="${VERCEL_GIT_PREVIOUS_SHA:-}" -if [ -z "$BASE_SHA" ]; then - build "no diff base supplied, cannot tell which paths changed" -fi - -REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null)" -if [ -z "$REPO_ROOT" ]; then - build "not inside a git work tree, cannot tell which paths changed" -fi - -if ! git -C "$REPO_ROOT" cat-file -e "${BASE_SHA}^{commit}" 2>/dev/null; then - build "diff base ${BASE_SHA} is not in this clone, cannot tell which paths changed" -fi - -read -r -a WATCH_ARRAY <<<"$WATCH_PATHS" -if git -C "$REPO_ROOT" diff --quiet "$BASE_SHA" HEAD -- "${WATCH_ARRAY[@]}"; then - skip "no changes under '${WATCH_PATHS}' since ${BASE_SHA}" -fi - -build "changes under '${WATCH_PATHS}' since ${BASE_SHA}" diff --git a/scripts/vercel-build-gate.test.ts b/scripts/vercel-build-gate.test.ts deleted file mode 100644 index ec3d96e67..000000000 --- a/scripts/vercel-build-gate.test.ts +++ /dev/null @@ -1,240 +0,0 @@ -import { afterAll, beforeAll, describe, expect, test } from 'bun:test'; -import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; - -const GATE = join(import.meta.dir, 'vercel-build-gate.sh'); - -const SKIP = 0; -const BUILD = 1; - -const READY = JSON.stringify({ number: 7, draft: false, labels: [] }); -const DRAFT = JSON.stringify({ number: 7, draft: true, labels: [] }); -const DRAFT_LABELLED = JSON.stringify({ - number: 7, - draft: true, - labels: [{ name: 'preview' }], -}); -const READY_BLOCKED = JSON.stringify({ - number: 7, - draft: false, - labels: [{ name: 'no-preview' }], -}); - -let sandbox: string; -let stubBin: string; -let repo: string; -let firstCommit: string; - -function git(cwd: string, ...args: string[]): string { - const result = Bun.spawnSync(['git', ...args], { cwd }); - if (result.exitCode !== 0) { - throw new Error(`git ${args.join(' ')}: ${result.stderr.toString()}`); - } - return result.stdout.toString().trim(); -} - -function runGate(env: Record, cwd: string = repo): { code: number; log: string } { - const result = Bun.spawnSync(['bash', GATE], { - cwd, - env: { - PATH: `${stubBin}:${process.env['PATH'] ?? ''}`, - HOME: process.env['HOME'] ?? '', - ...env, - }, - }); - return { - code: result.exitCode ?? -1, - log: result.stderr.toString(), - }; -} - -const withPullRequest = (json: string, extra: Record = {}) => ({ - VERCEL_ENV: 'preview', - VERCEL_GIT_PULL_REQUEST_ID: '7', - VERCEL_GIT_REPO_OWNER: 'Noveum', - VERCEL_GIT_REPO_SLUG: 'orbit', - BUILD_GATE_GITHUB_TOKEN: 'token', - MOCK_PR_JSON: json, - ...extra, -}); - -beforeAll(() => { - sandbox = mkdtempSync(join(tmpdir(), 'build-gate-')); - - stubBin = join(sandbox, 'bin'); - mkdirSync(stubBin); - const stub = join(stubBin, 'curl'); - writeFileSync( - stub, - [ - '#!/usr/bin/env bash', - 'if [ -n "$MOCK_CURL_FAIL" ]; then exit 7; fi', - 'printf "%s" "$MOCK_PR_JSON"', - '', - ].join('\n'), - ); - chmodSync(stub, 0o755); - - repo = join(sandbox, 'repo'); - mkdirSync(join(repo, 'apps', 'web'), { recursive: true }); - mkdirSync(join(repo, 'apps', 'realtime'), { recursive: true }); - mkdirSync(join(repo, 'packages'), { recursive: true }); - writeFileSync(join(repo, 'apps', 'web', 'page.tsx'), 'web\n'); - writeFileSync(join(repo, 'apps', 'realtime', 'server.ts'), 'realtime\n'); - writeFileSync(join(repo, 'tsconfig.base.json'), '{}\n'); - - git(repo, 'init', '-q', '.'); - git(repo, 'config', 'user.email', 'gate@test.invalid'); - git(repo, 'config', 'user.name', 'gate'); - git(repo, 'add', '-A'); - git(repo, 'commit', '-qm', 'base'); - firstCommit = git(repo, 'rev-parse', 'HEAD'); - - writeFileSync(join(repo, 'apps', 'web', 'page.tsx'), 'web changed\n'); - git(repo, 'commit', '-qam', 'touch apps/web'); -}); - -afterAll(() => { - rmSync(sandbox, { recursive: true, force: true }); -}); - -describe('vercel build gate', () => { - test('production always builds', () => { - expect(runGate({ VERCEL_ENV: 'production' }).code).toBe(BUILD); - }); - - test('absent system environment variables fail open', () => { - const { code, log } = runGate({ VERCEL_ENV: 'preview' }); - expect(code).toBe(BUILD); - expect(log).toContain('system environment variables'); - }); - - test('a branch with no pull request skips', () => { - expect( - runGate({ - VERCEL_ENV: 'preview', - VERCEL_GIT_REPO_OWNER: 'Noveum', - VERCEL_GIT_REPO_SLUG: 'orbit', - VERCEL_GIT_PULL_REQUEST_ID: '', - }).code, - ).toBe(SKIP); - }); - - test('a missing token fails open', () => { - expect( - runGate({ - VERCEL_ENV: 'preview', - VERCEL_GIT_REPO_OWNER: 'Noveum', - VERCEL_GIT_REPO_SLUG: 'orbit', - VERCEL_GIT_PULL_REQUEST_ID: '7', - }).code, - ).toBe(BUILD); - }); - - test('a draft pull request skips', () => { - expect(runGate(withPullRequest(DRAFT)).code).toBe(SKIP); - }); - - test('a ready pull request builds', () => { - expect(runGate(withPullRequest(READY, { BUILD_GATE_WATCH_PATHS: '.' })).code).toBe(BUILD); - }); - - test('the preview label builds a draft', () => { - expect(runGate(withPullRequest(DRAFT_LABELLED, { BUILD_GATE_WATCH_PATHS: '.' })).code).toBe( - BUILD, - ); - }); - - test('the no-preview label skips a ready pull request', () => { - expect(runGate(withPullRequest(READY_BLOCKED)).code).toBe(SKIP); - }); - - test('an incomplete pull request payload fails open rather than skipping', () => { - const partials = [ - JSON.stringify({ draft: true }), - JSON.stringify({ draft: true, number: 7 }), - JSON.stringify({ draft: true, number: 0, labels: [] }), - JSON.stringify({ draft: true, number: 7, labels: [{}] }), - JSON.stringify({ draft: true, number: 7, labels: 'preview' }), - JSON.stringify({ message: 'Not Found' }), - ]; - - for (const payload of partials) { - expect(runGate(withPullRequest(payload)).code).toBe(BUILD); - } - }); - - test('an unreadable API response fails open', () => { - expect(runGate(withPullRequest('not json')).code).toBe(BUILD); - expect(runGate(withPullRequest('')).code).toBe(BUILD); - expect(runGate(withPullRequest(READY, { MOCK_CURL_FAIL: '1' })).code).toBe(BUILD); - }); - - describe('path filter', () => { - const base = () => ({ VERCEL_GIT_PREVIOUS_SHA: firstCommit }); - - test('builds when a watched path changed', () => { - expect( - runGate( - withPullRequest(READY, { - ...base(), - BUILD_GATE_WATCH_PATHS: 'apps/web', - }), - ).code, - ).toBe(BUILD); - }); - - test('skips when nothing under the watched paths changed', () => { - expect( - runGate( - withPullRequest(READY, { - ...base(), - BUILD_GATE_WATCH_PATHS: 'packages tsconfig.base.json', - }), - ).code, - ).toBe(SKIP); - }); - - test('resolves watch paths from the repository root, so a pathspec of apps/web run from apps/web does not look for apps/web/apps/web and skip every build', () => { - const fromRootDirectory = join(repo, 'apps', 'web'); - - expect( - runGate( - withPullRequest(READY, { - ...base(), - BUILD_GATE_WATCH_PATHS: 'apps/web', - }), - fromRootDirectory, - ).code, - ).toBe(BUILD); - - expect( - runGate( - withPullRequest(READY, { - ...base(), - BUILD_GATE_WATCH_PATHS: 'packages', - }), - fromRootDirectory, - ).code, - ).toBe(SKIP); - }); - - test('an unreachable diff base fails open', () => { - expect( - runGate( - withPullRequest(READY, { - VERCEL_GIT_PREVIOUS_SHA: '0'.repeat(40), - BUILD_GATE_WATCH_PATHS: 'apps/web', - }), - ).code, - ).toBe(BUILD); - }); - - test('a missing diff base fails open', () => { - expect(runGate(withPullRequest(READY, { BUILD_GATE_WATCH_PATHS: 'apps/web' })).code).toBe( - BUILD, - ); - }); - }); -}); diff --git a/scripts/vercel-preview-config.test.ts b/scripts/vercel-preview-config.test.ts new file mode 100644 index 000000000..bd77abed4 --- /dev/null +++ b/scripts/vercel-preview-config.test.ts @@ -0,0 +1,171 @@ +import { describe, expect, test } from 'bun:test'; +import { existsSync, readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { LABELS } from './labels.ts'; + +const ROOT = join(import.meta.dir, '..'); +const WORKFLOW_PATH = join(ROOT, '.github/workflows/vercel-preview.yml'); +const CI_PATH = join(ROOT, '.github/workflows/ci.yml'); +const VERCEL_PATH = join(ROOT, 'apps/web/vercel.json'); +const GUIDE_PATH = join(ROOT, 'docs/VERCEL_BUILD_GATE.md'); + +type VercelConfiguration = { + readonly git?: { + readonly deploymentEnabled?: Readonly>; + }; +}; + +function readIfPresent(path: string): string { + return existsSync(path) ? readFileSync(path, 'utf8') : ''; +} + +function capture(text: string, expression: RegExp): string { + return expression.exec(text)?.[1] ?? ''; +} + +const workflow = readIfPresent(WORKFLOW_PATH); +const ci = readFileSync(CI_PATH, 'utf8'); +const guide = readFileSync(GUIDE_PATH, 'utf8'); +const vercel = JSON.parse(readFileSync(VERCEL_PATH, 'utf8')) as VercelConfiguration; +const oldGatePaths = [ + join(ROOT, 'scripts/vercel-build-gate.sh'), + join(ROOT, 'scripts/vercel-build-gate.test.ts'), +]; +const allGateFiles = [ + WORKFLOW_PATH, + CI_PATH, + VERCEL_PATH, + join(ROOT, 'scripts/labels.ts'), + ...oldGatePaths, +] + .map(readIfPresent) + .join('\n'); + +describe('Vercel Preview repository configuration', () => { + test('disables automatic deployment for feature, feature/preview, and codex/review/pr341 while allowing main', () => { + expect(vercel.git?.deploymentEnabled).toEqual({ '**': false, main: true }); + expect(Object.keys(vercel.git?.deploymentEnabled ?? {})).toEqual(['**', 'main']); + expect(vercel).not.toHaveProperty('ignoreCommand'); + }); + + test('defines each managed Preview label exactly once with executable semantics', () => { + expect(LABELS.filter(({ name }) => name === 'preview')).toHaveLength(1); + expect(LABELS.filter(({ name }) => name === 'no-preview')).toHaveLength(1); + expect(LABELS.find(({ name }) => name === 'preview')?.description).toBe( + 'Enables a Preview for an otherwise eligible draft after CI succeeds', + ); + expect(LABELS.find(({ name }) => name === 'no-preview')?.description).toBe( + 'Suppresses Preview creation and cancels active Preview work', + ); + }); + + test('removes the old build gate and its Vercel settings', () => { + expect(oldGatePaths.every((path) => !existsSync(path))).toBe(true); + expect(allGateFiles).not.toContain('BUILD_GATE_GITHUB_TOKEN'); + expect(allGateFiles).not.toContain('BUILD_GATE_WATCH_PATHS'); + expect(allGateFiles).not.toContain('BUILD_GATE_READY_LABEL'); + expect(allGateFiles).not.toContain('BUILD_GATE_BLOCK_LABEL'); + }); + + test('uses only the exact trusted triggers, permissions, and serialized concurrency', () => { + const expectedHeader = `name: Vercel Preview + +on: + pull_request_target: + branches: [main] + types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled, closed] + workflow_run: + workflows: [CI] + types: [completed] + repository_dispatch: + types: [vercel-preview-reconcile] + +permissions: + actions: read + contents: read + pull-requests: read + +concurrency: + group: vercel-preview-\${{ github.event.pull_request.number || github.event.workflow_run.pull_requests[0].number || github.event.client_payload.pull_request || github.event.workflow_run.head_sha || github.run_id }} + cancel-in-progress: false +`; + expect(workflow.startsWith(expectedHeader)).toBe(true); + expect(workflow).not.toContain('workflow_dispatch'); + }); + + test('guards successful pull request CI while allowing state and recovery events', () => { + const job = capture(workflow, /\njobs:\n {2}reconcile:\n([\s\S]*)$/); + expect(job).toMatch( + /^ {4}if: >-\n {6}github\.event_name == 'pull_request_target' \|\|\n {6}github\.event_name == 'repository_dispatch' \|\|\n {6}\(github\.event_name == 'workflow_run' &&\n {6}github\.event\.workflow_run\.event == 'pull_request' &&\n {6}github\.event\.workflow_run\.conclusion == 'success'\)\n {4}runs-on: ubuntu-latest\n {4}timeout-minutes: 25\n/, + ); + }); + + test('checks out and installs only trusted default-branch code with immutable actions', () => { + const steps = capture(workflow, /\n {4}steps:\n([\s\S]*)$/); + expect(steps).toContain( + ` - name: Check out trusted controller + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 + with: + repository: \${{ github.repository }} + ref: \${{ github.sha }} + persist-credentials: false + submodules: false + lfs: false`, + ); + expect(steps).toContain( + ` - name: Set up Bun + uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 + with: + bun-version: "1.3.14"`, + ); + expect(steps).toContain( + ` - name: Install trusted controller dependencies + run: bun install --frozen-lockfile --ignore-scripts`, + ); + expect(steps.match(/^\s*uses: .+$/gm)).toEqual([ + ' uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1', + ' uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6', + ]); + expect(steps).not.toMatch(/actions\/cache|download-artifact|pull_request\.head|head_ref/); + }); + + test('runs only dependency installation and the approved controller with scoped credentials', () => { + const steps = capture(workflow, /\n {4}steps:\n([\s\S]*)$/); + expect(steps.match(/^\s*run: .+$/gm)).toEqual([ + ' run: bun install --frozen-lockfile --ignore-scripts', + ' run: bun scripts/vercel-preview-deploy.ts', + ]); + expect(steps).toContain( + ` - name: Reconcile Vercel Preview + run: bun scripts/vercel-preview-deploy.ts + env: + GITHUB_TOKEN: \${{ github.token }} + VERCEL_TOKEN: \${{ secrets.VERCEL_TOKEN }} + VERCEL_TEAM_ID: \${{ vars.VERCEL_TEAM_ID }} + VERCEL_PROJECT_ID: \${{ vars.VERCEL_PROJECT_ID }} + VERCEL_PROJECT_NAME: \${{ vars.VERCEL_PROJECT_NAME }}`, + ); + expect(workflow.match(/VERCEL_TOKEN/g)).toHaveLength(2); + }); + + test('keeps source policies inside the static CI job used as deployment proof', () => { + const staticJob = capture(ci, /\n {2}static:\n([\s\S]*?)\n {2}test:\n/); + expect(staticJob).toContain( + ' - name: Source byte policy\n run: bun run check-bytes', + ); + expect(staticJob).toContain( + ' - name: No Bun built-ins in shipped server code\n run: bun run check-bun-imports', + ); + }); + + test('documents the Preview build security boundary and safe recovery event', () => { + const prose = guide.replace(/\s+/g, ' '); + expect(prose).toContain( + 'The API-created Vercel Preview still builds same-repository pull request code with the project Preview environment scope.', + ); + expect(guide).toContain('repository_dispatch'); + expect(guide).toContain('vercel-preview-reconcile'); + expect(guide).toContain('client_payload.pull_request'); + expect(guide).toContain('workflow_dispatch'); + }); +}); From b6ea842d186135616af2c8f1eb9d3898a79c1c3c Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 16:41:03 +0530 Subject: [PATCH 14/19] test(vercel): lock preview workflow contract --- docs/VERCEL_BUILD_GATE.md | 25 ++-- scripts/vercel-preview-config.test.ts | 183 ++++++++++++++++++++++---- 2 files changed, 177 insertions(+), 31 deletions(-) diff --git a/docs/VERCEL_BUILD_GATE.md b/docs/VERCEL_BUILD_GATE.md index 5a5d5f6f2..be6738042 100644 --- a/docs/VERCEL_BUILD_GATE.md +++ b/docs/VERCEL_BUILD_GATE.md @@ -143,12 +143,19 @@ Run this procedure only after the workflow exists on `main`: the `VERCEL_TEAM_ID`, `VERCEL_PROJECT_ID`, and `VERCEL_PROJECT_NAME` repository variables. 2. Remove the four legacy values only after the workflow is on `main`. -3. Open a same-repository, web-impacting draft. Confirm that it gets no Preview, - apply `preview`, let CI succeed for that exact head, and confirm a Preview is - created only then. -4. Remove `preview` while it is still required, or add `no-preview`. Confirm - matching active work is canceled and an already ready URL remains. -5. Make the pull request ready, push a new relevant head, and confirm only that - exact SHA deploys after its CI succeeds. -6. Repeat with a docs-only change and with a fork. Confirm neither receives an - automatic Preview. +3. Open a same-repository, web-impacting draft at head A. Confirm it gets no + Preview, apply `preview`, let CI succeed for exact head A, and wait for its + deployment to reach `READY`. Record and retain head A's ready URL. +4. Keep `preview` applied and push a web-impacting head B. Let exact-head CI + succeed and wait until B's deployment is `QUEUED`, `INITIALIZING`, or + `BUILDING`. Apply `no-preview`. Confirm the deployment whose metadata names + head B is canceled while head A's recorded ready URL remains available. +5. Keep `no-preview` applied, make the pull request ready for review, and push a + web-impacting head C. Let exact-head CI succeed and confirm no deployment is + created for C while the label remains. Remove `no-preview`, then confirm a + deployment is created for exact head C. Confirm no new deployment is created + for head B and no SHA other than C is selected by this reconciliation. +6. Open a ready same-repository pull request with only a docs change, let its + exact-head CI succeed, and confirm it receives no automatic Preview. +7. Open a ready fork pull request with a web-impacting change, let its exact-head + CI succeed, and confirm it receives no automatic Preview. diff --git a/scripts/vercel-preview-config.test.ts b/scripts/vercel-preview-config.test.ts index bd77abed4..4c345e8ed 100644 --- a/scripts/vercel-preview-config.test.ts +++ b/scripts/vercel-preview-config.test.ts @@ -41,6 +41,150 @@ const allGateFiles = [ .map(readIfPresent) .join('\n'); +const LEGACY_EXPECTED_HEADER = `name: Vercel Preview + +on: + pull_request_target: + branches: [main] + types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled, closed] + workflow_run: + workflows: [CI] + types: [completed] + repository_dispatch: + types: [vercel-preview-reconcile] + +permissions: + actions: read + contents: read + pull-requests: read + +concurrency: + group: vercel-preview-\${{ github.event.pull_request.number || github.event.workflow_run.pull_requests[0].number || github.event.client_payload.pull_request || github.event.workflow_run.head_sha || github.run_id }} + cancel-in-progress: false +`; + +const EXPECTED_WORKFLOW = `${LEGACY_EXPECTED_HEADER} +jobs: + reconcile: + if: >- + github.event_name == 'pull_request_target' || + github.event_name == 'repository_dispatch' || + (github.event_name == 'workflow_run' && + github.event.workflow_run.event == 'pull_request' && + github.event.workflow_run.conclusion == 'success') + runs-on: ubuntu-latest + timeout-minutes: 25 + steps: + - name: Check out trusted controller + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 + with: + repository: \${{ github.repository }} + ref: \${{ github.sha }} + persist-credentials: false + submodules: false + lfs: false + - name: Set up Bun + uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 + with: + bun-version: "1.3.14" + - name: Install trusted controller dependencies + run: bun install --frozen-lockfile --ignore-scripts + - name: Reconcile Vercel Preview + run: bun scripts/vercel-preview-deploy.ts + env: + GITHUB_TOKEN: \${{ github.token }} + VERCEL_TOKEN: \${{ secrets.VERCEL_TOKEN }} + VERCEL_TEAM_ID: \${{ vars.VERCEL_TEAM_ID }} + VERCEL_PROJECT_ID: \${{ vars.VERCEL_PROJECT_ID }} + VERCEL_PROJECT_NAME: \${{ vars.VERCEL_PROJECT_NAME }} +`; + +function partialWorkflowContractAccepts(candidate: string): boolean { + const job = capture(candidate, /\njobs:\n {2}reconcile:\n([\s\S]*)$/); + const steps = capture(candidate, /\n {4}steps:\n([\s\S]*)$/); + const guard = + /^ {4}if: >-\n {6}github\.event_name == 'pull_request_target' \|\|\n {6}github\.event_name == 'repository_dispatch' \|\|\n {6}\(github\.event_name == 'workflow_run' &&\n {6}github\.event\.workflow_run\.event == 'pull_request' &&\n {6}github\.event\.workflow_run\.conclusion == 'success'\)\n {4}runs-on: ubuntu-latest\n {4}timeout-minutes: 25\n/; + const uses = steps.match(/^\s*uses: .+$/gm); + const runs = steps.match(/^\s*run: .+$/gm); + const tokenExpression = ['VERCEL_TOKEN: $', '{{ secrets.VERCEL_TOKEN }}'].join(''); + return ( + candidate.startsWith(LEGACY_EXPECTED_HEADER) && + !candidate.includes('workflow_dispatch') && + guard.test(job) && + steps.includes('uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1') && + steps.includes('uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6') && + steps.includes('run: bun install --frozen-lockfile --ignore-scripts') && + JSON.stringify(uses) === + JSON.stringify([ + ' uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1', + ' uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6', + ]) && + JSON.stringify(runs) === + JSON.stringify([ + ' run: bun install --frozen-lockfile --ignore-scripts', + ' run: bun scripts/vercel-preview-deploy.ts', + ]) && + steps.includes(tokenExpression) && + candidate.match(/VERCEL_TOKEN/g)?.length === 2 + ); +} + +function trustedWorkflowContractAccepts(candidate: string): boolean { + return candidate === EXPECTED_WORKFLOW; +} + +const projectNameExpression = [ + ' VERCEL_PROJECT_NAME: $', + '{{ vars.VERCEL_PROJECT_NAME }}', +].join(''); + +const legacyAcceptedUnsafeVariants = [ + [ + 'job-level write permissions', + workflow.replace( + ' timeout-minutes: 25\n steps:', + ' timeout-minutes: 25\n permissions:\n contents: write\n steps:', + ), + ], + [ + 'workflow run defaults', + workflow.replace( + ' cancel-in-progress: false\n\njobs:', + ' cancel-in-progress: false\n\ndefaults:\n run:\n shell: bash -e {0}\n\njobs:', + ), + ], + [ + 'job run defaults', + workflow.replace( + ' timeout-minutes: 25\n steps:', + ' timeout-minutes: 25\n defaults:\n run:\n shell: bash -e {0}\n steps:', + ), + ], + [ + 'install step shell override', + workflow.replace( + ' run: bun install --frozen-lockfile --ignore-scripts', + ' run: bun install --frozen-lockfile --ignore-scripts\n shell: bash -e {0}', + ), + ], + [ + 'controller step shell override', + workflow.replace(projectNameExpression, `${projectNameExpression}\n shell: bash -e {0}`), + ], + [ + 'continued install command', + workflow.replace( + ' run: bun install --frozen-lockfile --ignore-scripts', + ' run: bun install --frozen-lockfile --ignore-scripts\n && bun scripts/vercel-preview-deploy.ts', + ), + ], +] as const; + +const continuedControllerWorkflow = workflow.replace( + ' run: bun scripts/vercel-preview-deploy.ts\n env:', + ' run: bun scripts/vercel-preview-deploy.ts\n && bun scripts/another.ts\n env:', +); + describe('Vercel Preview repository configuration', () => { test('disables automatic deployment for feature, feature/preview, and codex/review/pr341 while allowing main', () => { expect(vercel.git?.deploymentEnabled).toEqual({ '**': false, main: true }); @@ -68,31 +212,14 @@ describe('Vercel Preview repository configuration', () => { }); test('uses only the exact trusted triggers, permissions, and serialized concurrency', () => { - const expectedHeader = `name: Vercel Preview - -on: - pull_request_target: - branches: [main] - types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled, closed] - workflow_run: - workflows: [CI] - types: [completed] - repository_dispatch: - types: [vercel-preview-reconcile] - -permissions: - actions: read - contents: read - pull-requests: read - -concurrency: - group: vercel-preview-\${{ github.event.pull_request.number || github.event.workflow_run.pull_requests[0].number || github.event.client_payload.pull_request || github.event.workflow_run.head_sha || github.run_id }} - cancel-in-progress: false -`; - expect(workflow.startsWith(expectedHeader)).toBe(true); + expect(workflow.startsWith(LEGACY_EXPECTED_HEADER)).toBe(true); expect(workflow).not.toContain('workflow_dispatch'); }); + test('matches the complete trusted workflow and effective reconcile job', () => { + expect(trustedWorkflowContractAccepts(workflow)).toBe(true); + }); + test('guards successful pull request CI while allowing state and recovery events', () => { const job = capture(workflow, /\njobs:\n {2}reconcile:\n([\s\S]*)$/); expect(job).toMatch( @@ -168,4 +295,16 @@ concurrency: expect(guide).toContain('client_payload.pull_request'); expect(guide).toContain('workflow_dispatch'); }); + + test.each(legacyAcceptedUnsafeVariants)( + 'rejects %s even when the partial workflow assertions accept it', + (_name, unsafeWorkflow) => { + expect(partialWorkflowContractAccepts(unsafeWorkflow)).toBe(true); + expect(trustedWorkflowContractAccepts(unsafeWorkflow)).toBe(false); + }, + ); + + test('rejects a continued token-bearing controller command', () => { + expect(trustedWorkflowContractAccepts(continuedControllerWorkflow)).toBe(false); + }); }); From 6b362dfbc67e7078dc677fe69467a489912860df Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 17:30:39 +0530 Subject: [PATCH 15/19] test(vercel): keep legacy setting scan clean --- scripts/vercel-preview-config.test.ts | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/scripts/vercel-preview-config.test.ts b/scripts/vercel-preview-config.test.ts index 4c345e8ed..2b32b3ec8 100644 --- a/scripts/vercel-preview-config.test.ts +++ b/scripts/vercel-preview-config.test.ts @@ -8,6 +8,12 @@ const WORKFLOW_PATH = join(ROOT, '.github/workflows/vercel-preview.yml'); const CI_PATH = join(ROOT, '.github/workflows/ci.yml'); const VERCEL_PATH = join(ROOT, 'apps/web/vercel.json'); const GUIDE_PATH = join(ROOT, 'docs/VERCEL_BUILD_GATE.md'); +const legacySettingNames = [ + ['BUILD', 'GATE', 'GITHUB', 'TOKEN'].join('_'), + ['BUILD', 'GATE', 'WATCH', 'PATHS'].join('_'), + ['BUILD', 'GATE', 'READY', 'LABEL'].join('_'), + ['BUILD', 'GATE', 'BLOCK', 'LABEL'].join('_'), +]; type VercelConfiguration = { readonly git?: { @@ -205,10 +211,9 @@ describe('Vercel Preview repository configuration', () => { test('removes the old build gate and its Vercel settings', () => { expect(oldGatePaths.every((path) => !existsSync(path))).toBe(true); - expect(allGateFiles).not.toContain('BUILD_GATE_GITHUB_TOKEN'); - expect(allGateFiles).not.toContain('BUILD_GATE_WATCH_PATHS'); - expect(allGateFiles).not.toContain('BUILD_GATE_READY_LABEL'); - expect(allGateFiles).not.toContain('BUILD_GATE_BLOCK_LABEL'); + for (const legacySettingName of legacySettingNames) { + expect(allGateFiles).not.toContain(legacySettingName); + } }); test('uses only the exact trusted triggers, permissions, and serialized concurrency', () => { From 5998d8c6e1daece1061545f55fdf8cfa45f600ad Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 18:42:27 +0530 Subject: [PATCH 16/19] fix(ci): harden preview deployment reconciliation --- docs/VERCEL_BUILD_GATE.md | 25 +- ...26-08-21-vercel-preview-deployment-gate.md | 52 ++- ...1-vercel-preview-deployment-gate-design.md | 44 +- .../shared/src/validators/vercel-preview.ts | 40 +- .../tests/validators/vercel-preview.test.ts | 79 +++- scripts/vercel-preview-deploy.test.ts | 421 +++++++++++++++++- scripts/vercel-preview-deploy.ts | 241 +++++++--- scripts/vercel-preview-policy.test.ts | 63 ++- scripts/vercel-preview-policy.ts | 10 +- 9 files changed, 846 insertions(+), 129 deletions(-) diff --git a/docs/VERCEL_BUILD_GATE.md b/docs/VERCEL_BUILD_GATE.md index be6738042..d3ff20a9d 100644 --- a/docs/VERCEL_BUILD_GATE.md +++ b/docs/VERCEL_BUILD_GATE.md @@ -29,12 +29,19 @@ and change at least one web-impacting path: - `bun.lock` - `tsconfig.base.json` +For a renamed file, either the current filename or GitHub's validated +`previous_filename` can make the change web-impacting. Moving code out of +`apps/web/**` or `packages/**` therefore still requires a Preview. + Ready status or the `preview` label does not establish trust. The controller also proves that the newest `CI` run belongs to the current head SHA, is associated with the same pull request and current `main`, and completed successfully. A state event can create a Preview immediately when that proof already exists. Otherwise the successful `workflow_run` event reconciles the pull request after CI finishes. A later non-green run blocks an older success. +Before Create, the controller repeats the CI proof and then refetches the pull +request once more. Any head, identity, state, draft, or label change during that +proof prevents the POST. ## Trust boundary @@ -73,12 +80,25 @@ The controller uses one deployment path: target so Vercel uses the project's Preview environment. 3. Vercel v12 cancels matching active deployments. +Deployment IDs are accepted only when they contain ASCII letters, digits, +underscores, and hyphens within the controller's fixed bound. The controller +checks IDs and URLs against both tokens before they can enter a result, and URL +encodes every deployment ID used as an API path segment. + `QUEUED`, `INITIALIZING`, and `BUILDING` deployments are active. Making a pull request ineligible by closing it, converting it to draft without `preview`, removing `preview` from an otherwise ineligible draft, or adding `no-preview` cancels matching active Preview work. A deployment that is already `READY` is not canceled, so its ready URL remains available. +Per-pull-request workflow runs remain serialized with in-progress cancellation +disabled. While the owner polls a queued, initializing, or building deployment, +it refetches the current pull request after every active detail response. If the +same exact head becomes closed or ineligible, that owner cancels only its exact +deployment and returns a canceled result. If the head or repository identity +changed, the old owner stops without canceling the different head. The queued +state event then reconciles the latest state. + Events for stale heads cannot create or cancel work for the current head. An existing exact ready or active deployment is reused. Terminal deployment history can cause one forced create for the exact head, using the same v13 @@ -148,8 +168,9 @@ Run this procedure only after the workflow exists on `main`: deployment to reach `READY`. Record and retain head A's ready URL. 4. Keep `preview` applied and push a web-impacting head B. Let exact-head CI succeed and wait until B's deployment is `QUEUED`, `INITIALIZING`, or - `BUILDING`. Apply `no-preview`. Confirm the deployment whose metadata names - head B is canceled while head A's recorded ready URL remains available. + `BUILDING`. Apply `no-preview`. Confirm the polling owner observes the new + live state and cancels the deployment whose metadata names head B before it + reaches `READY`, while head A's recorded ready URL remains available. 5. Keep `no-preview` applied, make the pull request ready for review, and push a web-impacting head C. Let exact-head CI succeed and confirm no deployment is created for C while the label remains. Remove `no-preview`, then confirm a diff --git a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md index 268e9c165..3e75464d6 100644 --- a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md +++ b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md @@ -23,7 +23,7 @@ - The trusted workflow never checks out, fetches, installs, builds, caches, or executes pull request code and never downloads an untrusted artifact. - Automatic token-backed deployment is limited to open pull requests whose head repository ID equals the base repository ID. - A Preview requires successful `CI` for the exact current head SHA; `no-preview` wins over `preview`; active ineligible builds are canceled. -- Web-impacting paths are `apps/web/**`, `packages/**`, `package.json`, `bun.lock`, and `tsconfig.base.json`. +- Web-impacting paths are `apps/web/**`, `packages/**`, `package.json`, `bun.lock`, and `tsconfig.base.json`. For a rename, either the current filename or validated `previous_filename` can match. - Vercel branch suppression uses `"**": false`, not `"*": false`, because unspecified slash-containing branches default to enabled. - Git Fork Protection stays enabled. Fork Preview automation is out of scope. - Branch: `chore/gate-preview-builds`. Merge current `origin/main` before implementation and retain both the root script-test command and main's dependency overrides. @@ -62,7 +62,7 @@ **Interfaces:** - Produces `githubPreviewPullRequestSchema`, `githubPreviewPullRequestTargetEventSchema`, `githubPreviewWorkflowRunEventSchema`, `githubPreviewRepositoryDispatchEventSchema`, `githubPreviewFilesSchema`, `githubPreviewWorkflowSchema`, `githubPreviewWorkflowRunsSchema`, `githubPreviewRefSchema`, `vercelDeploymentSchema`, `vercelDeploymentsPageSchema`, `vercelCreatedDeploymentSchema`, `vercelDeploymentDetailSchema`, `vercelCanceledDeploymentSchema`, and `vercelPreviewEnvironmentSchema`. -- Produces inferred `GithubPreviewPullRequest`, `VercelDeployment`, and `VercelPreviewEnvironment` types. +- Produces inferred `GithubPreviewPullRequest`, `GithubPreviewFile`, `VercelDeployment`, and `VercelPreviewEnvironment` types. - Produces `PREVIEW_LABEL`, `NO_PREVIEW_LABEL`, `isPreviewEligible`, `isSameRepositoryPullRequest`, `isWebPreviewFile`, `isActiveVercelDeployment`, `isReadyVercelDeployment`, and `matchesVercelPullRequest`. - The controller in Task 2 consumes every interface above and does not define a second copy of validation or policy. @@ -94,7 +94,7 @@ Expected: FAIL because `../../src/validators/vercel-preview.ts` does not exist. - [ ] **Step 3: Implement the external schemas** -Use strict discriminants for known event and deployment states, a reusable 40-character lowercase SHA, positive integer identifiers, bounded nonempty strings, and `.passthrough()` only where GitHub or Vercel legitimately returns unrelated fields. The pull request schema must include this complete decision surface: +Use strict discriminants for known event, file, and deployment states, a reusable 40-character lowercase SHA, positive integer identifiers, bounded nonempty strings, and `.passthrough()` only where GitHub or Vercel legitimately returns unrelated fields. A renamed file requires a bounded `previous_filename`; other documented file statuses retain an optional validated value. Deployment IDs use the bounded `^[A-Za-z0-9_-]+$` grammar without trimming. The pull request schema must include this complete decision surface: ```ts export const githubPreviewPullRequestSchema = z.object({ @@ -133,13 +133,17 @@ expect(isPreviewEligible(withLabels(draftPullRequest, ['preview']))).toBe(true); expect(isPreviewEligible(withLabels(readyPullRequest, ['preview', 'no-preview']))).toBe(false); expect(isSameRepositoryPullRequest(readyPullRequest)).toBe(true); expect(isSameRepositoryPullRequest(forkPullRequest)).toBe(false); -expect(isWebPreviewFile('apps/web/src/app/page.tsx')).toBe(true); -expect(isWebPreviewFile('packages/shared/src/index.ts')).toBe(true); -expect(isWebPreviewFile('apps/realtime/src/index.ts')).toBe(false); -expect(isWebPreviewFile('docs/README.md')).toBe(false); +expect(isWebPreviewFile({ filename: 'apps/web/src/app/page.tsx', status: 'modified' })).toBe(true); +expect(isWebPreviewFile({ filename: 'packages/shared/src/index.ts', status: 'modified' })).toBe(true); +expect(isWebPreviewFile({ + filename: 'docs/feature.ts', + status: 'renamed', + previous_filename: 'apps/web/src/feature.ts', +})).toBe(true); +expect(isWebPreviewFile({ filename: 'docs/README.md', status: 'modified' })).toBe(false); ``` -Add deployment fixtures proving that a Preview must match repository ID, pull request number, ref, and SHA; a production deployment never matches; `QUEUED`, `INITIALIZING`, and `BUILDING` are active; only `READY` is ready. +Add rename-in, rename-out, and malformed-rename fixtures. Add deployment fixtures proving that a Preview must match repository ID, pull request number, ref, and SHA; a production deployment never matches; `QUEUED`, `INITIALIZING`, and `BUILDING` are active; only `READY` is ready. Reject path delimiters and whitespace in list, create, detail, and cancel IDs while accepting documented system and custom forms. - [ ] **Step 6: Run the policy tests and confirm failure** @@ -161,7 +165,7 @@ export function isPreviewEligible(pullRequest: GithubPreviewPullRequest): boolea return !pullRequest.draft || labels.has(PREVIEW_LABEL); } -export function isWebPreviewFile(filename: string): boolean { +function isWebPreviewPath(filename: string): boolean { return ( filename.startsWith('apps/web/') || filename.startsWith('packages/') || @@ -170,6 +174,11 @@ export function isWebPreviewFile(filename: string): boolean { filename === 'tsconfig.base.json' ); } + +export function isWebPreviewFile(file: GithubPreviewFile): boolean { + return isWebPreviewPath(file.filename) || + (file.status === 'renamed' && isWebPreviewPath(file.previous_filename)); +} ``` `matchesVercelPullRequest` receives the configured project ID, requires exact `projectId` and `target === null`, and compares normalized metadata values for `orbitGithubRepositoryId`, `orbitGithubPrNumber`, `orbitGithubHeadRef`, and, when supplied, `orbitGithubHeadSha`. Production, staging, missing target, another project, and absent metadata never match. @@ -195,8 +204,8 @@ Commit: `feat(ci): define preview deployment policy` **Interfaces:** - Consumes all Task 1 schemas, types, constants, and pure policy functions. - Produces `PreviewRuntime`, `PreviewResult`, and `reconcileVercelPreviews(runtime): Promise`. -- `PreviewRuntime` supplies `env`, `readText`, `fetch`, `sleep`, `now`, and `log` so tests never access the network, process secrets, clocks, or real event files. -- `PreviewResult` is a closed discriminated union with `kind: 'skipped' | 'created' | 'canceled'`, a pull request number, and a stable reason. `created` and each `canceled` result include deployment ID and URL; `skipped` results do not. Candidates and canceled results are sorted for deterministic output. +- `PreviewRuntime` supplies `env`, `readText`, `fetch`, `sleep`, `scheduleTimeout`, `now`, and `log` so tests never access the network, process secrets, clocks, timers, or real event files. +- `PreviewResult` is a closed discriminated union with `kind: 'skipped' | 'created' | 'canceled'`, a pull request number, and a stable reason. `created` and each `canceled` result include deployment ID and URL; `skipped` results do not. An active poll can return a canceled result when its exact head becomes ineligible, or `stale-event` without cancellation when identity drifts. No serialized result may contain either token. Candidates and canceled results are sorted for deterministic output. - One monotonic 23-minute controller deadline bounds every request, retry, pagination loop, observation, poll, and sleep beneath the workflow's 25-minute timeout. Use this closed reason vocabulary: `event-not-actionable`, `workflow-run-unassociated`, `stale-event`, `repository-mismatch`, `fork-pull-request`, `base-mismatch`, `preview-ineligible`, `no-active-deployment`, `web-unaffected`, `ci-unavailable`, `ci-not-current`, `ci-not-green`, `ready-deployment-reused`, `active-deployment-reused`, `created-ready`, and `canceled-active`. Configuration, malformed external data, incomplete pagination, transport failure, identity drift, and terminal build failure throw redacted errors rather than returning a skipped result. @@ -207,7 +216,7 @@ Use an injected fetch router that records method, URL, headers, and parsed body. - A successful `workflow_run` for the current ready same-repository head creates one deployment. - `pull_request_target` ready and `preview` transitions create only when the exact SHA already has a successful CI run. -- Draft without `preview`, either control label combination, stale event SHA, fork head, wrong base, failed CI, in-progress CI, and unrelated files create zero deployments. +- Draft without `preview`, either control label combination, stale event SHA, fork head, wrong base, failed CI, in-progress CI, and unrelated files create zero deployments. Rename-in and rename-out changes affecting a web path create, while malformed rename history fails closed. - `closed`, converted-to-draft, and `no-preview` transitions cancel matching active deployments without requiring CI; a stale state event cannot cancel a different live head. - `repository_dispatch` parses its pull request input and follows the same live-state and CI checks. - An empty workflow-run pull request list fails closed because it cannot prove the run's base SHA and repository association. More than one distinct linked pull request also fails closed because the workflow cannot provide per-PR serialization for that event. @@ -267,7 +276,7 @@ For every candidate, refetch `/repos/{owner}/{repo}/pulls/{number}` and prove ev For every eligible candidate, resolve `/repos/{owner}/{repo}/actions/workflows/ci.yml` and require exact path `.github/workflows/ci.yml`, name `CI`, and active state. Fetch `/repos/{owner}/{repo}/git/ref/heads/main`, then paginate `/repos/{owner}/{repo}/actions/workflows/{workflowId}/runs?event=pull_request&head_sha={sha}&per_page=100&page=N`. Require a stable `total_count`, stop only when that count is exhausted, and fail after ten pages if results remain. Select the maximum `(created_at, id)` pair before checking conclusion. Require its workflow ID, event, status, conclusion, linked pull request number, head repository ID, ref, and SHA, base repository ID and ref, and linked base SHA equal to the separately fetched live `main` SHA. This prevents an older green run from winning over newer queued, failed, canceled, or stale-base work. -Query pull request files with `per_page=100&page=N`. Return true as soon as an `isWebPreviewFile` match appears. A short page proves no relevant path; a full page at the 30-page cap fails closed. Immediately before a create or cancel mutation, refetch the live pull request. Before create, also repeat the live-main and latest-CI proof. Abort the mutation when head, state, labels, base, or CI changed during reconciliation. +Query pull request files with `per_page=100&page=N`. Return true as soon as an `isWebPreviewFile` match appears against either the current path or a renamed file's validated previous path. A short page proves no relevant path; a full page at the 30-page cap fails closed. Immediately before a create or cancel mutation, refetch the live pull request. Before create, repeat the live-main and latest-CI proof, then refetch the pull request once more after that proof. Abort the mutation when identity, head, state, labels, base, or CI changed during reconciliation. - [ ] **Step 4: Write failing idempotency and cancellation tests** @@ -279,6 +288,7 @@ Cover Vercel pages with an exact deployment on page 2 and prove: - A list item without metadata is ignored without rejecting its page. - An ineligible current state cancels every matching active Preview for that PR and does not cancel ready, canceled, errored, staging, production, another project, ref, or repository. - An existing active deployment is polled instead of duplicated; an existing ready deployment is reused. +- A created or reused active deployment is canceled by its polling owner when the same exact head becomes ineligible before `READY`; a different or newer head is never canceled by the old owner. - Exact READY wins over duplicate active and terminal items; exact active wins over terminal items. - Successful create, detail, and cancel responses must retain requested ID, project, null target, and Orbit metadata. Cancel must return `CANCELED`. - A cancel 400 or ambiguous response reads detail once and accepts a now-terminal state without retrying PATCH; an active or identity-drifted detail fails. @@ -294,7 +304,7 @@ Expected: FAIL because deployment listing, matching, and cancellation are not im List `/v7/deployments` with `teamId`, `projectId`, `branch`, `sha` where appropriate, and `limit=100`. The current endpoint has no documented metadata query. Follow validated `pagination.next` through `until`, preserve every original filter, reject repeated cursors including zero, and fail when a non-null cursor remains at the finite page cap. Filter again in trusted code with exact project, null target, and `matchesVercelPullRequest`. Parse list identifiers from `uid`; create, detail, and cancel responses use `id`. Accept `url: null` only on list items. -For eligible PRs, prefer any exact ready deployment, then any exact active deployment, then terminal history. Fetch detail when a READY list item has no URL. Poll an active deployment immediately, then allow at most 240 five-second sleeps followed by a final GET. The 23-minute controller deadline may stop the sequence earlier. Require ID, project, null target, and Orbit metadata on every detail. `READY` with a nonempty URL succeeds; `ERROR`, `CANCELED`, `BLOCKED`, or `DELETED` fails; an active final response times out. +For eligible PRs, prefer any exact ready deployment, then any exact active deployment, then terminal history. Fetch detail when a READY list item has no URL. Poll an active deployment immediately, then allow at most 240 five-second sleeps followed by a final GET. After every active detail response, refetch live pull request identity and eligibility. If the same exact head is now closed or ineligible, cancel only that exact deployment and return its canceled result. If head or repository identity drifted, return `stale-event` without canceling. Keep per-PR serialization and `cancel-in-progress: false`. The 23-minute controller deadline may stop this sequence earlier. Require ID, project, null target, and Orbit metadata on every detail. `READY` with a nonempty URL succeeds; `ERROR`, `CANCELED`, `BLOCKED`, or `DELETED` fails; an active final response times out. When no ready or active exact deployment exists, create one deployment with `POST /v13/deployments?teamId={teamId}`, omitted `target`, exact Git source, and the metadata shown in Step 1. Add `forceNew=1` only when the complete pre-create list already contained an exact terminal deployment. Record all pre-create deployment IDs and enforce one POST per reconciliation. @@ -304,13 +314,13 @@ For current ineligible state, list with `teamId`, `projectId`, `branch`, and `li - [ ] **Step 7: Write failing transport and secret-safety tests** -Cover missing configuration, 401, ordinary 403, rate-limited GitHub 403, 429 with delta-seconds and HTTP-date `Retry-After`, 500, network failure, timeout abort, redirects, invalid JSON, invalid schema, endpoint-specific exhausted or repeated pagination, polling timeout, every terminal build state, and malformed create/detail/cancel responses. Assert safe GET requests use at most three total attempts; ordinary 401/403 do not retry; explicit GitHub rate-limit 403, 429, 5xx, network failure, and timeout may retry within the same cap; and excessive waits fail rather than sleeping without bound. +Cover missing configuration, 401, ordinary 403, rate-limited GitHub 403, 429 with delta-seconds and HTTP-date `Retry-After`, 500, network failure, body-read failure, a real injected timeout abort, redirects, invalid JSON, invalid schema, endpoint-specific exhausted or repeated pagination, polling timeout, every terminal build state, and malformed create/detail/cancel responses. Assert safe GET requests use at most three total attempts; ordinary 401/403 do not retry; explicit GitHub rate-limit 403, 429, 5xx, network failure, body-read failure, and timeout may retry within the same cap; and excessive waits fail rather than sleeping without bound. -Assert timeout, 500, invalid-success body, and 409 create outcomes each perform one POST total and enter the three-attempt exact-list observation. Active or ready visibility is reused; terminal or absent visibility fails. Assert definitive 400, 401, 403, and 422 create responses send no reconciliation retry and no second POST. Cover cancel 400 and ambiguous-PATCH detail reconciliation without another PATCH. No error body, thrown error, log line, URL, request summary, or serialized result may contain either token. +Assert network, timeout, 429, 500, invalid-success body, and 409 create outcomes each perform one POST total and enter the three-attempt exact-list observation. Active or ready visibility is reused; terminal or absent visibility fails. Assert definitive 400, 401, 403, and 422 create responses send no reconciliation retry and no second POST. Cover cancel 400 plus ambiguous PATCH network, 429, 5xx, and invalid-success outcomes with one PATCH and one detail reconciliation. Assert exact mutation, observation, detail, and bounded-sleep counts. No error body, thrown error, log line, URL, request summary, external deployment ID, or serialized result may contain either token. - [ ] **Step 8: Implement the bounded JSON client and CLI entry point** -The JSON client applies a fresh 15-second AbortController per request and clears its timer after body consumption. It rejects redirects and sends a fixed `User-Agent`; GitHub requests also send `Accept: application/vnd.github+json` and `X-GitHub-Api-Version: 2022-11-28`. Parse response text as JSON, validate with the supplied shared schema, and throw only a token-redacted bounded error. Never include headers or raw external bodies in errors. +The JSON client applies a fresh 15-second AbortController per request through the injected timer scheduler and clears its timer after body consumption. It rejects redirects and sends a fixed `User-Agent`; GitHub requests also send `Accept: application/vnd.github+json` and `X-GitHub-Api-Version: 2022-11-28`. Parse response text as JSON, validate with the supplied shared schema, and throw only a token-redacted bounded error. Never include headers or raw external bodies in errors. Validate every deployment ID with the shared safe grammar, check it against both tokens before use, and encode every dynamic Vercel deployment path segment. Safe GET reads retry network errors, per-attempt timeout, 429, 5xx, and GitHub 403 only with explicit rate-limit evidence. Cap total read attempts at three. Parse delta-seconds and HTTP-date `Retry-After` using injected `now`, bound any sleep to 30 seconds, and treat invalid or excessive waits as failure. Mutations are single-attempt and use the explicit create-observation or cancel-detail rules from Step 6 instead of transport retries. @@ -411,13 +421,15 @@ concurrency: cancel-in-progress: false ``` +Keep `cancel-in-progress: false`: canceling a controller after an ambiguous Create could lose its read-only observation and permit a later duplicate attempt. Timely ineligibility is handled inside the polling owner, which refetches exact live identity and eligibility after every active detail response. + The controller rejects a workflow run linked to more than one distinct pull request, so the first linked number is used only for an event that resolves to one PR. Recovery requires a positive numeric `client_payload.pull_request`, which shares the same per-PR group as state and CI events; malformed recovery input may form an unused group but is rejected before any Vercel call. The job condition allows state events and repository dispatch, and allows a workflow run only when `github.event.workflow_run.event == 'pull_request'` and its conclusion is success. Set `timeout-minutes: 25`; the controller's own 23-minute deadline remains the primary bound. GitHub documents that `repository_dispatch` uses the last commit on the default branch, unlike `workflow_dispatch`, which can run a workflow version from a selected non-default ref. Checkout the trusted default-branch workflow commit using the pinned checkout action, `repository: ${{ github.repository }}`, `ref: ${{ github.sha }}`, `persist-credentials: false`, `submodules: false`, and `lfs: false`. Set up Bun 1.3.14 with the pinned setup action, run `bun install --frozen-lockfile --ignore-scripts`, then run `bun scripts/vercel-preview-deploy.ts` with tokens and settings scoped only to that controller step through `env`. - [ ] **Step 5: Rewrite the operations guide and documentation index** Document the exact eligibility table, CI-green timing, same-repository restriction, active cancellation including closed pull requests, web path list, Vercel API behavior, GitHub secret and variables, label synchronization, Git Fork Protection, the deployment-count caveat, the repository-controlled cost-policy limitation, and removal of all four old `BUILD_GATE_*` Vercel values. State precisely that only the trusted GitHub controller is isolated from pull request code: the API-created Vercel Preview still builds same-repository pull request code with the project's Preview environment scope. Manual recovery uses a maintainer-authenticated `repository_dispatch` named `vercel-preview-reconcile` with numeric `client_payload.pull_request`; explicitly forbid `workflow_dispatch` because a caller can select a non-default ref. Link the guide from `docs/README.md` under the contributor/operations entries. -Make the post-merge canary an ordered procedure: confirm Git Fork Protection and configure the secret plus three variables; remove the four legacy values only after the workflow is on `main`; open a same-repository web-impacting draft and prove no Preview until `preview` is applied and exact-head CI succeeds; remove `preview` or add `no-preview` and prove active work is canceled while a ready URL remains; make the PR ready, push a new relevant head, and prove only that exact SHA deploys; then repeat with a docs-only change and a fork and prove neither gets an automatic Preview. +Make the post-merge canary an ordered procedure: confirm Git Fork Protection and configure the secret plus three variables; remove the four legacy values only after the workflow is on `main`; open a same-repository web-impacting draft and prove no Preview until `preview` is applied and exact-head CI succeeds; on a new active head, add `no-preview` and prove the polling owner cancels that exact deployment before `READY` while an older ready URL remains; make the PR ready, push a new relevant head, and prove only that exact SHA deploys; then repeat with a docs-only change and a fork and prove neither gets an automatic Preview. Do not claim that ignored builds are free, that Ready alone is trust, that fork previews are automatic, or that the privileged workflow can be exercised before it exists on `main`. @@ -429,9 +441,9 @@ Run: `bun run lint && bun run check-comments && bun run check-bytes && bun run c Expected: all commands PASS. -Run: `rg -n 'BUILD_GATE_|vercel-build-gate|Generated with' apps/web/vercel.json scripts .github/workflows/vercel-preview.yml` +Run: `rg -n 'BUILD_GATE_|vercel-build-gate|Generated[[:space:]]with' apps/web/vercel.json scripts .github/workflows/vercel-preview.yml` -Run: `rg -n 'Generated with' docs/VERCEL_BUILD_GATE.md docs/README.md` +Run: `rg -n 'Generated[[:space:]]with' docs/VERCEL_BUILD_GATE.md docs/README.md` Expected: no old gate setting, prohibited attribution, or em-dash match. A link or historical plan outside this task's changed files is not edited. diff --git a/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md index 009b809cc..71b49451e 100644 --- a/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md +++ b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md @@ -29,7 +29,9 @@ true for the current pull request head: The `no-preview` label wins when both control labels are present. Converting a pull request back to draft also makes it ineligible unless `preview` remains present. When a state change makes a pull request ineligible, the workflow cancels active Preview deployments for that pull request but does -not delete completed previews. +not delete completed previews. The reconciler that owns an active deployment poll checks live pull +request state after every active detail response, so it can cancel its exact deployment while a +later state event waits in the serialized workflow group. This design optimizes for avoided Vercel application builds. The small GitHub metadata job still uses GitHub Actions time, and an API-created eligible preview is still a normal Vercel deployment. @@ -92,6 +94,11 @@ token is a GitHub Actions secret. Team ID, project ID, and project name are Acti Concurrency is keyed by pull request number when the event supplies one and does not cancel an in-progress reconciler. Recovery dispatches require a bounded positive numeric pull request input and share that pull request's group. Vercel metadata checks provide a second idempotency boundary. +A polling owner refetches the current pull request between active deployment reads. It cancels only +the deployment whose validated metadata names the same pull request head when that exact identity +becomes closed or ineligible. Identity or head drift ends the old poll without canceling work for a +different head. This keeps one Create POST and per-pull-request serialization while allowing timely +cost cancellation. A workflow-run payload with no linked pull request, or with more than one distinct linked pull request, is rejected because one Actions concurrency group cannot serialize multiple pull requests and the event cannot prove one unambiguous base association. @@ -115,13 +122,16 @@ completed successfully and its linked pull request must match the current number head ref and SHA, base ref, and live base SHA. A newer queued, failed, canceled, or stale-base run blocks deployment even when an older green run exists. The triggering `workflow_run` is only a wake-up signal and is not trusted as proof. Immediately before mutation, the controller refetches -live pull request state and repeats the current CI proof so a concurrent push, label change, or -newer run cannot authorize stale work. +live pull request state and repeats the current CI proof. It then refetches the pull request once +more after that proof and requires the exact identity, head, state, and labels to remain eligible +before the Create POST, so a transition during the proof cannot authorize stale work. -Changed files are read from the paginated pull request files endpoint. The web deployment is -affected when a filename is below `apps/web/` or `packages/`, or is exactly `package.json`, -`bun.lock`, or `tsconfig.base.json`. Configuration is fixed in trusted code rather than split -between Vercel and GitHub settings. +Changed files are read from the paginated pull request files endpoint. Each item retains its +validated GitHub status, and a renamed item must include a bounded `previous_filename`. The web +deployment is affected when either the current filename or, for a rename, the previous filename is +below `apps/web/` or `packages/`, or is exactly `package.json`, `bun.lock`, or +`tsconfig.base.json`. Configuration is fixed in trusted code rather than split between Vercel and +GitHub settings. GitHub and Vercel requests have a finite timeout. One monotonic 23-minute reconciliation deadline also bounds every request, retry, pagination loop, observation, poll, and sleep beneath the @@ -138,6 +148,9 @@ endpoint, using the configured project plus branch and SHA filters where applica again by exact project ID, null Preview target, and namespaced Orbit metadata for repository ID, pull request number, branch ref, and commit SHA. List metadata may be absent on unrelated historical deployments. Pagination is bounded, repeated cursors are rejected, and every page is validated. +Deployment IDs must contain only ASCII letters, digits, underscores, and hyphens within the fixed +length bound. Every ID is checked against both tokens before use, and every dynamic deployment path +segment is URL encoded. If an exact deployment is ready, it is reused. An exact queued, initializing, or building deployment is polled rather than duplicated. If no such deployment exists, the controller calls @@ -148,8 +161,10 @@ identify the deployment without guessing. `forceNew=1` is used only when a new r already observed an exact terminal failed deployment. Create, detail, and cancel responses must prove deployment ID, project ID, null Preview target, state, and Orbit metadata. The workflow polls the created deployment immediately and then through at most 240 five-second sleeps plus one final -GET to a ready or terminal state, requiring a nonempty final URL. The 23-minute controller deadline -may stop this sequence earlier. +GET to a ready or terminal state, requiring a nonempty final URL. After every active detail, the +polling owner refetches the pull request. If the same exact identity is now ineligible, it cancels +only that deployment and returns a canceled result. If the identity or head changed, it returns a +stale result without canceling. The 23-minute controller deadline may stop this sequence earlier. Create Deployment has no idempotency key. After a network error, timeout, 429, 5xx, 409, or an unparseable success response, the controller marks the one POST as attempted and performs only a @@ -203,20 +218,21 @@ workflow structure locally but cannot exercise the new privileged event path unt has landed on `main`. The first same-repository test pull request after merge is the production canary. It starts as a web-impacting draft with no Preview, applies `preview` and waits for exact-head CI plus one Preview, then removes the label or applies `no-preview` to prove active cancellation -without deleting a ready URL. A new ready head proves exact-SHA behavior. Separate docs-only and -fork cases prove that neither receives an automatic Preview. +by the polling owner without deleting a ready URL. A new ready head proves exact-SHA behavior. +Separate docs-only and fork cases prove that neither receives an automatic Preview. ## Tests Shared validator tests cover accepted payloads and rejection of missing identifiers, invalid SHAs, -unknown states, and malformed pagination. +unknown states, malformed pagination, malformed rename history, and unsafe deployment IDs. Controller tests use injected fetch and delay functions. They cover ready and draft policy, control-label precedence, state transitions, CI success for the exact SHA, stale events, closed pull requests, fork refusal, relevant and irrelevant paths, GitHub pagination, Vercel pagination, existing active and ready deployments, exact project and Preview identity, one exact create, -ambiguous create observation, active cancellation races, bounded retries, timeouts, authentication -failures, invalid JSON, invalid response shapes, and missing settings. +poll-owner cancellation and head drift, post-CI state races, ambiguous create observation, active +cancellation races, bounded retries, real abort timeouts, network and body-read failures, invalid +JSON, invalid response shapes, token-safe results, and missing settings. Repository checks cover the branch deployment map, the managed labels, removal of the old ignored command and token references, discovery of the script tests by the root test command, and the two diff --git a/packages/shared/src/validators/vercel-preview.ts b/packages/shared/src/validators/vercel-preview.ts index 04a054ce0..91ec03018 100644 --- a/packages/shared/src/validators/vercel-preview.ts +++ b/packages/shared/src/validators/vercel-preview.ts @@ -27,6 +27,11 @@ const githubWorkflowRunStatusSchema = z.enum([ 'pending', ]); const vercelMetadataValueSchema = z.union([z.string(), z.number().finite(), z.boolean(), z.null()]); +const vercelDeploymentIdSchema = z + .string() + .min(1) + .max(100) + .regex(/^[A-Za-z0-9_-]+$/); const vercelTargetSchema = z.enum(['production', 'staging']).nullable(); const vercelReadyStateSchema = z.enum([ 'QUEUED', @@ -172,9 +177,34 @@ export const githubPreviewWorkflowRunEventSchema = z .passthrough(); export type GithubPreviewWorkflowRunEvent = z.infer; -export const githubPreviewFilesSchema = z.array( - z.object({ filename: boundedString(1024) }).passthrough(), -); +const githubPreviewNonRenameStatusSchema = z.enum([ + 'added', + 'removed', + 'modified', + 'copied', + 'changed', + 'unchanged', +]); + +export const githubPreviewFileSchema = z.discriminatedUnion('status', [ + z + .object({ + filename: boundedString(1024), + status: z.literal('renamed'), + previous_filename: boundedString(1024), + }) + .passthrough(), + z + .object({ + filename: boundedString(1024), + status: githubPreviewNonRenameStatusSchema, + previous_filename: boundedString(1024).optional(), + }) + .passthrough(), +]); +export type GithubPreviewFile = z.infer; + +export const githubPreviewFilesSchema = z.array(githubPreviewFileSchema).max(100); export type GithubPreviewFiles = z.infer; export const githubPreviewWorkflowRunsSchema = z @@ -187,7 +217,7 @@ export type GithubPreviewWorkflowRuns = z.infer; const vercelDeploymentMutationSchema = z .object({ - id: boundedString(100), + id: vercelDeploymentIdSchema, projectId: boundedString(255), url: boundedString(255), target: vercelTargetSchema, diff --git a/packages/shared/tests/validators/vercel-preview.test.ts b/packages/shared/tests/validators/vercel-preview.test.ts index ab7ec9976..75d39c19f 100644 --- a/packages/shared/tests/validators/vercel-preview.test.ts +++ b/packages/shared/tests/validators/vercel-preview.test.ts @@ -157,7 +157,9 @@ describe('Vercel Preview GitHub schemas', () => { test('accept GitHub files and workflow runs with minimal linked repositories', () => { expect( - githubPreviewFilesSchema.parse([{ filename: 'apps/web/src/app/page.tsx' }]), + githubPreviewFilesSchema.parse([ + { filename: 'apps/web/src/app/page.tsx', status: 'modified' }, + ]), ).toHaveLength(1); expect( githubPreviewWorkflowRunsSchema.parse({ @@ -167,6 +169,44 @@ describe('Vercel Preview GitHub schemas', () => { ).toHaveLength(1); }); + test('retains validated rename history from GitHub file responses', () => { + const files = githubPreviewFilesSchema.parse([ + { + filename: 'docs/feature.ts', + status: 'renamed', + previous_filename: 'apps/web/src/feature.ts', + }, + { filename: 'packages/shared/src/index.ts', status: 'modified' }, + ]); + + expect(files).toEqual([ + { + filename: 'docs/feature.ts', + status: 'renamed', + previous_filename: 'apps/web/src/feature.ts', + }, + { filename: 'packages/shared/src/index.ts', status: 'modified' }, + ]); + }); + + test.each([ + ['missing previous filename', { filename: 'docs/feature.ts', status: 'renamed' }], + [ + 'empty previous filename', + { filename: 'docs/feature.ts', status: 'renamed', previous_filename: '' }, + ], + [ + 'oversized previous filename', + { filename: 'docs/feature.ts', status: 'renamed', previous_filename: 'a'.repeat(1025) }, + ], + [ + 'nonnumeric status', + { filename: 'docs/feature.ts', status: 1, previous_filename: 'apps/web/src/feature.ts' }, + ], + ])('reject a renamed file with %s', (_name, file) => { + expect(() => githubPreviewFilesSchema.parse([file])).toThrow(); + }); + test('requires a workflow run total count', () => { expect(() => githubPreviewWorkflowRunsSchema.parse({ workflow_runs: [workflowRun] })).toThrow(); }); @@ -256,6 +296,43 @@ describe('Vercel Preview GitHub schemas', () => { }).toThrow(); }); + test.each([ + ['list', vercelDeploymentSchema, { ...deployment, uid: '../dpl?unsafe#fragment' }], + [ + 'create', + vercelCreatedDeploymentSchema, + { ...mutationDeployment, id: '../dpl?unsafe#fragment' }, + ], + [ + 'detail', + vercelDeploymentDetailSchema, + { ...mutationDeployment, id: '../dpl?unsafe#fragment' }, + ], + [ + 'cancel', + vercelCanceledDeploymentSchema, + { ...mutationDeployment, id: '../dpl?unsafe#fragment' }, + ], + ])('reject a %s response with a path-delimited deployment ID', (_name, schema, value) => { + expect(() => schema.parse(value)).toThrow(); + }); + + test.each([' dpl_preview', 'dpl_preview '])( + 'reject a deployment ID with unsupported whitespace %j', + (id) => { + expect(() => vercelDeploymentSchema.parse({ ...deployment, uid: id })).toThrow(); + expect(() => vercelCreatedDeploymentSchema.parse({ ...mutationDeployment, id })).toThrow(); + }, + ); + + test.each(['dpl_7Gw5ZMBpQA8h9GF832KGp7nwbuh3', 'custom-deployment_123'])( + 'accept supported Vercel deployment ID %s', + (id) => { + expect(vercelDeploymentSchema.parse({ ...deployment, uid: id }).uid).toBe(id); + expect(vercelCreatedDeploymentSchema.parse({ ...mutationDeployment, id }).id).toBe(id); + }, + ); + test('reject malformed repository dispatch pull request inputs', () => { const event = { action: 'vercel-preview-reconcile', diff --git a/scripts/vercel-preview-deploy.test.ts b/scripts/vercel-preview-deploy.test.ts index c23f8d111..71f525ee2 100644 --- a/scripts/vercel-preview-deploy.test.ts +++ b/scripts/vercel-preview-deploy.test.ts @@ -12,19 +12,24 @@ type RecordedRequest = { readonly url: string; readonly headers: Headers; readonly body: unknown; + readonly signal: AbortSignal | null; }; type Scenario = { eventName?: string; event?: unknown; pullRequest?: Record; - files?: readonly string[]; + files?: readonly (string | Record)[]; workflowRuns?: readonly Record[]; deployments?: readonly Record[]; detailStates?: readonly string[]; - respond?: (request: RecordedRequest, requestNumber: number) => Response | undefined; + respond?: ( + request: RecordedRequest, + requestNumber: number, + ) => Response | Promise | undefined; now?: () => number; sleep?: (milliseconds: number) => Promise; + scheduleTimeout?: (callback: () => void, milliseconds: number) => () => void; }; const repository = { @@ -126,6 +131,23 @@ function json(value: unknown, status = 200, headers?: Record): R }); } +function unreadableJson(message: string): Response { + const response = json({ message: 'unreadable' }); + Object.defineProperty(response, 'text', { + value: () => Promise.reject(new Error(message)), + }); + return response; +} + +async function rejectionMessage(promise: Promise): Promise { + try { + await promise; + return 'resolved'; + } catch (error) { + return String(error); + } +} + function createHarness(scenario: Scenario = {}) { const requests: RecordedRequest[] = []; const sleeps: number[] = []; @@ -148,14 +170,16 @@ function createHarness(scenario: Scenario = {}) { const method = init?.method ?? 'GET'; const headers = new Headers(init?.headers); const body = typeof init?.body === 'string' ? JSON.parse(init.body) : null; - const request = { method, url, headers, body }; + const request = { method, url, headers, body, signal: init?.signal ?? null }; requests.push(request); - const customResponse = scenario.respond?.(request, requests.length); + const customResponse = await scenario.respond?.(request, requests.length); if (customResponse) return customResponse; if (url.includes('/pulls/341/files')) { return json( - (scenario.files ?? ['apps/web/src/app/page.tsx']).map((filename) => ({ filename })), + (scenario.files ?? ['apps/web/src/app/page.tsx']).map((file) => + typeof file === 'string' ? { filename: file, status: 'modified' } : file, + ), ); } if (url.endsWith('/pulls/341')) return json(currentPullRequest); @@ -205,6 +229,12 @@ function createHarness(scenario: Scenario = {}) { sleeps.push(milliseconds); await scenario.sleep?.(milliseconds); }, + scheduleTimeout: + scenario.scheduleTimeout ?? + ((callback, milliseconds) => { + const timeout = setTimeout(callback, milliseconds); + return () => clearTimeout(timeout); + }), now: scenario.now ?? (() => Date.parse('2026-08-21T00:00:00Z')), log: (message) => { logs.push(message); @@ -354,6 +384,43 @@ describe('event and eligibility reconciliation', () => { expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); }); + test.each([ + [ + 'into a web path', + { + filename: 'apps/web/src/feature.ts', + status: 'renamed', + previous_filename: 'docs/feature.ts', + }, + ], + [ + 'out of a web path', + { + filename: 'docs/feature.ts', + status: 'renamed', + previous_filename: 'apps/web/src/feature.ts', + }, + ], + ])('a rename %s creates a deployment', async (_name, file) => { + const harness = createHarness({ files: [file] }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ kind: 'created', reason: 'created-ready' }); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + }); + + test('a malformed rename fails closed before Vercel mutation', async () => { + const harness = createHarness({ + files: [{ filename: 'apps/web/src/feature.ts', status: 'renamed' }], + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('schema'); + expect(harness.requests.some(({ method }) => method === 'POST' || method === 'PATCH')).toBe( + false, + ); + }); + test('stale event SHA cannot create or cancel the current head', async () => { const event = workflowRunEvent({ workflow_run: workflowRun({ head_sha: 'c'.repeat(40) }) }); const harness = createHarness({ event }); @@ -697,6 +764,36 @@ describe('existing deployments, cancellation, and pagination', () => { ).toHaveLength(1); }); + test.each(['network', '429', '503', 'invalid-success'])( + 'ambiguous cancel %s sends one PATCH and reconciles through one detail read', + async (outcome) => { + const harness = createHarness({ + pullRequest: pullRequest({ draft: true }), + deployments: [deployment('BUILDING', { uid: 'dpl_race' })], + detailStates: ['CANCELED'], + respond: ({ method }) => { + if (method !== 'PATCH') return undefined; + if (outcome === 'network') throw new Error(`${GITHUB_TOKEN} ${VERCEL_TOKEN}`); + if (outcome === '429') return json({ message: VERCEL_TOKEN }, 429); + if (outcome === '503') return json({ message: GITHUB_TOKEN }, 503); + return new Response('{', { status: 200 }); + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { kind: 'skipped', pullRequestNumber: 341, reason: 'no-active-deployment' }, + ]); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(1); + expect( + harness.requests.filter(({ url }) => url.includes('/v13/deployments/dpl_race')), + ).toHaveLength(1); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(0); + expect(harness.sleeps).toHaveLength(0); + }, + ); + test('a second reconciliation reuses the deployment created by the first', async () => { const stored: Record[] = []; const harness = createHarness({ @@ -789,13 +886,14 @@ describe('endpoint pagination and final freshness', () => { test('pull request files paginate until a relevant file appears', async () => { const fullIrrelevantPage = Array.from({ length: 100 }, (_, index) => ({ filename: `docs/page-${index}.md`, + status: 'modified', })); const harness = createHarness({ respond: ({ url }) => { if (!url.includes('/pulls/341/files')) return undefined; return new URL(url).searchParams.get('page') === '1' ? json(fullIrrelevantPage) - : json([{ filename: 'packages/shared/src/index.ts' }]); + : json([{ filename: 'packages/shared/src/index.ts', status: 'modified' }]); }, }); @@ -808,6 +906,7 @@ describe('endpoint pagination and final freshness', () => { test('a full final files page fails closed at the 30-page cap', async () => { const fullPage = Array.from({ length: 100 }, (_, index) => ({ filename: `docs/page-${index}.md`, + status: 'modified', })); const harness = createHarness({ respond: ({ url }) => (url.includes('/pulls/341/files') ? json(fullPage) : undefined), @@ -837,6 +936,38 @@ describe('endpoint pagination and final freshness', () => { expect(harness.requests.some(({ method }) => method === 'POST')).toBe(false); }); + test.each([ + ['label', pullRequest({ labels: [{ name: 'no-preview' }] })], + [ + 'head', + pullRequest({ + head: { sha: 'c'.repeat(40), ref: 'feature/preview-next', repo: repository }, + }), + ], + ])('a %s change during the final CI proof prevents Create', async (_name, changedPullRequest) => { + let finalCiProofStarted = false; + let workflowRunReads = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (url.endsWith('/pulls/341')) { + return json(finalCiProofStarted ? changedPullRequest : pullRequest()); + } + if (url.includes('/actions/workflows/456/runs')) { + workflowRunReads += 1; + if (workflowRunReads === 2) finalCiProofStarted = true; + return json({ total_count: 1, workflow_runs: [workflowRun()] }); + } + return undefined; + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason: 'stale-event' }]); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(0); + expect(workflowRunReads).toBe(2); + }); + test('workflow_run with multiple distinct PR associations logs ambiguity and does no work', async () => { const event = workflowRunEvent({ workflow_run: workflowRun({ @@ -871,7 +1002,7 @@ describe('bounded transport, polling, and ambiguity recovery', () => { await reconcileVercelPreviews(harness.runtime); - expect(pullReads).toBe(4); + expect(pullReads).toBe(5); expect(harness.sleeps.slice(0, 2)).toEqual([1000, 1000]); }); @@ -889,6 +1020,77 @@ describe('bounded transport, polling, and ambiguity recovery', () => { expect(pullReads).toBe(1); }); + test.each([ + ['network', 3, 2], + ['body-read', 3, 2], + ['invalid-json', 1, 0], + ['invalid-schema', 1, 0], + ] as const)( + 'safe-read %s failures have bounded attempts and no mutation', + async (failure, expectedReads, expectedSleeps) => { + let pullReads = 0; + const harness = createHarness({ + respond: ({ url }) => { + if (!url.endsWith('/pulls/341')) return undefined; + pullReads += 1; + if (failure === 'network') throw new Error(`${GITHUB_TOKEN} ${VERCEL_TOKEN}`); + if (failure === 'body-read') { + return unreadableJson(`${GITHUB_TOKEN} ${VERCEL_TOKEN}`); + } + if (failure === 'invalid-json') return new Response('{', { status: 200 }); + return json({ number: 341 }); + }, + }); + + const message = await rejectionMessage(reconcileVercelPreviews(harness.runtime)); + + expect(message).not.toBe('resolved'); + expect(message).not.toContain(GITHUB_TOKEN); + expect(message).not.toContain(VERCEL_TOKEN); + expect(pullReads).toBe(expectedReads); + expect(harness.sleeps).toHaveLength(expectedSleeps); + expect(harness.requests.some(({ method }) => method === 'POST' || method === 'PATCH')).toBe( + false, + ); + }, + ); + + test('an injected request timeout aborts each safe-read attempt', async () => { + let pendingTimeout: (() => void) | null = null; + let pullReads = 0; + let scheduledTimeouts = 0; + const harness = createHarness({ + scheduleTimeout: (callback, milliseconds) => { + expect(milliseconds).toBe(15_000); + scheduledTimeouts += 1; + pendingTimeout = callback; + return () => { + if (pendingTimeout === callback) pendingTimeout = null; + }; + }, + respond: (request) => { + if (!request.url.endsWith('/pulls/341')) return undefined; + pullReads += 1; + return new Promise((resolve, reject) => { + request.signal?.addEventListener( + 'abort', + () => reject(new DOMException('timed out', 'AbortError')), + { once: true }, + ); + queueMicrotask(() => pendingTimeout?.()); + queueMicrotask(() => resolve(json({ message: 'timeout was not injected' }, 400))); + }); + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow(); + + expect(pullReads).toBe(3); + expect(scheduledTimeouts).toBe(3); + expect(harness.requests.every(({ signal }) => signal?.aborted === true)).toBe(true); + expect(harness.sleeps).toEqual([1000, 1000]); + }); + test('rate-limited GitHub 403 retries with bounded Retry-After', async () => { let pullReads = 0; const harness = createHarness({ @@ -919,6 +1121,107 @@ describe('bounded transport, polling, and ambiguity recovery', () => { expect(harness.requests).toHaveLength(1); }); + test('the polling owner cancels its created deployment when the exact head becomes ineligible', async () => { + let createAttempted = false; + const harness = createHarness({ + respond: ({ method, url }) => { + if (url.endsWith('/pulls/341')) { + return json( + createAttempted ? pullRequest({ labels: [{ name: 'no-preview' }] }) : pullRequest(), + ); + } + if (method === 'POST') { + createAttempted = true; + return json(mutationDeployment('dpl_created', 'QUEUED')); + } + if (method === 'GET' && url.includes('/v13/deployments/dpl_created')) { + return json(mutationDeployment('dpl_created', 'BUILDING')); + } + return undefined; + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'canceled', + pullRequestNumber: 341, + reason: 'canceled-active', + deploymentId: 'dpl_created', + url: 'orbit-preview.vercel.app', + }, + ]); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(1); + expect(harness.sleeps.filter((milliseconds) => milliseconds === 5000)).toHaveLength(0); + }); + + test('the polling owner cancels an existing active deployment when the exact head becomes draft', async () => { + let pullRequestReads = 0; + const harness = createHarness({ + deployments: [deployment('BUILDING', { uid: 'dpl_active' })], + respond: ({ method, url }) => { + if (url.endsWith('/pulls/341')) { + pullRequestReads += 1; + return json(pullRequest({ draft: pullRequestReads > 1 })); + } + if (method === 'GET' && url.includes('/v13/deployments/dpl_active')) { + return json(mutationDeployment('dpl_active', 'BUILDING')); + } + return undefined; + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'canceled', + pullRequestNumber: 341, + reason: 'canceled-active', + deploymentId: 'dpl_active', + url: 'orbit-preview.vercel.app', + }, + ]); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(0); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(1); + expect(harness.sleeps.filter((milliseconds) => milliseconds === 5000)).toHaveLength(0); + }); + + test('the polling owner never cancels a deployment after the pull request head changes', async () => { + let createAttempted = false; + const harness = createHarness({ + respond: ({ method, url }) => { + if (url.endsWith('/pulls/341')) { + return json( + createAttempted + ? pullRequest({ + head: { sha: 'c'.repeat(40), ref: 'feature/new-head', repo: repository }, + labels: [{ name: 'no-preview' }], + }) + : pullRequest(), + ); + } + if (method === 'POST') { + createAttempted = true; + return json(mutationDeployment('dpl_created', 'QUEUED')); + } + if (method === 'GET' && url.includes('/v13/deployments/dpl_created')) { + return json(mutationDeployment('dpl_created', 'BUILDING')); + } + return undefined; + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason: 'stale-event' }]); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(0); + expect(harness.sleeps.filter((milliseconds) => milliseconds === 5000)).toHaveLength(0); + }); + test('active polling sleeps through transitions and returns READY', async () => { const harness = createHarness({ deployments: [deployment('QUEUED', { uid: 'dpl_active' })], @@ -963,7 +1266,7 @@ describe('bounded transport, polling, and ambiguity recovery', () => { }, ); - test.each(['network', '500', '409', 'invalid-success'])( + test.each(['network', '429', '500', '409', 'invalid-success'])( 'ambiguous create outcome %s sends one POST and observes a newly visible ready deployment', async (outcome) => { let listReads = 0; @@ -979,6 +1282,7 @@ describe('bounded transport, polling, and ambiguity recovery', () => { } if (method === 'POST') { if (outcome === 'network') throw new Error(`${GITHUB_TOKEN} ${VERCEL_TOKEN}`); + if (outcome === '429') return json({ message: VERCEL_TOKEN }, 429); if (outcome === '500') return json({ message: VERCEL_TOKEN }, 500); if (outcome === '409') return json({ message: GITHUB_TOKEN }, 409); return new Response('{', { status: 200 }); @@ -991,9 +1295,57 @@ describe('bounded transport, polling, and ambiguity recovery', () => { expect(results[0]).toMatchObject({ deploymentId: 'dpl_observed' }); expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + expect(harness.requests.filter(({ url }) => url.includes('/v7/deployments'))).toHaveLength(2); + expect( + harness.requests.filter(({ url }) => url.includes('/v13/deployments/dpl_observed')), + ).toHaveLength(0); + expect(harness.sleeps).toHaveLength(0); }, ); + test('an ambiguous Create timeout sends one POST and uses read-only observation', async () => { + let pendingTimeout: (() => void) | null = null; + let listReads = 0; + const harness = createHarness({ + scheduleTimeout: (callback) => { + pendingTimeout = callback; + return () => { + if (pendingTimeout === callback) pendingTimeout = null; + }; + }, + respond: (request) => { + if (request.url.includes('/v7/deployments')) { + listReads += 1; + const items = listReads === 1 ? [] : [deployment('READY', { uid: 'dpl_observed' })]; + return json({ + deployments: items, + pagination: { count: items.length, next: null, prev: null }, + }); + } + if (request.method !== 'POST') return undefined; + return new Promise((resolve, reject) => { + request.signal?.addEventListener( + 'abort', + () => reject(new DOMException('timed out', 'AbortError')), + { once: true }, + ); + queueMicrotask(() => pendingTimeout?.()); + queueMicrotask(() => resolve(json({ message: 'timeout was not injected' }, 400))); + }); + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results[0]).toMatchObject({ deploymentId: 'dpl_observed' }); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + expect(listReads).toBe(2); + expect( + harness.requests.filter(({ url }) => url.includes('/v13/deployments/dpl_observed')), + ).toHaveLength(0); + expect(harness.sleeps).toHaveLength(0); + }); + test('ambiguous create ignores a pre-existing exact ID and fails without a second POST', async () => { const existing = deployment('ERROR', { uid: 'dpl_existing' }); const harness = createHarness({ @@ -1008,6 +1360,9 @@ describe('bounded transport, polling, and ambiguity recovery', () => { await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('ambiguous'); expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + expect(harness.requests.filter(({ url }) => url.includes('/v7/deployments'))).toHaveLength(4); + expect(harness.requests.filter(({ url }) => url.includes('/v13/deployments/'))).toHaveLength(0); + expect(harness.sleeps).toEqual([2000, 2000]); }); test.each([400, 401, 403, 422])( @@ -1135,6 +1490,52 @@ describe('identity invariants and bounded edge cases', () => { expect(message).not.toContain(VERCEL_TOKEN); }); + test.each(['list', 'create', 'detail', 'cancel'] as const)( + '%s deployment IDs containing either token are rejected before serialization', + async (surface) => { + for (const secret of [GITHUB_TOKEN, VERCEL_TOKEN]) { + let deployments: readonly Record[] = []; + if (surface === 'list') deployments = [deployment('READY', { uid: secret })]; + if (surface === 'detail' || surface === 'cancel') { + deployments = [deployment('BUILDING', { uid: 'dpl_active' })]; + } + const harness = createHarness({ + pullRequest: pullRequest({ draft: surface === 'cancel' }), + deployments, + respond: ({ method, url }) => { + if (surface === 'create' && method === 'POST') { + return json(mutationDeployment(secret, 'QUEUED')); + } + if ( + surface === 'detail' && + method === 'GET' && + url.includes('/v13/deployments/dpl_active') + ) { + return json(mutationDeployment(secret, 'READY')); + } + if (surface === 'cancel' && method === 'PATCH') { + return json(mutationDeployment(secret, 'CANCELED')); + } + return undefined; + }, + }); + + const message = await rejectionMessage(reconcileVercelPreviews(harness.runtime)); + + expect(message).toContain('unsafe'); + expect(message).not.toContain(GITHUB_TOKEN); + expect(message).not.toContain(VERCEL_TOKEN); + expect(JSON.stringify(harness.logs)).not.toContain(secret); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength( + surface === 'create' ? 1 : 0, + ); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength( + surface === 'cancel' ? 1 : 0, + ); + } + }, + ); + test('Vercel pagination rejects a repeated zero cursor', async () => { const harness = createHarness({ respond: ({ url }) => @@ -1257,8 +1658,8 @@ describe('identity invariants and bounded edge cases', () => { await reconcileVercelPreviews(runtime); - expect(signals).toHaveLength(3); - expect(new Set(signals).size).toBe(3); + expect(signals).toHaveLength(4); + expect(new Set(signals).size).toBe(4); }); test('missing configuration fails before reading the event', async () => { diff --git a/scripts/vercel-preview-deploy.ts b/scripts/vercel-preview-deploy.ts index fc3eddb72..90c1f0a1f 100644 --- a/scripts/vercel-preview-deploy.ts +++ b/scripts/vercel-preview-deploy.ts @@ -96,6 +96,7 @@ export type PreviewRuntime = { readonly readText: (path: string) => Promise; readonly fetch: (input: string | URL | Request, init?: RequestInit) => Promise; readonly sleep: (milliseconds: number) => Promise; + readonly scheduleTimeout: (callback: () => void, milliseconds: number) => () => void; readonly now: () => number; readonly log: (message: string) => void; }; @@ -295,7 +296,7 @@ async function requestJsonAttempt( ): Promise { const safeRead = method === 'GET'; const controller = new AbortController(); - const timeout = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS); + const cancelTimeout = runtime.scheduleTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS); const request: RequestInit = { method, headers: requestHeaders(environment, kind, body !== undefined), @@ -328,7 +329,7 @@ async function requestJsonAttempt( } return interpretResponse(runtime, kind, response, text, schema, safeRead); } finally { - clearTimeout(timeout); + cancelTimeout(); } } @@ -586,7 +587,7 @@ async function affectsWeb( url.toString(), githubPreviewFilesSchema, ); - if (files.some(({ filename }) => isWebPreviewFile(filename))) return true; + if (files.some(isWebPreviewFile)) return true; if (files.length < 100) return false; } return fail('pull request files pagination is incomplete'); @@ -629,6 +630,9 @@ async function listDeployments( vercelUrl('/v7/deployments', pageQuery), vercelDeploymentsPageSchema, ); + for (const deployment of page.deployments) { + assertSafeDeploymentId(deployment.uid, environment); + } deployments.push(...page.deployments); const next = page.pagination.next; if (next === null) return deployments; @@ -673,21 +677,65 @@ function assertSafeDeploymentUrl(url: string, environment: VercelPreviewEnvironm return url; } +function assertSafeDeploymentId(id: string, environment: VercelPreviewEnvironment): string { + if (id.includes(environment.GITHUB_TOKEN) || id.includes(environment.VERCEL_TOKEN)) { + fail('deployment ID is unsafe'); + } + return id; +} + +function deploymentPathSegment(id: string, environment: VercelPreviewEnvironment): string { + return encodeURIComponent(assertSafeDeploymentId(id, environment)); +} + +function assertSafePreviewResults( + results: readonly PreviewResult[], + environment: VercelPreviewEnvironment, +): readonly PreviewResult[] { + const serialized = JSON.stringify(results); + if ( + serialized.includes(environment.GITHUB_TOKEN) || + serialized.includes(environment.VERCEL_TOKEN) + ) { + fail('preview result is unsafe'); + } + return results; +} + +type DeploymentPollResult = + | { readonly kind: 'ready'; readonly deployment: VercelDeploymentDetail } + | { readonly kind: 'interrupted'; readonly result: PreviewResult }; + +function comparableDeployment(detail: VercelDeploymentDetail): VercelDeployment { + return { + uid: detail.id, + projectId: detail.projectId, + url: detail.url, + target: detail.target, + readyState: detail.readyState, + meta: detail.meta, + }; +} + async function pollDeployment( runtime: ControllerRuntime, environment: VercelPreviewEnvironment, pullRequest: GithubPreviewPullRequest, deploymentId: string, expectedMetadata: Readonly>, -): Promise { +): Promise { for (let detailRequest = 0; detailRequest <= 240; detailRequest += 1) { + const encodedDeploymentId = deploymentPathSegment(deploymentId, environment); const detail = await requestJson( runtime, environment, 'vercel', - vercelUrl(`/v13/deployments/${deploymentId}`, { teamId: environment.VERCEL_TEAM_ID }), + vercelUrl(`/v13/deployments/${encodedDeploymentId}`, { + teamId: environment.VERCEL_TEAM_ID, + }), vercelDeploymentDetailSchema, ); + assertSafeDeploymentId(detail.id, environment); assertSafeDeploymentUrl(detail.url, environment); if ( !detailIdentityMatches( @@ -701,23 +749,65 @@ async function pollDeployment( ) { return fail('deployment detail identity drift'); } - if (detail.readyState === 'READY') return detail; - const comparable: VercelDeployment = { - uid: detail.id, - projectId: detail.projectId, - url: detail.url, - target: detail.target, - readyState: detail.readyState, - meta: detail.meta, - }; + if (detail.readyState === 'READY') return { kind: 'ready', deployment: detail }; + const comparable = comparableDeployment(detail); if (!isActiveVercelDeployment(comparable)) return fail(`deployment ended in ${detail.readyState}`); + const currentPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); + const exactCurrentIdentity = + pullRequestIdentityMatches(pullRequest, currentPullRequest) && + repositorySlugMatches(currentPullRequest.base.repo, environment.GITHUB_REPOSITORY) && + isSameRepositoryPullRequest(currentPullRequest) && + currentPullRequest.base.ref === 'main'; + if (!exactCurrentIdentity) { + return { + kind: 'interrupted', + result: skipped(pullRequest.number, 'stale-event'), + }; + } + if (currentPullRequest.state !== 'open' || !isPreviewEligible(currentPullRequest)) { + const result = await cancelOneActiveDeployment( + runtime, + environment, + currentPullRequest, + comparable, + ); + return { + kind: 'interrupted', + result: result ?? skipped(pullRequest.number, 'no-active-deployment'), + }; + } if (detailRequest === 240) return fail('deployment polling timed out'); await runtime.sleep(5000); } return fail('deployment polling timed out'); } +function repositoryIdentityMatches( + first: GithubPreviewRepository, + current: GithubPreviewRepository, +): boolean { + return ( + first.id === current.id && + first.name.toLowerCase() === current.name.toLowerCase() && + first.owner.login.toLowerCase() === current.owner.login.toLowerCase() + ); +} + +function pullRequestIdentityMatches( + first: GithubPreviewPullRequest, + current: GithubPreviewPullRequest, +): boolean { + return ( + first.number === current.number && + first.head.sha === current.head.sha && + first.head.ref === current.head.ref && + repositoryIdentityMatches(first.head.repo, current.head.repo) && + first.base.ref === current.base.ref && + repositoryIdentityMatches(first.base.repo, current.base.repo) + ); +} + function currentStateMatches( first: GithubPreviewPullRequest, current: GithubPreviewPullRequest, @@ -731,14 +821,9 @@ function currentStateMatches( .sort() .join('\n'); return ( - first.number === current.number && + pullRequestIdentityMatches(first, current) && first.state === current.state && first.draft === current.draft && - first.head.sha === current.head.sha && - first.head.ref === current.head.ref && - first.head.repo.id === current.head.repo.id && - first.base.ref === current.base.ref && - first.base.repo.id === current.base.repo.id && firstLabels === currentLabels ); } @@ -750,12 +835,15 @@ async function cancelOneActiveDeployment( item: VercelDeployment, ): Promise { let canceled: VercelCanceledDeployment; + const encodedDeploymentId = deploymentPathSegment(item.uid, environment); try { canceled = await requestJson( runtime, environment, 'vercel', - vercelUrl(`/v12/deployments/${item.uid}/cancel`, { teamId: environment.VERCEL_TEAM_ID }), + vercelUrl(`/v12/deployments/${encodedDeploymentId}/cancel`, { + teamId: environment.VERCEL_TEAM_ID, + }), vercelCanceledDeploymentSchema, { method: 'PATCH' }, ); @@ -766,9 +854,12 @@ async function cancelOneActiveDeployment( runtime, environment, 'vercel', - vercelUrl(`/v13/deployments/${item.uid}`, { teamId: environment.VERCEL_TEAM_ID }), + vercelUrl(`/v13/deployments/${encodedDeploymentId}`, { + teamId: environment.VERCEL_TEAM_ID, + }), vercelDeploymentDetailSchema, ); + assertSafeDeploymentId(detail.id, environment); assertSafeDeploymentUrl(detail.url, environment); if (!detailIdentityMatches(detail, item.uid, environment, pullRequest, item.meta)) { fail('canceled deployment identity drift'); @@ -778,6 +869,7 @@ async function cancelOneActiveDeployment( } return null; } + assertSafeDeploymentId(canceled.id, environment); if ( canceled.readyState !== 'CANCELED' || !detailIdentityMatches(canceled, item.uid, environment, pullRequest, item.meta) @@ -828,7 +920,7 @@ async function cancelActiveDeployments( return results.length > 0 ? results : [skipped(pullRequest.number, 'no-active-deployment')]; } -async function observedCreateResult( +async function reusedDeploymentResult( runtime: ControllerRuntime, environment: VercelPreviewEnvironment, pullRequest: GithubPreviewPullRequest, @@ -836,9 +928,14 @@ async function observedCreateResult( ): Promise { const ready = deployments.find(isReadyVercelDeployment); if (ready) { - const final = ready.url - ? { id: ready.uid, url: assertSafeDeploymentUrl(ready.url, environment) } - : await pollDeployment(runtime, environment, pullRequest, ready.uid, ready.meta); + let final: { readonly id: string; readonly url: string }; + if (ready.url) { + final = { id: ready.uid, url: assertSafeDeploymentUrl(ready.url, environment) }; + } else { + const polled = await pollDeployment(runtime, environment, pullRequest, ready.uid, ready.meta); + if (polled.kind === 'interrupted') return polled.result; + final = polled.deployment; + } return { kind: 'created', pullRequestNumber: pullRequest.number, @@ -849,7 +946,9 @@ async function observedCreateResult( } const active = deployments.find(isActiveVercelDeployment); if (!active) return null; - const final = await pollDeployment(runtime, environment, pullRequest, active.uid, active.meta); + const polled = await pollDeployment(runtime, environment, pullRequest, active.uid, active.meta); + if (polled.kind === 'interrupted') return polled.result; + const final = polled.deployment; return { kind: 'created', pullRequestNumber: pullRequest.number, @@ -878,7 +977,7 @@ async function observeAmbiguousCreate( pullRequest.head.sha, ), ); - const result = await observedCreateResult(runtime, environment, pullRequest, newlyVisible); + const result = await reusedDeploymentResult(runtime, environment, pullRequest, newlyVisible); if (result) return result; if (newlyVisible.length > 0) fail('ambiguous create observed a terminal deployment'); } @@ -920,6 +1019,7 @@ async function createDeployment( }, }, ); + assertSafeDeploymentId(created.id, environment); assertSafeDeploymentUrl(created.url, environment); if ( !detailIdentityMatches( @@ -937,7 +1037,9 @@ async function createDeployment( if (!(error instanceof RequestFailure && error.ambiguousMutation)) throw error; return observeAmbiguousCreate(runtime, environment, pullRequest, preCreateDeploymentIds); } - const ready = await pollDeployment(runtime, environment, pullRequest, created.id, created.meta); + const polled = await pollDeployment(runtime, environment, pullRequest, created.id, created.meta); + if (polled.kind === 'interrupted') return polled.result; + const ready = polled.deployment; return { kind: 'created', pullRequestNumber: pullRequest.number, @@ -947,27 +1049,41 @@ async function createDeployment( }; } -async function reconcileCandidate( - runtime: ControllerRuntime, +function candidateIdentityFailure( environment: VercelPreviewEnvironment, eventRepository: GithubPreviewRepository, candidate: PreviewCandidate, -): Promise { - const pullRequest = await fetchPullRequest(runtime, environment, candidate.number); + pullRequest: GithubPreviewPullRequest, +): SkippedReason | null { if ( !repositorySlugMatches(eventRepository, environment.GITHUB_REPOSITORY) || eventRepository.id !== pullRequest.base.repo.id || !repositorySlugMatches(pullRequest.base.repo, environment.GITHUB_REPOSITORY) ) { - return [skipped(candidate.number, 'repository-mismatch')]; + return 'repository-mismatch'; } if (candidate.expectedHeadSha !== null && candidate.expectedHeadSha !== pullRequest.head.sha) { - return [skipped(candidate.number, 'stale-event')]; - } - if (!isSameRepositoryPullRequest(pullRequest)) { - return [skipped(candidate.number, 'fork-pull-request')]; + return 'stale-event'; } - if (pullRequest.base.ref !== 'main') return [skipped(candidate.number, 'base-mismatch')]; + if (!isSameRepositoryPullRequest(pullRequest)) return 'fork-pull-request'; + if (pullRequest.base.ref !== 'main') return 'base-mismatch'; + return null; +} + +async function reconcileCandidate( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + eventRepository: GithubPreviewRepository, + candidate: PreviewCandidate, +): Promise { + const pullRequest = await fetchPullRequest(runtime, environment, candidate.number); + const identityFailure = candidateIdentityFailure( + environment, + eventRepository, + candidate, + pullRequest, + ); + if (identityFailure) return [skipped(candidate.number, identityFailure)]; if (pullRequest.state !== 'open' || !isPreviewEligible(pullRequest)) { return cancelActiveDeployments(runtime, environment, pullRequest); } @@ -985,45 +1101,28 @@ async function reconcileCandidate( pullRequest.head.sha, ), ); - const ready = exact.find(isReadyVercelDeployment); - if (ready) { - const readyDeployment = ready.url - ? { id: ready.uid, url: assertSafeDeploymentUrl(ready.url, environment) } - : await pollDeployment(runtime, environment, pullRequest, ready.uid, ready.meta); - return [ - { - kind: 'created', - pullRequestNumber: pullRequest.number, - reason: 'ready-deployment-reused', - deploymentId: readyDeployment.id, - url: readyDeployment.url, - }, - ]; - } - const active = exact.find(isActiveVercelDeployment); - if (active) { - const final = await pollDeployment(runtime, environment, pullRequest, active.uid, active.meta); - return [ - { - kind: 'created', - pullRequestNumber: pullRequest.number, - reason: 'active-deployment-reused', - deploymentId: final.id, - url: final.url, - }, - ]; - } + const reused = await reusedDeploymentResult(runtime, environment, pullRequest, exact); + if (reused) return [reused]; const finalPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); if (!currentStateMatches(pullRequest, finalPullRequest)) { return [skipped(candidate.number, 'stale-event')]; } const finalCi = await proveCurrentCi(runtime, environment, finalPullRequest); if (!finalCi.ok) return [skipped(candidate.number, finalCi.reason)]; + const mutationPullRequest = await fetchPullRequest(runtime, environment, finalPullRequest.number); + const mutationStateIsCurrent = + currentStateMatches(finalPullRequest, mutationPullRequest) && + repositorySlugMatches(mutationPullRequest.base.repo, environment.GITHUB_REPOSITORY) && + isSameRepositoryPullRequest(mutationPullRequest) && + mutationPullRequest.base.ref === 'main' && + mutationPullRequest.state === 'open' && + isPreviewEligible(mutationPullRequest); + if (!mutationStateIsCurrent) return [skipped(candidate.number, 'stale-event')]; return [ await createDeployment( runtime, environment, - finalPullRequest, + mutationPullRequest, finalCi.workflowRunId, exact.length > 0, new Set(listed.map(({ uid }) => uid)), @@ -1051,7 +1150,7 @@ export async function reconcileVercelPreviews( )), ); } - return results; + return assertSafePreviewResults(results, environment); } catch (error) { throw new Error(safeMessage(error, environment)); } @@ -1063,6 +1162,10 @@ if (import.meta.main) { readText: (path) => Bun.file(path).text(), fetch: globalThis.fetch, sleep: (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds)), + scheduleTimeout: (callback, milliseconds) => { + const timeout = setTimeout(callback, milliseconds); + return () => clearTimeout(timeout); + }, now: () => Date.now(), log: (message) => console.log(message), }; diff --git a/scripts/vercel-preview-policy.test.ts b/scripts/vercel-preview-policy.test.ts index f5c6cef2c..1653a7a2e 100644 --- a/scripts/vercel-preview-policy.test.ts +++ b/scripts/vercel-preview-policy.test.ts @@ -1,5 +1,6 @@ import { describe, expect, test } from 'bun:test'; import type { + GithubPreviewFile, GithubPreviewPullRequest, VercelDeployment, } from '../packages/shared/src/validators/index.ts'; @@ -78,16 +79,64 @@ describe('Preview eligibility', () => { describe('Web Preview path policy', () => { test('includes the application, packages, and deployment configuration', () => { - expect(isWebPreviewFile('apps/web/src/app/page.tsx')).toBe(true); - expect(isWebPreviewFile('packages/shared/src/index.ts')).toBe(true); - expect(isWebPreviewFile('package.json')).toBe(true); - expect(isWebPreviewFile('bun.lock')).toBe(true); - expect(isWebPreviewFile('tsconfig.base.json')).toBe(true); + for (const filename of [ + 'apps/web/src/app/page.tsx', + 'packages/shared/src/index.ts', + 'package.json', + 'bun.lock', + 'tsconfig.base.json', + ]) { + expect(isWebPreviewFile({ filename, status: 'modified' })).toBe(true); + } }); test('excludes unrelated applications and documentation', () => { - expect(isWebPreviewFile('apps/realtime/src/index.ts')).toBe(false); - expect(isWebPreviewFile('docs/README.md')).toBe(false); + expect(isWebPreviewFile({ filename: 'apps/realtime/src/index.ts', status: 'modified' })).toBe( + false, + ); + expect(isWebPreviewFile({ filename: 'docs/README.md', status: 'modified' })).toBe(false); + }); + + test.each([ + [ + 'into the web application', + { + filename: 'apps/web/src/feature.ts', + status: 'renamed', + previous_filename: 'docs/feature.ts', + }, + ], + [ + 'out of the web application', + { + filename: 'docs/feature.ts', + status: 'renamed', + previous_filename: 'apps/web/src/feature.ts', + }, + ], + [ + 'out of a shared package', + { + filename: 'docs/shared.ts', + status: 'renamed', + previous_filename: 'packages/shared/src/shared.ts', + }, + ], + ] satisfies readonly (readonly [string, GithubPreviewFile])[])( + 'treats a rename %s as web-impacting', + (_name, file) => { + expect(isWebPreviewFile(file)).toBe(true); + }, + ); + + test('ignores an unrelated rename', () => { + expect( + isWebPreviewFile({ + filename: 'docs/new-name.md', + status: 'renamed', + previous_filename: 'docs/old-name.md', + }), + ).toBe(false); }); }); diff --git a/scripts/vercel-preview-policy.ts b/scripts/vercel-preview-policy.ts index 31413eb7d..66f36342b 100644 --- a/scripts/vercel-preview-policy.ts +++ b/scripts/vercel-preview-policy.ts @@ -1,4 +1,5 @@ import type { + GithubPreviewFile, GithubPreviewPullRequest, VercelDeployment, } from '../packages/shared/src/validators/index.ts'; @@ -16,7 +17,7 @@ export function isSameRepositoryPullRequest(pullRequest: GithubPreviewPullReques return pullRequest.head.repo.id === pullRequest.base.repo.id; } -export function isWebPreviewFile(filename: string): boolean { +function isWebPreviewPath(filename: string): boolean { return ( filename.startsWith('apps/web/') || filename.startsWith('packages/') || @@ -26,6 +27,13 @@ export function isWebPreviewFile(filename: string): boolean { ); } +export function isWebPreviewFile(file: GithubPreviewFile): boolean { + return ( + isWebPreviewPath(file.filename) || + (file.status === 'renamed' && isWebPreviewPath(file.previous_filename)) + ); +} + export function isActiveVercelDeployment(deployment: VercelDeployment): boolean { return ['QUEUED', 'INITIALIZING', 'BUILDING'].includes(deployment.readyState); } From 5adec4bd97ebc726f41cf6c1c03ae6d5c370af5b Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Fri, 21 Aug 2026 19:14:29 +0530 Subject: [PATCH 17/19] chore(ci): apply preview review cleanups --- ...26-08-21-vercel-preview-deployment-gate.md | 56 +++++++++---------- .../shared/src/validators/vercel-preview.ts | 2 +- scripts/vercel-preview-config.test.ts | 2 +- scripts/vercel-preview-deploy.test.ts | 7 +-- scripts/vercel-preview-deploy.ts | 9 +-- 5 files changed, 32 insertions(+), 44 deletions(-) diff --git a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md index 3e75464d6..ad8b14474 100644 --- a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md +++ b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md @@ -66,7 +66,7 @@ - Produces `PREVIEW_LABEL`, `NO_PREVIEW_LABEL`, `isPreviewEligible`, `isSameRepositoryPullRequest`, `isWebPreviewFile`, `isActiveVercelDeployment`, `isReadyVercelDeployment`, and `matchesVercelPullRequest`. - The controller in Task 2 consumes every interface above and does not define a second copy of validation or policy. -- [ ] **Step 1: Write failing validator tests** +- [x] **Step 1: Write failing validator tests** Create fixtures with a 40-character lowercase SHA and assert the schemas accept a complete open pull request, state event including `closed`, successful CI workflow event, repository-dispatch recovery input, GitHub file page, canonical workflow response, workflow-run page with `total_count`, live Git ref response, Vercel deployment page with `pagination.count`, create response, detail response, cancel response, and complete environment. Add rejection cases for a short SHA, missing repository ID, missing draft state, unknown Vercel ready state, nonnumeric recovery input, invalid pagination, and missing secrets. Workflow-run linked pull requests use GitHub's minimal repository shape with ID and name rather than the full repository shape with `owner.login`. Vercel list items require `projectId` but accept missing metadata as an empty record; mutation and detail responses require `id`, `projectId`, a nonempty URL, target, state, and metadata. @@ -86,13 +86,13 @@ expect(() => ).toThrow(); ``` -- [ ] **Step 2: Run the validator test and confirm failure** +- [x] **Step 2: Run the validator test and confirm failure** Run: `bun test packages/shared/tests/validators/vercel-preview.test.ts` Expected: FAIL because `../../src/validators/vercel-preview.ts` does not exist. -- [ ] **Step 3: Implement the external schemas** +- [x] **Step 3: Implement the external schemas** Use strict discriminants for known event, file, and deployment states, a reusable 40-character lowercase SHA, positive integer identifiers, bounded nonempty strings, and `.passthrough()` only where GitHub or Vercel legitimately returns unrelated fields. A renamed file requires a bounded `previous_filename`; other documented file statuses retain an optional validated value. Deployment IDs use the bounded `^[A-Za-z0-9_-]+$` grammar without trimming. The pull request schema must include this complete decision surface: @@ -116,13 +116,13 @@ export const githubPreviewPullRequestSchema = z.object({ The workflow-run schema must retain run `id`, `workflow_id`, `name`, `event`, `head_sha`, `status`, `conclusion`, creation time, and linked pull request head and base identity; the page retains nonnegative `total_count`. The live Git ref schema retains `object.sha`. Vercel metadata accepts string, number, boolean, or null values. Vercel pagination retains nonnegative `count` and finite nonnegative `next` and `prev` cursors or null. Export all inferred types and add `export * from './vercel-preview.ts';` to the validator index. -- [ ] **Step 4: Run the validator tests** +- [x] **Step 4: Run the validator tests** Run: `bun test packages/shared/tests/validators/vercel-preview.test.ts` Expected: PASS. -- [ ] **Step 5: Write failing pure-policy tests** +- [x] **Step 5: Write failing pure-policy tests** Cover this truth table and identity behavior: @@ -145,13 +145,13 @@ expect(isWebPreviewFile({ filename: 'docs/README.md', status: 'modified' })).toB Add rename-in, rename-out, and malformed-rename fixtures. Add deployment fixtures proving that a Preview must match repository ID, pull request number, ref, and SHA; a production deployment never matches; `QUEUED`, `INITIALIZING`, and `BUILDING` are active; only `READY` is ready. Reject path delimiters and whitespace in list, create, detail, and cancel IDs while accepting documented system and custom forms. -- [ ] **Step 6: Run the policy tests and confirm failure** +- [x] **Step 6: Run the policy tests and confirm failure** Run: `bun test scripts/vercel-preview-policy.test.ts` Expected: FAIL because `./vercel-preview-policy.ts` does not exist. -- [ ] **Step 7: Implement the pure policy** +- [x] **Step 7: Implement the pure policy** Use fixed label and path values: @@ -183,7 +183,7 @@ export function isWebPreviewFile(file: GithubPreviewFile): boolean { `matchesVercelPullRequest` receives the configured project ID, requires exact `projectId` and `target === null`, and compares normalized metadata values for `orbitGithubRepositoryId`, `orbitGithubPrNumber`, `orbitGithubHeadRef`, and, when supplied, `orbitGithubHeadSha`. Production, staging, missing target, another project, and absent metadata never match. -- [ ] **Step 8: Run Task 1 checks and commit** +- [x] **Step 8: Run Task 1 checks and commit** Run: `bun test packages/shared/tests/validators/vercel-preview.test.ts scripts/vercel-preview-policy.test.ts` @@ -208,9 +208,9 @@ Commit: `feat(ci): define preview deployment policy` - `PreviewResult` is a closed discriminated union with `kind: 'skipped' | 'created' | 'canceled'`, a pull request number, and a stable reason. `created` and each `canceled` result include deployment ID and URL; `skipped` results do not. An active poll can return a canceled result when its exact head becomes ineligible, or `stale-event` without cancellation when identity drifts. No serialized result may contain either token. Candidates and canceled results are sorted for deterministic output. - One monotonic 23-minute controller deadline bounds every request, retry, pagination loop, observation, poll, and sleep beneath the workflow's 25-minute timeout. -Use this closed reason vocabulary: `event-not-actionable`, `workflow-run-unassociated`, `stale-event`, `repository-mismatch`, `fork-pull-request`, `base-mismatch`, `preview-ineligible`, `no-active-deployment`, `web-unaffected`, `ci-unavailable`, `ci-not-current`, `ci-not-green`, `ready-deployment-reused`, `active-deployment-reused`, `created-ready`, and `canceled-active`. Configuration, malformed external data, incomplete pagination, transport failure, identity drift, and terminal build failure throw redacted errors rather than returning a skipped result. +Use this closed reason vocabulary: `event-not-actionable`, `workflow-run-unassociated`, `stale-event`, `repository-mismatch`, `fork-pull-request`, `base-mismatch`, `preview-ineligible`, `no-active-deployment`, `web-unaffected`, `ci-unavailable`, `ci-not-current`, `ci-not-green`, `ready-deployment-reused`, `active-deployment-reused`, `created-ready`, and `canceled-active`. Stale candidate identity and pre-creation pull request state changes return `stale-event`. Configuration, malformed external data, incomplete pagination, transport failure, deployment-response identity drift, and terminal build failure throw redacted errors rather than returning a skipped result. -- [ ] **Step 1: Write failing event and eligibility tests** +- [x] **Step 1: Write failing event and eligibility tests** Use an injected fetch router that records method, URL, headers, and parsed body. Cover: @@ -251,13 +251,13 @@ expect(createRequest.body).toEqual({ expect(createRequest.body).not.toHaveProperty('target'); ``` -- [ ] **Step 2: Run the focused tests and confirm failure** +- [x] **Step 2: Run the focused tests and confirm failure** Run: `bun test scripts/vercel-preview-deploy.test.ts --test-name-pattern 'event|eligibility|creates'` Expected: FAIL because `./vercel-preview-deploy.ts` does not exist. -- [ ] **Step 3: Implement event resolution and GitHub reconciliation** +- [x] **Step 3: Implement event resolution and GitHub reconciliation** Parse `GITHUB_EVENT_NAME`, read `GITHUB_EVENT_PATH`, and select the matching event schema. Resolve candidates as follows: @@ -278,7 +278,7 @@ For every eligible candidate, resolve `/repos/{owner}/{repo}/actions/workflows/c Query pull request files with `per_page=100&page=N`. Return true as soon as an `isWebPreviewFile` match appears against either the current path or a renamed file's validated previous path. A short page proves no relevant path; a full page at the 30-page cap fails closed. Immediately before a create or cancel mutation, refetch the live pull request. Before create, repeat the live-main and latest-CI proof, then refetch the pull request once more after that proof. Abort the mutation when identity, head, state, labels, base, or CI changed during reconciliation. -- [ ] **Step 4: Write failing idempotency and cancellation tests** +- [x] **Step 4: Write failing idempotency and cancellation tests** Cover Vercel pages with an exact deployment on page 2 and prove: @@ -294,13 +294,13 @@ Cover Vercel pages with an exact deployment on page 2 and prove: - A cancel 400 or ambiguous response reads detail once and accepts a now-terminal state without retrying PATCH; an active or identity-drifted detail fails. - A second reconciliation after creation sees the created metadata and is a no-op. -- [ ] **Step 5: Run the idempotency tests and confirm failure** +- [x] **Step 5: Run the idempotency tests and confirm failure** Run: `bun test scripts/vercel-preview-deploy.test.ts --test-name-pattern 'existing|cancel|duplicate|pagination'` Expected: FAIL because deployment listing, matching, and cancellation are not implemented. -- [ ] **Step 6: Implement bounded Vercel reconciliation** +- [x] **Step 6: Implement bounded Vercel reconciliation** List `/v7/deployments` with `teamId`, `projectId`, `branch`, `sha` where appropriate, and `limit=100`. The current endpoint has no documented metadata query. Follow validated `pagination.next` through `until`, preserve every original filter, reject repeated cursors including zero, and fail when a non-null cursor remains at the finite page cap. Filter again in trusted code with exact project, null target, and `matchesVercelPullRequest`. Parse list identifiers from `uid`; create, detail, and cancel responses use `id`. Accept `url: null` only on list items. @@ -312,13 +312,13 @@ An ambiguous create outcome is a network error, timeout, 429, 5xx, 409, or succe For current ineligible state, list with `teamId`, `projectId`, `branch`, and `limit=100` without SHA, then call `PATCH /v12/deployments/{id}/cancel?teamId={teamId}` only for locally matched active Preview deployments. Refetch and revalidate live pull request identity and ineligibility immediately before every individual PATCH. Never retry PATCH. Validate a successful cancel as the requested ID, project, null target, Orbit metadata, and `CANCELED`, then emit one `canceled-active` result. After a 400 or ambiguous PATCH, read v13 detail once; accept `CANCELED`, `READY`, or another terminal state as a completed race with no active spend, and fail if the deployment remains active or its identity drifted. Race-only reconciliation emits `no-active-deployment`; mixed reconciliation emits results only for deployments actually canceled. A CI failure does not cancel a previously ready Preview; cancellation is driven by current pull request state. -- [ ] **Step 7: Write failing transport and secret-safety tests** +- [x] **Step 7: Write failing transport and secret-safety tests** Cover missing configuration, 401, ordinary 403, rate-limited GitHub 403, 429 with delta-seconds and HTTP-date `Retry-After`, 500, network failure, body-read failure, a real injected timeout abort, redirects, invalid JSON, invalid schema, endpoint-specific exhausted or repeated pagination, polling timeout, every terminal build state, and malformed create/detail/cancel responses. Assert safe GET requests use at most three total attempts; ordinary 401/403 do not retry; explicit GitHub rate-limit 403, 429, 5xx, network failure, body-read failure, and timeout may retry within the same cap; and excessive waits fail rather than sleeping without bound. Assert network, timeout, 429, 500, invalid-success body, and 409 create outcomes each perform one POST total and enter the three-attempt exact-list observation. Active or ready visibility is reused; terminal or absent visibility fails. Assert definitive 400, 401, 403, and 422 create responses send no reconciliation retry and no second POST. Cover cancel 400 plus ambiguous PATCH network, 429, 5xx, and invalid-success outcomes with one PATCH and one detail reconciliation. Assert exact mutation, observation, detail, and bounded-sleep counts. No error body, thrown error, log line, URL, request summary, external deployment ID, or serialized result may contain either token. -- [ ] **Step 8: Implement the bounded JSON client and CLI entry point** +- [x] **Step 8: Implement the bounded JSON client and CLI entry point** The JSON client applies a fresh 15-second AbortController per request through the injected timer scheduler and clears its timer after body consumption. It rejects redirects and sends a fixed `User-Agent`; GitHub requests also send `Accept: application/vnd.github+json` and `X-GitHub-Api-Version: 2022-11-28`. Parse response text as JSON, validate with the supplied shared schema, and throw only a token-redacted bounded error. Never include headers or raw external bodies in errors. Validate every deployment ID with the shared safe grammar, check it against both tokens before use, and encode every dynamic Vercel deployment path segment. @@ -326,7 +326,7 @@ Safe GET reads retry network errors, per-attempt timeout, 429, 5xx, and GitHub 4 The executable path uses `Bun.file(path).text()`, global `fetch`, a timer-backed sleep, and `console.log`. Guard it with `if (import.meta.main)`, set `process.exitCode = 1` on error, and print only the redacted error message. -- [ ] **Step 9: Run Task 2 checks and commit** +- [x] **Step 9: Run Task 2 checks and commit** Run: `bun test scripts/vercel-preview-deploy.test.ts` @@ -357,7 +357,7 @@ Commit: `feat(ci): deploy previews after successful checks` - Uses immutable `actions/checkout` SHA `3d3c42e5aac5ba805825da76410c181273ba90b1` and immutable `oven-sh/setup-bun` SHA `0c5077e51419868618aeaa5fe8019c62421857d6`. - Produces managed labels named exactly `preview` and `no-preview`. -- [ ] **Step 1: Write failing repository configuration tests** +- [x] **Step 1: Write failing repository configuration tests** Parse `apps/web/vercel.json`, read the workflow and docs as text, and import `LABELS`. Assert the branch map and removal checks below, then use anchored assertions over the intentionally fixed workflow structure rather than global word searches: @@ -371,15 +371,15 @@ expect(allGateFiles).not.toContain('BUILD_GATE_GITHUB_TOKEN'); Require the exact top-level read-only permissions; only the three intended triggers and their exact activity types; no `workflow_dispatch`; the workflow-run job guard for a `pull_request` source and `success` conclusion; `timeout-minutes: 25`; and `cancel-in-progress: false`. Require the exact immutable checkout and setup-Bun SHAs, checkout of `${{ github.repository }}` at `${{ github.sha }}`, disabled persisted credentials, submodules, and LFS, Bun 1.3.14, and `bun install --frozen-lockfile --ignore-scripts`. Prove no cache or artifact-download action exists, no pull request head/ref expression can select checkout code, and the controller is the only run step receiving `VERCEL_TOKEN`. -Assert the `static` job, rather than merely the whole CI file, contains named `bun run check-bytes` and `bun run check-bun-imports` steps. Assert each managed label description matches its executable semantics: `preview` enables an otherwise eligible draft after CI, while `no-preview` suppresses creation and cancels active Preview work. Also test the `**` rule against `feature`, `feature/preview`, and `codex/review/pr341` using the same minimatch semantics documented by Vercel. Do not add a dependency: implement the narrow expected assertion by checking that the configured key is exactly `**` and enumerate the branch examples in the test name. +Assert the `static` job, rather than merely the whole CI file, contains named `bun run check-bytes` and `bun run check-bun-imports` steps. Assert each managed label description matches its executable semantics: `preview` enables an otherwise eligible draft after CI, while `no-preview` suppresses creation and cancels active Preview work. Assert the deployment map is exactly `{ '**': false, main: true }`, with the wildcard rule before the `main` override, without claiming that the test executes Vercel's glob matcher. -- [ ] **Step 2: Run the configuration test and confirm failure** +- [x] **Step 2: Run the configuration test and confirm failure** Run: `bun test scripts/vercel-preview-config.test.ts` Expected: FAIL because the workflow and managed labels do not exist and `ignoreCommand` remains. -- [ ] **Step 3: Replace the Vercel gate and manage labels** +- [x] **Step 3: Replace the Vercel gate and manage labels** Remove `ignoreCommand` from `apps/web/vercel.json` and add: @@ -396,7 +396,7 @@ Add `preview` and `no-preview` beside the status labels in `scripts/labels.ts`, Add named `Source byte policy` and `No Bun built-ins in shipped server code` steps to the `static` job in `.github/workflows/ci.yml`, invoking `bun run check-bytes` and `bun run check-bun-imports`. This makes the canonical `CI` success used by the dispatcher cover every non-test verification command from `bun run verify`. -- [ ] **Step 4: Add the default-branch workflow** +- [x] **Step 4: Add the default-branch workflow** Create a workflow named `Vercel Preview` with this trigger and trust boundary: @@ -425,7 +425,7 @@ Keep `cancel-in-progress: false`: canceling a controller after an ambiguous Crea The controller rejects a workflow run linked to more than one distinct pull request, so the first linked number is used only for an event that resolves to one PR. Recovery requires a positive numeric `client_payload.pull_request`, which shares the same per-PR group as state and CI events; malformed recovery input may form an unused group but is rejected before any Vercel call. The job condition allows state events and repository dispatch, and allows a workflow run only when `github.event.workflow_run.event == 'pull_request'` and its conclusion is success. Set `timeout-minutes: 25`; the controller's own 23-minute deadline remains the primary bound. GitHub documents that `repository_dispatch` uses the last commit on the default branch, unlike `workflow_dispatch`, which can run a workflow version from a selected non-default ref. Checkout the trusted default-branch workflow commit using the pinned checkout action, `repository: ${{ github.repository }}`, `ref: ${{ github.sha }}`, `persist-credentials: false`, `submodules: false`, and `lfs: false`. Set up Bun 1.3.14 with the pinned setup action, run `bun install --frozen-lockfile --ignore-scripts`, then run `bun scripts/vercel-preview-deploy.ts` with tokens and settings scoped only to that controller step through `env`. -- [ ] **Step 5: Rewrite the operations guide and documentation index** +- [x] **Step 5: Rewrite the operations guide and documentation index** Document the exact eligibility table, CI-green timing, same-repository restriction, active cancellation including closed pull requests, web path list, Vercel API behavior, GitHub secret and variables, label synchronization, Git Fork Protection, the deployment-count caveat, the repository-controlled cost-policy limitation, and removal of all four old `BUILD_GATE_*` Vercel values. State precisely that only the trusted GitHub controller is isolated from pull request code: the API-created Vercel Preview still builds same-repository pull request code with the project's Preview environment scope. Manual recovery uses a maintainer-authenticated `repository_dispatch` named `vercel-preview-reconcile` with numeric `client_payload.pull_request`; explicitly forbid `workflow_dispatch` because a caller can select a non-default ref. Link the guide from `docs/README.md` under the contributor/operations entries. @@ -433,7 +433,7 @@ Make the post-merge canary an ordered procedure: confirm Git Fork Protection and Do not claim that ignored builds are free, that Ready alone is trust, that fork previews are automatic, or that the privileged workflow can be exercised before it exists on `main`. -- [ ] **Step 6: Run Task 3 checks and commit** +- [x] **Step 6: Run Task 3 checks and commit** Run: `bun test scripts/vercel-preview-config.test.ts scripts/vercel-preview-policy.test.ts scripts/vercel-preview-deploy.test.ts packages/shared/tests/validators/vercel-preview.test.ts` @@ -461,7 +461,7 @@ Commit: `chore(vercel): gate previews after CI` - Consumes the complete branch from Tasks 1 through 3. - Produces a branch merged with current `origin/main`, a green local verification record, an updated remote branch, an accurate PR body, and resolved addressed threads. -- [ ] **Step 1: Merge the latest main and rerun focused tests** +- [x] **Step 1: Merge the latest main and rerun focused tests** Run: `git fetch origin main && git merge --no-edit origin/main` @@ -479,7 +479,7 @@ Run: `ORBIT_TEST_LANE=preview-gate bun run verify` Expected: lint, comment policy, source-byte check, Bun-import check, dependency dedupe, all typechecks, and all tests PASS. If a database service is unavailable, start the repository's existing infrastructure and initialize only the isolated `preview-gate` test lane before rerunning. -- [ ] **Step 3: Review the final diff and operational text** +- [x] **Step 3: Review the final diff and operational text** Run: `git diff --check origin/main...HEAD` @@ -489,7 +489,7 @@ Run the attribution and em-dash scan against every changed text file. Run the fo Expected: no whitespace error, prohibited attribution, em dash, or old token/config reference in changed files except the operations guide's explicit removal instructions for the four legacy setting names. -- [ ] **Step 4: Push and update pull request 341** +- [x] **Step 4: Push and update pull request 341** Push `chore/gate-preview-builds` after all local checks pass. Replace the PR body with the final motivation, architecture, security boundary, setup requirements, test evidence, and the honest post-merge canary limitation. Remove the stale test count and all attribution. diff --git a/packages/shared/src/validators/vercel-preview.ts b/packages/shared/src/validators/vercel-preview.ts index 91ec03018..1bea87394 100644 --- a/packages/shared/src/validators/vercel-preview.ts +++ b/packages/shared/src/validators/vercel-preview.ts @@ -162,7 +162,7 @@ const githubPreviewWorkflowRunSchema = z head_sha: gitShaSchema, status: githubWorkflowRunStatusSchema, conclusion: githubWorkflowConclusionSchema, - created_at: z.string().datetime({ offset: true }), + created_at: z.iso.datetime({ offset: true }), pull_requests: z.array(githubPreviewWorkflowPullRequestSchema).max(100), }) .passthrough(); diff --git a/scripts/vercel-preview-config.test.ts b/scripts/vercel-preview-config.test.ts index 2b32b3ec8..1ae25dfa5 100644 --- a/scripts/vercel-preview-config.test.ts +++ b/scripts/vercel-preview-config.test.ts @@ -192,7 +192,7 @@ const continuedControllerWorkflow = workflow.replace( ); describe('Vercel Preview repository configuration', () => { - test('disables automatic deployment for feature, feature/preview, and codex/review/pr341 while allowing main', () => { + test('configures automatic Git deployments only for main without an ignore command', () => { expect(vercel.git?.deploymentEnabled).toEqual({ '**': false, main: true }); expect(Object.keys(vercel.git?.deploymentEnabled ?? {})).toEqual(['**', 'main']); expect(vercel).not.toHaveProperty('ignoreCommand'); diff --git a/scripts/vercel-preview-deploy.test.ts b/scripts/vercel-preview-deploy.test.ts index 71f525ee2..003e90ab5 100644 --- a/scripts/vercel-preview-deploy.test.ts +++ b/scripts/vercel-preview-deploy.test.ts @@ -1638,12 +1638,7 @@ describe('identity invariants and bounded edge cases', () => { respond: (request) => { if (!request.url.endsWith('/pulls/341')) return undefined; pullReads += 1; - const matching = harness.requests.at(-1); - const signalRequest = matching; - if (signalRequest) { - const requestSignal = request.headers; - expect(requestSignal.get('authorization')).toBe(`Bearer ${GITHUB_TOKEN}`); - } + expect(request.headers.get('authorization')).toBe(`Bearer ${GITHUB_TOKEN}`); return pullReads === 1 ? json({ message: 'temporary' }, 500) : json(pullRequest()); }, }); diff --git a/scripts/vercel-preview-deploy.ts b/scripts/vercel-preview-deploy.ts index 90c1f0a1f..7dabcfa21 100644 --- a/scripts/vercel-preview-deploy.ts +++ b/scripts/vercel-preview-deploy.ts @@ -651,14 +651,7 @@ function detailIdentityMatches( expectedMetadata: Readonly>, headSha?: string, ): boolean { - const comparable: VercelDeployment = { - uid: detail.id, - projectId: detail.projectId, - url: detail.url, - target: detail.target, - readyState: detail.readyState, - meta: detail.meta, - }; + const comparable = comparableDeployment(detail); return ( detail.id === id && ORBIT_METADATA_KEYS.every((key) => { From 618ecb672d525505eb11ee126c795290470ff55d Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Mon, 24 Aug 2026 00:51:01 +0530 Subject: [PATCH 18/19] fix(ci): cancel superseded preview deployments --- .github/workflows/vercel-preview.yml | 2 +- docs/VERCEL_BUILD_GATE.md | 28 +- ...26-08-21-vercel-preview-deployment-gate.md | 17 +- ...1-vercel-preview-deployment-gate-design.md | 25 +- .../shared/src/validators/vercel-preview.ts | 10 + .../tests/validators/vercel-preview.test.ts | 19 ++ scripts/vercel-preview-config.test.ts | 2 +- scripts/vercel-preview-deploy.test.ts | 283 +++++++++++++++++- scripts/vercel-preview-deploy.ts | 195 ++++++++++-- 9 files changed, 513 insertions(+), 68 deletions(-) diff --git a/.github/workflows/vercel-preview.yml b/.github/workflows/vercel-preview.yml index a88518f87..004a66487 100644 --- a/.github/workflows/vercel-preview.yml +++ b/.github/workflows/vercel-preview.yml @@ -3,7 +3,7 @@ name: Vercel Preview on: pull_request_target: branches: [main] - types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled, closed] + types: [opened, reopened, synchronize, ready_for_review, converted_to_draft, labeled, unlabeled, closed] workflow_run: workflows: [CI] types: [completed] diff --git a/docs/VERCEL_BUILD_GATE.md b/docs/VERCEL_BUILD_GATE.md index d3ff20a9d..eaa19f791 100644 --- a/docs/VERCEL_BUILD_GATE.md +++ b/docs/VERCEL_BUILD_GATE.md @@ -73,12 +73,16 @@ canceled attempt is free. The controller uses one deployment path: -1. Vercel v7 lists Preview deployments by team, project, branch, and, when - creating, exact head SHA. -2. Vercel v13 creates or reads a deployment with the same-repository GitHub +1. Vercel v7 lists Preview deployments by team, project, and branch before CI + to find active work from prior heads. It lists by exact head SHA again when + creating or reusing current-head work. +2. Vercel v9 reads the configured project immediately before every mutation. + The validated project ID, name, and account ID must match the configured + project and team. +3. Vercel v13 creates or reads a deployment with the same-repository GitHub repository ID, head ref, exact head SHA, and Orbit metadata. It omits a target so Vercel uses the project's Preview environment. -3. Vercel v12 cancels matching active deployments. +4. Vercel v12 cancels matching active deployments. Deployment IDs are accepted only when they contain ASCII letters, digits, underscores, and hyphens within the controller's fixed bound. The controller @@ -95,9 +99,19 @@ Per-pull-request workflow runs remain serialized with in-progress cancellation disabled. While the owner polls a queued, initializing, or building deployment, it refetches the current pull request after every active detail response. If the same exact head becomes closed or ineligible, that owner cancels only its exact -deployment and returns a canceled result. If the head or repository identity -changed, the old owner stops without canceling the different head. The queued -state event then reconciles the latest state. +deployment and returns a canceled result. If only the head SHA changed while the +pull request, repositories, base, and head ref remain identical, the old owner +cancels its superseded active deployment. Any other identity change returns a +stale event without mutation. + +`pull_request_target` includes `synchronize`, so a pushed head immediately runs +the trusted controller. Every trusted current-candidate reconciliation also +sweeps active prior-head work before checking current-head CI. A candidate for +cancellation must have a different valid 40-character hexadecimal SHA and exact +configured project, repository ID, pull request number, and head ref metadata. +This repeated sweep lets later CI and repository-dispatch events recover after a +transient synchronize failure or an interrupted polling owner. READY prior-head +deployments are retained, so their URLs remain available. Events for stale heads cannot create or cancel work for the current head. An existing exact ready or active deployment is reused. Terminal deployment diff --git a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md index ad8b14474..06eaff3c9 100644 --- a/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md +++ b/docs/superpowers/plans/2026-08-21-vercel-preview-deployment-gate.md @@ -208,7 +208,7 @@ Commit: `feat(ci): define preview deployment policy` - `PreviewResult` is a closed discriminated union with `kind: 'skipped' | 'created' | 'canceled'`, a pull request number, and a stable reason. `created` and each `canceled` result include deployment ID and URL; `skipped` results do not. An active poll can return a canceled result when its exact head becomes ineligible, or `stale-event` without cancellation when identity drifts. No serialized result may contain either token. Candidates and canceled results are sorted for deterministic output. - One monotonic 23-minute controller deadline bounds every request, retry, pagination loop, observation, poll, and sleep beneath the workflow's 25-minute timeout. -Use this closed reason vocabulary: `event-not-actionable`, `workflow-run-unassociated`, `stale-event`, `repository-mismatch`, `fork-pull-request`, `base-mismatch`, `preview-ineligible`, `no-active-deployment`, `web-unaffected`, `ci-unavailable`, `ci-not-current`, `ci-not-green`, `ready-deployment-reused`, `active-deployment-reused`, `created-ready`, and `canceled-active`. Stale candidate identity and pre-creation pull request state changes return `stale-event`. Configuration, malformed external data, incomplete pagination, transport failure, deployment-response identity drift, and terminal build failure throw redacted errors rather than returning a skipped result. +Use this closed reason vocabulary: `event-not-actionable`, `workflow-run-unassociated`, `stale-event`, `repository-mismatch`, `fork-pull-request`, `base-mismatch`, `preview-eligible`, `no-active-deployment`, `web-unaffected`, `ci-unavailable`, `ci-not-current`, `ci-not-green`, `ready-deployment-reused`, `active-deployment-reused`, `created-ready`, and `canceled-active`. Stale candidate identity and pre-creation pull request state changes return `stale-event`. A cancellation abandoned because the live pull request became eligible returns `preview-eligible`. Configuration, malformed external data, incomplete pagination, transport failure, deployment-response identity drift, and terminal build failure throw redacted errors rather than returning a skipped result. - [x] **Step 1: Write failing event and eligibility tests** @@ -268,7 +268,7 @@ type PreviewCandidate = { }; ``` -- `pull_request_target`: one candidate from the matching event and embedded PR number plus event head SHA; `closed` is an accepted cancellation transition. +- `pull_request_target`: one candidate from the matching event and embedded PR number plus event head SHA; `closed` and `synchronize` are accepted transitions. - `workflow_run`: no candidates unless the workflow is `CI`, source event is `pull_request`, conclusion is `success`, action is `completed`, and the payload links exactly one distinct pull request; duplicate links to that same number are deduplicated. Empty or ambiguous linked lists log a stable reason and return no candidates. - `repository_dispatch`: one candidate from the validated positive integer `client_payload.pull_request` and no expected SHA. @@ -288,7 +288,8 @@ Cover Vercel pages with an exact deployment on page 2 and prove: - A list item without metadata is ignored without rejecting its page. - An ineligible current state cancels every matching active Preview for that PR and does not cancel ready, canceled, errored, staging, production, another project, ref, or repository. - An existing active deployment is polled instead of duplicated; an existing ready deployment is reused. -- A created or reused active deployment is canceled by its polling owner when the same exact head becomes ineligible before `READY`; a different or newer head is never canceled by the old owner. +- A created or reused active deployment is canceled by its polling owner when the same exact head becomes ineligible before `READY`. When only the SHA changes for the same head ref and pull request identity, the owner cancels its superseded active deployment. Other identity changes return `stale-event` without cancellation. +- Every trusted eligible reconciliation cancels only validated active prior-head deployments with a different valid SHA and exact project, repository, pull request, and head ref metadata. READY prior-head deployments remain available. - Exact READY wins over duplicate active and terminal items; exact active wins over terminal items. - Successful create, detail, and cancel responses must retain requested ID, project, null target, and Orbit metadata. Cancel must return `CANCELED`. - A cancel 400 or ambiguous response reads detail once and accepts a now-terminal state without retrying PATCH; an active or identity-drifted detail fails. @@ -304,10 +305,14 @@ Expected: FAIL because deployment listing, matching, and cancellation are not im List `/v7/deployments` with `teamId`, `projectId`, `branch`, `sha` where appropriate, and `limit=100`. The current endpoint has no documented metadata query. Follow validated `pagination.next` through `until`, preserve every original filter, reject repeated cursors including zero, and fail when a non-null cursor remains at the finite page cap. Filter again in trusted code with exact project, null target, and `matchesVercelPullRequest`. Parse list identifiers from `uid`; create, detail, and cancel responses use `id`. Accept `url: null` only on list items. -For eligible PRs, prefer any exact ready deployment, then any exact active deployment, then terminal history. Fetch detail when a READY list item has no URL. Poll an active deployment immediately, then allow at most 240 five-second sleeps followed by a final GET. After every active detail response, refetch live pull request identity and eligibility. If the same exact head is now closed or ineligible, cancel only that exact deployment and return its canceled result. If head or repository identity drifted, return `stale-event` without canceling. Keep per-PR serialization and `cancel-in-progress: false`. The 23-minute controller deadline may stop this sequence earlier. Require ID, project, null target, and Orbit metadata on every detail. `READY` with a nonempty URL succeeds; `ERROR`, `CANCELED`, `BLOCKED`, or `DELETED` fails; an active final response times out. +Before current-head CI, list the branch without SHA and cancel active prior-head deployments only when their metadata carries a different valid 40-character hexadecimal SHA and exact project, repository ID, pull request number, and head ref. Run this sweep for every trusted current candidate so `workflow_run` and `repository_dispatch` repair a missed `synchronize` cleanup. Leave every non-active state, including READY, unchanged. + +For eligible PRs, prefer any exact ready deployment, then any exact active deployment, then terminal history. Fetch detail when a READY list item has no URL. Poll an active deployment immediately, then allow at most 240 five-second sleeps followed by a final GET. After every active detail response, refetch live pull request identity and eligibility. If the same exact head is now closed or ineligible, cancel only that exact deployment and return its canceled result. If only the SHA changed for the same pull request identity and head ref, cancel the superseded active deployment. Other identity drift returns `stale-event` without canceling. Keep per-PR serialization and `cancel-in-progress: false`. The 23-minute controller deadline may stop this sequence earlier. Require ID, project, null target, and Orbit metadata on every detail. `READY` with a nonempty URL succeeds; `ERROR`, `CANCELED`, `BLOCKED`, or `DELETED` fails; an active final response times out. When no ready or active exact deployment exists, create one deployment with `POST /v13/deployments?teamId={teamId}`, omitted `target`, exact Git source, and the metadata shown in Step 1. Add `forceNew=1` only when the complete pre-create list already contained an exact terminal deployment. Record all pre-create deployment IDs and enforce one POST per reconciliation. +Immediately before every Create or Cancel mutation, read `/v9/projects/{projectId}?teamId={teamId}` and validate the response. Require its project ID, project name, and account ID to equal the configured project ID, project name, and team ID. Repeat this proof at each mutation boundary so a long poll cannot rely on stale project ownership. + An ambiguous create outcome is a network error, timeout, 429, 5xx, 409, or successful response that cannot be parsed, validated, or matched to the requested identity. Ordinary 4xx responses are definitive. After ambiguity, set `createAttempted` and make a second POST impossible. Run three exact-list observation attempts separated by two seconds. Reuse only a newly visible ready or active ID that was not in the pre-create set. A new terminal deployment or no new exact ID fails visibly. A later event starts a new reconciliation from a complete list and may decide independently. For current ineligible state, list with `teamId`, `projectId`, `branch`, and `limit=100` without SHA, then call `PATCH /v12/deployments/{id}/cancel?teamId={teamId}` only for locally matched active Preview deployments. Refetch and revalidate live pull request identity and ineligibility immediately before every individual PATCH. Never retry PATCH. Validate a successful cancel as the requested ID, project, null target, Orbit metadata, and `CANCELED`, then emit one `canceled-active` result. After a 400 or ambiguous PATCH, read v13 detail once; accept `CANCELED`, `READY`, or another terminal state as a completed race with no active spend, and fail if the deployment remains active or its identity drifted. Race-only reconciliation emits `no-active-deployment`; mixed reconciliation emits results only for deployments actually canceled. A CI failure does not cancel a previously ready Preview; cancellation is driven by current pull request state. @@ -404,7 +409,7 @@ Create a workflow named `Vercel Preview` with this trigger and trust boundary: on: pull_request_target: branches: [main] - types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled, closed] + types: [opened, reopened, synchronize, ready_for_review, converted_to_draft, labeled, unlabeled, closed] workflow_run: workflows: [CI] types: [completed] @@ -421,7 +426,7 @@ concurrency: cancel-in-progress: false ``` -Keep `cancel-in-progress: false`: canceling a controller after an ambiguous Create could lose its read-only observation and permit a later duplicate attempt. Timely ineligibility is handled inside the polling owner, which refetches exact live identity and eligibility after every active detail response. +Keep `cancel-in-progress: false`: canceling a controller after an ambiguous Create could lose its read-only observation and permit a later duplicate attempt. Timely ineligibility and same-ref head supersession are handled inside the polling owner, which refetches exact live identity and eligibility after every active detail response. The controller rejects a workflow run linked to more than one distinct pull request, so the first linked number is used only for an event that resolves to one PR. Recovery requires a positive numeric `client_payload.pull_request`, which shares the same per-PR group as state and CI events; malformed recovery input may form an unused group but is rejected before any Vercel call. The job condition allows state events and repository dispatch, and allows a workflow run only when `github.event.workflow_run.event == 'pull_request'` and its conclusion is success. Set `timeout-minutes: 25`; the controller's own 23-minute deadline remains the primary bound. GitHub documents that `repository_dispatch` uses the last commit on the default branch, unlike `workflow_dispatch`, which can run a workflow version from a selected non-default ref. Checkout the trusted default-branch workflow commit using the pinned checkout action, `repository: ${{ github.repository }}`, `ref: ${{ github.sha }}`, `persist-credentials: false`, `submodules: false`, and `lfs: false`. Set up Bun 1.3.14 with the pinned setup action, run `bun install --frozen-lockfile --ignore-scripts`, then run `bun scripts/vercel-preview-deploy.ts` with tokens and settings scoped only to that controller step through `env`. diff --git a/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md index 71b49451e..fa467939e 100644 --- a/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md +++ b/docs/superpowers/specs/2026-08-21-vercel-preview-deployment-gate-design.md @@ -76,7 +76,7 @@ boundary. `.github/workflows/vercel-preview.yml` runs from the default branch through three trusted event paths: -- `pull_request_target` handles `opened`, `reopened`, `ready_for_review`, +- `pull_request_target` handles `opened`, `reopened`, `synchronize`, `ready_for_review`, `converted_to_draft`, `labeled`, `unlabeled`, and `closed` state transitions. - `workflow_run` handles a completed `CI` workflow and proceeds only when the source event was a pull request and the conclusion was success. @@ -96,9 +96,10 @@ in-progress reconciler. Recovery dispatches require a bounded positive numeric p and share that pull request's group. Vercel metadata checks provide a second idempotency boundary. A polling owner refetches the current pull request between active deployment reads. It cancels only the deployment whose validated metadata names the same pull request head when that exact identity -becomes closed or ineligible. Identity or head drift ends the old poll without canceling work for a -different head. This keeps one Create POST and per-pull-request serialization while allowing timely -cost cancellation. +becomes closed or ineligible. It also cancels that deployment when only the head SHA is superseded +and the pull request, repositories, base, and head ref still match. Other identity drift ends the old +poll without mutation. This keeps one Create POST and per-pull-request serialization while allowing +timely cost cancellation. A workflow-run payload with no linked pull request, or with more than one distinct linked pull request, is rejected because one Actions concurrency group cannot serialize multiple pull requests and the event cannot prove one unambiguous base association. @@ -115,6 +116,12 @@ when its recorded head SHA no longer equals the live pull request head. Stale ev Fork heads and pull requests targeting another repository or branch are also no-ops. A proven closed pull request follows the active-cancellation path without requiring CI. +Before CI, every trusted current-candidate reconciliation lists the branch and cancels only active +prior-head deployments with a different valid 40-character hexadecimal SHA and exact project, +repository ID, pull request number, and head ref metadata. This lets `synchronize` cancel promptly +and lets later CI or recovery events repair a missed cleanup. READY prior-head deployments remain +unchanged. + For every eligible event, the controller resolves the canonical `.github/workflows/ci.yml`, fetches the live `main` ref, and queries every bounded page of runs for the exact head SHA. It selects the maximum creation time and run ID rather than filtering to successful runs. The run must be @@ -152,6 +159,10 @@ Deployment IDs must contain only ASCII letters, digits, underscores, and hyphens length bound. Every ID is checked against both tokens before use, and every dynamic deployment path segment is URL encoded. +Immediately before each Create or Cancel request, the controller reads `/v9/projects/{projectId}` +through the configured team scope. The validated project ID, name, and account ID must match the +configured project and team, including after a long deployment poll. + If an exact deployment is ready, it is reused. An exact queued, initializing, or building deployment is polled rather than duplicated. If no such deployment exists, the controller calls Vercel's Create Deployment endpoint with the linked GitHub repository ID, exact branch ref, and @@ -163,8 +174,10 @@ prove deployment ID, project ID, null Preview target, state, and Orbit metadata. the created deployment immediately and then through at most 240 five-second sleeps plus one final GET to a ready or terminal state, requiring a nonempty final URL. After every active detail, the polling owner refetches the pull request. If the same exact identity is now ineligible, it cancels -only that deployment and returns a canceled result. If the identity or head changed, it returns a -stale result without canceling. The 23-minute controller deadline may stop this sequence earlier. +only that deployment and returns a canceled result. If only the SHA changed for the same head ref +and pull request identity, it cancels the superseded active deployment. Other identity changes +return a stale result without canceling. The 23-minute controller deadline may stop this sequence +earlier. Create Deployment has no idempotency key. After a network error, timeout, 429, 5xx, 409, or an unparseable success response, the controller marks the one POST as attempted and performs only a diff --git a/packages/shared/src/validators/vercel-preview.ts b/packages/shared/src/validators/vercel-preview.ts index 1bea87394..0983fd578 100644 --- a/packages/shared/src/validators/vercel-preview.ts +++ b/packages/shared/src/validators/vercel-preview.ts @@ -101,6 +101,7 @@ export const githubPreviewPullRequestTargetEventSchema = z action: z.enum([ 'opened', 'reopened', + 'synchronize', 'ready_for_review', 'converted_to_draft', 'labeled', @@ -239,6 +240,15 @@ export const vercelDeploymentsPageSchema = z .passthrough(); export type VercelDeploymentsPage = z.infer; +export const vercelProjectSchema = z + .object({ + id: boundedString(255), + name: boundedString(100), + accountId: boundedString(255), + }) + .passthrough(); +export type VercelProject = z.infer; + const vercelDeploymentMutationSchema = z .object({ id: vercelDeploymentIdSchema, diff --git a/packages/shared/tests/validators/vercel-preview.test.ts b/packages/shared/tests/validators/vercel-preview.test.ts index 75d39c19f..2cedd2a8e 100644 --- a/packages/shared/tests/validators/vercel-preview.test.ts +++ b/packages/shared/tests/validators/vercel-preview.test.ts @@ -14,6 +14,7 @@ import { vercelDeploymentSchema, vercelDeploymentsPageSchema, vercelPreviewEnvironmentSchema, + vercelProjectSchema, } from '../../src/validators/vercel-preview.ts'; const SHA = 'a'.repeat(40); @@ -135,6 +136,17 @@ describe('Vercel Preview GitHub schemas', () => { ).toMatchObject({ action: 'closed', number: pullRequest.number }); }); + test('accept a pull request synchronize event', () => { + expect( + githubPreviewPullRequestTargetEventSchema.parse({ + action: 'synchronize', + number: pullRequest.number, + pull_request: pullRequest, + repository, + }), + ).toMatchObject({ action: 'synchronize', number: pullRequest.number }); + }); + test('accept a successful CI workflow event', () => { expect( githubPreviewWorkflowRunEventSchema.parse({ @@ -237,6 +249,13 @@ describe('Vercel Preview GitHub schemas', () => { ).toBe('dpl_preview'); }); + test('validate the Vercel project identity boundary', () => { + expect( + vercelProjectSchema.parse({ id: 'prj_orbit', name: 'orbit', accountId: 'team_orbit' }), + ).toMatchObject({ id: 'prj_orbit', name: 'orbit', accountId: 'team_orbit' }); + expect(() => vercelProjectSchema.parse({ id: 'prj_orbit', name: 'orbit' })).toThrow(); + }); + test('accept a complete controller environment', () => { expect(vercelPreviewEnvironmentSchema.parse(environment).VERCEL_PROJECT_ID).toBe('prj_orbit'); }); diff --git a/scripts/vercel-preview-config.test.ts b/scripts/vercel-preview-config.test.ts index 1ae25dfa5..b11b9b995 100644 --- a/scripts/vercel-preview-config.test.ts +++ b/scripts/vercel-preview-config.test.ts @@ -52,7 +52,7 @@ const LEGACY_EXPECTED_HEADER = `name: Vercel Preview on: pull_request_target: branches: [main] - types: [opened, reopened, ready_for_review, converted_to_draft, labeled, unlabeled, closed] + types: [opened, reopened, synchronize, ready_for_review, converted_to_draft, labeled, unlabeled, closed] workflow_run: workflows: [CI] types: [completed] diff --git a/scripts/vercel-preview-deploy.test.ts b/scripts/vercel-preview-deploy.test.ts index 003e90ab5..719e17918 100644 --- a/scripts/vercel-preview-deploy.test.ts +++ b/scripts/vercel-preview-deploy.test.ts @@ -3,6 +3,7 @@ import type { PreviewRuntime } from './vercel-preview-deploy.ts'; import { reconcileVercelPreviews } from './vercel-preview-deploy.ts'; const SHA = 'a'.repeat(40); +const NEW_SHA = 'c'.repeat(40); const MAIN_SHA = 'b'.repeat(40); const GITHUB_TOKEN = 'github-secret-token'; const VERCEL_TOKEN = 'vercel-secret-token'; @@ -124,6 +125,15 @@ function mutationDeployment( return { id, ...rest, url: 'orbit-preview.vercel.app' }; } +function vercelProject(overrides: Record = {}) { + return { + id: 'prj_orbit', + name: 'orbit', + accountId: 'team_orbit', + ...overrides, + }; +} + function json(value: unknown, status = 200, headers?: Record): Response { return new Response(JSON.stringify(value), { status, @@ -191,6 +201,7 @@ function createHarness(scenario: Scenario = {}) { const runs = scenario.workflowRuns ?? [workflowRun()]; return json({ total_count: runs.length, workflow_runs: runs }); } + if (url.includes('/v9/projects/')) return json(vercelProject()); if (url.includes('/v7/deployments')) { const deployments = scenario.deployments ?? []; return json({ @@ -264,10 +275,13 @@ describe('event and eligibility reconciliation', () => { ); expect(creates).toHaveLength(1); const createRequest = creates[0]; - expect(createRequest?.method).toBe('POST'); - expect(createRequest?.url).toContain('/v13/deployments'); - expect(createRequest?.url).not.toContain('forceNew=1'); - expect(createRequest?.body).toEqual({ + if (createRequest === undefined) { + throw new Error('Expected one Vercel deployment creation request'); + } + expect(createRequest.method).toBe('POST'); + expect(createRequest.url).toContain('/v13/deployments'); + expect(createRequest.url).not.toContain('forceNew=1'); + expect(createRequest.body).toEqual({ name: 'orbit', project: 'prj_orbit', gitSource: { type: 'github', repoId: 123, ref: 'feature/preview', sha: SHA }, @@ -280,7 +294,20 @@ describe('event and eligibility reconciliation', () => { orbitGithubWorkflowRunId: '987654321', }, }); - expect(createRequest?.body).not.toHaveProperty('target'); + expect(createRequest.body).not.toHaveProperty('target'); + const projectLookupIndex = harness.requests.findIndex( + ({ method, url }) => method === 'GET' && url.includes('/v9/projects/prj_orbit'), + ); + expect(projectLookupIndex).toBeGreaterThanOrEqual(0); + expect(projectLookupIndex).toBeLessThan(harness.requests.indexOf(createRequest)); + expect( + new URL(harness.requests[projectLookupIndex]?.url ?? '').searchParams.get('teamId'), + ).toBe('team_orbit'); + expect( + harness.requests.filter( + ({ method, url }) => method === 'GET' && url.includes('/v9/projects/'), + ), + ).toHaveLength(1); }); test('repository_dispatch follows current eligibility and exact CI proof', async () => { @@ -610,6 +637,9 @@ describe('existing deployments, cancellation, and pagination', () => { const harness = createHarness({ respond: ({ url }) => { if (!url.includes('/v7/deployments')) return undefined; + if (!new URL(url).searchParams.has('sha')) { + return json({ deployments: [], pagination: { count: 0, next: null, prev: null } }); + } listPage += 1; if (listPage === 1) { return json({ deployments: [], pagination: { count: 0, next: 123, prev: null } }); @@ -628,16 +658,20 @@ describe('existing deployments, cancellation, and pagination', () => { deploymentId: 'dpl_page_two', }); const pages = harness.requests.filter(({ url }) => url.includes('/v7/deployments')); - expect(pages).toHaveLength(2); + const exactHeadPages = pages.filter(({ url }) => new URL(url).searchParams.has('sha')); + expect(pages).toHaveLength(3); + expect(exactHeadPages).toHaveLength(2); for (const request of pages) { const url = new URL(request.url); expect(url.searchParams.get('teamId')).toBe('team_orbit'); expect(url.searchParams.get('projectId')).toBe('prj_orbit'); expect(url.searchParams.get('branch')).toBe('feature/preview'); - expect(url.searchParams.get('sha')).toBe(SHA); expect(url.searchParams.get('limit')).toBe('100'); } - expect(new URL(pages[1]?.url ?? '').searchParams.get('until')).toBe('123'); + for (const request of exactHeadPages) { + expect(new URL(request.url).searchParams.get('sha')).toBe(SHA); + } + expect(new URL(exactHeadPages[1]?.url ?? '').searchParams.get('until')).toBe('123'); }); test('an exact active deployment wins over terminal history', async () => { @@ -703,6 +737,115 @@ describe('existing deployments, cancellation, and pagination', () => { expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(2); }); + test('synchronize cancels only validated active prior heads and retains ready history', async () => { + const currentPullRequest = pullRequest({ + head: { sha: NEW_SHA, ref: 'feature/preview', repo: repository }, + }); + const harness = createHarness({ + eventName: 'pull_request_target', + event: { + action: 'synchronize', + number: 341, + pull_request: currentPullRequest, + repository, + }, + pullRequest: currentPullRequest, + workflowRuns: [], + deployments: [ + deployment('BUILDING', { uid: 'dpl_prior' }), + deployment('READY', { uid: 'dpl_prior_ready' }), + deployment('BUILDING', { + uid: 'dpl_current', + meta: metadata({ orbitGithubHeadSha: NEW_SHA }), + }), + deployment('BUILDING', { uid: 'dpl_project', projectId: 'prj_other' }), + deployment('BUILDING', { + uid: 'dpl_repository', + meta: metadata({ orbitGithubRepositoryId: '999' }), + }), + deployment('BUILDING', { + uid: 'dpl_pull_request', + meta: metadata({ orbitGithubPrNumber: '342' }), + }), + deployment('BUILDING', { + uid: 'dpl_ref', + meta: metadata({ orbitGithubHeadRef: 'feature/other' }), + }), + deployment('BUILDING', { + uid: 'dpl_malformed_sha', + meta: metadata({ orbitGithubHeadSha: 'not-a-sha' }), + }), + ], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'canceled', + pullRequestNumber: 341, + reason: 'canceled-active', + deploymentId: 'dpl_prior', + url: 'orbit-preview.vercel.app', + }, + { kind: 'skipped', pullRequestNumber: 341, reason: 'ci-unavailable' }, + ]); + expect( + harness.requests + .filter(({ method }) => method === 'PATCH') + .map(({ url }) => url.split('/').at(-2)), + ).toEqual(['dpl_prior']); + expect( + harness.requests.some(({ url }) => url.includes('/v13/deployments/dpl_prior_ready')), + ).toBe(false); + }); + + test('repository dispatch recovers an active deployment left by a prior head', async () => { + const currentPullRequest = pullRequest({ + head: { sha: NEW_SHA, ref: 'feature/preview', repo: repository }, + }); + const harness = createHarness({ + eventName: 'repository_dispatch', + pullRequest: currentPullRequest, + workflowRuns: [], + deployments: [deployment('BUILDING', { uid: 'dpl_prior' })], + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'canceled', + pullRequestNumber: 341, + reason: 'canceled-active', + deploymentId: 'dpl_prior', + url: 'orbit-preview.vercel.app', + }, + { kind: 'skipped', pullRequestNumber: 341, reason: 'ci-unavailable' }, + ]); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(1); + }); + + test('cancellation reversal reports that the pull request became eligible', async () => { + let pullRequestReads = 0; + const harness = createHarness({ + pullRequest: pullRequest({ draft: true }), + deployments: [deployment('BUILDING', { uid: 'dpl_active' })], + respond: ({ url }) => { + if (!url.endsWith('/pulls/341')) return undefined; + pullRequestReads += 1; + return json(pullRequest({ draft: pullRequestReads === 1 })); + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { kind: 'skipped', pullRequestNumber: 341, reason: 'preview-eligible' }, + ]); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(0); + }); + test('eligibility changing after the first cancellation prevents a second PATCH', async () => { let canceled = false; const harness = createHarness({ @@ -1157,6 +1300,37 @@ describe('bounded transport, polling, and ambiguity recovery', () => { expect(harness.sleeps.filter((milliseconds) => milliseconds === 5000)).toHaveLength(0); }); + test('project identity drift before a poll-triggered cancel blocks the PATCH', async () => { + let createAttempted = false; + let projectReads = 0; + const harness = createHarness({ + respond: ({ method, url }) => { + if (url.includes('/v9/projects/')) { + projectReads += 1; + return json(vercelProject(projectReads === 1 ? {} : { accountId: 'team_transferred' })); + } + if (url.endsWith('/pulls/341')) { + return json( + createAttempted ? pullRequest({ labels: [{ name: 'no-preview' }] }) : pullRequest(), + ); + } + if (method === 'POST') { + createAttempted = true; + return json(mutationDeployment('dpl_created', 'QUEUED')); + } + if (method === 'GET' && url.includes('/v13/deployments/dpl_created')) { + return json(mutationDeployment('dpl_created', 'BUILDING')); + } + return undefined; + }, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('project identity'); + expect(projectReads).toBe(2); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(0); + }); + test('the polling owner cancels an existing active deployment when the exact head becomes draft', async () => { let pullRequestReads = 0; const harness = createHarness({ @@ -1189,7 +1363,7 @@ describe('bounded transport, polling, and ambiguity recovery', () => { expect(harness.sleeps.filter((milliseconds) => milliseconds === 5000)).toHaveLength(0); }); - test('the polling owner never cancels a deployment after the pull request head changes', async () => { + test('the polling owner cancels its active deployment after a same-ref head change', async () => { let createAttempted = false; const harness = createHarness({ respond: ({ method, url }) => { @@ -1197,8 +1371,47 @@ describe('bounded transport, polling, and ambiguity recovery', () => { return json( createAttempted ? pullRequest({ - head: { sha: 'c'.repeat(40), ref: 'feature/new-head', repo: repository }, - labels: [{ name: 'no-preview' }], + head: { sha: NEW_SHA, ref: 'feature/preview', repo: repository }, + }) + : pullRequest(), + ); + } + if (method === 'POST') { + createAttempted = true; + return json(mutationDeployment('dpl_created', 'QUEUED')); + } + if (method === 'GET' && url.includes('/v13/deployments/dpl_created')) { + return json(mutationDeployment('dpl_created', 'BUILDING')); + } + return undefined; + }, + }); + + const results = await reconcileVercelPreviews(harness.runtime); + + expect(results).toEqual([ + { + kind: 'canceled', + pullRequestNumber: 341, + reason: 'canceled-active', + deploymentId: 'dpl_created', + url: 'orbit-preview.vercel.app', + }, + ]); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(1); + expect(harness.sleeps.filter((milliseconds) => milliseconds === 5000)).toHaveLength(0); + }); + + test('the polling owner does not cancel after the pull request head ref changes', async () => { + let createAttempted = false; + const harness = createHarness({ + respond: ({ method, url }) => { + if (url.endsWith('/pulls/341')) { + return json( + createAttempted + ? pullRequest({ + head: { sha: NEW_SHA, ref: 'feature/new-head', repo: repository }, }) : pullRequest(), ); @@ -1219,7 +1432,6 @@ describe('bounded transport, polling, and ambiguity recovery', () => { expect(results).toEqual([{ kind: 'skipped', pullRequestNumber: 341, reason: 'stale-event' }]); expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(0); - expect(harness.sleeps.filter((milliseconds) => milliseconds === 5000)).toHaveLength(0); }); test('active polling sleeps through transitions and returns READY', async () => { @@ -1273,6 +1485,9 @@ describe('bounded transport, polling, and ambiguity recovery', () => { const harness = createHarness({ respond: ({ method, url }) => { if (url.includes('/v7/deployments')) { + if (!new URL(url).searchParams.has('sha')) { + return json({ deployments: [], pagination: { count: 0, next: null, prev: null } }); + } listReads += 1; const items = listReads === 1 ? [] : [deployment('READY', { uid: 'dpl_observed' })]; return json({ @@ -1295,7 +1510,8 @@ describe('bounded transport, polling, and ambiguity recovery', () => { expect(results[0]).toMatchObject({ deploymentId: 'dpl_observed' }); expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); - expect(harness.requests.filter(({ url }) => url.includes('/v7/deployments'))).toHaveLength(2); + expect(listReads).toBe(2); + expect(harness.requests.filter(({ url }) => url.includes('/v7/deployments'))).toHaveLength(3); expect( harness.requests.filter(({ url }) => url.includes('/v13/deployments/dpl_observed')), ).toHaveLength(0); @@ -1315,6 +1531,9 @@ describe('bounded transport, polling, and ambiguity recovery', () => { }, respond: (request) => { if (request.url.includes('/v7/deployments')) { + if (!new URL(request.url).searchParams.has('sha')) { + return json({ deployments: [], pagination: { count: 0, next: null, prev: null } }); + } listReads += 1; const items = listReads === 1 ? [] : [deployment('READY', { uid: 'dpl_observed' })]; return json({ @@ -1340,6 +1559,7 @@ describe('bounded transport, polling, and ambiguity recovery', () => { expect(results[0]).toMatchObject({ deploymentId: 'dpl_observed' }); expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); expect(listReads).toBe(2); + expect(harness.requests.filter(({ url }) => url.includes('/v7/deployments'))).toHaveLength(3); expect( harness.requests.filter(({ url }) => url.includes('/v13/deployments/dpl_observed')), ).toHaveLength(0); @@ -1360,7 +1580,7 @@ describe('bounded transport, polling, and ambiguity recovery', () => { await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('ambiguous'); expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); - expect(harness.requests.filter(({ url }) => url.includes('/v7/deployments'))).toHaveLength(4); + expect(harness.requests.filter(({ url }) => url.includes('/v7/deployments'))).toHaveLength(5); expect(harness.requests.filter(({ url }) => url.includes('/v13/deployments/'))).toHaveLength(0); expect(harness.sleeps).toEqual([2000, 2000]); }); @@ -1379,7 +1599,7 @@ describe('bounded transport, polling, and ambiguity recovery', () => { await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow(String(status)); expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(1); - expect(listReads).toBe(1); + expect(listReads).toBe(2); }, ); @@ -1420,6 +1640,39 @@ describe('bounded transport, polling, and ambiguity recovery', () => { }); describe('identity invariants and bounded edge cases', () => { + test.each([ + ['project ID', { id: 'prj_other' }, 'project identity'], + ['project name', { name: 'other' }, 'project identity'], + ['team ID', { accountId: 'team_other' }, 'project identity'], + ['response schema', { accountId: undefined }, 'schema'], + ])( + 'a mismatched Vercel %s blocks create before mutation', + async (_name, projectOverrides, message) => { + const harness = createHarness({ + respond: ({ url }) => + url.includes('/v9/projects/') ? json(vercelProject(projectOverrides)) : undefined, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow(message); + expect(harness.requests.filter(({ method }) => method === 'POST')).toHaveLength(0); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(0); + }, + ); + + test('a mismatched Vercel project blocks cancellation before mutation', async () => { + const harness = createHarness({ + pullRequest: pullRequest({ draft: true }), + deployments: [deployment('BUILDING', { uid: 'dpl_active' })], + respond: ({ url }) => + url.includes('/v9/projects/') + ? json(vercelProject({ accountId: 'team_other' })) + : undefined, + }); + + await expect(reconcileVercelPreviews(harness.runtime)).rejects.toThrow('project identity'); + expect(harness.requests.filter(({ method }) => method === 'PATCH')).toHaveLength(0); + }); + test.each([ ['wrong project', { projectId: 'prj_other' }], ['non-null target', { target: 'production' }], diff --git a/scripts/vercel-preview-deploy.ts b/scripts/vercel-preview-deploy.ts index 7dabcfa21..f37c91433 100644 --- a/scripts/vercel-preview-deploy.ts +++ b/scripts/vercel-preview-deploy.ts @@ -10,6 +10,7 @@ import { githubPreviewWorkflowRunEventSchema, githubPreviewWorkflowRunsSchema, githubPreviewWorkflowSchema, + gitShaSchema, type VercelCanceledDeployment, type VercelCreatedDeployment, type VercelDeployment, @@ -20,6 +21,7 @@ import { vercelDeploymentDetailSchema, vercelDeploymentsPageSchema, vercelPreviewEnvironmentSchema, + vercelProjectSchema, } from '../packages/shared/src/validators/index.ts'; import { isActiveVercelDeployment, @@ -54,7 +56,7 @@ type PreviewReason = | 'repository-mismatch' | 'fork-pull-request' | 'base-mismatch' - | 'preview-ineligible' + | 'preview-eligible' | 'no-active-deployment' | 'web-unaffected' | 'ci-unavailable' @@ -374,6 +376,27 @@ function vercelUrl(path: string, query: Readonly>): strin return url.toString(); } +async function verifyVercelProjectIdentity( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, +): Promise { + const projectId = encodeURIComponent(environment.VERCEL_PROJECT_ID); + const project = await requestJson( + runtime, + environment, + 'vercel', + vercelUrl(`/v9/projects/${projectId}`, { teamId: environment.VERCEL_TEAM_ID }), + vercelProjectSchema, + ); + if ( + project.id !== environment.VERCEL_PROJECT_ID || + project.name !== environment.VERCEL_PROJECT_NAME || + project.accountId !== environment.VERCEL_TEAM_ID + ) { + fail('Vercel project identity mismatch'); + } +} + function repositorySlugMatches( repository: GithubPreviewRepository, configuredSlug: string, @@ -710,6 +733,49 @@ function comparableDeployment(detail: VercelDeploymentDetail): VercelDeployment }; } +function trustedCurrentPullRequest( + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, +): boolean { + return ( + repositorySlugMatches(pullRequest.base.repo, environment.GITHUB_REPOSITORY) && + isSameRepositoryPullRequest(pullRequest) && + pullRequest.base.ref === 'main' + ); +} + +async function cancelPolledDeployment( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, + deployment: VercelDeployment, +): Promise { + return ( + (await cancelOneActiveDeployment(runtime, environment, pullRequest, deployment)) ?? + skipped(pullRequest.number, 'no-active-deployment') + ); +} + +async function pollingInterruption( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + originalPullRequest: GithubPreviewPullRequest, + currentPullRequest: GithubPreviewPullRequest, + deployment: VercelDeployment, +): Promise { + if (!trustedCurrentPullRequest(environment, currentPullRequest)) { + return skipped(originalPullRequest.number, 'stale-event'); + } + if (pullRequestIdentityMatches(originalPullRequest, currentPullRequest)) { + if (currentPullRequest.state === 'open' && isPreviewEligible(currentPullRequest)) return null; + return await cancelPolledDeployment(runtime, environment, currentPullRequest, deployment); + } + if (!supersedingHeadMatches(originalPullRequest, currentPullRequest)) { + return skipped(originalPullRequest.number, 'stale-event'); + } + return await cancelPolledDeployment(runtime, environment, originalPullRequest, deployment); +} + async function pollDeployment( runtime: ControllerRuntime, environment: VercelPreviewEnvironment, @@ -747,29 +813,14 @@ async function pollDeployment( if (!isActiveVercelDeployment(comparable)) return fail(`deployment ended in ${detail.readyState}`); const currentPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); - const exactCurrentIdentity = - pullRequestIdentityMatches(pullRequest, currentPullRequest) && - repositorySlugMatches(currentPullRequest.base.repo, environment.GITHUB_REPOSITORY) && - isSameRepositoryPullRequest(currentPullRequest) && - currentPullRequest.base.ref === 'main'; - if (!exactCurrentIdentity) { - return { - kind: 'interrupted', - result: skipped(pullRequest.number, 'stale-event'), - }; - } - if (currentPullRequest.state !== 'open' || !isPreviewEligible(currentPullRequest)) { - const result = await cancelOneActiveDeployment( - runtime, - environment, - currentPullRequest, - comparable, - ); - return { - kind: 'interrupted', - result: result ?? skipped(pullRequest.number, 'no-active-deployment'), - }; - } + const interruption = await pollingInterruption( + runtime, + environment, + pullRequest, + currentPullRequest, + comparable, + ); + if (interruption) return { kind: 'interrupted', result: interruption }; if (detailRequest === 240) return fail('deployment polling timed out'); await runtime.sleep(5000); } @@ -801,6 +852,20 @@ function pullRequestIdentityMatches( ); } +function supersedingHeadMatches( + first: GithubPreviewPullRequest, + current: GithubPreviewPullRequest, +): boolean { + return ( + first.number === current.number && + first.head.sha !== current.head.sha && + first.head.ref === current.head.ref && + repositoryIdentityMatches(first.head.repo, current.head.repo) && + first.base.ref === current.base.ref && + repositoryIdentityMatches(first.base.repo, current.base.repo) + ); +} + function currentStateMatches( first: GithubPreviewPullRequest, current: GithubPreviewPullRequest, @@ -829,6 +894,7 @@ async function cancelOneActiveDeployment( ): Promise { let canceled: VercelCanceledDeployment; const encodedDeploymentId = deploymentPathSegment(item.uid, environment); + await verifyVercelProjectIdentity(runtime, environment); try { canceled = await requestJson( runtime, @@ -878,6 +944,64 @@ async function cancelOneActiveDeployment( }; } +type PriorHeadCancellation = { + readonly current: boolean; + readonly results: readonly PreviewResult[]; +}; + +function isActivePriorHeadDeployment( + deployment: VercelDeployment, + pullRequest: GithubPreviewPullRequest, + projectId: string, +): boolean { + const headSha = deployment.meta['orbitGithubHeadSha']; + const parsedHeadSha = gitShaSchema.safeParse(headSha); + return ( + isActiveVercelDeployment(deployment) && + matchesVercelPullRequest(deployment, pullRequest, projectId) && + parsedHeadSha.success && + parsedHeadSha.data !== pullRequest.head.sha + ); +} + +async function cancelPriorHeadDeployments( + runtime: ControllerRuntime, + environment: VercelPreviewEnvironment, + pullRequest: GithubPreviewPullRequest, +): Promise { + const listed = await listDeployments(runtime, environment, pullRequest, false); + const activePriorHeads = listed + .filter((deployment) => + isActivePriorHeadDeployment(deployment, pullRequest, environment.VERCEL_PROJECT_ID), + ) + .sort((left, right) => left.uid.localeCompare(right.uid)); + const results: PreviewResult[] = []; + for (const deployment of activePriorHeads) { + const currentPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); + const stateIsCurrent = + currentStateMatches(pullRequest, currentPullRequest) && + repositorySlugMatches(currentPullRequest.base.repo, environment.GITHUB_REPOSITORY) && + isSameRepositoryPullRequest(currentPullRequest) && + currentPullRequest.base.ref === 'main' && + currentPullRequest.state === 'open' && + isPreviewEligible(currentPullRequest); + if (!stateIsCurrent) { + return { + current: false, + results: results.length > 0 ? results : [skipped(pullRequest.number, 'stale-event')], + }; + } + const result = await cancelOneActiveDeployment( + runtime, + environment, + currentPullRequest, + deployment, + ); + if (result) results.push(result); + } + return { current: true, results }; +} + async function cancelActiveDeployments( runtime: ControllerRuntime, environment: VercelPreviewEnvironment, @@ -898,14 +1022,14 @@ async function cancelActiveDeployments( for (const item of active) { const finalPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); const identityIsCurrent = - currentStateMatches(pullRequest, finalPullRequest) && + pullRequestIdentityMatches(pullRequest, finalPullRequest) && repositorySlugMatches(finalPullRequest.base.repo, environment.GITHUB_REPOSITORY) && isSameRepositoryPullRequest(finalPullRequest); if (!identityIsCurrent) { return results.length > 0 ? results : [skipped(pullRequest.number, 'stale-event')]; } if (finalPullRequest.state === 'open' && isPreviewEligible(finalPullRequest)) { - return results.length > 0 ? results : [skipped(pullRequest.number, 'preview-ineligible')]; + return results.length > 0 ? results : [skipped(pullRequest.number, 'preview-eligible')]; } const result = await cancelOneActiveDeployment(runtime, environment, finalPullRequest, item); if (result) results.push(result); @@ -986,6 +1110,7 @@ async function createDeployment( preCreateDeploymentIds: ReadonlySet, ): Promise { const metadata = deploymentMetadata(pullRequest, workflowRunId); + await verifyVercelProjectIdentity(runtime, environment); let created: VercelCreatedDeployment; try { created = await requestJson( @@ -1080,10 +1205,13 @@ async function reconcileCandidate( if (pullRequest.state !== 'open' || !isPreviewEligible(pullRequest)) { return cancelActiveDeployments(runtime, environment, pullRequest); } + const priorHeadCancellation = await cancelPriorHeadDeployments(runtime, environment, pullRequest); + if (!priorHeadCancellation.current) return priorHeadCancellation.results; + const priorHeadResults = priorHeadCancellation.results; const ci = await proveCurrentCi(runtime, environment, pullRequest); - if (!ci.ok) return [skipped(candidate.number, ci.reason)]; + if (!ci.ok) return [...priorHeadResults, skipped(candidate.number, ci.reason)]; if (!(await affectsWeb(runtime, environment, pullRequest.number))) { - return [skipped(candidate.number, 'web-unaffected')]; + return [...priorHeadResults, skipped(candidate.number, 'web-unaffected')]; } const listed = await listDeployments(runtime, environment, pullRequest, true); const exact = listed.filter((item) => @@ -1095,13 +1223,13 @@ async function reconcileCandidate( ), ); const reused = await reusedDeploymentResult(runtime, environment, pullRequest, exact); - if (reused) return [reused]; + if (reused) return [...priorHeadResults, reused]; const finalPullRequest = await fetchPullRequest(runtime, environment, pullRequest.number); if (!currentStateMatches(pullRequest, finalPullRequest)) { - return [skipped(candidate.number, 'stale-event')]; + return [...priorHeadResults, skipped(candidate.number, 'stale-event')]; } const finalCi = await proveCurrentCi(runtime, environment, finalPullRequest); - if (!finalCi.ok) return [skipped(candidate.number, finalCi.reason)]; + if (!finalCi.ok) return [...priorHeadResults, skipped(candidate.number, finalCi.reason)]; const mutationPullRequest = await fetchPullRequest(runtime, environment, finalPullRequest.number); const mutationStateIsCurrent = currentStateMatches(finalPullRequest, mutationPullRequest) && @@ -1110,8 +1238,11 @@ async function reconcileCandidate( mutationPullRequest.base.ref === 'main' && mutationPullRequest.state === 'open' && isPreviewEligible(mutationPullRequest); - if (!mutationStateIsCurrent) return [skipped(candidate.number, 'stale-event')]; + if (!mutationStateIsCurrent) { + return [...priorHeadResults, skipped(candidate.number, 'stale-event')]; + } return [ + ...priorHeadResults, await createDeployment( runtime, environment, From 6556e05ff6343fa4aad9036cc2b557dbcb74145a Mon Sep 17 00:00:00 2001 From: shashank agarwal Date: Wed, 26 Aug 2026 01:31:56 +0530 Subject: [PATCH 19/19] fix(ci): declare read-only workflow permissions --- .github/workflows/ci.yml | 5 +++-- scripts/vercel-preview-config.test.ts | 5 +++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fe3ffa3d8..d6fcd9977 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -6,6 +6,9 @@ on: pull_request: workflow_dispatch: +permissions: + contents: read + concurrency: group: ci-${{ github.ref }} cancel-in-progress: true @@ -135,8 +138,6 @@ jobs: build: name: Build runs-on: ubuntu-latest - permissions: - contents: read env: DATABASE_URL: postgres://orbit:orbit@localhost:5433/orbit REDIS_URL: redis://localhost:6380 diff --git a/scripts/vercel-preview-config.test.ts b/scripts/vercel-preview-config.test.ts index b11b9b995..ecc5c81de 100644 --- a/scripts/vercel-preview-config.test.ts +++ b/scripts/vercel-preview-config.test.ts @@ -290,6 +290,11 @@ describe('Vercel Preview repository configuration', () => { ); }); + test('gives every CI job only read access to repository contents', () => { + expect(ci).toContain('\npermissions:\n contents: read\n\nconcurrency:\n'); + expect(ci.match(/^ *permissions:/gm)).toEqual(['permissions:']); + }); + test('documents the Preview build security boundary and safe recovery event', () => { const prose = guide.replace(/\s+/g, ' '); expect(prose).toContain(