-
Notifications
You must be signed in to change notification settings - Fork 13
chore(vercel): gate preview builds on pull request readiness #341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
imshashank
wants to merge
27
commits into
main
Choose a base branch
from
chore/gate-preview-builds
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 3 commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
e311851
chore(vercel): gate preview builds on pull request readiness
imshashank 8620b54
test(vercel): cover the build gate, and watch the root tsconfig
imshashank 2da9f89
fix(vercel): require a complete pull request payload before skipping
imshashank 2a78c65
docs(vercel): design the preview deployment gate
imshashank 918908f
Merge remote-tracking branch 'origin/main' into chore/gate-preview-bu…
imshashank e255616
docs(vercel): tighten the preview gate plan
imshashank ae5fefc
feat(ci): define preview deployment policy
imshashank 2e359cd
fix(ci): harden preview event schemas
imshashank 756ea88
fix(ci): validate preview deployment identity
imshashank a861e38
docs(vercel): finalize the preview controller contract
imshashank 89201b1
feat(ci): deploy previews after successful checks
imshashank cb219bf
fix(ci): harden preview reconciliation
imshashank 73a42cc
docs(vercel): harden preview workflow plan
imshashank d4fe89e
chore(vercel): gate previews after CI
imshashank b6ea842
test(vercel): lock preview workflow contract
imshashank 6b362df
test(vercel): keep legacy setting scan clean
imshashank 5998d8c
fix(ci): harden preview deployment reconciliation
imshashank 5adec4b
chore(ci): apply preview review cleanups
imshashank 618ecb6
fix(ci): cancel superseded preview deployments
imshashank dee4f81
Merge remote-tracking branch 'source/main' into chore/gate-preview-bu…
imshashank 6556e05
fix(ci): declare read-only workflow permissions
imshashank 99d8c63
Merge main into chore/gate-preview-builds
imshashank 82c9a10
Merge remote-tracking branch 'origin/main' into codex/review-pr-341
imshashank 96494ca
fix(preview): revalidate immediately before mutations
imshashank d3685c4
Merge remote-tracking branch 'origin/main' into codex/review-pr-341
imshashank b03c1ad
fix(preview): reject Dependabot deployment events
imshashank 6ed5558
Merge remote-tracking branch 'origin/main' into codex/review-pr-341
imshashank File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| # Vercel build gate | ||
|
|
||
| Preview deployments only build once a pull request is marked **Ready for review**. | ||
| Production always builds. Two labels override the rule: `preview` builds a draft, | ||
| and `no-preview` suppresses a pull request that is ready. | ||
|
|
||
| ## Why the gate exists | ||
|
|
||
| Orbit ran **700 deployments in the 22 days** after the project was created on | ||
| 2026-07-28, peaking at 131 in a single day, and 77% of them were previews. | ||
| Across the team, 81% of 4,393 deployments in the 90 days to 2026-08-19 were | ||
| previews, and builds were $110 of the $506.93 August invoice. | ||
|
|
||
| ## How it works | ||
|
|
||
| `apps/web/vercel.json` points Vercel's Ignored Build Step at | ||
| `scripts/vercel-build-gate.sh`. | ||
|
|
||
| **Exit codes are inverted from intuition: `exit 0` skips the build, `exit 1` runs it.** | ||
|
|
||
| Decision order: | ||
|
|
||
| | Condition | Result | | ||
| |---|---| | ||
| | `VERCEL_ENV=production` | build | | ||
| | system environment variables not exposed | build | | ||
| | branch has no open PR | skip | | ||
| | PR labelled `no-preview` | skip | | ||
| | PR labelled `preview` | build (even while draft) | | ||
| | PR is a draft | skip | | ||
| | PR is ready for review | build, subject to the path filter | | ||
| | nothing changed under `BUILD_GATE_WATCH_PATHS` | skip | | ||
|
|
||
| Every failure path - missing token, GitHub API error, unparseable response, | ||
| unreachable diff base - **builds**. The gate never silently withholds a | ||
| deployment because something broke. | ||
|
|
||
| The metadata check has to come before the pull request check, and the order is | ||
| load-bearing. `VERCEL_GIT_PULL_REQUEST_ID` is empty both when a branch genuinely | ||
| has no pull request *and* when system environment variables are not exposed at | ||
| all. Testing the PR id first would read the second case as the first and skip | ||
| every preview in the project, silently, which is the one behaviour this gate | ||
| must never have. `VERCEL_GIT_REPO_OWNER` and `VERCEL_GIT_REPO_SLUG` are set | ||
| whenever the variables are exposed, regardless of pull request state, so they | ||
| are what distinguishes the two. | ||
|
|
||
| ## The button | ||
|
|
||
| Open the PR as a **draft** while you work. Commits accumulate with zero builds. | ||
| When you want a preview, click **Ready for review** - that is the button. Adding | ||
| the `preview` label also works if you want previews while staying in draft. | ||
|
|
||
| ## Setup per project | ||
|
|
||
| 1. Project Settings → Environment Variables → tick **Enable access to System | ||
| Environment Variables**. The gate needs `VERCEL_GIT_PULL_REQUEST_ID`, | ||
| `VERCEL_GIT_REPO_OWNER`, `VERCEL_GIT_REPO_SLUG` and `VERCEL_GIT_PREVIOUS_SHA`. | ||
| Note that `VERCEL_GIT_PREVIOUS_SHA` is *only* exposed when an Ignored Build | ||
| Step is configured. | ||
| 2. Add `BUILD_GATE_GITHUB_TOKEN` - a fine-grained token with **Pull requests: | ||
| read** on the repo. Without it the gate fails open and every push builds. | ||
| 3. Optionally add `BUILD_GATE_WATCH_PATHS` (space separated, repo-relative) to | ||
| skip builds when nothing under those paths changed. | ||
|
|
||
| ## Monorepo path filtering | ||
|
|
||
| This repo holds two apps. Only `apps/web` is deployed to Vercel, so a push that | ||
| only touches `apps/realtime` has nothing to preview. `apps/web/vercel.json` | ||
| therefore supplies a default: | ||
|
|
||
| ```sh | ||
| BUILD_GATE_WATCH_PATHS="apps/web packages package.json bun.lock tsconfig.base.json" | ||
| ``` | ||
|
|
||
| Setting the variable in project settings overrides that default. | ||
|
|
||
| Watch paths are **repo-relative**, and the script resolves them against | ||
| `git rev-parse --show-toplevel` rather than the working directory. This matters: | ||
| Vercel runs the Ignored Build Step from the project's **Root Directory**, so for | ||
| a project rooted at `apps/web` a plain `git diff -- apps/web` looks for | ||
| `apps/web/apps/web`, finds nothing, and skips every build. Test any change to | ||
| this script from a subdirectory, not just from the repo root. | ||
|
|
||
| The diff base is `VERCEL_GIT_PREVIOUS_SHA`, the last **successfully deployed** | ||
| commit - not `HEAD^`. `HEAD^` is wrong whenever more than one commit lands at | ||
| once, which is the normal case for a squash merge or a batch of pushes. If that | ||
| SHA is missing from Vercel's shallow clone the gate builds rather than guessing. | ||
|
|
||
| ## Testing changes to the gate | ||
|
|
||
| The script shells out to `curl` and `git`, so it is testable by putting a stub | ||
| `curl` earlier on `PATH`. See the harness used when this landed - it covers | ||
| production, draft, ready, both labels, a missing token, API failure, malformed | ||
| JSON, and the path filter against real git history. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| #!/usr/bin/env bash | ||
| set -uo pipefail | ||
|
|
||
| WATCH_PATHS="${BUILD_GATE_WATCH_PATHS:-.}" | ||
| READY_LABEL="${BUILD_GATE_READY_LABEL:-preview}" | ||
| BLOCK_LABEL="${BUILD_GATE_BLOCK_LABEL:-no-preview}" | ||
|
|
||
| announce() { echo "[build-gate] $*" >&2; } | ||
| build() { announce "BUILD - $1"; exit 1; } | ||
|
greptile-apps[bot] marked this conversation as resolved.
Outdated
|
||
| skip() { announce "SKIP - $1"; exit 0; } | ||
|
|
||
| if [ "${VERCEL_ENV:-}" = "production" ]; then | ||
| build "production deployment" | ||
| fi | ||
|
|
||
| REPO_OWNER="${VERCEL_GIT_REPO_OWNER:-}" | ||
| REPO_SLUG="${VERCEL_GIT_REPO_SLUG:-}" | ||
| if [ -z "$REPO_OWNER" ] || [ -z "$REPO_SLUG" ]; then | ||
| build "system environment variables are not exposed to this build so pull request state is unreadable, failing open" | ||
| fi | ||
|
|
||
| PR_ID="${VERCEL_GIT_PULL_REQUEST_ID:-}" | ||
| if [ -z "$PR_ID" ]; then | ||
| skip "branch has no open pull request, nothing to preview yet" | ||
| fi | ||
|
|
||
| if [ -z "${BUILD_GATE_GITHUB_TOKEN:-}" ]; then | ||
| build "BUILD_GATE_GITHUB_TOKEN is unset so pull request state cannot be read, failing open" | ||
| fi | ||
|
|
||
| if ! PR_JSON="$(curl -sS --max-time 15 \ | ||
| -H "Authorization: Bearer ${BUILD_GATE_GITHUB_TOKEN}" \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| -H "X-GitHub-Api-Version: 2022-11-28" \ | ||
| "https://api.github.com/repos/${REPO_OWNER}/${REPO_SLUG}/pulls/${PR_ID}")"; then | ||
| build "GitHub API unreachable, failing open" | ||
| fi | ||
|
|
||
| if [ -z "$PR_JSON" ]; then | ||
| build "GitHub API returned an empty response, failing open" | ||
| fi | ||
|
|
||
| VERDICT="$( | ||
| PR_JSON="$PR_JSON" READY_LABEL="$READY_LABEL" BLOCK_LABEL="$BLOCK_LABEL" node -e ' | ||
| try { | ||
| const pr = JSON.parse(process.env.PR_JSON); | ||
| const wellFormed = | ||
| pr && | ||
| typeof pr.draft === "boolean" && | ||
| Number.isInteger(pr.number) && | ||
| pr.number > 0 && | ||
| Array.isArray(pr.labels) && | ||
| pr.labels.every((label) => label && typeof label.name === "string"); | ||
| if (!wellFormed) { | ||
| console.log("unknown:pull request payload was not a complete pull request"); | ||
| process.exit(0); | ||
| } | ||
| const labels = pr.labels.map((label) => label.name.toLowerCase()); | ||
| if (labels.includes(process.env.BLOCK_LABEL.toLowerCase())) { | ||
| console.log(`skip:pull request ${pr.number} carries the ${process.env.BLOCK_LABEL} label`); | ||
| } else if (labels.includes(process.env.READY_LABEL.toLowerCase())) { | ||
| console.log(`build:pull request ${pr.number} carries the ${process.env.READY_LABEL} label`); | ||
| } else if (pr.draft) { | ||
| console.log(`skip:pull request ${pr.number} is still a draft, mark it ready for review to start previews`); | ||
| } else { | ||
| console.log(`build:pull request ${pr.number} is ready for review`); | ||
| } | ||
| } catch (error) { | ||
| console.log(`unknown:${error.message}`); | ||
| } | ||
| ' | ||
| )" | ||
|
|
||
| REASON="$(printf '%s' "$VERDICT" | cut -d: -f2-)" | ||
| case "$(printf '%s' "$VERDICT" | cut -d: -f1)" in | ||
| skip) skip "$REASON" ;; | ||
| build) announce "gate passed, $REASON" ;; | ||
| *) build "pull request state could not be evaluated, failing open" ;; | ||
| esac | ||
|
|
||
| BASE_SHA="${VERCEL_GIT_PREVIOUS_SHA:-}" | ||
| if [ -z "$BASE_SHA" ]; then | ||
| build "no diff base supplied, cannot tell which paths changed" | ||
| fi | ||
|
|
||
| REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null)" | ||
| if [ -z "$REPO_ROOT" ]; then | ||
| build "not inside a git work tree, cannot tell which paths changed" | ||
| fi | ||
|
|
||
| if ! git -C "$REPO_ROOT" cat-file -e "${BASE_SHA}^{commit}" 2>/dev/null; then | ||
| build "diff base ${BASE_SHA} is not in this clone, cannot tell which paths changed" | ||
| fi | ||
|
|
||
| read -r -a WATCH_ARRAY <<<"$WATCH_PATHS" | ||
| if git -C "$REPO_ROOT" diff --quiet "$BASE_SHA" HEAD -- "${WATCH_ARRAY[@]}"; then | ||
| skip "no changes under '${WATCH_PATHS}' since ${BASE_SHA}" | ||
| fi | ||
|
|
||
| build "changes under '${WATCH_PATHS}' since ${BASE_SHA}" | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.