feat(backend): add match-based email nudges and Clerk-gated admin dashboard

AGENTS.md

@@ -65,21 +65,23 @@ Each job post contains:
 - **Source** — LinkedIn or X, with direct link to original post
 - **Score** — community votes (upvote/downvote)
 - **Comments count**
-- **Posted time** — relative (e.g. "2h ago"), auto-expires after 7 days
+- **Posted time** — relative (e.g. "2h ago"), auto-expires after 30 days
 
 ## Backend
 
 The backend is a Hono app with Prisma + PostgreSQL. It provides:
 - Post ingestion via Apify (LinkedIn/X scraping)
 - Recruiter suggestions (community submissions)
 - Voting on job posts
-- Admin endpoints (protected by `ADMIN_SECRET`)
+- Admin endpoints (`ADMIN_SECRET` bearer, or a Clerk admin per `ADMIN_USER_IDS` / `ADMIN_EMAILS`)
 - AI-powered post enrichment via Groq
 
 The frontend fetches jobs from the backend API. The API contract is aligned with the `Job` interface in `frontend/src/lib/data.ts`.
 
 Ingest path: `ingestPosts` in `backend/src/lib/apify.ts` -> `classifyPost` (`llm-classifier.ts`, regex fallback in `classifier.ts`) -> `inferCompany` (`extract-company.ts`) -> deterministic gates in `backend/src/lib/ingest-gates.ts` -> `prisma.job.create`. The feed is engineering-only by default (`INGEST_ROLE_FAMILIES`); skip reasons and the `scripts/reclassify-jobs.ts` cleanup are documented under "Ingest quality gates" in `DEPLOYMENT.md`. Add any new junk pattern as a failing test in `backend/tests/` first; the gates are pure functions and must never call the LLM.
 
+Email nudges: `runCampaign` in `backend/src/lib/nudge-send.ts` -> `selectJobsForUser` (`nudge-select.ts`, pure: re-applies `evaluateJobGates`, lifts SDE-3/5+ year roles to senior, excludes jobs already sent) -> `renderNudgeEmail` (`nudge-render.ts`, plain HTML + text) -> `EmailProvider` (`email.ts`, Resend or dry run without `RESEND_API_KEY`). Recipients are Clerk users with a Mongo profile; their emails come from Clerk, never from the resume. Admin access is `ADMIN_USER_IDS` / `ADMIN_EMAILS` (`admin-auth.ts`); the dashboard at `frontend/src/app/admin/` never sees `ADMIN_SECRET`. Operations, env vars, DNS and the cron list are under "Match-based email nudges" in `DEPLOYMENT.md`. Tests mock Prisma with `backend/tests/helpers/fake-prisma.ts`.
+
 Frontend config (`frontend/src/lib/config.ts`) centralizes `BACKEND_URL` and `API_KEY` — all client-side fetches import from there.
 
 ## Deployment (Vercel)

DEPLOYMENT.md

@@ -98,8 +98,26 @@ npx vercel env add APIFY_MAX_CONCURRENT production
 # Value: 5
 npx vercel env add INGEST_ROLE_FAMILIES production
 # Optional. Value: engineering,ai_ml (default). Comma-separated RoleFamily values allowed into the feed.
+
+# Email nudges (see "Match-based email nudges" below)
+npx vercel env add RESEND_API_KEY production
+# Optional. Without it every send is a dry run: rendered and recorded, nothing leaves the server.
+npx vercel env add RESEND_WEBHOOK_SECRET production
+# Signing secret of the Resend webhook endpoint (starts with whsec_). Without it POST /api/email/webhook returns 503.
+npx vercel env add EMAIL_FROM production
+# Value: SkipTheBoard Jobs <jobs@mail.skiptheboard.in> (default when unset)
+npx vercel env add UNSUBSCRIBE_SECRET production
+# Random 32+ char string. Signs the one-click unsubscribe links; rotating it invalidates links in already-sent emails.
+npx vercel env add ADMIN_USER_IDS production
+# Comma-separated Clerk user ids allowed into /admin. Either this or ADMIN_EMAILS must list the captain.
+npx vercel env add ADMIN_EMAILS production
+# Comma-separated primary Clerk emails allowed into /admin (case-insensitive). Resolved through @clerk/backend.
+npx vercel env add FRONTEND_URL production
+# Value: https://skiptheboard.in — origin used in email links and the /go redirect fallback.
 ```
 
+`CLERK_SECRET_KEY` and `MONGODB_URI` are already required for sign-in and profiles; the nudges reuse them for recipient emails and resume profiles.
+
 **`CORS_ORIGIN`** must be a comma-separated list of allowed origins:
 
 ```
@@ -138,7 +156,7 @@ npm run dev
 
 ### Daily recruiter scrape
 
-`backend/vercel.json` registers a Vercel Cron that hits `GET /api/cron/scrape` every 15 minutes. Vercel sends `Authorization: Bearer $CRON_SECRET` when that env var is set.
+`backend/vercel.json` registers a Vercel Cron that hits `GET /api/cron/scrape` once a day at 04:00 UTC. Vercel sends `Authorization: Bearer $CRON_SECRET` when that env var is set.
 
 The endpoint starts at most 5 due recruiters per tick (Apify Free concurrent cap), then if more remain it waits 10 minutes and starts the next 5, until everyone due that day is scraped. Unfinished Apify runs stay `RUNNING` and are ingested on a later tick. Set `APIFY_WAIT_MS` (default 180000) if a manual admin scrape should wait longer in-request.
 
@@ -151,6 +169,34 @@ curl -X POST https://backend-umber-nu-43.vercel.app/api/cron/scrape \
 
 Use `?once=1` to scrape a single due recruiter and stop.
 
+### Match-based email nudges
+
+Signed-in users with a resume profile get an email with the jobs on the board that best match their resume. Everything lives behind `backend/src/lib/email.ts` (provider), `nudge-select.ts` (pure ranking, senior override, gates), `nudge-render.ts` (plain HTML + text template) and `nudge-send.ts` (campaign runner). The send path never calls an LLM.
+
+**Crons (`backend/vercel.json`, 3 total, all daily or slower as Vercel Hobby requires):**
+
+| Path | Schedule (UTC) | What it does |
+|------|----------------|--------------|
+| `/api/cron/expire-jobs` | `0 3 * * *` | Delete jobs older than `JOB_EXPIRY_DAYS`. |
+| `/api/cron/scrape` | `0 4 * * *` | Daily recruiter scrape (drains through GitHub Actions). |
+| `/api/cron/nudges` | `30 2 * * *` | 08:00 IST. On Mondays it creates and runs the weekly campaign for every subscribed user; on other days it runs a daily campaign only for users who chose daily. Hobby fires it within the hour. |
+
+A run processes recipients in pages of 100 (one Resend batch call each). If the 240 s budget (`NUDGE_BUDGET_MS`) runs out it POSTs itself `/api/cron/nudges?campaign=<id>` with `CRON_SECRET` via `waitUntil`, so a long send continues in a fresh invocation. The `(campaign, user)` unique index makes any retry safe. **The schedule starts paused.** On a fresh deployment there is no `nudges.schedule_paused` row in `app_settings`, and the cron treats that as paused: it creates no campaign and nothing goes out on its own until an admin presses "Resume schedule" under Send controls (which stores `false`). "Pause schedule" stores `true` again. "Send now" and test sends from the dashboard work either way.
+
+**Dry run.** With no `RESEND_API_KEY`, sends are rendered and recorded with status `dry_run` and the admin preview, test send and campaigns all work. Nothing is ever sent from tests.
+
+**Resend setup.**
+
+1. Domain `mail.skiptheboard.in` is added in Resend in region **ap-northeast-1 (Tokyo)**. Its DNS records are already in place. Because skiptheboard.in's nameservers are Vercel's (`ns1/ns2.vercel-dns.com`), any email DNS change goes into **Vercel → Domains → skiptheboard.in → DNS Records**, not GoDaddy. The records Resend requires are: a TXT at `resend._domainkey.mail` (DKIM), an MX plus a TXT at `send.mail` (SPF / return path), and a TXT at `_dmarc` (`v=DMARC1; p=none; rua=mailto:<reporting inbox>`, tighten to `p=quarantine` after a few clean weeks). Copy the exact values from the Resend domain page.
+2. Webhook: in Resend → Webhooks add `https://backend-umber-nu-43.vercel.app/api/email/webhook` with the events `email.delivered`, `email.opened`, `email.clicked`, `email.bounced`, `email.complained` (sent and delivery_delayed are accepted and ignored). Paste the endpoint's signing secret into `RESEND_WEBHOOK_SECRET` and redeploy. Signatures are verified with Svix; events are deduplicated by `svix-id`.
+3. Every email carries `List-Unsubscribe` and `List-Unsubscribe-Post: List-Unsubscribe=One-Click` (RFC 8058) pointing at `POST /api/email/unsubscribe?t=<signed token>`, plus a footer link to `https://skiptheboard.in/unsubscribe?t=…` (the page asks for one click before it acts). Test sends carry neither the headers nor a working footer link. Hard bounces and complaints unsubscribe the user automatically.
+
+**Click tracking.** Job links go to `https://skiptheboard.in/go/<sendId>/<jobId>`; the Next route forwards to the backend `GET /go/...` (rewrite in `vercel.json`), which logs a `nudge_clicks` row and 302s to the LinkedIn post. Site links carry `utm_source=nudge&utm_medium=email&utm_campaign=<campaign key>`.
+
+**Admin dashboard.** `https://skiptheboard.in/admin` renders only for Clerk users in `ADMIN_USER_IDS` / `ADMIN_EMAILS`; every `/api/admin/*` call is checked on the backend (the machine `ADMIN_SECRET` still works for curl and scripts, but is never sent to a browser). Tabs: Subscribers, Email performance, Jobs & site, Send controls (preview any user, test send, send now, pause/resume), Experiments (2+ variants with weights and an optional holdout, results side by side), Submissions, Hiring managers.
+
+**Migration.** `backend/prisma/migrations/20261002090000_add_email_nudges` adds `email_preferences`, `campaigns`, `campaign_variants`, `nudge_sends`, `nudge_clicks`, `email_events` and `app_settings`. Apply with `npx prisma migrate deploy` against production before the first send.
+
 ### Ingest quality gates
 
 Every scraped post passes through deterministic gates in `backend/src/lib/ingest-gates.ts` before it becomes a `Job`. A post is skipped, and counted in `ApifyRunLog.errorMsg` (for example `not_job=3,off_target=2,duplicate=1`), for one of these reasons:

README.md

@@ -66,13 +66,14 @@ npm run dev
 **Environment variables:**
 - `DATABASE_URL` — PostgreSQL connection string
 - `API_KEY` — Shared API key for feed endpoints
-- `ADMIN_SECRET` — Bearer token for admin endpoints
+- `ADMIN_SECRET` — Machine bearer token for admin endpoints (scripts and curl; the `/admin` dashboard uses Clerk with `ADMIN_USER_IDS` / `ADMIN_EMAILS` instead)
 - `APIFY_TOKEN` — Apify API token for LinkedIn/X scraping (`APIFY_API_KEY` is accepted as a fallback)
 - `APIFY_MAX_CONCURRENT` — Max simultaneous Apify runs (default 5, Apify free-plan cap)
 - `GROQ_API_KEY` — Groq API key for AI-powered post enrichment
 - `GROQ_MODEL` — Optional Groq model override (default `openai/gpt-oss-120b`)
 - `INGEST_ROLE_FAMILIES` — Role families allowed into the feed, comma-separated (default `engineering,ai_ml`); see the ingest quality gates in [DEPLOYMENT.md](./DEPLOYMENT.md)
 - `CORS_ORIGIN` — Allowed frontend origin(s), comma-separated
+- Email nudge and admin access variables (`RESEND_API_KEY`, `UNSUBSCRIBE_SECRET`, `ADMIN_EMAILS`, …) are listed in [DEPLOYMENT.md](./DEPLOYMENT.md)
 
 ## API Endpoints
 

backend/package.json

@@ -18,7 +18,9 @@
     "@vercel/functions": "^3.9.3",
     "hono": "^4.7.0",
     "mongodb": "^7.4.0",
+    "resend": "^6.32.0",
     "stripe": "^22.3.0",
+    "svix": "^2.6.1",
     "unpdf": "^1.6.2",
     "zod": "^4.4.3"
   },

backend/prisma/migrations/20261002090000_add_email_nudges/migration.sql

@@ -0,0 +1,172 @@
+-- CreateEnum
+CREATE TYPE "EmailFrequency" AS ENUM ('weekly', 'daily');
+
+-- CreateEnum
+CREATE TYPE "CampaignKind" AS ENUM ('weekly', 'daily', 'manual', 'experiment');
+
+-- CreateEnum
+CREATE TYPE "CampaignStatus" AS ENUM ('draft', 'running', 'completed');
+
+-- CreateEnum
+CREATE TYPE "SendStatus" AS ENUM ('queued', 'dry_run', 'sent', 'delivered', 'bounced', 'complained', 'failed', 'skipped', 'holdout');
+
+-- CreateTable
+CREATE TABLE "email_preferences" (
+    "id" SERIAL NOT NULL,
+    "user_id" TEXT NOT NULL,
+    "email" TEXT,
+    "subscribed" BOOLEAN NOT NULL DEFAULT true,
+    "frequency" "EmailFrequency" NOT NULL DEFAULT 'weekly',
+    "paused_until" TIMESTAMP(3),
+    "unsubscribed_at" TIMESTAMP(3),
+    "unsubscribe_reason" TEXT,
+    "unsubscribe_token" TEXT NOT NULL,
+    "last_sent_at" TIMESTAMP(3),
+    "created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    "updated_at" TIMESTAMP(3) NOT NULL,
+
+    CONSTRAINT "email_preferences_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateTable
+CREATE TABLE "campaigns" (
+    "id" SERIAL NOT NULL,
+    "key" TEXT NOT NULL,
+    "name" TEXT NOT NULL,
+    "kind" "CampaignKind" NOT NULL,
+    "status" "CampaignStatus" NOT NULL DEFAULT 'draft',
+    "job_count" INTEGER NOT NULL DEFAULT 5,
+    "created_by" TEXT,
+    "started_at" TIMESTAMP(3),
+    "completed_at" TIMESTAMP(3),
+    "created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+
+    CONSTRAINT "campaigns_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateTable
+CREATE TABLE "campaign_variants" (
+    "id" SERIAL NOT NULL,
+    "campaign_id" INTEGER NOT NULL,
+    "key" TEXT NOT NULL,
+    "name" TEXT NOT NULL,
+    "weight" INTEGER NOT NULL,
+    "is_holdout" BOOLEAN NOT NULL DEFAULT false,
+    "subject" TEXT,
+    "intro" TEXT,
+    "job_count" INTEGER,
+
+    CONSTRAINT "campaign_variants_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateTable
+CREATE TABLE "nudge_sends" (
+    "id" TEXT NOT NULL,
+    "campaign_id" INTEGER NOT NULL,
+    "variant_id" INTEGER,
+    "user_id" TEXT NOT NULL,
+    "email" TEXT NOT NULL,
+    "status" "SendStatus" NOT NULL,
+    "provider_message_id" TEXT,
+    "job_ids" INTEGER[],
+    "subject" TEXT,
+    "error" TEXT,
+    "sent_at" TIMESTAMP(3),
+    "delivered_at" TIMESTAMP(3),
+    "opened_at" TIMESTAMP(3),
+    "clicked_at" TIMESTAMP(3),
+    "bounced_at" TIMESTAMP(3),
+    "complained_at" TIMESTAMP(3),
+    "unsubscribed_at" TIMESTAMP(3),
+    "created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+
+    CONSTRAINT "nudge_sends_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateTable
+CREATE TABLE "nudge_clicks" (
+    "id" SERIAL NOT NULL,
+    "send_id" TEXT NOT NULL,
+    "job_id" INTEGER,
+    "source" TEXT NOT NULL,
+    "url" TEXT,
+    "created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+
+    CONSTRAINT "nudge_clicks_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateTable
+CREATE TABLE "email_events" (
+    "id" SERIAL NOT NULL,
+    "provider_event_id" TEXT NOT NULL,
+    "type" TEXT NOT NULL,
+    "message_id" TEXT,
+    "payload" JSONB NOT NULL,
+    "created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+
+    CONSTRAINT "email_events_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateTable
+CREATE TABLE "app_settings" (
+    "key" TEXT NOT NULL,
+    "value" TEXT NOT NULL,
+    "updated_at" TIMESTAMP(3) NOT NULL,
+
+    CONSTRAINT "app_settings_pkey" PRIMARY KEY ("key")
+);
+
+-- CreateIndex
+CREATE UNIQUE INDEX "email_preferences_user_id_key" ON "email_preferences"("user_id");
+
+-- CreateIndex
+CREATE UNIQUE INDEX "email_preferences_unsubscribe_token_key" ON "email_preferences"("unsubscribe_token");
+
+-- CreateIndex
+CREATE INDEX "email_preferences_subscribed_frequency_idx" ON "email_preferences"("subscribed", "frequency");
+
+-- CreateIndex
+CREATE UNIQUE INDEX "campaigns_key_key" ON "campaigns"("key");
+
+-- CreateIndex
+CREATE INDEX "campaigns_status_idx" ON "campaigns"("status");
+
+-- CreateIndex
+CREATE UNIQUE INDEX "campaign_variants_campaign_id_key_key" ON "campaign_variants"("campaign_id", "key");
+
+-- CreateIndex
+CREATE UNIQUE INDEX "nudge_sends_provider_message_id_key" ON "nudge_sends"("provider_message_id");
+
+-- CreateIndex
+CREATE INDEX "nudge_sends_user_id_idx" ON "nudge_sends"("user_id");
+
+-- CreateIndex
+CREATE INDEX "nudge_sends_status_idx" ON "nudge_sends"("status");
+
+-- CreateIndex
+CREATE UNIQUE INDEX "nudge_sends_campaign_id_user_id_key" ON "nudge_sends"("campaign_id", "user_id");
+
+-- CreateIndex
+CREATE INDEX "nudge_clicks_send_id_idx" ON "nudge_clicks"("send_id");
+
+-- CreateIndex
+CREATE INDEX "nudge_clicks_job_id_idx" ON "nudge_clicks"("job_id");
+
+-- CreateIndex
+CREATE UNIQUE INDEX "email_events_provider_event_id_key" ON "email_events"("provider_event_id");
+
+-- CreateIndex
+CREATE INDEX "email_events_message_id_idx" ON "email_events"("message_id");
+
+-- AddForeignKey
+ALTER TABLE "campaign_variants" ADD CONSTRAINT "campaign_variants_campaign_id_fkey" FOREIGN KEY ("campaign_id") REFERENCES "campaigns"("id") ON DELETE CASCADE ON UPDATE CASCADE;
+
+-- AddForeignKey
+ALTER TABLE "nudge_sends" ADD CONSTRAINT "nudge_sends_campaign_id_fkey" FOREIGN KEY ("campaign_id") REFERENCES "campaigns"("id") ON DELETE CASCADE ON UPDATE CASCADE;
+
+-- AddForeignKey
+ALTER TABLE "nudge_sends" ADD CONSTRAINT "nudge_sends_variant_id_fkey" FOREIGN KEY ("variant_id") REFERENCES "campaign_variants"("id") ON DELETE SET NULL ON UPDATE CASCADE;
+
+-- AddForeignKey
+ALTER TABLE "nudge_clicks" ADD CONSTRAINT "nudge_clicks_send_id_fkey" FOREIGN KEY ("send_id") REFERENCES "nudge_sends"("id") ON DELETE CASCADE ON UPDATE CASCADE;
+