feat(api): add opt-out usage metrics (spec 133)

[arielshad]

What

Opt-out usage metrics for Shep (spec 133).

Every process (CLI, daemon, feature-agent workers) records about ten content-free events into a local SQLite telemetry_outbox. Only the long-lived server process sends them: the shep start daemon or shep ui. Events go to PostHog's EU /batch/ endpoint over plain fetch.

Controls:

• shep telemetry on|off|identity|contact|status|show
• a Settings section
• a first-run notice in both the CLI and web onboarding

Why

This implements specs/133-telemetry/, the "Metrics" section of the T3 Code product review. Today nobody can tell which areas, commands or agents are used, or whether runs end in merged PRs. The north star is merged PRs per weekly-active install.

The product owner fixed these decisions:

• Usage events are on by default. A notice shows once, in the CLI (stderr) and in web onboarding, with one-click Turn off.
• Always off under CI, DO_NOT_TRACK=1, SHEP_TELEMETRY_DISABLED=1 and test runners.
• Identity is opt-in. Every event carries a random installId.
  • Only after the user turns Include my identity on (off by default) do events also carry:
    • a SHA-256 hash of the agent account id
    • the GitHub username (gh api user)
    • the GitHub owners of the repos' remotes
  • Both notices list these fields under their own heading and say how to turn them on: shep telemetry identity on, or the switch in the notice and in Settings.
  • The setting defaults to off everywhere: the settings default, the migration column default and the mapper fallback all read one constant.
• Contact consent. A separate The Shep team may contact me on GitHub flag (off by default) is sent as a property.
• Agent-agnostic identity. Where to find the account id is data on the agent catalog: an accountIdSource on the Claude and Codex entries. Core has no if (agentType === …).
• Events. install.heartbeat, cli.command, web.area.viewed, feature.created, feature.run.finished, pr.opened, pr.merged, decision.answered (defined only), onboarding.step, error.unhandled.
  • The event names are a TypeSpec enum.
  • TelemetryEventPropertyMap types each event's properties, so a content property does not compile.
• Trust promise. The README Trust and safety section, the FAQ and CLAUDE.md now say what is sent. The full list is in docs/telemetry.md.

Delivery details:

• Each event gets a UUID.
• A failed batch backs off with jitter (2 s, doubling up to 5 min) and is dropped after 5 attempts.
• The outbox is capped at 5,000 events.
• Batches are claimed with a two-minute lease, so the daemon and shep ui never send the same batch twice.
• pr.opened / pr.merged are once-keyed per feature: the merge node, both PR-sync paths and the webhook all see a merge, and it counts once.
• Identity is attached at send time, so turning identity off also covers events already queued.
• Turning metrics off deletes everything queued.

Owner actions before release

1. PostHog project key.
   • Set SHEP_POSTHOG_KEY in the release environment, or fill BUILT_IN_POSTHOG_KEY in packages/core/src/infrastructure/services/telemetry/telemetry-config.ts.
   • Until then nothing is sent; events wait in the capped outbox.
2. PostHog project setting. Turn on Discard client IP data. GeoIP is already disabled per event.

The GDPR question in the spec is resolved by the owner's decision to make identity opt-in.

Screenshots / Recording

Storybook captures from pnpm build:storybook, in light and dark.

Web onboarding notice (one-click Turn off; identity fields listed apart, identity off by default)	Settings → Usage metrics

When the environment forces metrics off (DO_NOT_TRACK), the switches are locked:

The CLI first-run notice, printed once to stderr:

Shep collects usage metrics
To learn which features are used, Shep sends usage events to the Shep team (PostHog, EU region). It never sends prompts, code, file paths, repository or branch names, feature titles or error messages.
What is sent:
  - a random install id
  - which commands, pages and features you use, and how runs end
  - Shep version, operating system and Node.js version
  - whether the Shep team may contact you on GitHub (no, unless you allow it)
Sent only if you turn identity on (off by default):
  - a SHA-256 hash of your Claude or Codex account id
  - your GitHub username (read with gh)
  - the GitHub owners (users or organisations) of the repositories you use with Shep
These identify you, so they stay off unless you turn them on: shep telemetry identity on
Turn metrics off: shep telemetry off (or set DO_NOT_TRACK=1). See what is sent: shep telemetry show

Testing

All of these ran locally on the final tree, after merging main:

• pnpm lint, pnpm format:check, pnpm typecheck, pnpm typecheck:web and pnpm lint:web pass.
• pnpm generate leaves no diff.
• pnpm test:unit: 1,319 files, 14,830 tests, 0 failed.
• pnpm test:int passes.
• pnpm build and pnpm build:storybook succeed, and check:stories is OK.

TDD, RED first. The identity default had RED tests in four places:

• the settings defaults
• the migration column default (PRAGMA table_info)
• the mapper fallback in both directions
• the status use case with no stored telemetry

The split notice had its own RED tests: a domain field-list test, the use case, the CLI hook output order, and the web identity section.

Earlier coverage:

• Domain: env-state matrix, backoff and buckets, stack-frame reduction, owner parsing.
• Use cases: flush, backoff, drop, opt-out clears, no key sends nothing, the identity toggle strips identity from queued events, heartbeat, notice.
• Adapters: the PostHog body against a stubbed fetch; the identity reader against a temp home.
• Persistence: the outbox against real SQLite; a settings round-trip with two different non-default writes.
• DI: a bootstrap token test.
• Emit sites, CLI and web: component and hook tests.

End to end: I posted the real wire body shape to https://eu.i.posthog.com/batch/ and got 200; a malformed body gets 400. PostHog answers 200 for any key, so this proves the format, not a project.

Checklist

• pnpm lint passes
• pnpm format:check passes
• pnpm typecheck passes
• pnpm test:unit and pnpm test:int pass
• pnpm build succeeds
• (UI only) pnpm build:storybook succeeds; every new component has a colocated .stories.tsx
• (Domain changes) pnpm generate ran; output.ts and apis/json-schema/ are committed
• (New use case) Tests landed RED-first per the TDD guide
• No domain/ or application/ file imports anything from infrastructure/
• Commit messages follow Conventional Commits
• Updated LESSONS.md

Notes for reviewers

• Migration number. The migration is now 168-create-telemetry.ts: feat(agents): one Decision primitive rendered inline everywhere (spec 134, part 1) #922 owns 166, and feat(agents): background agents ask with a recommendation and a deadline (spec 134, part 2) #924 takes 167.
• Files over the 300-line rule. settings-page-client.tsx, settings.mapper.ts, create-feature.use-case.ts, feature-agent-worker.ts, merge.node.ts and agent-catalog.ts were already past the limit. This PR only adds injection and one call to each. New code lives in its own modules. Splitting the large files is left for a dedicated refactor.
• IClock registration moved to register-services, so every container has a clock.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y84mkYCEBtd3RMaSvpSLz5

[arielshad]

Unit Tests (windows-latest) on 5d8c7f8: 14,683 tests passed. The job still failed because the Vitest worker running tests/unit/infrastructure/services/agents/dev-server-agent/dev-server-agent.service.test.ts exited unexpectedly ("Worker forks emitted error", no stack). Every other file reported (1306/1307).

• That test only uses in-memory mocks and spawns no processes. This PR doesn't touch it or the service it tests.
• It passes 16/16 locally, in 5 out of 5 runs.
• The same job was green on ac8979f.

The job is running again on the new head 70a0529. I'll investigate the worker exit if it recurs there.

---

Generated by Claude Code