Skip to content

feat(cli): report every command run to a debug function in astryx.config - #4812

Open
josephfarina wants to merge 2 commits into
mainfrom
feat/cli-debug
Open

feat(cli): report every command run to a debug function in astryx.config#4812
josephfarina wants to merge 2 commits into
mainfrom
feat/cli-debug

Conversation

@josephfarina

Copy link
Copy Markdown
Contributor

Summary

Set debug in astryx.config and that function receives one event per command run:

export default {
  debug: event => appendFileSync('runs.ndjson', JSON.stringify(event) + '\n'),
};

Setting it is the whole opt-in — there is no flag, no env var, and no astryx debug command. Leave it out and nothing is recorded and nothing is stored anywhere.

Each event carries the command, its arguments and flags (with their Commander source, so a typed flag is distinguishable from a default), the outcome, exit code, duration, error code, an environment snapshot including which coding agent invoked the CLI, and everything the command printed to stdout and stderr.

That last part is the reason to have this at all: you can see the answer a user actually got, not only what they asked for. astryx build "analytics dashboard" records the prompt and the composition kit returned, which is the pair you need to judge whether the answer was any good.

Second commit is independent: astryx init --json now works.

Where the seams are, and why

Coverage is the whole point, so the choices are all in service of not missing failures.

  • Delivery happens at process.on('exit'). cliError() calls process.exit() synchronously, so a try/finally around an action never runs on an error and postAction hooks are skipped. Anything hooked to normal completion would report successes and almost no failures.
  • The handler loads before Commander parses. Parse errors and --help short-circuit before any hook runs, so anywhere later would leave exactly the failures you most want reported with nowhere to report them.
  • Signals are handled separately, because exit does not fire for them — a theme build --watch someone Ctrl-Cs would otherwise vanish. The handler removes itself and re-raises, so Ctrl-C still terminates and the parent still sees true signal death.
  • Capture is a tee on process.stdout.write / process.stderr.write. One seam catches emit(), JSON envelopes, cliError, Commander's own help text, and direct writes.

All 24 commands are covered without any of them knowing this exists; there is no per-command wiring to forget.

Contract

DebugEvent is published from @astryxdesign/cli/debug with a sealed zod validator (parseDebugEvent) drift-locked to the type, so the recorder cannot add a field without publishing it. schemaVersion is a literal, so widening it turns every consumer's branch into a compile error rather than a silent misread.

Handlers run synchronously at exit and receive a copy. One that throws — or mutates what it was given — can neither fail the command nor affect anything else. A returned promise is never awaited, so network delivery from inside the handler will not work; write a file or spawn a detached child.

Values are scrubbed before delivery: home paths become ~, project paths become relative, emails and URL credentials are removed, options with sensitive-looking names are dropped, and recognizable credential formats (GitHub, Slack, AWS, JWT, bearer) are stripped wherever they appear. Verified on a real session: zero home paths and zero username occurrences across 37 recorded runs.

The known limit, documented rather than hidden: a high-entropy string with no recognizable format under an ordinary option name is recorded as written, because it is indistinguishable from a filename.

Also fixed

A pre-existing gap this uncovered: installJsonShim did one recursive walk at startup, so any command registered afterwards silently kept Commander's default _exit — dropping out of the --json contract and out of parse-error attribution. It now shims commands as they join the tree, removing the ordering requirement entirely.

Test plan

  • pnpm -F @astryxdesign/cli test — 2645 passing. The 9 failures are pre-existing on main (verified by stashing and re-running): macOS case-insensitive-filesystem issues and cwd leakage between parallel workers.
  • typecheck:strict, typecheck:json-api, typecheck:authoring clean — the last of those is the drift-lock proving the published type and the zod schema infer identically.
  • eslint clean; pnpm -F @astryxdesign/cli readme regenerated; drift, manifest and readme-gen gates pass.
  • Every outcome verified end to end against the real binary: success, handled error, parse error, help, unknown command, rejected --json, and Ctrl-C.
  • Output capture verified byte-for-byte — 21150 bytes recorded against 21150 on the terminal — and --json stdout still parses as a single envelope.
  • Overhead with no debug set is within noise (219–233ms with a config present vs 216–224ms without).
  • Hardened against an adversarial chaos run; each finding mutation-tested before its fix landed, so reverting a fix individually turns exactly its test red.

Made with Cursor

josephfarina and others added 2 commits August 7, 2026 16:01
…onfig

Set `debug` in astryx.config and that function receives one event per
command run — the command, its arguments and flags (with their Commander
source, so a typed flag is distinguishable from a default), the outcome,
exit code, duration, error code, an environment snapshot, and everything
the command printed to stdout and stderr. Setting it is the whole opt-in;
leave it out and nothing is recorded and nothing is stored.

Capturing the output is what makes the record useful for improving the
CLI rather than just counting invocations: you can see the answer a user
actually got, not only what they asked for.

Coverage is the point, so the seams are chosen for it. Events are
delivered from a `process.on('exit')` listener because `cliError()` exits
synchronously — anything hooked to normal completion would report
successes and almost no failures. The handler is loaded before Commander
parses, because parse errors and `--help` short-circuit before any hook
runs. Signals are handled separately since `exit` does not fire for them,
and the handler removes itself and re-raises so Ctrl-C still terminates.

`DebugEvent` is published from `@astryxdesign/cli/debug` with a sealed
zod validator drift-locked to the type, so the recorder cannot add a
field without publishing it. Handlers run synchronously and receive a
copy: one that throws, or mutates what it was given, can neither fail the
command nor affect anything else. Values are scrubbed before delivery.

Also fixes a pre-existing gap this uncovered: installJsonShim now shims
commands as they join the command tree rather than in one walk at
startup, so a command registered later can no longer fall out of the
--json contract or lose parse-error attribution.

Co-authored-by: Cursor <cursoragent@cursor.com>
`astryx init --json` emits its install receipt as a standard envelope —
`init.run` with the mode, features run, agent-doc files written, any soft
docsError and the template outcome, or `init.remove` for --remove-agents.
Human output is suppressed so stdout carries only the envelope, and the
exit code is unchanged from human mode.

init was the last side-effecting command still refused by the --json
gate. That gate exists so a command cannot write half a project and only
then report that --json is unsupported; since init() already returned a
typed receipt, the fix was to emit it rather than keep refusing. theme
and layout remain off the allowlist, but both are command groups with no
output of their own.

The tests that used init as their example of an unsupported command move
to theme, which still is one.

Co-authored-by: Cursor <cursoragent@cursor.com>
@vercel

vercel Bot commented Aug 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
astryx Ignored Ignored Aug 7, 2026 11:02pm

Request Review

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant