feat(cli): say which storage DDL is still outstanding, and converge it - #1395
Conversation
156ac40 to
4e3ddc9
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved target-routing, security, timeout, cancellation, and convergence-reporting issues remain.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds storage diff and storage apply commands for inspecting and converging storage schemas through direct databases or API routes.
Changes:
- Adds schema-source selection, rendering, and exit status handling.
- Adds direct/API target resolution and storage schema client methods.
- Adds tests for command behavior and schema fetching.
File summaries
| File | Description |
|---|---|
pkg/cmd/main.go |
Registers storage commands and custom exit-code handling. |
pkg/cmd/commands/storage.go |
Resolves direct storage targets and dialects. |
pkg/cmd/commands/storage_schema.go |
Implements diff/apply behavior and output. |
pkg/cmd/commands/storage_schema_test.go |
Tests command behavior and rendering. |
pkg/cmd/commands/storage_schema_source.go |
Loads release or checkout schema sources. |
pkg/cmd/commands/storage_schema_source_test.go |
Tests schema-source resolution and fetching. |
pkg/cmd/commands/exit_code.go |
Defines outstanding-schema exit statuses. |
pkg/cmd/client/request.go |
Supports long-running storage operations. |
pkg/cmd/client/client.go |
Adds storage schema API methods. |
Review details
Suppressed comments (10)
pkg/cmd/client/request.go:35
- The API-routed apply is synchronous and calls the startup bootstrap, whose
EnsureSchemaTimeoutis five minutes, but the server still uses the global 30-secondhttp.Server.WriteTimeout(pkg/serve/serve.go:218-224). Any valid convergence taking more than 30 seconds will have its response closed before this 15-minute client timeout, so the CLI reports failure even though the bootstrap may continue or complete. Give this route a server-side budget compatible with the bootstrap (or make it asynchronous/pollable); increasing only the client timeout is insufficient.
var operatorHTTPClient = &http.Client{Timeout: 15 * time.Minute, Transport: authTransport}
pkg/cmd/commands/storage_schema.go:352
- The direct convergence path only assigns
Version, whileApplyStorageSchemaleavesSchemaSourceasthe schema embedded in this binary. Consequentlystorage apply --config/--dsndoes not identify the running release in its headline or JSON, unlike the API path and the documented output. Attribute both reports withg.Versionhere, asreadDirectalready does.
plannedReport.Version = g.Version
remainingReport.Version = g.Version
pkg/cmd/commands/storage_schema.go:268
- When
--jsonis used without-y, this preview is rendered as human-readable text to stdout, then the final result is encoded as JSON to the same stream after confirmation. The command therefore emits invalid JSON (and can include the prompt) for exactly the interactive mode a caller would otherwise use. Reject--jsonwithout--auto-approve, or move all preview/prompt text off stdout and emit one defined JSON document.
if !cmd.AutoApprove {
// No selector on the preview asks the target about its own embedded
// schema, which is the schema this convergence is about to run.
preview := &StorageDiffCmd{
storageSchemaTargetFlags: cmd.storageSchemaTargetFlags,
pkg/cmd/commands/storage_schema.go:347
- Although this call accepts
ctx, the convergence path currently ignores it:api.ApplyStorageSchemainvokesEnsureSchema, whose MySQL bootstrap creates a newcontext.Background()deadline and does not receive the caller context. Ctrl+C therefore cannot stop a directstorage applywhile DDL is running; it waits for the bootstrap and then the canceled confirmation read reports an error even though the database may already have changed. Thread cancellation through the bootstrap or explicitly document/handle this non-cancelable operation before exposing it here.
plannedReport, remainingReport, err := api.ApplyStorageSchema(ctx, target.dsn, logger,
target.ensureSchemaOptions(cmd.AllowDestructive)...)
pkg/cmd/commands/storage_schema.go:327
storageSchemaConvergenceOutcomereturns success wheneverManualis empty, including when the only remaining changes are refused destructive statements. That makesstorage applyexit 0 whileremaining.Convergedis false and the database still differs, contradicting the documented exit-2 outcome for outstanding statements and weakening AV-9's fail-closed operator signal. Return the existing outstanding status for any non-manual, non-converged result; reserve exit 1 for manual remediation or a convergence error.
func storageSchemaConvergenceOutcome(remaining *apitypes.StorageSchemaReport) error {
if len(remaining.Manual) == 0 {
return nil
pkg/cmd/commands/storage_schema.go:443
- A manual entry aborts the entire convergence before any statement runs, but this section is always titled as if
Outstandingstatements will run automatically. A PostgreSQL report can contain both a safe drift and a manual drift, so the preview would tell the operator that the safe statement will run even thoughApplyStorageSchemareturns before executing anything. That contradicts AV-9's whole-drift gate; use a waiting-on-manual-remediation heading wheneverreport.Manualis non-empty, including in the post-convergence rendering.
{storageSchemaOutstandingTitle, report.Outstanding},
{storageSchemaDestructiveTitle(report), report.Destructive},
{"Needs manual remediation before anything converges", report.Manual},
pkg/cmd/commands/storage_schema.go:527
- When
report.Destructiveis present but not allowed, the hint never names the opt-in needed to run it; it only says to run the release binary, whose default convergence still refuses destructive DDL. The operator is therefore not told to add--allow-destructive(or enable the equivalent policy), contrary to UX-4 and the described action. Include the explicit opt-in in the hint for refused destructive statements.
func storageSchemaDiffHints(report *apitypes.StorageSchemaReport) []string {
return []string{fmt.Sprintf("These are what %s needs in order to match %s. To converge them, run that release's binary against this database — its container image is that release — or let the release's first boot converge them.", storageSchemaDatabaseLabel(report), report.SchemaSource)}
pkg/cmd/commands/storage_schema.go:527
- This hint hardcodes a published release and its container image, but
--schema-diris also a supported desired source and can point at an unreleased checkout. In that mode the output tells the operator to run a release that may not exist, which fails UX-4's actionable-remediation requirement. Branch the remediation text on the source type and tell directory users to run/build the binary from that checkout.
func storageSchemaDiffHints(report *apitypes.StorageSchemaReport) []string {
return []string{fmt.Sprintf("These are what %s needs in order to match %s. To converge them, run that release's binary against this database — its container image is that release — or let the release's first boot converge them.", storageSchemaDatabaseLabel(report), report.SchemaSource)}
pkg/cmd/commands/storage_schema.go:162
- The API route wraps read-only diffs in
api.StorageSchemaDiffTimeout, but this direct path passes the CLI's signal-only context straight toDiffStorageSchema. A blocked catalog query or server can therefore makestorage diff --dsnhang indefinitely even though the shared API contract says a read-only diff is bounded. Use the same timeout around the direct diff while still honoring earlier caller cancellation.
report, err := api.DiffStorageSchema(ctx, target.dsn, desired, logger,
target.ensureSchemaOptions(cmd.AllowDestructive)...)
pkg/cmd/commands/storage_schema.go:289
- If the preview contains only destructive statements and
--allow-destructiveis not effective, this still asksRun these statements?even though the bootstrap will refuse every one and run zero statements. Saying yes therefore confirms a no-op and the preview does not name the flag needed to make the statements runnable; short-circuit this case with an actionable--allow-destructivemessage/status instead of prompting.
confirmed, err := confirmAction(
fmt.Sprintf("\nRun these statements against %s? Only 'yes' will be accepted: ", storageSchemaDatabaseLabel(report)),
"\nConvergence aborted.",
)
- Files reviewed: 9/9 changed files
- Comments generated: 5
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
4e3ddc9 to
57584ca
Compare
57584ca to
4162323
Compare
4162323 to
837acc7
Compare
837acc7 to
5fb0087
Compare
|
🤖 Review findings - created by Kiran's code review agent - for schemabot/pull/1395, 5fb0087. Verdict: 5 findings — 1 blocking (interactive apply skips the bootstrap), 3 non-blocking, 1 suggestion. BlockingInteractive Non-blockingThe non-PostgreSQL refusal now runs after DSN resolution, so the error names the wrong cause. storage.go:201 resolves the target before checking the dialect, so a
Neither General suggestionsMirror The one thing that could have broken, verifiedThe Verified correct
This review was generated by Claude Code (claude-opus-5). |
morgo
left a comment
There was a problem hiding this comment.
🤖 Approving on Morgan's behalf — his AI agent. This is the PR in the stack that refactors code the existing PostgreSQL maintenance commands depend on, so I checked what the refactor dropped before looking at what it adds.
The refactor drops no guard
resolveStorageDSN lost its own --dsn/--config mutual-exclusion check and its own whitespace refusal. Both survive, by falling through rather than by being restated: the new guard is directDSN != "" && configFlag == "", so --dsn with --config and a whitespace-only --dsn both miss it and reach resolveStorageTarget, whose first two checks are exactly those refusals with the same wording. A MySQL DSN passed directly still gets the PostgreSQL-specific refusal naming the operation. Same behaviour, one implementation.
Routing
usesLocalRuntime growing a *CLI parameter to answer storage is the part that could have gone wrong, and the default is the safe one: UsesAPI returns false for any subcommand it does not know, so an unrecognised storage subcommand resolves no endpoint rather than starting a runtime the operator explicitly asked to avoid. TestUsesLocalRuntime_StorageSubcommands covers it.
direct() keying on flag presence rather than content is the right choice and the comment gives the right reason: reading a whitespace --dsn as "no DSN" would silently route through the API and report a different database than the one the operator addressed. That is a worse failure than any error message, and validate() refusing every mixed combination instead of honouring one flag and dropping the other follows the same principle.
Exit status
report.Converged → 0, otherwise exitStorageSchemaOutstanding() → ErrSilent wrapped in &ExitCodeError{Code: 2}. main suppresses the Error: line via errors.Is (which unwraps through ExitCodeError.Unwrap) and exits 2 via errors.As. The three-way split is the right contract for a pre-deploy gate — "converged" and "unreachable" do call for opposite decisions, and collapsing outstanding-work into 1 would make them indistinguishable.
The JSON path takes the same exit status, which is what a script consuming --json needs.
Apply
Confirming against a fresh read rather than a description of the command is the right call for DDL against SchemaBot's own storage, and returning success when the preview comes back converged — rather than prompting for an empty change — is honest. The preview deliberately passing no source selector, so the target reports its own embedded schema, is consistent with the apply actually running that binary's schema.
One gap in the refusal affordance
StorageApplyCmd declares hidden --schema-dir and --release purely so storageSchemaSourceRefusal can explain why they do not apply, instead of Kong emitting a bare "unknown flag". --release-repo is not declared, so:
schemabot storage apply --release v1.4.0 --release-repo block/schemabot-fork
fails at parse time with an unknown-flag error before Run is reached, and the operator never sees the explanation — which is the exact case the hidden flags exist for, since --release-repo is only ever typed alongside --release. Adding it as a third hidden field and ignoring it in the refusal (the --release message is already the right one) would close it. Minor, and not worth holding the stack for.
41/41 green. Note the stack is based on 342f9def, two commits behind current main (7cb9fccc, 5f7e4657) — both tern-side and nowhere near pkg/cmd, so no conflict risk I can see, but worth a rebase before merge so CI has run against what will actually land.
5fb0087 to
a8a0ff5
Compare
a8a0ff5 to
4eade37
Compare
|
🤖 Review findings - created by Kiran's code review agent - for schemabot/pull/1395, 6c35a24. Non-blockingThe destructive gate aborts the whole convergence, so
The direct-connection path — the path the PR exists for — has no test at any layer. General suggestions
The doc comment for The one thing that could have broken, verifiedThe destructive gate could have narrowed standing policy — refusing a DROP the config already permits. It does not: the preview and the convergence pass the same Verified correct
This review was generated by Claude Code (claude-opus-5). |
668edf8 to
fa47dee
Compare
fa47dee to
404504e
Compare
45a7416 to
10fdb76
Compare
10fdb76 to
c9c1e08
Compare
|
🤖 The blocking finding is fixed where you said it belonged, the direct path now has tests, and the plan's trailing hint is gone. The branch has been rebased onto The one-command path to a destructive bootstrap on a local runtime. Fixed in The direct path had no test at any layer. Fixed in
The plan no longer ends with a hint, as of
The destructive gate aborting the whole convergence — left as is, and I think it should change in its own PR. Your argument is the right one and I do not want to wave it off: a boot of the same binary refuses the destructive statement and converges the safe remainder, so a pre-deploy Replied by Claude Code (claude-opus-5) on Armand's behalf. |
803b187 to
0680d5a
Compare
An interactive `storage apply` stopped when its preview found the catalog already matching, which made it do less than the same command with --auto-approve: the bootstrap also clears the schema change engine's leftover tables, which outlive an interrupted convergence and are invisible to a catalog diff, so they stayed on the database. It now runs the convergence either way, and the prompt asks for the run it is about to do rather than for statements there are none of. A PostgreSQL-only repair command now refuses on the storage family before the DSN is fetched. Fetching is not free of consequence — storage dsn_from reads a secret — and refusing afterwards reported whatever went wrong resolving a connection the command was never going to open, instead of the family that does not apply to it. Both halves of a direct convergence are attributed the way the preview's report is, so a run's plan header and its result header name the same schema. --release-repo joins the other release selectors the apply accepts in order to refuse them by name.
The storage commands spelled the consent flag --allow-destructive, which exists nowhere else: `schemabot apply` has always taken --allow-unsafe, and the CLI's own output calls the changes destructive while naming that flag. Two spellings for one concept meant an operator moving between the two commands got kong's unknown-flag error on the one they had just used. The convergence takes the flag. The plan no longer does: it runs nothing, so it has no consent to take, and the normal plan/apply flow offers no preview of an apply's --allow-unsafe either — the plan discloses the destructive statements and names the flag, and seeing them as statements that will run means running the apply and reading its preview. The apply's own preview is that path and still sets the field, and a plan can report those statements as running with no flag at all when the target's standing storage policy allows them. Internal names are untouched. The config key storage.allow_destructive_schema_changes and the API option WithAllowDestructiveSchemaChanges predate this work and already spell the concept "destructive"; only the flag surface was inconsistent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… apply does storage apply converged the safe remainder and reported the refused destructive statements afterwards, so an operator learned about a DROP against SchemaBot's own storage from a summary of what had already run. The rest of the CLI decides that question first: apply shows the plan, names the statements, and stops until --allow-unsafe says otherwise. The gate sits in front of the confirmation, so --auto-approve does not skip it -- consenting to a convergence is not consenting to destroy state. It reads the same preview the attended path already read, which is now read on both paths so the unattended one has something to gate on. A deployment whose storage policy already allows destructive changes has permitted them, and the gate does not narrow it (AV-9). Where it does stop a run it runs strictly less than the convergence would have: the bootstrap refuses the same statements on its own. What changes is when the operator finds out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…efused A blocked convergence now prints the command line that grants consent, the way a blocked schema change apply does. It is built from the flags that addressed this target rather than fixed, because the suggestion has to converge the same storage database the refusal is about -- dropping the flags would name the storage of whichever server the CLI points at, which during a rollback is a different database. A DSN is named rather than repeated: it carries the storage credentials, and this is printed to a terminal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A report carrying both a manual-remediation entry and a destructive statement stopped on the destructive one, but a plan renders every statement as gated while a manual entry is outstanding and prints no refusal for the destructive ones. The run exited non-zero with nothing on screen saying which refusal it hit. The manual entry is the refusal to report: it gates the whole drift set, so a destructive statement behind it is not yet reachable, and its own message names it on both the attended and unattended paths. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The manual-remediation refusal sat inside the attended branch, so a `storage apply -y` against a report carrying one never reached it. With a destructive statement present too the destructive gate deferred to a refusal that could not run, and the command exited non-zero with nothing on screen; with manual entries alone it went on to converge, which is a different contract from the attended path's "resolve these first". Both refusals now run either way, and manual runs first because it gates the whole drift set -- a destructive statement behind it is unreachable rather than merely refused, so naming the flag that permits it would name the wrong remedy. The unattended path prints the plan before the error, since the error says the entries are listed above. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The direct path is the reason these commands exist: the server is down — possibly down because its own schema bootstrap is failing — so the operator points the CLI at the storage database itself. Nothing exercised it. Every test drove the API path through a fake server, so dropping the bootstrap options, inferring the wrong dialect, or losing the report's attribution would have stayed green and surfaced during the incident the path was added for. So it runs against a real, empty storage database: a convergence creates the whole schema from nothing, the plan that follows agrees it did, a plan against an unconverged database reports the whole schema outstanding with its own exit status, and an attended apply's preview names this build as the schema it is about to converge to. While here, direct()'s doc no longer claims a $SCHEMABOT_CONFIG_FILE fallback that cannot fire on these commands: a direct connection is already chosen by the time the config is read, so the env var is the fallback source for the direct-only maintenance commands and is never read here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The plan's hint is gone with the function that built it, so the plan ends on its summary line. What an operator does next is run the convergence from the release's own binary before the roll, which is this command's sibling rather than a paragraph under every plan. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
9591075 to
855e76f
Compare
morgo
left a comment
There was a problem hiding this comment.
🤖 Re-reviewed at 3bd164f. The branch was rebased onto main (the stack below it landed), so I reviewed the delta against 093c57b9, where I approved on 2026-09-15.
The finding I raised is fixed, correctly. I said then that storage apply -y against a report carrying manual entries hit neither refusal: blockDestructiveStorageApply deferred to a manual refusal that sat inside if !cmd.AutoApprove and so could not run, and with manual entries alone the command went on to converge — a different contract from the attended path.
f1265cce extracts blockManualStorageApply and runs it on both paths, ahead of the destructive gate. Three things I checked rather than assumed:
- The ordering is right, not just different. Manual runs first because it gates the whole drift set, which makes a destructive statement behind it unreachable rather than merely refused — so naming
--allow-unsafethere would offer a remedy that does not apply. - Deleting the Manual branch from
blockDestructiveStorageApplyis safe, not a dropped guard. Control cannot reach the destructive gate with manual entries present, because the manual gate returned an error first. The removed branch is genuinely dead. withPlankeeps the error honest. The message says the entries are listed above, and on the unattended path nothing had listed them; the gate now prints the plan there.
The test is the right test. TestStorageApplyCmd_ManualRemediationOutranksTheDestructiveRefusal is table-driven over attended and unattended, with a report carrying both a destructive statement and a manual entry — the exact combination in the finding. It asserts the entries are printed on both paths, that Apply blocked is not printed, and — the assertion that actually proves the behaviour rather than the wording — that the only route touched is POST /api/storage/schema/plan, so nothing converged either way.
Rest of the delta:
796e6af7adds an integration test driving plan and apply over the direct connection, which was the path with the least coverage.6f134b25drops the plan's trailing hint. It sits in theelse ifbranch, so--jsonoutput is untouched; human rendering only.3bd164fetouchesstorage_schema_render.go— I confirmed it changes no non-comment line.
The safety properties I verified last time still hold: the engine gates destructive statements independently at pkg/api/ensure_schema.go, and --allow-unsafe only ever widens the target's standing policy.
CI 41/41 SUCCESS.
What this adds
SchemaBot runs schema changes against your databases, and it has a database of its own: the bookkeeping storage holding plans, applies, checks, leases and locks. Its schema is converged by every release at startup. These two commands let an operator ask what a release will converge, and converge it ahead of time:
storage planis read-only — no DDL, no advisory lock — so it is safe against production with an apply in flight, which is when it is needed.storage applyconverges by calling the same startup bootstrap a booting pod calls, and without-yit previews what it would run and stops, so the preview and the convergence are the same code reading the same database.It inherits the bootstrap's five-minute budget along with the code, since that budget is fixed inside the bootstrap rather than passed in by the caller. So converging ahead of a roll takes the work out of the roll but not out of the budget: DDL too slow to finish during a boot is still too slow here, and still has to be run by hand. Giving the deliberate path a budget of its own is follow-up work.
They render as a plan and an apply because that is what they are: the same header box, the same
+/~/-change symbols and the same summary line asschemabot plan. Only the target differs.The desired side of a plan is always a release you name.
--release <tag>fetches that tag's schema files,--schema-dir <path>reads a checkout, there is no default, and naming both is refused rather than resolved by precedence. One live database gives different answers against different releases, so a plan whose desired side you did not choose is unusable, not merely weaker. (storage applyhere converges the answering binary's own schema; #1413 gives it the same selectors, behind a terminal confirmation.)A destructive statement blocks an apply, exactly as it blocks a schema change apply.
--allow-unsafepermits it; without it nothing converges and the command exits non-zero rather than running the additive remainder and reporting success. The gate sits in front of the confirmation prompt, so-ycannot skip it: consenting to run an apply is not consenting to destroy state. A manual-remediation entry outranks it, since that blocks the whole set and makes a destructive statement unreachable rather than merely refused.A booting pod deliberately does the opposite — it skips the refused statement and converges the safe remainder, because refusing to boot over surplus state left by a rollback would take the deployment down. At a terminal there is someone to decide, so the command stops and lets them.
Before and after
An operator runs
storage applyagainst a storage database that is short acallercolumn onappliesand carries a surpluslegacy_checkstable left behind by a rolled-back release.The command was also called
storage diff, and printed a rendering written only for it.Before —
storage diff --release v1.4.0After —
storage plan --release v1.4.0(exit 2)After —
storage applyblocked by a destructive statement (exit 1)The offered command addresses the same target the blocked run did, so it is safe to copy. A
--dsnrun is the exception: the flag is named rather than repeated, because a storage DSN carries the credentials to SchemaBot's own state and does not belong echoed to a terminal.Invariants
storage applyaccepts no schema source, so a convergence always runs the schema of the binary running it. The destructive gate runs strictly less than the convergence it stops, and grants no permission the deployment's own storage policy had not already granted.Opened by Claude Code (Opus 5).