feat(cli): let storage apply converge to a release you name, before it rolls

docs/invariants.md

@@ -317,9 +317,11 @@ Every convergence of SchemaBot's own storage — at startup, or on an operator's
 unless destroying storage state was explicitly permitted, and decides before it writes. Additivity
 is the whole rule: a statement that loses data and one that removes a schema object without losing
 any are both refusals, because the exposure is the fleet reading that storage, not the rows alone.
-Nothing about which surface asked changes that: an operator's command runs the
-bootstrap rather than a second implementation of it, converges the schema of the binary running it,
-and cannot narrow the permission a deployment already granted. On MySQL such a statement is refused
+Nothing about which surface asked changes that: an operator's command runs the bootstrap rather than
+a second implementation of it, and cannot narrow the permission a deployment already granted. Which
+schema a convergence runs is the operator's to name, so that the storage a release needs can be in
+place before the first pod of it starts; naming one supplies the files the bootstrap computes from
+and nothing else, and every gate above applies to it unchanged. On MySQL such a statement is refused
 unless destructive storage changes are explicitly allowed, and the verdict is the one the plan
 already carries, read rather than re-derived, so the boot refuses exactly what the operator-facing
 plan flagged. The verdict is per statement and the differ emits one combined statement per table, so
@@ -335,7 +337,12 @@ the unsafe verdict the engine's plan reports (`pkg/engine/spirit/spirit.go`) and
 partition that reduces a flagged statement to its additions (`pkg/ddl/additive.go`), which the
 operator-facing storage schema surface reads rather than reimplements (`pkg/api/storage_schema.go`);
 the instance's own storage is the only target a remote caller can address, and the deployment's
-permission is only ever widened, in the adapter that answers for it (`pkg/serve/storage_schema.go`).
+permission is only ever widened, in the adapter that answers for it (`pkg/serve/storage_schema.go`);
+and a named schema is refused apart from the files it names on every transport that carries one
+(`pkg/api/storage_schema_handlers.go`, `pkg/serve/storage_schema.go`), is not converged at all
+until the target has confirmed it would use it (`pkg/api/storage_schema_handlers.go`), and is
+converged by the CLI only behind a confirmation it refuses to skip
+(`pkg/cmd/commands/storage_schema.go`).
 
 ### AV-10: Anything the PR can do, the CLI can do
 

docs/release.md

@@ -284,9 +284,11 @@ schemabot storage plan --deployment west -e production --release v1.4.0
 
 `storage plan` requires the release to be named — `--release`, or `--schema-dir`
 pointed at a checkout — so there is no way to get an answer about a release you
-did not choose. `storage apply` is the command that runs the schema embedded in
-the binary running it, and it takes no selector at all. Operators pre-creating an
-index ahead of the roll should also read [Deploying a release that changes the
+did not choose. `storage apply` takes the same selectors and converges what they
+name, which is how an operator readies the storage before the release that needs
+it rolls; naming one is confirmed at a terminal and refused with
+`--auto-approve`. Operators pre-creating an index ahead of the roll should also
+read [Deploying a release that changes the
 storage
 schema](./storage-schema.md#deploying-a-release-that-changes-the-storage-schema)
 — on MySQL an index drop is refused like any other destructive statement, but

docs/storage-schema.md

@@ -11,6 +11,7 @@
 - [Name the release, and the storage database](#name-the-release-and-the-storage-database)
 - [Converge it](#converge-it)
 - [While it runs](#while-it-runs)
+- [Converging a release before it rolls](#converging-a-release-before-it-rolls)
 - [Deploying a release that changes the storage schema](#deploying-a-release-that-changes-the-storage-schema)
 - [When a pod will not start](#when-a-pod-will-not-start)
 - [Pre-creating indexes on a long-lived database](#pre-creating-indexes-on-a-long-lived-database)
@@ -59,18 +60,22 @@ the startup path                        the CLI path
                    └────────────────┬─────────────────┘
                                     ▼
                   the storage database, converged to the
-                  schema of the binary that ran it, and to
-                  nothing else
+                  schema the run was given: the running
+                  binary's own, or a release an operator named
 ```
 
 Four consequences worth holding onto:
 
 - **`storage apply` is what a boot does**, not an equivalent of it: the same
   differ, the same refusal, the same lock. Converging ahead of a roll cannot
   disagree with what the roll then does, and concurrent runs serialize.
-- **A binary converges its own schema, never another's.** Neither the command
-  line nor the RPC behind it can hand it another release's files, which stops an
-  older binary from treating a newer release's tables as state to prune.
+- **Which schema it converges is yours to name.** With no selector it runs the
+  files the answering binary carries, which is what that binary's own next boot
+  would run. With `--release` or `--schema-dir` it runs that release's files
+  instead, so the storage a release needs can be in place before its first pod
+  starts. Naming a release supplies the files and nothing else: every refusal
+  that guards a boot guards this too, so an older binary handed a newer
+  release's files still cannot prune the newer tables.
 - **Convergence is bounded, and the budget follows the path, not the code.** A
   boot gets five minutes, covering the lock wait and the DDL, because a pod
   converging is a pod not yet serving, and one holding the lock keeps the others
@@ -235,23 +240,28 @@ that only reads the exit status waits for a convergence that is never coming.
 ## Name the release, and the storage database
 
 The live side of a plan is always a read of the database. The desired side is
-one release's schema *files*. One selector is required, and naming both is
-refused rather than resolved by precedence:
+one release's schema *files*, named the same way on both commands: `storage
+plan` requires a selector, and `storage apply` converges the answering binary's
+own schema when given none. Naming both selectors is refused rather than
+resolved by precedence:
 
 - **`--release <tag>`** fetches that tag's `pkg/schema/<dialect>/` files over the
   repository's contents API, for the dialect the live storage runs.
   `--release-repo` points at a fork or mirror (`block/schemabot` by default),
   `GITHUB_API_URL` at a different API host, and `GITHUB_TOKEN` or `GH_TOKEN`
-  authorizes the fetch.
+  authorizes the fetch. Where `GITHUB_API_URL` names a plaintext host, a plan
+  warns and a convergence is refused: anything on that path can rewrite the
+  files before they arrive, and a convergence runs whatever arrives. A host on
+  this machine is exempt, having no network path.
 - **`--schema-dir <path>`** reads `*.sql` from a checkout or an extracted image
   layer, needs no network, and is how you ask about a commit that was never
   tagged. Point it at the dialect directory (`pkg/schema/mysql`), not its parent.
 
-There is no default because the same storage is converged against the release
+A plan has no default because the same storage is converged against the release
 that is running and short of the release about to roll, and an operator who
-assumed the wrong one either rolls into a failing bootstrap or converges storage
-they did not mean to touch. `storage apply` takes neither selector, because a
-convergence runs the schema of the binary running it.
+assumed the wrong one either misreads the report or converges storage they did
+not mean to touch. What the answering binary carries is not a selector either:
+through the API, which release that is depends on which pod took the call.
 
 Which storage database either command reads is stated rather than discovered:
 
@@ -278,9 +288,12 @@ access](auth.md#what-read-and-write-access-include)).
 
 `storage apply` runs the bootstrap described above, so two operators running it
 at once serialize the way two booting pods do. It prints the plan, asks for a
-literal `yes`, and reports what ran, naming the schema embedded in the binary
-that ran it; `--auto-approve` (`-y`) skips the prompt for scripted maintenance.
-A convergence that leaves statements behind prints them as a plan underneath.
+literal `yes`, and reports what ran, naming the schema it converged: the
+answering binary's own, or the release the command named. `--auto-approve`
+(`-y`) skips the prompt for scripted maintenance, and is refused when a release
+is named, for the reason in [Converging a release before it
+rolls](#converging-a-release-before-it-rolls). A convergence that leaves
+statements behind prints them as a plan underneath.
 
 What it does not share with a boot is the budget. A boot gives up after five
 minutes, because a converging pod is not yet serving; this runs under an hour,
@@ -347,7 +360,10 @@ not, so a maintenance script keyed on the status gets them right:
 
 Both non-zero refusals mean the same thing: nothing converged, and a person has
 to decide something before anything does. A script keyed on the status can
-therefore treat them alike and re-read the plan for which one it hit.
+therefore treat them alike and re-read the plan for which one it hit. Under
+`--json` a refusal comes back in the shape a convergence does, so nothing has to
+be re-read to learn which statements were refused: nothing ran, so the planned
+and the remaining halves of the report are the same.
 
 The one place a refused destructive statement is *not* a failure is a startup
 bootstrap, which skips it and converges the safe remainder (AV-9). That
@@ -473,6 +489,88 @@ The line only ever appears when a convergence is detected. Its absence is not a
 statement that the database is idle: the same read answers "nothing running"
 and "could not tell", so a plan never claims the second as the first.
 
+## Converging a release before it rolls
+
+Name the release and `storage apply` converges *its* files, from whatever CLI
+you have to hand:
+
+```bash
+schemabot storage apply --release v1.4.0 --dsn "$STORAGE_DSN"
+```
+
+This is the command's reason for existing: the storage a release needs has to be
+there before the first pod of that release starts, and the pod that would
+otherwise converge it is the one that cannot start until it is.
+
+It is confirmed at a terminal, and `--auto-approve` is refused with it. Naming a
+release substitutes files the answering binary never carried, so until that
+release is deployed the fleet boots against storage holding more than it
+declares, and the confirmation's notice says what that costs for this
+deployment. On MySQL every one of those boots refuses the removals and logs
+them, so the pre-applied state survives and the fleet warns about it until the
+deploy; a release from before indexes were protected is the exception ([What is
+never automatic](#what-is-never-automatic)). The case to watch is a MySQL
+deployment that has set
+[`allow_destructive_schema_changes`](configuration.md#allow_destructive_schema_changes):
+its boots converge destructively, so what you pre-apply is dropped by the next
+pod to start, and this belongs in the deploy rather than ahead of it. On
+PostgreSQL the boots never compute a removal at all, whatever that deployment
+has configured, so the state survives unremarked. Converge close to the deploy,
+re-run `storage plan` just before it, and leave unattended runs to a job with no
+selector, which converges the answering binary's own schema.
+
+What the notice reports is the deployment's own standing policy, never the flags
+on your command. Passing `--allow-unsafe` widens what your convergence may run
+and moves nothing about the pods around you: their boots read the deployment's
+config and have never heard of your request, so state they would have refused to
+drop they still refuse to drop.
+
+Sometimes there is no policy to report, and the notice says so rather than
+picking the reassuring half of it. A `--dsn` target has no deployment config
+behind it, and a target answering from a release older than the field reports
+nothing for it either. The deployment behind either may be of either kind, so
+confirm which before treating pre-applied state as safe until the deploy — check
+that deployment's
+[`allow_destructive_schema_changes`](configuration.md#allow_destructive_schema_changes),
+or converge as part of the deploy instead. PostgreSQL needs no such check: its
+bootstrap is additive-only on every release, so the notice answers from the
+dialect.
+
+A target too old to accept a named schema is refused rather than answered, and
+the two hops the request crosses refuse it differently.
+
+The server is what compares. `--release` and `--schema-dir` reach a data plane
+over gRPC, where a release that predates the fields drops them and works from
+its own embedded schema — silently, since that is what proto does with a field
+it does not know. The report says which schema it used, so the server checks
+that against the one you asked for. A convergence is checked *before* it runs,
+with a diff, because DDL cannot be taken back; it is checked again on the answer,
+because a deployment is many pods and a roll makes them different releases.
+Either refusal is a 502 naming the schema the target actually used.
+
+The CLI-to-server hop refuses earlier and says something else. That body is
+strictly decoded, so a SchemaBot server predating these fields rejects the
+request outright with a 400 about an unknown field, rather than ignoring them.
+The hop fails closed either way; the error just does not mention schemas.
+
+Upgrade that target, or address a release that carries the schema you are
+converging.
+
+The convergence gets no more permission than a boot: a statement that would drop
+a storage table or column is refused as it is at startup, and a PostgreSQL
+column shape the convergence will not run still stops the whole set.
+
+What that does **not** cover is a `--schema-dir` missing files, and the two
+dialects fail it in opposite directions. On MySQL an omitted table reads as
+surplus, so the plan is full of drops and the convergence refuses them: the cost
+is a report that overstates what is surplus, and a table is only lost if someone
+then permits it. On PostgreSQL the convergence is additive-only and walks only
+the files it was given, so an omitted table is not reported at all — a set
+missing most of a release's tables converges cleanly and reports converged, for
+storage that is not ready. Point `--schema-dir` at a release's whole
+`pkg/schema/<dialect>` directory rather than at a hand-assembled subset;
+`--release` cannot get this wrong, because it fetches the directory whole.
+
 ## Deploying a release that changes the storage schema
 
 Every startup converges the storage schema on its own, so the routine case needs
@@ -491,20 +589,23 @@ starting.
 2. **Decide whether the boot should do it.** Additive DDL inside the five-minute
    budget is fine when the tables are small, and not when they are not: on MySQL
    an index added to an existing storage table copies the table, and every pod in
-   the roll pays it. Converge once, ahead of the roll, from a binary of the new
-   release, or that release's image as a one-shot job.
+   the roll pays it. Converge once, ahead of the roll, naming the release you are
+   about to roll: the files come from the tag, so any binary you have to hand
+   converges it.
 
    ```bash
-   schemabot storage apply --dsn "$STORAGE_DSN"   # from the new release's binary
+   schemabot storage apply --release v1.4.0 --dsn "$STORAGE_DSN"
    ```
 
-   This takes the work out of the roll *and* out of the boot's budget: `apply`
-   is the same bootstrap running under an hour rather than five minutes, so a
-   build a boot could not have finished is exactly what this step is for.
-   Expect the bootstrap lock to be held throughout — pods booting in that
-   window will not come up, which is why this belongs ahead of a roll rather
-   than during one. A build too slow even for the hour has to be created by
-   hand instead; see [Pre-creating indexes on a long-lived
+   It prompts, with the notice from [Converging a release before it
+   rolls](#converging-a-release-before-it-rolls), and `--auto-approve` is
+   refused here. This takes the work out of the roll *and* out of the boot's
+   budget: `apply` is the same bootstrap running under an hour rather than five
+   minutes, so a build a boot could not have finished is exactly what this step
+   is for. Expect the bootstrap lock to be held throughout — pods booting in
+   that window will not come up, which is why this belongs ahead of a roll
+   rather than during one. A build too slow even for the hour has to be created
+   by hand instead; see [Pre-creating indexes on a long-lived
    database](#pre-creating-indexes-on-a-long-lived-database).
 
 3. **On MySQL, if you converged ahead of the roll, re-check right before it.** A
@@ -541,10 +642,10 @@ schemabot storage plan --release v1.4.0 --dsn "$STORAGE_DSN"
 
 The statements it prints are the ones the pod is failing on. With no network to
 the repository, `--schema-dir` reads the same files from a checkout of that
-release; with neither, `storage apply --dsn "$STORAGE_DSN"` prints what the
-binary in your hand would run and waits for a `yes`. It clears the statements
-under the same advisory lock the pods are contending for, and the pod starts on
-its next backoff. Two failures look similar in the logs and are not:
+release, on either command: swapping `plan` for `apply` prints them and waits
+for a `yes`. It clears the statements under the same advisory lock the pods are
+contending for, and the pod starts on its next backoff. Two failures look
+similar in the logs and are not:
 
 - **The convergence is refusing something.** The plan names it: a destructive
   statement, or a PostgreSQL column shape needing manual remediation. Nothing