feat(api): answer storage schema requests for the storage a server booted on - #1404
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Critical target-pinning and credential-redaction issues, along with additional correctness and coverage gaps, remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Implements storage-schema plan/apply RPCs for the database the server booted against, including target checks, HTTP/gRPC wiring, and version attribution.
Changes:
- Captures and validates boot storage identity while allowing credential rotation.
- Adds storage-schema planning and convergence behavior.
- Updates build, target, adapter, and integration tests.
File summaries
| File | Review summary |
|---|---|
pkg/serve/storage_target.go |
Adds storage identity parsing. Critical (3 votes): parsing bypasses DSN redaction and may leak credentials in logs. |
pkg/serve/storage_target_test.go |
Tests target parsing, credential rotation, and target movement behavior. |
pkg/serve/storage_schema.go |
Implements schema plan/apply handling. Moderate (1 vote each): source-only requests are misclassified; caller validation occurs too late; malformed supplied DDL is classified as internal; successful/cancellation paths lack coverage. Nit (1 vote): update the AV-9 enforcement pointer. |
pkg/serve/storage_schema_test.go |
Tests adapter validation, policies, and target handling. |
pkg/serve/serve.go |
Records boot state and wires services. Critical (1 vote): pool reloads can switch storage targets. Moderate (1 vote): missing end-to-end registration coverage. Nit (1 vote): update the AV-9 enforcement pointer. |
pkg/serve/serve_storage_integration_test.go |
Updates storage boot integration coverage. |
pkg/serve/serve_build_test.go |
Updates server construction fixtures. |
Review details
Suppressed comments (7)
pkg/serve/serve.go:542
- The new Build wiring is not covered end to end: the added tests exercise
storageSchemaAdapterdirectly and exercise HTTP/gRPC routing with fakes, but no test builds a server and verifies that its local HTTP handler and registered gRPC service expose this concrete adapter. A regression that drops eitherSetStorageSchemaServiceorWithStorageSchemaServicewould therefore leave the advertised data-plane surface unavailable while all current tests pass. Add a storage-backed build/integration test for both registrations.
storageSchema, err := srv.newStorageSchemaService()
if err != nil {
return nil, fmt.Errorf("build storage schema service: %w", err)
}
srv.storageSchema = storageSchema
svc.SetStorageSchemaService(storageSchema)
pkg/serve/serve.go:542
- This introduces the data-plane binding that makes AV-9's "the binary running it" guarantee true, but
docs/invariants.md:327-329still lists only the API bootstrap and storage-schema implementation as enforcement. Update the canonical AV-9 enforcement pointer to include this serving adapter/boot-target check; otherwise the registry does not identify the new safety gate.
storageSchema, err := srv.newStorageSchemaService()
if err != nil {
return nil, fmt.Errorf("build storage schema service: %w", err)
}
srv.storageSchema = storageSchema
svc.SetStorageSchemaService(storageSchema)
pkg/serve/storage_schema.go:193
- This adds the data-plane side of AV-9:
checkBootTargetis what prevents the storage surface from reading or converging a DSN different from the database this instance booted on. The AV-9 registry still lists onlypkg/api/ensure_schema*.goandpkg/api/storage_schema.goin itsEnforced:line, so the invariant documentation no longer identifies all of the enforcement added here. Please update that pointer indocs/invariants.mdin the same change.
func (a *storageSchemaAdapter) checkBootTarget(dsn string) error {
resolved, err := storageTargetFor(a.dialect, dsn)
if err != nil {
return fmt.Errorf("read the storage target the current configuration names: %w", err)
}
pkg/serve/storage_schema.go:106
- This branch treats every request with zero
schema_filesas a request for the running binary's embedded schema. AStorageSchemaPlanRequestcan also arrive directly over gRPC with onlyschema_sourceset (the HTTP validator is not on that path), so the source is silently ignored and the response is attributed to the wrong schema instead of returningInvalidArgument. Distinguish an actually empty request from a source-without-files before defaulting to the embedded schema.
if len(files) == 0 {
pkg/serve/storage_schema.go:81
- The caller's schema is validated only after resolving and checking the mutable storage DSN. If the request has an invalid schema (for example, files without a source) while the DSN is unreadable or has moved,
targetreturns a non-sentinel error first, so the gRPC wrapper reportsInternaland hides the fixable caller error. ValidatedesiredSchemabeforetargetso caller input consistently reachesErrInvalidStorageSchemaRequest.
dsn, opts, err := a.target(req.GetAllowDestructive())
if err != nil {
return nil, err
}
desired, err := a.desiredSchema(req)
pkg/serve/storage_schema.go:90
- When the supplied file contents are syntactically invalid or violate a dialect's schema-file shape,
api.PlanStorageSchemareturns a plain parser/shape error here. Because onlydesiredSchema's metadata errors are wrapped withtern.ErrInvalidStorageSchemaRequest, the gRPC/HTTP layers classify these caller-authored files as Internal and hide the actionable reason. Propagate the caller-error sentinel from the validation/parser path (without wrapping storage/dial failures) so malformed supplied DDL is returned as InvalidArgument.
report, err := api.PlanStorageSchema(diffCtx, dsn, desired, a.logger, opts...)
if err != nil {
return nil, fmt.Errorf("diff storage schema (dialect %s) against %s: %w", a.dialect, desired.Description, err)
pkg/serve/storage_schema.go:142
- The new adapter methods are only tested on pre-database branches (
targetanddesiredSchema); no test exercises a successful plan/apply through this adapter or the documented context split where the two plans observe cancellation butEnsureSchemacontinues. That leaves the central data-plane behavior and option/version wiring unverified. Add an integration test (or seams around the API calls) covering the successful RPC path and cancellation during each phase.
planned, remaining, err := api.ApplyStorageSchema(ctx, dsn, a.logger, opts...)
if err != nil {
return nil, fmt.Errorf("converge storage schema (dialect %s): %w", a.dialect, err)
}
planned.AttributeTo(a.version)
remaining.AttributeTo(a.version)
- Files reviewed: 7/7 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
9e56f32 to
79956e6
Compare
|
🤖 Review findings - created by Kiran's code review agent - for schemabot/pull/1404, 79956e6. Non-blockingNo test covers the wiring this PR exists to add. Neither General suggestionsThe plan timeout is applied twice. The one thing that could have broken, verified
Verified correct
This review was generated by Claude Code (claude-opus-5). |
79956e6 to
beea1c4
Compare
beea1c4 to
45e303e
Compare
|
🤖 Both addressed in No test covers the wiring this PR exists to add. Fixed, and your mutation is the one the new test is built around: The plan timeout is applied twice. Removed the outer one. The inner bound in Replied by Claude Code (claude-opus-5) on Armand's behalf. |
|
🤖 Review findings - created by Kiran's code review agent - for schemabot/pull/1404, 45e303e. Non-blockingThe boot-target pin compares the host spelling byte-for-byte, so the same database under a differently-cased hostname reads as "the storage moved." Nothing binds the storage pool to No test builds a General suggestionsA schema that passes file-name validation but fails the dialect parser is reported as Internal/500 pointing at the data plane's logs. The one thing that could have broken, verified
Verified correct
This review was generated by Claude Code (claude-opus-5). |
b8f853f to
d6a3471
Compare
|
🤖 Review findings - created by Kiran's code review agent - for schemabot/pull/1404, d6a3471. Non-blockingA caller's unparseable schema file returns 500/Internal instead of the promised 400/InvalidArgument. AV-9's Enforced: citation does not name the file that now enforces it. The PostgreSQL boot-target pin compares only the DSN's first host. Nothing tests that The one thing that could have broken, verifiedPinning the storage DSN at boot could have wedged credential rotation: if Verified correct
This review was generated by Claude Code (claude-opus-5). |
d6a3471 to
9d62c4f
Compare
|
🤖 Review findings - created by Kiran's code review agent - for schemabot/pull/1404, 9d62c4f. Non-blockingThe PostgreSQL boot target ignores the schema namespace, so a The gRPC half of "one adapter on both surfaces" has no test. Mutating General suggestionsNothing asserts that
The one thing that could have broken, verifiedThe pinned-DSN guard is the load-bearing mechanism: if it could wedge or mislead the pool, an auth-refused dial would take storage down. Verified it cannot — Verified correct
This review was generated by Claude Code (claude-opus-5). |
6d05348 to
776ad0c
Compare
|
🤖 Three rounds — Both "untested wiring" findings reproduce. I ran them:
A caller's unparseable schema still answers 500. Confirmed at AV-9's Enforced: line does not name the file that now enforces it. Confirmed: it names the two bootstrappers and
The host comparison is byte-for-byte. Confirmed — The two latent ones I am leaving as latent, deliberately. The PostgreSQL pin reading only All six confirmed items are owed work, not disputed findings — tracked, and they land before anything ships this surface. Thanks for holding the line on the gRPC registration across three rounds; the test I wrote for it was the wrong test and only the third repetition made that obvious. Replied by Claude Code (claude-opus-5) on Armand's behalf. |
Nothing failed when the HTTP registration was removed, which is the whole point of the wiring: one adapter has to reach the routes an operator calls and the field the gRPC endpoint registers from, or a surface answers every request as unsupported and reads as a data plane too old to serve it. The wiring now has a name, and a test that fails when either half goes away. The diff also bounded itself twice, once here and once inside PlanStorageSchema, which left two places to change a budget that has one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…port A container's mapped port accepts on the host as soon as Docker creates the mapping, so wait.ForListeningPort is satisfied before PgBouncer binds inside the container. The forwarder then resets the early connection, which reaches the client as a reset partway through the startup handshake rather than as a refused dial -- so a test that opened the pooled DSN first thing failed on a transport error instead of exercising the pooling behavior it asserts. The upstream PostgreSQL container was already gated on a real query. Gate the pooler in front of it the same way, which additionally cannot pass until PgBouncer reaches that upstream and authenticates. The probe and the DSN returned to callers are built by one helper so the probe performs the same handshake the tests do. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
776ad0c to
6dcdfaa
Compare
…ool to one pinned reload Six findings from review of the storage schema adapter, all on the same theme: a fact this instance knows should not reach an operator as something else. A hostname resolves case insensitively, so a secret rewritten with the endpoint in another case was reading as a database that had moved: the reload was refused, and the pool stayed on credentials that no longer authenticate for a database it was already connected to. The host now folds; the database name does not, because it is case sensitive on both engines. The pool's reload is no longer a parameter. Every storage pool must re-resolve through the pinned DSN, so openStoragePool builds it rather than accepting one, and a caller handing it the raw resolver no longer compiles. Caller-supplied schema content is parsed at the door with the dialect's real parser. It would have failed several layers down inside the differ, where a parse error is indistinguishable from the storage database being unreachable — so an operator who mistyped a file was told the server was broken. RegisterGRPC's threading of the adapter is now covered by driving the RPC over a real connection, with both wirings checked: an adapter answers, and no adapter refuses as Unimplemented. The unidentifiable-build sentinel stops at the adapter. It belongs in a log field, where a query for the field finds the pod; in a report it read as a release named "unknown". AV-9's Enforced line now names pkg/serve/storage_schema.go, where the instance's own storage is the only addressable target and the deployment's destructive permission is only ever widened. The invariant is upheld, not changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ted server AZ-6 says local hosting never permits a destructive storage bootstrap, and until now the only way to ask for one was the config key ValidateLocalConfig refuses. The storage schema surface adds a second way: a per-request opt-in that widens the deployment's policy. On a deployed server that is the explicit operator consent AV-9 asks for, arriving through a command an admin had to issue. On a local host it is not the same thing — the local runtime has no authorization that can say who issued it, and a local host can be pointed at a real deployment's storage — so the request is refused rather than honored. It refuses instead of running the safe remainder under a flag it ignored: an operator who asked for the destructive statements has to learn their opt-in did not apply, not read a convergence report that looks as though it did. The refusal names the way forward that does converge everything else. Local hosting travels with the server rather than being re-derived: RunLocal claims it through an unexported option, so neither a config file nor an embedder can assert the boundaries AZ-6 grants without accepting them, or deny them to drop them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Why this matters
The RPCs exist one layer down; this is the data plane implementing them. A server answers storage schema requests for the storage it is actually running on, and refuses to answer for anything else.
What it does
The database is pinned at boot, and credentials are not.
StorageDSN()reads mutable environment, files and structured references, so re-resolving it per call means a config edit that moves storage from database A to B silently makes plan and apply read and write B while the instance is still initialized against A. SoBuildrecords the target it booted against, and every request checks against it:The identity is deliberately address plus database name and nothing else, so a rotated password still compares equal. That asymmetry is the point and it matches the pool: the reloadable storage pool re-resolves its own DSN only after an authentication failure, so credentials are allowed to move underneath a running instance and the database is not. The refusal names both sides, because an operator who sees it has either edited the wrong config or is talking to the wrong pod, and the message has to be enough to tell which:
A caller fault reads as a caller fault. Validation failures on a supplied schema — a missing source, a file name that is a path, an empty file — are wrapped in
tern.ErrInvalidStorageSchemaRequestand surface asInvalidArgumentcarrying the full message, since the text is the caller's own input and naming the offending file is the only way they can fix it. Everything else staysInternalwith a fixed summary pointing at the logs, so a dial failure or a DSN fragment never reaches a client.Apply observes the caller's context on the plans either side of the convergence, and not on the convergence. That is deliberate.
EnsureSchematakes no context and bounds itself withEnsureSchemaTimeout; a convergence abandoned part-way because a caller hung up leaves the storage schema between two releases with nobody watching, while running it out leaves a state the next plan can describe exactly. A caller that disconnects stops waiting for an answer rather than stopping the work.The same pin is enforced one level lower, on the pool itself. The reloadable pool's callback re-resolves the DSN after an authentication failure, and an unguarded callback means one auth failure is all it takes for a config that now names another database to answer the dial — leaving the server on storage it never bootstrapped while the adapter refuses requests because its own pin still says otherwise. Two components disagreeing about which database the process is on is worse than either being wrong, so the callback permits a rotated credential and refuses a moved database, failing the connection rather than relocating the server.
A host binary that embeds SchemaBot and omits
WithBuildInfonow gets its module-graph version stored on the server, not just added to the logger — otherwise every report from that instance is unattributable to a build, which defeats the reason a report names a version at all.Invariants
AV-9, extends. Enforcement reaches the data plane: the adapter is bound at construction to the storage its instance booted on, so no caller — not the control plane, not an operator — can point a convergence at a different database. A later layer (#1413) lets a caller name which schema converges; the binding this layer adds is what keeps that from also naming which database.
AZ-5, upholds. The adapter answers only for the database the instance booted against, and the pool refuses to dial a database that moved underneath it. A config edit cannot silently relocate either one.
Opened by Claude Code (Opus 5).