From d218543a37e5b9851bf4d59fb24bb76a976884f1 Mon Sep 17 00:00:00 2001 From: Kiran Muddukrishna Date: Thu, 10 Sep 2026 14:35:07 +1000 Subject: [PATCH 1/5] capabilities: make the embedded YAML matrix the source of the support tables docs/capabilities.md was hand-edited Markdown, so its tiers, marks, and owner vocabulary could drift from the verdict reasons the engine emits and from the contract in docs/capabilities-contract.md, and no machine consumer could read it. pkg/capabilities embeds capabilities.yaml (53 rows across 7 areas) and exposes it as typed rows behind a small API. Loading validates the closed vocabularies (tier, mark, backend, owner) and the cross-field rules against the real verdict.Reasons(), so a row cannot name a reason the engine does not have or carry an owner outside the tier that allows one. A renderer emits the marked matrix tables and the summary counts between markers in docs/capabilities.md; `make gen-capabilities` regenerates them and a unit test fails when the checked-in document is stale or a marker is malformed. Regeneration against the current document is a no-op and the table cell text is unchanged. This completes step 1 of the capabilities contract; the contract doc, SAFETY.md periphery table, and go.mod (yaml.v3 promoted to a direct dependency) are updated to match. --- Makefile | 5 +- SAFETY.md | 1 + docs/capabilities-contract.md | 2 + docs/capabilities.md | 19 + go.mod | 2 +- internal/cmd/gen-capabilities/main.go | 29 ++ pkg/capabilities/capabilities.go | 218 +++++++++ pkg/capabilities/capabilities.yaml | 637 ++++++++++++++++++++++++++ pkg/capabilities/capabilities_test.go | 80 ++++ 9 files changed, 991 insertions(+), 2 deletions(-) create mode 100644 internal/cmd/gen-capabilities/main.go create mode 100644 pkg/capabilities/capabilities.go create mode 100644 pkg/capabilities/capabilities.yaml create mode 100644 pkg/capabilities/capabilities_test.go diff --git a/Makefile b/Makefile index 2f19470..f0454a5 100644 --- a/Makefile +++ b/Makefile @@ -14,7 +14,10 @@ PG_DSN_LOCAL = postgres://$(PG_USER):$(PG_PASSWORD)@localhost:$(PG_PORT)/$(PG_DA # Which corpus replay project to run (replay//project.conf). REPLAY_PROJECT ?= buzz -.PHONY: build test test-unit test-db test-supported-postgres test-aws-boundary lint setup db-up db-down demos clean demo demo-seed demo-check replay replay-refresh replay-down +.PHONY: build gen-capabilities test test-unit test-db test-supported-postgres test-aws-boundary lint setup db-up db-down demos clean demo demo-seed demo-check replay replay-refresh replay-down + +gen-capabilities: + $(GO) run ./internal/cmd/gen-capabilities build: $(GO) build -o bin/pg-sprite ./cmd/pg-sprite diff --git a/SAFETY.md b/SAFETY.md index 91f4895..ce2563f 100644 --- a/SAFETY.md +++ b/SAFETY.md @@ -29,6 +29,7 @@ The invariant registry (invariant IDs referenced below) lives in | `pkg/schemachange` — orchestrator, **cutover swap + fidelity gate** | ✅ core | package contract exists; orchestrator planned | LK-2, LK-4, ST-5 | | `pkg/statement`, `pkg/planner`, `pkg/schemadiff`, `pkg/router`, `pkg/plan`, `pkg/lint`, `pkg/suggest` — classify/diff/route/report | ❌ periphery¹ | `pkg/statement` (parse boundary), `pkg/schemadiff` (introspect/diff via scratch execute-and-introspect), `pkg/planner` (classifier), `pkg/router` (backend assignment + availability policy), `pkg/plan` (versioned dry-run plan report), `pkg/lint` (offline typed findings), and `pkg/suggest` (advisory rewrites with typed caveats) exist (Phases 2.1–2.5) | (CO-7 holds at the parse boundary) | | `pkg/verdict` — structured outcome contract, rendering, exit codes | ❌ periphery | exists (Phase 1) | — | +| `pkg/capabilities` — embedded, validated support matrix and Markdown rendering | ❌ periphery | exists | — | | `pkg/diffplan` — desired schema → routed convergence plan, the declarative front door as a library (the CLI `diff` and embedding orchestrators share it) | ❌ periphery | exists | — | | `pkg/migrate` — one gated statement → resolve, classify, route, execute → one verdict; the imperative front door as a library (the CLI `migrate` and embedding orchestrators share it), plus the desired-state execution loop (`RunDesired`: derive the convergence plan, admit it as a whole, run each planned statement back through the same pipeline) | ❌ periphery² | exists | — | | `internal/cli` — CLI, flags, help, prompts | ❌ periphery | `migrate`, `status`, `diff`, `fmt`, `lint`, and `suggest` exist | — | diff --git a/docs/capabilities-contract.md b/docs/capabilities-contract.md index 6a91085..c976ba4 100644 --- a/docs/capabilities-contract.md +++ b/docs/capabilities-contract.md @@ -263,6 +263,8 @@ does not change any capability, tier, refusal, or runtime behavior. Implementation order is: +**Status:** step 1 is complete; steps 2–4 remain planned. + 1. add the typed package, `pkg/capabilities/capabilities.yaml`, validator, generator, and markers together, making the repository single-source on day one; 2. add `pg-sprite capabilities`, including `--json` and the embedded binary version; diff --git a/docs/capabilities.md b/docs/capabilities.md index b4e0de2..58c31a6 100644 --- a/docs/capabilities.md +++ b/docs/capabilities.md @@ -138,8 +138,13 @@ the canonical example. ## Support matrix +> **Editing the matrix:** edit `../pkg/capabilities/capabilities.yaml`, then run +> `make gen-capabilities`; do not edit the generated regions below by hand. + + **53 operations: 17 supported today, 20 planned behind a typed refusal, 14 out of scope by design, and 2 with no online mechanism in PostgreSQL to build on.** + Status legend: ✅ T1 (supported today) · 🟡 T2 (planned; typed refusal today) · ⚪ T3 (out of scope; **no online-safety problem** — run directly, or through whatever @@ -149,6 +154,7 @@ review the object warrants) · ### Column changes + | Operation | Status | Engine path | Online-safety problem? | Behavior and why | | --- | --- | --- | --- | --- | | `ADD COLUMN` (no default, or constant default) | ✅ | native, as-is | Yes | Metadata-only / fast default (PG 11+); executes instantly under bounded locks | @@ -162,9 +168,11 @@ review the object warrants) · | `SET NOT NULL` | ✅ | native, safer sequence | Yes | Executed as the native four-step pattern: `ADD CONSTRAINT ... CHECK (col IS NOT NULL) NOT VALID` → online `VALIDATE` → `SET NOT NULL` (catalog flip, PG 12+) → drop the scaffold check | | `RENAME COLUMN` / `RENAME TABLE` | ✅ | native, as-is | Yes | Metadata-only for PostgreSQL but **app-breaking** across deployed instances; executed with a typed reason so lint/plan consumers can steer away | | `SET TABLESPACE` | 🟡 | copy-and-swap | Yes | Physical relocation is a rewrite; copy-and-swap route | + ### Constraints + | Operation | Status | Engine path | Online-safety problem? | Behavior and why | | --- | --- | --- | --- | --- | | `ADD PRIMARY KEY` / `ADD UNIQUE` (plain key columns) | ✅ | native, safer sequence | Yes | Rewritten to the online sequence: `CREATE UNIQUE INDEX CONCURRENTLY` → `ADD CONSTRAINT ... USING INDEX` | @@ -173,9 +181,11 @@ review the object warrants) · | `ADD FOREIGN KEY ... NOT VALID` on a **partitioned parent** | 🟡 | native, planned flow | Yes | PostgreSQL supports this only from version 18; refused on 14–17 | | `EXCLUDE` constraints (and unrecognized constraint forms) | ❌ | — | Yes — unsolvable today | No online pattern exists in PostgreSQL — the build scans under `ACCESS EXCLUSIVE` with no `NOT VALID`/`USING INDEX` equivalent. Refused; revisit only if PostgreSQL grows one | | `DROP CONSTRAINT` | ✅ | native, as-is | Yes | Metadata-only; flagged **destructive** | + ### Indexes + | Operation | Status | Engine path | Online-safety problem? | Behavior and why | | --- | --- | --- | --- | --- | | `CREATE [UNIQUE] INDEX` on a plain table — including partial, expression, covering (`INCLUDE`), GIN/GiST/BRIN | ✅ | native, safer sequence | Yes | Executed as (or rewritten to) `CREATE INDEX CONCURRENTLY`, with validity verification, typed invalid-index outcomes, and a proven recovery for abandoned leftovers (`RebuildAbandonedIndex`, or `DropAbandonedIndex` to remove the leftover without rebuilding; both library-only; [runbook](invalid-index-recovery.md)) | @@ -183,9 +193,11 @@ review the object warrants) · | `REINDEX` | ✅ | native, safer sequence | Yes | Rewritten to `REINDEX ... CONCURRENTLY` | | Index build on a **partitioned parent** | 🟡 | native, planned flow | Yes | PostgreSQL has no parent-level `CONCURRENTLY`; the blocking form is refused by policy (`--force` does not bypass it). The partition-aware flow — `CREATE INDEX ON ONLY` → per-partition CIC → `ATTACH PARTITION`, with crash-resume per leaf — is planned | | `ADD CONSTRAINT ... USING INDEX` on a partitioned parent | ❌ | — | Yes — unsolvable today | PostgreSQL does not support adopting an index on a partitioned parent in any supported version; refused before execution | + ### Partitioned tables + | Operation | Status | Engine path | Online-safety problem? | Behavior and why | | --- | --- | --- | --- | --- | | `CREATE TABLE ... PARTITION OF` | 🟡 | native, planned flow | Yes | Typed refusal at both doors: the imperative door does not take `CREATE TABLE`, and the declarative create path refuses the form at plan time and re-checks it at apply — attaching a partition takes a brief `ACCESS EXCLUSIVE` on the **parent**, which the greenfield absence proof does not cover. The partition-aware flow is planned | @@ -193,9 +205,11 @@ review the object warrants) · | `DETACH PARTITION [CONCURRENTLY]` | ✅ | native, safer sequence | Yes | `CONCURRENTLY` is the idiom; the blocking form is rewritten to it | | Partitioned parents in the **declarative model** | 🟡 | native, planned flow | Yes | Typed refusal: the model does not yet carry partition keys, and rendering a partitioned parent as a plain `CREATE TABLE` would be silently wrong | | Partitioned tables in **copy-and-swap** | 🟡 | copy-and-swap | Yes | Root-vs-leaf publication semantics and per-partition swap; sequenced after the copy engine core | + ### The declarative model (desired files, diff, pull) + | Table shape | Status | Engine path | Online-safety problem? | Behavior and why | | --- | --- | --- | --- | --- | | Plain tables + their indexes | ✅ | native, as-is | Yes | `diff`, `pull`, and desired-file rendering round-trip the canonical model. The model carries each index's validity (`pg_index.indisvalid`): on a plain table an invalid entry is a concurrent build that did not finish — abandoned, or still running — so a live entry with the desired name and definition does not deliver the desired index, and `diff` plans it as a `create-index` change. `diff` never emits a drop for an invalid entry, whatever the desired file says about its name: a plain `DROP INDEX` blocks the table and cannot tell abandoned debris from a build still in progress. A `pull`ed baseline of such a table therefore re-diffs to that one rebuild rather than to zero. On a partitioned parent an invalid index means unattached partition indexes, not an unfinished build, so validity plays no part in the parent's comparison | @@ -205,9 +219,11 @@ review the object warrants) · | Explicit column collations | 🟡 | native, planned flow | Yes | Typed refusal: dropping a `COLLATE` clause from a rendered baseline silently changes sort order and index semantics; a collation delta cannot converge without a rewrite | | Columns whose default uses a sequence the column does not own | 🟡 | native, planned flow | Yes | Typed refusal: in a desired-state model that sequence exists only inside the scratch transaction, so no derived plan can reference it. Column-owned (`serial`-style) sequences are fine | | Greenfield `CREATE TABLE` apply (the table does not exist yet — a fresh database or a new table in a live one) | ✅ | native, as-is | Yes — a `REFERENCES` clause would take a brief `SHARE ROW EXCLUSIVE` on each **referenced** live table, but desired files refuse foreign keys today, so no live table is locked | Desired-state execution creates the table: `CheckTableAbsent` verifies the table relation and composite-type name are free, the executor verifies every relation name the desired file states (explicit index names and first-choice constraint-index and column-sequence names) is free in the schema, and `CheckCreatePrivileges` verifies the role can create there. It then runs the `CREATE TABLE` and index builds as brief bounded steps under the engine's budgets. An occupied claimed name is a typed `create-collision` refusal before execution — drop or rename the occupant, name a constraint's index explicitly, or for a sequence use an explicitly named sequence or a non-serial column. Duplicate-name SQLSTATEs backstop races for explicit names; for server-chosen names, the probe narrows the race to the time-of-check window, and after the `CREATE TABLE` commits the executor reads the constraint-index and sequence names the table actually owns and compares them against the claimed first-choice names — a name taken inside the window makes the server pick a suffixed replacement, which surfaces as a typed `create-name-mismatch` failure at step 1 with the born table left in place for an operator to rename the relation or drop, then re-diff (a read of the owned names that does not complete is the same step-1 failure as `create-names-unverified`, never a pass). `PARTITION OF`, `INHERITS`, `LIKE`, `OF`, `IF NOT EXISTS`, and in-set duplicate names refuse at plan time and are re-checked at apply, while `REFERENCES` and `CONCURRENTLY` are refused upstream at desired-file parse and re-checked at admission as defense in depth | + ### Types and non-table objects + | Object / operation | Status | Engine path | Online-safety problem? | Behavior and why | | --- | --- | --- | --- | --- | | Enum-typed columns on plain tables | 🟡 | native, planned flow | Yes | Tolerance end to end (introspection already canonicalizes via `format_type`; desired-file admission and transaction-scoped scratch-schema mechanics are being verified) | @@ -222,9 +238,11 @@ review the object warrants) · | Grants, roles, row-level-security policies | 🔵 | — | No — provisioning / IaC | Access control, not table shape; belongs to provisioning (see [engine-role.md](engine-role.md) for what the *engine's own* role needs) | | Standalone sequences | ⚪ | — | No — owner tooling | Transactional catalog work on an object with no readers-and-writers problem | | Publications, subscriptions | 🔵 | — | No — replication provisioning / IaC | Replication provisioning, not table shape (`ALTER PUBLICATION ... ADD TABLE` also takes `SHARE UPDATE EXCLUSIVE` on the table) | + ### Data and whole-table operations + | Operation | Status | Engine path | Online-safety problem? | Behavior and why | | --- | --- | --- | --- | --- | | `DROP TABLE` | ⚪ | — | No — owner tooling, through a reviewed process | Discards the table and its data in one brief `ACCESS EXCLUSIVE` step: there is nothing online for an engine to make safer, only an irreversible decision an operator must own. Both front doors refuse it — the imperative door as an unsupported statement kind, and the declarative diff is single-table scoped, so a live table with no desired file is not in its view. pg-sprite never plans or executes it; accounting for undeclared tables is the whole-schema owner's job, described under [Deliberately operator-owned](#deliberately-operator-owned) | @@ -234,6 +252,7 @@ review the object warrants) · | Whole-schema convergence (apply a directory of desired files, dependency-ordered) | 🔵 | — | No — convergence planners (pg-schema-diff, pgschema, pgdelta) | Convergence planning across objects is a planner's job; pg-sprite stays the execution engine for the table-shape subset | | Versioned schema-change-file workflow (Flyway-style ordered scripts) | 🔵 | — | No — versioned-script runners (Flyway-style) | Declarative-only by design; see [vision.md](vision.md) | | Expand/contract dual-schema versions (pgroll/reshape style) | 🔵 | — | No — pgroll/reshape own this model | Rejected: application invisibility is a core invariant; see [vision.md](vision.md) | + ## Peers share these limits — for different reasons diff --git a/go.mod b/go.mod index 632faf5..cd5e634 100644 --- a/go.mod +++ b/go.mod @@ -16,6 +16,7 @@ require ( github.com/testcontainers/testcontainers-go v0.43.0 github.com/testcontainers/testcontainers-go/modules/postgres v0.43.0 github.com/wasilibs/go-pgquery v0.0.0-20260728010200-155ebad2880e + gopkg.in/yaml.v3 v3.0.1 ) require ( @@ -78,5 +79,4 @@ require ( golang.org/x/sys v0.45.0 // indirect golang.org/x/text v0.37.0 // indirect google.golang.org/protobuf v1.36.11 // indirect - gopkg.in/yaml.v3 v3.0.1 // indirect ) diff --git a/internal/cmd/gen-capabilities/main.go b/internal/cmd/gen-capabilities/main.go new file mode 100644 index 0000000..7759513 --- /dev/null +++ b/internal/cmd/gen-capabilities/main.go @@ -0,0 +1,29 @@ +// Command gen-capabilities renders the checked-in support matrix from embedded YAML. +package main + +import ( + "fmt" + "os" + + "github.com/block/pg-sprite/pkg/capabilities" +) + +func main() { + const path = "docs/capabilities.md" + rows, err := capabilities.Rows() + if err != nil { + fail(err) + } + input, err := os.ReadFile(path) + if err != nil { + fail(err) + } + output, err := capabilities.RenderDocument(input, rows) + if err != nil { + fail(err) + } + if err = os.WriteFile(path, output, 0o644); err != nil { + fail(err) + } +} +func fail(err error) { fmt.Fprintln(os.Stderr, err); os.Exit(1) } diff --git a/pkg/capabilities/capabilities.go b/pkg/capabilities/capabilities.go new file mode 100644 index 0000000..2b53558 --- /dev/null +++ b/pkg/capabilities/capabilities.go @@ -0,0 +1,218 @@ +// Package capabilities provides the embedded, validated support matrix and its Markdown renderer. +package capabilities + +import ( + "bytes" + _ "embed" + "fmt" + "regexp" + "strings" + + "github.com/block/pg-sprite/pkg/verdict" + "gopkg.in/yaml.v3" +) + +//go:embed capabilities.yaml +var source []byte + +// Area identifies one matrix table. +type Area string + +// Tier describes whether a capability is supported, planned, or out of scope. +type Tier string + +// StatusMark is the human-readable mark associated with a tier. +type StatusMark string + +// EnginePath identifies how the engine executes or plans to execute an operation. +type EnginePath string + +// FrontDoorStatus describes admission through one CLI front door. +type FrontDoorStatus string + +// Closed values used by capability rows. +const ( + AreaColumnChanges Area = "column_changes" + AreaConstraints Area = "constraints" + AreaIndexes Area = "indexes" + AreaPartitionedTables Area = "partitioned_tables" + AreaDeclarativeModel Area = "declarative_model" + AreaTypesAndNonTableObjects Area = "types_and_non_table_objects" + AreaDataAndWholeTableOperations Area = "data_and_whole_table_operations" + TierOne Tier = "t1" + TierTwo Tier = "t2" + TierThree Tier = "t3" + StatusSupported StatusMark = "✅" + StatusPlanned StatusMark = "🟡" + StatusNoSafetyProblem StatusMark = "⚪" + StatusOtherTool StatusMark = "🔵" + StatusNoOnlineMechanism StatusMark = "❌" + PathNativeAsIs EnginePath = "native_as_is" + PathNativeSaferSequence EnginePath = "native_safer_sequence" + PathNativePlannedFlow EnginePath = "native_planned_flow" + PathCopyAndSwap EnginePath = "copy_and_swap" + PathNone EnginePath = "none" + DoorSupported FrontDoorStatus = "supported" + DoorRefused FrontDoorStatus = "refused" + DoorNotApplicable FrontDoorStatus = "not_applicable" +) + +// FrontDoors records admission through the imperative and declarative interfaces. +type FrontDoors struct { + Migrate FrontDoorStatus `yaml:"migrate" json:"migrate"` + Diff FrontDoorStatus `yaml:"diff" json:"diff"` +} + +// Row is one machine-readable support-matrix entry. +type Row struct { + ID string `yaml:"id" json:"id"` + Area Area `yaml:"area" json:"area"` + Operation string `yaml:"operation" json:"operation"` + Tier Tier `yaml:"tier" json:"tier"` + StatusMark StatusMark `yaml:"status_mark" json:"status_mark"` + EnginePath EnginePath `yaml:"engine_path" json:"engine_path"` + OnlineSafetyProblem bool `yaml:"online_safety_problem" json:"online_safety_problem"` + OnlineSafetyDetail string `yaml:"online_safety_detail,omitempty" json:"online_safety_detail,omitempty"` + OwningToolClass string `yaml:"owning_tool_class,omitempty" json:"owning_tool_class,omitempty"` + FrontDoors FrontDoors `yaml:"front_doors" json:"front_doors"` + RefusalReason verdict.Reason `yaml:"refusal_reason,omitempty" json:"refusal_reason,omitempty"` + ReasonNotes string `yaml:"reason_notes" json:"reason_notes"` +} + +type document struct { + Rows []Row `yaml:"rows"` +} + +// Load parses and validates capability YAML. +func Load(data []byte) ([]Row, error) { + var doc document + decoder := yaml.NewDecoder(bytes.NewReader(data)) + decoder.KnownFields(true) + if err := decoder.Decode(&doc); err != nil { + return nil, fmt.Errorf("parse capabilities YAML: %w", err) + } + if err := Validate(doc.Rows); err != nil { + return nil, err + } + return doc.Rows, nil +} + +// Rows returns the validated embedded capability rows in display order. +func Rows() ([]Row, error) { return Load(source) } + +var idPattern = regexp.MustCompile(`^[a-z0-9]+(?:-[a-z0-9]+)*$`) + +// Validate checks closed vocabularies and cross-field invariants. +func Validate(rows []Row) error { + areas := set(AreaColumnChanges, AreaConstraints, AreaIndexes, AreaPartitionedTables, AreaDeclarativeModel, AreaTypesAndNonTableObjects, AreaDataAndWholeTableOperations) + tiers := set(TierOne, TierTwo, TierThree) + marks := set(StatusSupported, StatusPlanned, StatusNoSafetyProblem, StatusOtherTool, StatusNoOnlineMechanism) + paths := set(PathNativeAsIs, PathNativeSaferSequence, PathNativePlannedFlow, PathCopyAndSwap, PathNone) + doors := set(DoorSupported, DoorRefused, DoorNotApplicable) + reasons := map[verdict.Reason]bool{} + for _, reason := range verdict.Reasons() { + reasons[reason] = true + } + ids := map[string]bool{} + for i, row := range rows { + prefix := fmt.Sprintf("row %d", i+1) + if !idPattern.MatchString(row.ID) { + return fmt.Errorf("%s: id %q is not lowercase kebab-case", prefix, row.ID) + } + if ids[row.ID] { + return fmt.Errorf("%s: duplicate id %q", prefix, row.ID) + } + ids[row.ID] = true + if !areas[row.Area] || !tiers[row.Tier] || !marks[row.StatusMark] || !paths[row.EnginePath] || !doors[row.FrontDoors.Migrate] || !doors[row.FrontDoors.Diff] { + return fmt.Errorf("%s: unknown enum value", prefix) + } + if row.Operation == "" || row.ReasonNotes == "" { + return fmt.Errorf("%s: operation and reason_notes are required", prefix) + } + expected := map[StatusMark]Tier{StatusSupported: TierOne, StatusPlanned: TierTwo, StatusNoSafetyProblem: TierThree, StatusOtherTool: TierThree, StatusNoOnlineMechanism: TierThree}[row.StatusMark] + if row.Tier != expected { + return fmt.Errorf("%s: status_mark does not agree with tier", prefix) + } + if row.Tier == TierThree && row.EnginePath != PathNone { + return fmt.Errorf("%s: T3 requires engine_path none", prefix) + } + if row.EnginePath == PathNone && row.Tier != TierThree { + return fmt.Errorf("%s: engine_path none requires T3", prefix) + } + if !row.OnlineSafetyProblem && row.OwningToolClass == "" { + return fmt.Errorf("%s: No row requires owning_tool_class", prefix) + } + if row.OnlineSafetyProblem && row.OwningToolClass != "" { + return fmt.Errorf("%s: Yes row cannot name owning_tool_class", prefix) + } + refused := row.FrontDoors.Migrate == DoorRefused || row.FrontDoors.Diff == DoorRefused + if refused != (row.RefusalReason != "") { + return fmt.Errorf("%s: refusal_reason must be present iff a front door is refused", prefix) + } + if row.RefusalReason != "" && !reasons[row.RefusalReason] { + return fmt.Errorf("%s: unknown refusal_reason %q", prefix, row.RefusalReason) + } + } + return nil +} + +func set[T comparable](values ...T) map[T]bool { + out := make(map[T]bool, len(values)) + for _, v := range values { + out[v] = true + } + return out +} + +// RenderDocument replaces every marked generated region in a capabilities document. +func RenderDocument(input []byte, rows []Row) ([]byte, error) { + areaHeadings := map[Area]string{AreaColumnChanges: "Operation", AreaConstraints: "Operation", AreaIndexes: "Operation", AreaPartitionedTables: "Operation", AreaDeclarativeModel: "Table shape", AreaTypesAndNonTableObjects: "Object / operation", AreaDataAndWholeTableOperations: "Operation"} + pathLabels := map[EnginePath]string{PathNativeAsIs: "native, as-is", PathNativeSaferSequence: "native, safer sequence", PathNativePlannedFlow: "native, planned flow", PathCopyAndSwap: "copy-and-swap", PathNone: "—"} + out := append([]byte(nil), input...) + counts := map[StatusMark]int{} + tiers := map[Tier]int{} + for _, r := range rows { + counts[r.StatusMark]++ + tiers[r.Tier]++ + } + summary := fmt.Sprintf("**%d operations: %d supported today, %d planned behind a typed refusal, %d out of scope\nby design, and %d with no online mechanism in PostgreSQL to build on.**", len(rows), tiers[TierOne], tiers[TierTwo], counts[StatusNoSafetyProblem]+counts[StatusOtherTool], counts[StatusNoOnlineMechanism]) + var err error + out, err = replaceRegion(out, "summary", summary) + if err != nil { + return nil, err + } + for _, area := range []Area{AreaColumnChanges, AreaConstraints, AreaIndexes, AreaPartitionedTables, AreaDeclarativeModel, AreaTypesAndNonTableObjects, AreaDataAndWholeTableOperations} { + var b strings.Builder + fmt.Fprintf(&b, "| %s | Status | Engine path | Online-safety problem? | Behavior and why |\n| --- | --- | --- | --- | --- |\n", areaHeadings[area]) + for _, r := range rows { + if r.Area != area { + continue + } + answer := "Yes" + if !r.OnlineSafetyProblem { + answer = "No — " + r.OwningToolClass + } else if r.OnlineSafetyDetail != "" { + answer += " — " + r.OnlineSafetyDetail + } + fmt.Fprintf(&b, "| %s | %s | %s | %s | %s |\n", r.Operation, r.StatusMark, pathLabels[r.EnginePath], answer, r.ReasonNotes) + } + out, err = replaceRegion(out, string(area), strings.TrimSuffix(b.String(), "\n")) + if err != nil { + return nil, err + } + } + return out, nil +} + +func replaceRegion(doc []byte, name, content string) ([]byte, error) { + begin := []byte("") + end := []byte("") + bi := bytes.Index(doc, begin) + ei := bytes.Index(doc, end) + if bi < 0 || ei < 0 || ei < bi || bytes.Contains(doc[bi+len(begin):], begin) || bytes.Contains(doc[ei+len(end):], end) { + return nil, fmt.Errorf("missing, unbalanced, or duplicate capability markers for %s", name) + } + start := bi + len(begin) + replacement := []byte("\n" + content + "\n") + return append(append(append([]byte(nil), doc[:start]...), replacement...), doc[ei:]...), nil +} diff --git a/pkg/capabilities/capabilities.yaml b/pkg/capabilities/capabilities.yaml new file mode 100644 index 0000000..fd17947 --- /dev/null +++ b/pkg/capabilities/capabilities.yaml @@ -0,0 +1,637 @@ +rows: + - id: add-column-no-default-or-constant-default + area: "column_changes" + operation: "`ADD COLUMN` (no default, or constant default)" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Metadata-only / fast default (PG 11+); executes instantly under bounded locks" + - id: add-column-with-volatile-default-now-gen-random-uuid + area: "column_changes" + operation: "`ADD COLUMN` with volatile default (`now()`, `gen_random_uuid()`, …)" + tier: "t2" + status_mark: "🟡" + engine_path: "copy_and_swap" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: backend-unavailable + reason_notes: "Table rewrite; routes to copy-and-swap and is refused until that engine lands" + - id: add-column-generated-stored + area: "column_changes" + operation: "`ADD COLUMN ... GENERATED ... STORED`" + tier: "t2" + status_mark: "🟡" + engine_path: "copy_and_swap" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: backend-unavailable + reason_notes: "Table rewrite; copy-and-swap route. The copy engine must **recompute, never copy,** generated columns on the shadow table" + - id: add-column-with-inline-unique-primary-key-references-check + area: "column_changes" + operation: "`ADD COLUMN` with inline `UNIQUE`/`PRIMARY KEY`/`REFERENCES`/`CHECK`" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "The inline constraint does its index build or validation scan under the `ADD COLUMN`'s `ACCESS EXCLUSIVE` lock; refused with guidance to add the column first, then build the constraint online" + - id: drop-column + area: "column_changes" + operation: "`DROP COLUMN`" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Metadata-only; flagged **destructive** in the plan report" + - id: alter-column-type-binary-coercible-proven-against-live-column-facts + area: "column_changes" + operation: "`ALTER COLUMN TYPE`, binary-coercible (proven against live column facts)" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Catalog relabel, e.g. `varchar(50)` → `varchar(100)`, `varchar` → `text`; PostgreSQL itself refuses the change when a view, rule, or `STORED` generated column depends on the column — see [binary-coercible-type-changes.md](binary-coercible-type-changes.md#no-rewrite-is-not-no-cost)" + - id: alter-column-type-general-or-with-using + area: "column_changes" + operation: "`ALTER COLUMN TYPE`, general (or with `USING`)" + tier: "t2" + status_mark: "🟡" + engine_path: "copy_and_swap" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: backend-unavailable + reason_notes: "Table rewrite; copy-and-swap route, refused today" + - id: set-default-drop-default-drop-not-null + area: "column_changes" + operation: "`SET DEFAULT` / `DROP DEFAULT` / `DROP NOT NULL`" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Metadata-only" + - id: set-not-null + area: "column_changes" + operation: "`SET NOT NULL`" + tier: "t1" + status_mark: "✅" + engine_path: "native_safer_sequence" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Executed as the native four-step pattern: `ADD CONSTRAINT ... CHECK (col IS NOT NULL) NOT VALID` → online `VALIDATE` → `SET NOT NULL` (catalog flip, PG 12+) → drop the scaffold check" + - id: rename-column-rename-table + area: "column_changes" + operation: "`RENAME COLUMN` / `RENAME TABLE`" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Metadata-only for PostgreSQL but **app-breaking** across deployed instances; executed with a typed reason so lint/plan consumers can steer away" + - id: set-tablespace + area: "column_changes" + operation: "`SET TABLESPACE`" + tier: "t2" + status_mark: "🟡" + engine_path: "copy_and_swap" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: backend-unavailable + reason_notes: "Physical relocation is a rewrite; copy-and-swap route" + - id: add-primary-key-add-unique-plain-key-columns + area: "constraints" + operation: "`ADD PRIMARY KEY` / `ADD UNIQUE` (plain key columns)" + tier: "t1" + status_mark: "✅" + engine_path: "native_safer_sequence" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Rewritten to the online sequence: `CREATE UNIQUE INDEX CONCURRENTLY` → `ADD CONSTRAINT ... USING INDEX`" + - id: add-check-add-foreign-key-imperative + area: "constraints" + operation: "`ADD CHECK` / `ADD FOREIGN KEY` (imperative)" + tier: "t1" + status_mark: "✅" + engine_path: "native_safer_sequence" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Rewritten to the online sequence: `ADD CONSTRAINT ... NOT VALID` (brief metadata lock) → `VALIDATE CONSTRAINT` (writes keep flowing during the scan)" + - id: add-constraint-not-valid-using-index-validate-constraint + area: "constraints" + operation: "`ADD CONSTRAINT ... NOT VALID` / `... USING INDEX` / `VALIDATE CONSTRAINT`" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Already the online idiom; executed as-is" + - id: add-foreign-key-not-valid-on-a-partitioned-parent + area: "constraints" + operation: "`ADD FOREIGN KEY ... NOT VALID` on a **partitioned parent**" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "PostgreSQL supports this only from version 18; refused on 14–17" + - id: exclude-constraints-and-unrecognized-constraint-forms + area: "constraints" + operation: "`EXCLUDE` constraints (and unrecognized constraint forms)" + tier: "t3" + status_mark: "❌" + engine_path: "none" + online_safety_problem: true + online_safety_detail: "unsolvable today" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "No online pattern exists in PostgreSQL — the build scans under `ACCESS EXCLUSIVE` with no `NOT VALID`/`USING INDEX` equivalent. Refused; revisit only if PostgreSQL grows one" + - id: drop-constraint + area: "constraints" + operation: "`DROP CONSTRAINT`" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Metadata-only; flagged **destructive**" + - id: create-unique-index-on-a-plain-table-including-partial-expression-covering-include-gin-gist-brin + area: "indexes" + operation: "`CREATE [UNIQUE] INDEX` on a plain table — including partial, expression, covering (`INCLUDE`), GIN/GiST/BRIN" + tier: "t1" + status_mark: "✅" + engine_path: "native_safer_sequence" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Executed as (or rewritten to) `CREATE INDEX CONCURRENTLY`, with validity verification, typed invalid-index outcomes, and a proven recovery for abandoned leftovers (`RebuildAbandonedIndex`, or `DropAbandonedIndex` to remove the leftover without rebuilding; both library-only; [runbook](invalid-index-recovery.md))" + - id: drop-index + area: "indexes" + operation: "`DROP INDEX`" + tier: "t1" + status_mark: "✅" + engine_path: "native_safer_sequence" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Rewritten to `DROP INDEX CONCURRENTLY`; flagged **destructive**" + - id: reindex + area: "indexes" + operation: "`REINDEX`" + tier: "t1" + status_mark: "✅" + engine_path: "native_safer_sequence" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Rewritten to `REINDEX ... CONCURRENTLY`" + - id: index-build-on-a-partitioned-parent + area: "indexes" + operation: "Index build on a **partitioned parent**" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "PostgreSQL has no parent-level `CONCURRENTLY`; the blocking form is refused by policy (`--force` does not bypass it). The partition-aware flow — `CREATE INDEX ON ONLY` → per-partition CIC → `ATTACH PARTITION`, with crash-resume per leaf — is planned" + - id: add-constraint-using-index-on-a-partitioned-parent + area: "indexes" + operation: "`ADD CONSTRAINT ... USING INDEX` on a partitioned parent" + tier: "t3" + status_mark: "❌" + engine_path: "none" + online_safety_problem: true + online_safety_detail: "unsolvable today" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "PostgreSQL does not support adopting an index on a partitioned parent in any supported version; refused before execution" + - id: create-table-partition-of + area: "partitioned_tables" + operation: "`CREATE TABLE ... PARTITION OF`" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Typed refusal at both doors: the imperative door does not take `CREATE TABLE`, and the declarative create path refuses the form at plan time and re-checks it at apply — attaching a partition takes a brief `ACCESS EXCLUSIVE` on the **parent**, which the greenfield absence proof does not cover. The partition-aware flow is planned" + - id: attach-partition + area: "partitioned_tables" + operation: "`ATTACH PARTITION`" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "Executed; the safer idiom (pre-prove the bound with a validated `CHECK` so the attach skips its scan) is surfaced as guidance. A classify-first flow that constructs the proof itself is planned" + - id: detach-partition-concurrently + area: "partitioned_tables" + operation: "`DETACH PARTITION [CONCURRENTLY]`" + tier: "t1" + status_mark: "✅" + engine_path: "native_safer_sequence" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "`CONCURRENTLY` is the idiom; the blocking form is rewritten to it" + - id: partitioned-parents-in-the-declarative-model + area: "partitioned_tables" + operation: "Partitioned parents in the **declarative model**" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Typed refusal: the model does not yet carry partition keys, and rendering a partitioned parent as a plain `CREATE TABLE` would be silently wrong" + - id: partitioned-tables-in-copy-and-swap + area: "partitioned_tables" + operation: "Partitioned tables in **copy-and-swap**" + tier: "t2" + status_mark: "🟡" + engine_path: "copy_and_swap" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: backend-unavailable + reason_notes: "Root-vs-leaf publication semantics and per-partition swap; sequenced after the copy engine core" + - id: plain-tables-their-indexes + area: "declarative_model" + operation: "Plain tables + their indexes" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + front_doors: + migrate: supported + diff: supported + reason_notes: "`diff`, `pull`, and desired-file rendering round-trip the canonical model. The model carries each index's validity (`pg_index.indisvalid`): on a plain table an invalid entry is a concurrent build that did not finish — abandoned, or still running — so a live entry with the desired name and definition does not deliver the desired index, and `diff` plans it as a `create-index` change. `diff` never emits a drop for an invalid entry, whatever the desired file says about its name: a plain `DROP INDEX` blocks the table and cannot tell abandoned debris from a build still in progress. A `pull`ed baseline of such a table therefore re-diffs to that one rebuild rather than to zero. On a partitioned parent an invalid index means unattached partition indexes, not an unfinished build, so validity plays no part in the parent's comparison" + - id: classic-table-inheritance-inherits + area: "declarative_model" + operation: "Classic table inheritance (`INHERITS`)" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Typed refusal for both parents and children: the model cannot express inheritance edges, and flattening inherited columns would produce a silently lossy baseline" + - id: tables-that-own-or-are-referenced-by-foreign-keys + area: "declarative_model" + operation: "Tables that own **or are referenced by** foreign keys" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Typed refusal on both sides — an incoming FK cannot be expressed in the table's own desired file, and a lossy description would be worse than none. Declarative FK support (composite keys as the primary case, two-phase `NOT VALID` → `VALIDATE` execution) is planned" + - id: unlogged-tables + area: "declarative_model" + operation: "Unlogged tables" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Typed refusal: persistence is not modeled, converging it (`SET LOGGED`) is a full rewrite, and rendering the table as plain `CREATE TABLE` would silently change crash-safety" + - id: explicit-column-collations + area: "declarative_model" + operation: "Explicit column collations" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Typed refusal: dropping a `COLLATE` clause from a rendered baseline silently changes sort order and index semantics; a collation delta cannot converge without a rewrite" + - id: columns-whose-default-uses-a-sequence-the-column-does-not-own + area: "declarative_model" + operation: "Columns whose default uses a sequence the column does not own" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Typed refusal: in a desired-state model that sequence exists only inside the scratch transaction, so no derived plan can reference it. Column-owned (`serial`-style) sequences are fine" + - id: greenfield-create-table-apply-the-table-does-not-exist-yet-a-fresh-database-or-a-new-table-in-a-live-one + area: "declarative_model" + operation: "Greenfield `CREATE TABLE` apply (the table does not exist yet — a fresh database or a new table in a live one)" + tier: "t1" + status_mark: "✅" + engine_path: "native_as_is" + online_safety_problem: true + online_safety_detail: "a `REFERENCES` clause would take a brief `SHARE ROW EXCLUSIVE` on each **referenced** live table, but desired files refuse foreign keys today, so no live table is locked" + front_doors: + migrate: supported + diff: supported + reason_notes: "Desired-state execution creates the table: `CheckTableAbsent` verifies the table relation and composite-type name are free, the executor verifies every relation name the desired file states (explicit index names and first-choice constraint-index and column-sequence names) is free in the schema, and `CheckCreatePrivileges` verifies the role can create there. It then runs the `CREATE TABLE` and index builds as brief bounded steps under the engine's budgets. An occupied claimed name is a typed `create-collision` refusal before execution — drop or rename the occupant, name a constraint's index explicitly, or for a sequence use an explicitly named sequence or a non-serial column. Duplicate-name SQLSTATEs backstop races for explicit names; for server-chosen names, the probe narrows the race to the time-of-check window, and after the `CREATE TABLE` commits the executor reads the constraint-index and sequence names the table actually owns and compares them against the claimed first-choice names — a name taken inside the window makes the server pick a suffixed replacement, which surfaces as a typed `create-name-mismatch` failure at step 1 with the born table left in place for an operator to rename the relation or drop, then re-diff (a read of the owned names that does not complete is the same step-1 failure as `create-names-unverified`, never a pass). `PARTITION OF`, `INHERITS`, `LIKE`, `OF`, `IF NOT EXISTS`, and in-set duplicate names refuse at plan time and are re-checked at apply, while `REFERENCES` and `CONCURRENTLY` are refused upstream at desired-file parse and re-checked at admission as defense in depth" + - id: enum-typed-columns-on-plain-tables + area: "types_and_non_table_objects" + operation: "Enum-typed columns on plain tables" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Tolerance end to end (introspection already canonicalizes via `format_type`; desired-file admission and transaction-scoped scratch-schema mechanics are being verified)" + - id: alter-type-add-value + area: "types_and_non_table_objects" + operation: "`ALTER TYPE ... ADD VALUE`" + tier: "t2" + status_mark: "🟡" + engine_path: "native_planned_flow" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Metadata-only and online-safe (PG 14+ allows it in a transaction; the value is usable after commit) — planned as an owned operation. No peer online executor owns it" + - id: enum-value-rename-removal + area: "types_and_non_table_objects" + operation: "Enum value rename / removal" + tier: "t2" + status_mark: "🟡" + engine_path: "copy_and_swap" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: backend-unavailable + reason_notes: "PostgreSQL has no `DROP VALUE`; this is a type swap + table rewrite — routes to a typed refusal toward copy-and-swap" + - id: enum-domain-type-creation-and-drop + area: "types_and_non_table_objects" + operation: "Enum/domain type creation and drop" + tier: "t3" + status_mark: "⚪" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "owner tooling (psql, shipped with the code change)" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Bootstrap/catalog work with no concurrent-access problem; owner tooling applies it in the same change that ships the code" + - id: views-materialized-views-create-and-replace + area: "types_and_non_table_objects" + operation: "Views, materialized views (create and replace)" + tier: "t3" + status_mark: "⚪" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "owner tooling" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Transactional catalog work, but `CREATE OR REPLACE VIEW` takes a brief `ACCESS EXCLUSIVE` on the view and queues behind in-flight readers — run it under a `lock_timeout`" + - id: refresh-materialized-view + area: "types_and_non_table_objects" + operation: "`REFRESH MATERIALIZED VIEW`" + tier: "t3" + status_mark: "🔵" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "data jobs / owner tooling" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "A data operation, not catalog work: the plain form holds `ACCESS EXCLUSIVE` on the matview for the whole rebuild (`CONCURRENTLY` needs a unique index and trades the lock for churn). Scheduling refreshes belongs to data jobs" + - id: pl-pgsql-function-bodies-create-or-replace-function + area: "types_and_non_table_objects" + operation: "PL/pgSQL function bodies (`CREATE OR REPLACE FUNCTION`)" + tier: "t3" + status_mark: "⚪" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "owner tooling" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Transactional catalog work that takes no lock on any relation; nothing for an online engine to add. No peer online executor owns it either" + - id: triggers-create-trigger + area: "types_and_non_table_objects" + operation: "Triggers (`CREATE TRIGGER`)" + tier: "t3" + status_mark: "⚪" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "owner tooling" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Catalog work — no scan, no rewrite — but it takes a brief `SHARE ROW EXCLUSIVE` on the table, queues behind long-running queries, and blocks writers while it waits — run it under a `lock_timeout`" + - id: extensions-create-extension + area: "types_and_non_table_objects" + operation: "Extensions (`CREATE EXTENSION`)" + tier: "t3" + status_mark: "⚪" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "owner tooling" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Same: catalog bootstrap, owner tooling" + - id: grants-roles-row-level-security-policies + area: "types_and_non_table_objects" + operation: "Grants, roles, row-level-security policies" + tier: "t3" + status_mark: "🔵" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "provisioning / IaC" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Access control, not table shape; belongs to provisioning (see [engine-role.md](engine-role.md) for what the *engine's own* role needs)" + - id: standalone-sequences + area: "types_and_non_table_objects" + operation: "Standalone sequences" + tier: "t3" + status_mark: "⚪" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "owner tooling" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Transactional catalog work on an object with no readers-and-writers problem" + - id: publications-subscriptions + area: "types_and_non_table_objects" + operation: "Publications, subscriptions" + tier: "t3" + status_mark: "🔵" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "replication provisioning / IaC" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Replication provisioning, not table shape (`ALTER PUBLICATION ... ADD TABLE` also takes `SHARE UPDATE EXCLUSIVE` on the table)" + - id: drop-table + area: "data_and_whole_table_operations" + operation: "`DROP TABLE`" + tier: "t3" + status_mark: "⚪" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "owner tooling, through a reviewed process" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Discards the table and its data in one brief `ACCESS EXCLUSIVE` step: there is nothing online for an engine to make safer, only an irreversible decision an operator must own. Both front doors refuse it — the imperative door as an unsupported statement kind, and the declarative diff is single-table scoped, so a live table with no desired file is not in its view. pg-sprite never plans or executes it; accounting for undeclared tables is the whole-schema owner's job, described under [Deliberately operator-owned](#deliberately-operator-owned)" + - id: data-backfills-update-delete-batches-dml-of-any-kind + area: "data_and_whole_table_operations" + operation: "Data backfills, `UPDATE`/`DELETE` batches, DML of any kind" + tier: "t3" + status_mark: "🔵" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "data-change runners, application batch jobs" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "pg-sprite changes table *shape*, never table *contents*. Versioned-script runners and application jobs own data changes" + - id: column-transform-expressions-during-a-copy-and-swap-rewrite + area: "data_and_whole_table_operations" + operation: "Column-transform expressions during a copy-and-swap rewrite" + tier: "t2" + status_mark: "🟡" + engine_path: "copy_and_swap" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: backend-unavailable + reason_notes: "The one principled exception: when a rewrite is already copying every row, deriving a new column's value by expression is part of the shape change, not a data job. Planned as part of the copy engine" + - id: online-table-rebuild-with-no-shape-change-bloat-reclamation + area: "data_and_whole_table_operations" + operation: "Online table rebuild with no shape change (bloat reclamation)" + tier: "t2" + status_mark: "🟡" + engine_path: "copy_and_swap" + online_safety_problem: true + front_doors: + migrate: refused + diff: refused + refusal_reason: backend-unavailable + reason_notes: "A copy-and-swap with an identical target shape — the pg_repack use case with checksum-gated cutover and crash-resume. Planned once the copy engine lands" + - id: whole-schema-convergence-apply-a-directory-of-desired-files-dependency-ordered + area: "data_and_whole_table_operations" + operation: "Whole-schema convergence (apply a directory of desired files, dependency-ordered)" + tier: "t3" + status_mark: "🔵" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "convergence planners (pg-schema-diff, pgschema, pgdelta)" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Convergence planning across objects is a planner's job; pg-sprite stays the execution engine for the table-shape subset" + - id: versioned-schema-change-file-workflow-flyway-style-ordered-scripts + area: "data_and_whole_table_operations" + operation: "Versioned schema-change-file workflow (Flyway-style ordered scripts)" + tier: "t3" + status_mark: "🔵" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "versioned-script runners (Flyway-style)" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Declarative-only by design; see [vision.md](vision.md)" + - id: expand-contract-dual-schema-versions-pgroll-reshape-style + area: "data_and_whole_table_operations" + operation: "Expand/contract dual-schema versions (pgroll/reshape style)" + tier: "t3" + status_mark: "🔵" + engine_path: "none" + online_safety_problem: false + owning_tool_class: "pgroll/reshape own this model" + front_doors: + migrate: refused + diff: refused + refusal_reason: unsupported-statement + reason_notes: "Rejected: application invisibility is a core invariant; see [vision.md](vision.md)" diff --git a/pkg/capabilities/capabilities_test.go b/pkg/capabilities/capabilities_test.go new file mode 100644 index 0000000..fe5daec --- /dev/null +++ b/pkg/capabilities/capabilities_test.go @@ -0,0 +1,80 @@ +package capabilities + +import ( + "os" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func validRow() Row { + return Row{ID: "example", Area: AreaColumnChanges, Operation: "example", Tier: TierOne, StatusMark: StatusSupported, EnginePath: PathNativeAsIs, OnlineSafetyProblem: true, FrontDoors: FrontDoors{Migrate: DoorSupported, Diff: DoorSupported}, ReasonNotes: "notes"} +} + +func TestEmbeddedCapabilitiesValidate(t *testing.T) { + rows, err := Rows() + require.NoError(t, err) + assert.Len(t, rows, 53) +} + +func TestLoadRejectsInvalidYAML(t *testing.T) { + _, err := Load([]byte("rows: [")) + assert.ErrorContains(t, err, "parse capabilities YAML") +} + +func TestValidateRules(t *testing.T) { + tests := map[string]func(*Row){ + "unknown area": func(r *Row) { r.Area = "unknown" }, + "unknown tier": func(r *Row) { r.Tier = "unknown" }, + "unknown status mark": func(r *Row) { r.StatusMark = "unknown" }, + "unknown engine path": func(r *Row) { r.EnginePath = "unknown" }, + "unknown front door status": func(r *Row) { r.FrontDoors.Migrate = "unknown" }, + "duplicate id": func(r *Row) {}, + "invalid id": func(r *Row) { r.ID = "Not valid" }, + "required text": func(r *Row) { r.Operation = "" }, + "mark agrees with tier": func(r *Row) { r.Tier = TierTwo }, + "T3 path is none": func(r *Row) { + r.Tier = TierThree + r.StatusMark = StatusOtherTool + r.EnginePath = PathNativeAsIs + r.OnlineSafetyProblem = false + r.OwningToolClass = "owner" + }, + "none only on T3": func(r *Row) { r.EnginePath = PathNone }, + "No names owner": func(r *Row) { r.OnlineSafetyProblem = false }, + "Yes omits owner": func(r *Row) { r.OwningToolClass = "owner" }, + "refusal reason iff refused": func(r *Row) { r.FrontDoors.Diff = DoorRefused }, + "known refusal reason": func(r *Row) { r.FrontDoors.Diff = DoorRefused; r.RefusalReason = "unknown" }, + } + for name, mutate := range tests { + t.Run(name, func(t *testing.T) { + row := validRow() + mutate(&row) + rows := []Row{row} + if name == "duplicate id" { + rows = append(rows, row) + } + assert.Error(t, Validate(rows)) + }) + } +} + +func TestCheckedInMarkdownIsGenerated(t *testing.T) { + rows, err := Rows() + require.NoError(t, err) + input, err := os.ReadFile("../../docs/capabilities.md") + require.NoError(t, err) + output, err := RenderDocument(input, rows) + require.NoError(t, err) + assert.Equal(t, input, output) +} + +func TestRenderDocumentRejectsBadMarkers(t *testing.T) { + rows, err := Rows() + require.NoError(t, err) + for _, doc := range []string{"", ""} { + _, err := RenderDocument([]byte(doc), rows) + assert.Error(t, err) + } +} From 5413ce147089d62680feec1097af81f5f9404275 Mon Sep 17 00:00:00 2001 From: Kiran Muddukrishna Date: Thu, 10 Sep 2026 15:17:00 +1000 Subject: [PATCH 2/5] ci: fail when capabilities Markdown disagrees with YAML The capabilities YAML is authoritative, but a stale generated page could still merge because CI did not compare the checked-in rendering with generator output. Add a regenerate-and-diff Make target and run it unconditionally in CI so both docs-only and code changes are covered. Repeat the check in the release sweep so the tagged tree cannot publish a stale capabilities matrix. --- .github/workflows/ci.yml | 19 +++++++++++++++++-- .github/workflows/release.yml | 4 ++++ Makefile | 9 ++++++++- docs/capabilities-contract.md | 12 +++++++----- 4 files changed, 36 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c37bc7b..a0cd652 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -84,6 +84,21 @@ jobs: go-version-file: go.mod - run: make test-unit + # The checked-in capabilities page is generated from YAML. Run this on + # docs-only and code changes alike so neither side of that contract can + # merge with stale generated tables. + capabilities: + name: capabilities matrix is generated + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + with: + persist-credentials: false + - uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0 + with: + go-version-file: go.mod + - run: make check-capabilities + lint: needs: changes if: needs.changes.outputs.code == 'true' @@ -213,10 +228,10 @@ jobs: # Single required status for branch protection ("all-green" is the # context to require). Succeeds when nothing failed — including # docs-only PRs, where the code-gated jobs were skipped but the unit - # job (and its docs guards) still had to pass. + # and capabilities jobs still had to pass. all-green: if: always() - needs: [changes, unit, lint, build, test, demo] + needs: [changes, unit, capabilities, lint, build, test, demo] runs-on: ubuntu-latest steps: - name: Check job results diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e5d79ba..c9fce43 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -41,6 +41,10 @@ jobs: # previous workflow could have poisoned. cache: false + # A tag cannot ship a capabilities matrix that disagrees with its YAML. + - name: Check the capabilities matrix + run: make check-capabilities + # Belt and braces on top of the ancestry gate: re-run the suite against # the exact tree being released. - name: Test the tagged tree diff --git a/Makefile b/Makefile index f0454a5..943b46f 100644 --- a/Makefile +++ b/Makefile @@ -14,11 +14,18 @@ PG_DSN_LOCAL = postgres://$(PG_USER):$(PG_PASSWORD)@localhost:$(PG_PORT)/$(PG_DA # Which corpus replay project to run (replay//project.conf). REPLAY_PROJECT ?= buzz -.PHONY: build gen-capabilities test test-unit test-db test-supported-postgres test-aws-boundary lint setup db-up db-down demos clean demo demo-seed demo-check replay replay-refresh replay-down +.PHONY: build gen-capabilities check-capabilities test test-unit test-db test-supported-postgres test-aws-boundary lint setup db-up db-down demos clean demo demo-seed demo-check replay replay-refresh replay-down gen-capabilities: $(GO) run ./internal/cmd/gen-capabilities +# Regenerate the capabilities page and leave any stale output visible for review. +check-capabilities: gen-capabilities + @if ! git diff --exit-code -- docs/capabilities.md; then \ + echo "docs/capabilities.md disagrees with pkg/capabilities/capabilities.yaml; run make gen-capabilities and commit the result" >&2; \ + exit 1; \ + fi + build: $(GO) build -o bin/pg-sprite ./cmd/pg-sprite $(GO) build ./... diff --git a/docs/capabilities-contract.md b/docs/capabilities-contract.md index c976ba4..74756c8 100644 --- a/docs/capabilities-contract.md +++ b/docs/capabilities-contract.md @@ -154,10 +154,12 @@ markers — the introduction, tier explanation, legend, peer comparison, refusal and operator recipes — remains hand-written. Generated output is deterministic: source order is display order, formatting has no timestamps, and a second generation is a no-op. -The generator lands with the YAML file, not later. A Make target runs its `go run` -entry point. CI runs that target and then fails unless `git diff --exit-code` is empty. -The test validates semantics; regenerate-and-diff proves the checked-in human page is -the rendering of the validated data. +The generator lands with the YAML file, not later. `make check-capabilities` runs its +`go run` entry point and fails unless the generated page has an empty git diff. The +unconditional capabilities job in `.github/workflows/ci.yml` applies that gate to code +and docs-only changes, and `.github/workflows/release.yml` repeats it for the tagged tree +before the test sweep. The test validates semantics; regenerate-and-diff proves the +checked-in human page is the rendering of the validated data. The capability-statement rule still applies beyond the generated matrix. A behavior change updates the YAML, [limitations.md](limitations.md), and the README's short @@ -263,7 +265,7 @@ does not change any capability, tier, refusal, or runtime behavior. Implementation order is: -**Status:** step 1 is complete; steps 2–4 remain planned. +**Status:** steps 1 and 3 are complete; steps 2 and 4 remain planned. 1. add the typed package, `pkg/capabilities/capabilities.yaml`, validator, generator, and markers together, making the repository single-source on day one; From 69b6cdfddad67bc551ec25d8c58a15ddc6b579ec Mon Sep 17 00:00:00 2001 From: Kiran Muddukrishna Date: Fri, 11 Sep 2026 05:50:00 +1000 Subject: [PATCH 3/5] ci: gate on the generator's own edits and run it beside the unit tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `git diff --exit-code` on the capabilities page conflated stale generated regions with any uncommitted edit to the hand-written prose, which regeneration leaves untouched, so the local gate could fail with advice that changes nothing. Snapshot the page, regenerate, and compare the two: only what the generator rewrote can trip the gate, and the printed diff shows exactly that. The separate CI job re-asserted what the unit job already proves through `TestCheckedInMarkdownIsGenerated`, at the cost of a runner and a checkout. Run `make check-capabilities` as a step of the always-on unit job instead: that keeps the `go run` entry point and Make target release.yml depends on exercised on every PR without a second job. 🤖 Generated with Amp (Claude Opus 4.6) --- .github/workflows/ci.yml | 24 ++++++++---------------- Makefile | 15 +++++++++++---- docs/capabilities-contract.md | 14 ++++++++------ 3 files changed, 27 insertions(+), 26 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a0cd652..a46687d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -72,6 +72,12 @@ jobs: # would switch the guards off precisely when prose changes. The job is # cheap enough that running it redundantly alongside the matrix on code # PRs costs less than maintaining a list of which packages are guards. + # + # The capabilities gate rides along for the same reason. The semantic + # assertion — the checked-in capabilities page is the rendering of the + # YAML — is already a unit test; the make target adds the generator's + # own `go run` entry point, which release.yml runs against the tagged + # tree, so a broken target fails a PR here rather than a release. unit: name: unit tests (no Docker; docs guards) runs-on: ubuntu-latest @@ -83,20 +89,6 @@ jobs: with: go-version-file: go.mod - run: make test-unit - - # The checked-in capabilities page is generated from YAML. Run this on - # docs-only and code changes alike so neither side of that contract can - # merge with stale generated tables. - capabilities: - name: capabilities matrix is generated - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 - with: - persist-credentials: false - - uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0 - with: - go-version-file: go.mod - run: make check-capabilities lint: @@ -228,10 +220,10 @@ jobs: # Single required status for branch protection ("all-green" is the # context to require). Succeeds when nothing failed — including # docs-only PRs, where the code-gated jobs were skipped but the unit - # and capabilities jobs still had to pass. + # job still had to pass. all-green: if: always() - needs: [changes, unit, capabilities, lint, build, test, demo] + needs: [changes, unit, lint, build, test, demo] runs-on: ubuntu-latest steps: - name: Check job results diff --git a/Makefile b/Makefile index 943b46f..3963619 100644 --- a/Makefile +++ b/Makefile @@ -19,12 +19,19 @@ REPLAY_PROJECT ?= buzz gen-capabilities: $(GO) run ./internal/cmd/gen-capabilities -# Regenerate the capabilities page and leave any stale output visible for review. -check-capabilities: gen-capabilities - @if ! git diff --exit-code -- docs/capabilities.md; then \ +# Regenerate the capabilities page and fail if regeneration changed it. Only the +# generator's own edits count, so an uncommitted edit to the hand-written prose +# outside the marker regions does not trip the gate; the stale generated output +# is left in place, with the diff printed, for review. +check-capabilities: + @before=$$(mktemp); cp docs/capabilities.md "$$before"; \ + $(GO) run ./internal/cmd/gen-capabilities || { rm -f "$$before"; exit 1; }; \ + if ! diff -u --label docs/capabilities.md --label regenerated "$$before" docs/capabilities.md; then \ + rm -f "$$before"; \ echo "docs/capabilities.md disagrees with pkg/capabilities/capabilities.yaml; run make gen-capabilities and commit the result" >&2; \ exit 1; \ - fi + fi; \ + rm -f "$$before" build: $(GO) build -o bin/pg-sprite ./cmd/pg-sprite diff --git a/docs/capabilities-contract.md b/docs/capabilities-contract.md index 74756c8..e264735 100644 --- a/docs/capabilities-contract.md +++ b/docs/capabilities-contract.md @@ -74,7 +74,7 @@ path on T3 rows. │ go:embed │ regenerate │ committed ▼ ▼ ▼ ┌───────────────────────┐ ┌─────────────────────────────────────────────────────┐ -│ pkg/capabilities │ │ CI gate: regenerate, then require an empty git diff │ +│ pkg/capabilities │ │ CI gate: regenerate, then require a no-op rewrite │ └───────────┬───────────┘ └─────────────────────────────────────────────────────┘ │ ▼ @@ -155,11 +155,13 @@ and operator recipes — remains hand-written. Generated output is deterministic order is display order, formatting has no timestamps, and a second generation is a no-op. The generator lands with the YAML file, not later. `make check-capabilities` runs its -`go run` entry point and fails unless the generated page has an empty git diff. The -unconditional capabilities job in `.github/workflows/ci.yml` applies that gate to code -and docs-only changes, and `.github/workflows/release.yml` repeats it for the tagged tree -before the test sweep. The test validates semantics; regenerate-and-diff proves the -checked-in human page is the rendering of the validated data. +`go run` entry point and fails unless regeneration leaves the page byte-identical; only +the generator's own edits count, so an uncommitted edit to the hand-written prose does +not trip it. The unconditional unit job in `.github/workflows/ci.yml` runs that target +beside the unit tests on code and docs-only changes alike, and +`.github/workflows/release.yml` repeats it for the tagged tree before the test sweep. The +test validates semantics; regenerate-and-diff proves the checked-in human page is the +rendering of the validated data. The capability-statement rule still applies beyond the generated matrix. A behavior change updates the YAML, [limitations.md](limitations.md), and the README's short From d75dfeda4d87d9eb35a1e1e3f726186a3d1d5636 Mon Sep 17 00:00:00 2001 From: Kiran Muddukrishna Date: Fri, 11 Sep 2026 07:53:39 +1000 Subject: [PATCH 4/5] ci: make the capabilities gate a pure check and run it first MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address the review of the capabilities CI gate: - check-capabilities restores the committed page when regeneration changes it, so a rerun fails the same way instead of passing on the fix the first run left behind, and `make gen-capabilities` is the only command that writes docs/capabilities.md. The failure message and the behavior now agree. - Both targets run one shared GEN_CAPABILITIES command, so the gate cannot drift from the command it tells the operator to run. - The unit job runs the gate before the unit suite: the generator's only smoke test is one `go run` and a diff, and it should report even when an unrelated test fails. This matches release.yml's order. - The all-green comment keeps the clause naming the docs guards and now the capabilities gate, which is the property that makes the gate safe on docs-only PRs. - The contract doc marks completion per step, so parallel step PRs edit their own line instead of rewriting one shared status sentence, and its gate paragraph records that a failing check has no side effect. 🤖 Generated with Amp (Claude Opus 4.6) --- .github/workflows/ci.yml | 6 ++++-- Makefile | 17 +++++++++++------ docs/capabilities-contract.md | 22 +++++++++++----------- 3 files changed, 26 insertions(+), 19 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a46687d..e877e16 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -88,8 +88,10 @@ jobs: - uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0 with: go-version-file: go.mod - - run: make test-unit + # The gate is one `go run` and a diff; it goes first so a broken + # generator is reported even when an unrelated unit test fails. - run: make check-capabilities + - run: make test-unit lint: needs: changes @@ -220,7 +222,7 @@ jobs: # Single required status for branch protection ("all-green" is the # context to require). Succeeds when nothing failed — including # docs-only PRs, where the code-gated jobs were skipped but the unit - # job still had to pass. + # job (its docs guards and the capabilities gate) still had to pass. all-green: if: always() needs: [changes, unit, lint, build, test, demo] diff --git a/Makefile b/Makefile index 9fcd250..2bf9515 100644 --- a/Makefile +++ b/Makefile @@ -54,19 +54,24 @@ lint: golangci-lint run # Regenerate the marked regions of docs/capabilities.md from the embedded -# matrix (pkg/capabilities/capabilities.yaml); CI fails if they drift. +# matrix (pkg/capabilities/capabilities.yaml); CI fails if they drift. The +# gate below runs the same command, so the two cannot drift apart. +GEN_CAPABILITIES = $(GO) run ./internal/cmd/gen-capabilities + gen-capabilities: - $(GO) run ./internal/cmd/gen-capabilities + $(GEN_CAPABILITIES) # Regenerate the capabilities page and fail if regeneration changed it. Only the # generator's own edits count, so an uncommitted edit to the hand-written prose -# outside the marker regions does not trip the gate; the stale generated output -# is left in place, with the diff printed, for review. +# outside the marker regions does not trip the gate. The target is a pure +# check: on failure it prints the diff and puts the committed page back, so a +# rerun fails the same way and `make gen-capabilities` is the only command +# that writes the page. check-capabilities: @before=$$(mktemp); cp docs/capabilities.md "$$before"; \ - $(GO) run ./internal/cmd/gen-capabilities || { rm -f "$$before"; exit 1; }; \ + $(GEN_CAPABILITIES) || { rm -f "$$before"; exit 1; }; \ if ! diff -u --label docs/capabilities.md --label regenerated "$$before" docs/capabilities.md; then \ - rm -f "$$before"; \ + cp "$$before" docs/capabilities.md; rm -f "$$before"; \ echo "docs/capabilities.md disagrees with pkg/capabilities/capabilities.yaml; run make gen-capabilities and commit the result" >&2; \ exit 1; \ fi; \ diff --git a/docs/capabilities-contract.md b/docs/capabilities-contract.md index e264735..e2693e0 100644 --- a/docs/capabilities-contract.md +++ b/docs/capabilities-contract.md @@ -157,11 +157,12 @@ order is display order, formatting has no timestamps, and a second generation is The generator lands with the YAML file, not later. `make check-capabilities` runs its `go run` entry point and fails unless regeneration leaves the page byte-identical; only the generator's own edits count, so an uncommitted edit to the hand-written prose does -not trip it. The unconditional unit job in `.github/workflows/ci.yml` runs that target -beside the unit tests on code and docs-only changes alike, and -`.github/workflows/release.yml` repeats it for the tagged tree before the test sweep. The -test validates semantics; regenerate-and-diff proves the checked-in human page is the -rendering of the validated data. +not trip it, and a failing run restores the committed page so the check has no side +effect and `make gen-capabilities` is the one command that writes it. The unconditional +unit job in `.github/workflows/ci.yml` runs that target beside the unit tests on code and +docs-only changes alike, and `.github/workflows/release.yml` repeats it for the tagged +tree before the test sweep. The test validates semantics; regenerate-and-diff proves the +checked-in human page is the rendering of the validated data. The capability-statement rule still applies beyond the generated matrix. A behavior change updates the YAML, [limitations.md](limitations.md), and the README's short @@ -265,15 +266,14 @@ artifact. The generated `docs/capabilities.md` remains the human-facing home. This decision does not build sortable HTML tables or a documentation site. It also does not change any capability, tier, refusal, or runtime behavior. -Implementation order is: - -**Status:** steps 1 and 3 are complete; steps 2 and 4 remain planned. +Implementation order is, with each step marked as it ships: 1. add the typed package, `pkg/capabilities/capabilities.yaml`, validator, generator, - and markers together, making the repository single-source on day one; + and markers together, making the repository single-source on day one; *(done)* 2. add `pg-sprite capabilities`, including `--json` and the embedded binary version; -3. add the regenerate-and-diff CI gate to the normal pipeline; and -4. add documentation and `jq` recipes for consumers. + *(pending)* +3. add the regenerate-and-diff CI gate to the normal pipeline; and *(done)* +4. add documentation and `jq` recipes for consumers. *(pending)* The generator is part of the first step rather than a cleanup step: there is never an intermediate state in which two hand-maintained matrices are authoritative. From 5ba612a6ccb3cfc28235bd1dea15d29cb568ac8e Mon Sep 17 00:00:00 2001 From: Kiran Muddukrishna Date: Fri, 11 Sep 2026 17:48:35 +1000 Subject: [PATCH 5/5] ci: restore the capabilities page on a generator failure too The check's generator-failure branch now copies the pre-run page back before exiting, so a write that fails partway cannot leave a mangled page behind; the diff branch already did this. The Makefile comment and the contract doc describe the restore as "the page as it was before the run" rather than "the committed page": the recipe snapshots the working tree, not git, so an uncommitted edit survives a failing run unchanged. --- Makefile | 8 ++++---- docs/capabilities-contract.md | 5 +++-- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index 29a6d07..1605a48 100644 --- a/Makefile +++ b/Makefile @@ -64,12 +64,12 @@ gen-capabilities: # Regenerate the capabilities page and fail if regeneration changed it. Only the # generator's own edits count, so an uncommitted edit to the hand-written prose # outside the marker regions does not trip the gate. The target is a pure -# check: on failure it prints the diff and puts the committed page back, so a -# rerun fails the same way and `make gen-capabilities` is the only command -# that writes the page. +# check: on failure — a generator error or a diff — it puts the page back as it +# was before the run, so a rerun fails the same way and `make gen-capabilities` +# is the only command that writes the page. check-capabilities: @before=$$(mktemp); cp docs/capabilities.md "$$before"; \ - $(GEN_CAPABILITIES) || { rm -f "$$before"; exit 1; }; \ + $(GEN_CAPABILITIES) || { cp "$$before" docs/capabilities.md; rm -f "$$before"; exit 1; }; \ if ! diff -u --label docs/capabilities.md --label regenerated "$$before" docs/capabilities.md; then \ cp "$$before" docs/capabilities.md; rm -f "$$before"; \ echo "docs/capabilities.md disagrees with pkg/capabilities/capabilities.yaml; run make gen-capabilities and commit the result" >&2; \ diff --git a/docs/capabilities-contract.md b/docs/capabilities-contract.md index 829b589..b377898 100644 --- a/docs/capabilities-contract.md +++ b/docs/capabilities-contract.md @@ -157,8 +157,9 @@ order is display order, formatting has no timestamps, and a second generation is The generator lands with the YAML file, not later. `make check-capabilities` runs its `go run` entry point and fails unless regeneration leaves the page byte-identical; only the generator's own edits count, so an uncommitted edit to the hand-written prose does -not trip it, and a failing run restores the committed page so the check has no side -effect and `make gen-capabilities` is the one command that writes it. The unconditional +not trip it, and a failing run — a generator error or a diff — restores the page as it was +before the run, so the check has no side effect and `make gen-capabilities` is the one +command that writes it. The unconditional unit job in `.github/workflows/ci.yml` runs that target beside the unit tests on code and docs-only changes alike, and `.github/workflows/release.yml` repeats it for the tagged tree before the test sweep. The test validates semantics; regenerate-and-diff proves the