sqlfmt: omit generated columns from diff -r sql by vishnujayvel · Pull Request #11476 · dolthub/dolt

vishnujayvel · 2026-08-09T15:20:41Z

dolt diff -r sql no longer assigns generated columns in the INSERT and UPDATE statements it emits, so its output can be replayed. Previously the engine rejected those assignments; where the diff is emitted as DELETE+INSERT — keyless tables, and tables whose primary key is itself generated — replay deleted the old row and then failed to reinsert it, silently losing the row. dolt_patch()'s statement output is fixed by the same change.

schemaFromCreateTableStmt now populates Generated, Virtual, and OnUpdate on schema.Column, so IsGenerated() is correct for schemas built from SHOW CREATE TABLE.
SqlRowAsUpdateStmt skips generated columns, and GenerateDataDiffStatement drops them from colsToUpdate.
STORED and VIRTUAL generated columns are both covered, on primary-keyed and keyless tables.

Fix #11445
Close #11699

dolt diff -r sql was emitting STORED generated columns in INSERT and UPDATE statements, which the engine rejects on replay. Populate Column.Generated and Virtual in schemaFromCreateTableStmt so IsGenerated works on the CLI SQL-diff path, and skip generated columns in SqlRowAsUpdateStmt / GenerateDataDiffStatement (mirroring the existing INSERT filters). Fixes dolthub#11445

elianddb · 2026-08-19T16:13:50Z

@vishnujayvel Could you shorten the description to be similar to our teams' release notes? They come up as release notes and this change is relatively small.

vishnujayvel · 2026-08-19T16:33:04Z

Done — shortened the description to match the team's release-note style.

Thanks again for the repro in #11445 — it named both failure points, which is why this landed as a small diff.

I've parked the long-form root cause and test plan below so it stays on the PR for review without ending up in the release notes.

Original description (root cause + test plan)

dolt diff -r sql was writing generated columns into the INSERT and
UPDATE statements it emits. The engine rejects any assignment into a generated
column, so the SQL patch could not be replayed. Where the diff is emitted as
DELETE+INSERT — keyless tables, and tables whose primary key is itself a
generated column — replaying a patch deleted the old row and then failed the
INSERT, silently losing the row.

The same emission path backs the dolt_patch() table function, so its
statement output is fixed by the same change.

Fixes #11445

Thanks to @elianddb (Elian) for the clear repro and for tracing both failure
points in #11445.

Got

UPDATE `t` SET `a`=99,`g`=100 WHERE `id`=1;
INSERT INTO `t` (`id`,`a`,`g`) VALUES (2,20,21);

Want / after this change

UPDATE `t` SET `a`=99 WHERE `id`=1;
INSERT INTO `t` (`id`,`a`) VALUES (2,20);

Root cause (two parts)

Schema construction on the CLI SQL-diff path.
schemaFromCreateTableStmt (go/cmd/dolt/commands/diff.go) already
parsed col.Type.GeneratedExpr into a buffer, but wrote
Generated: "" and Virtual: false into schema.Column. That made
Column.IsGenerated() always false for schemas built from
SHOW CREATE TABLE, so the existing INSERT filters in
InsertStatementPrefix / SqlRowAsTupleString never fired on this path.
UPDATE emission never filtered generated columns.
SqlRowAsUpdateStmt (go/libraries/doltcore/sqle/sqlfmt/row_fmt.go —
not diff.go) only gated on colsToUpdate, unlike the INSERT helpers
which already skip col.IsGenerated(). Even with an honest schema,
UPDATE would still emit generated columns when their values changed
alongside base columns.

Fix

Populate Generated from the already-formatted expression buffer and
set Virtual from GeneratedExpr != nil && !Stored (and fill
OnUpdate from its parallel buffer while touching the same struct).
Skip col.IsGenerated() columns in SqlRowAsUpdateStmt, and drop them
from colsToUpdate in GenerateDataDiffStatement so the UPDATE column
set never includes them.

Test plan

Unit: TestSqlRowAsUpdateStmtSkipsGeneratedCols in
go/libraries/doltcore/sqle/sqlfmt/row_fmt_test.go, mirroring
TestInsertStatementPrefixSkipsGeneratedCols.
- Red on base: expected
  UPDATE ... SET a='x' WHERE id=1 vs actual
  UPDATE ... SET a='x',c='x!' WHERE id=1.
- Green at fix: pass.
Full package: go test ./libraries/doltcore/sqle/sqlfmt/ -count=1 → ok.
Integration: bats case
sql-diff: stored generated columns omitted from INSERT and UPDATE in
integration-tests/bats/sql-diff.bats — INSERT+UPDATE, assert g absent,
replay after dolt reset --hard, assert final rows. Passed with the
fixed binary on PATH, and fails against an unpatched binary. The full
sql-diff.bats file was not run to completion locally (slow host); its
first three cases (INSERT/UPDATE/DELETE reconcile) pass with the fixed
binary, and CI covers the rest.
Manual end-to-end repro (same as dolt diff -r sql emits statements that assign generated columns #11445 Got/Want): fixed binary omits
g from both statements; replay succeeds; rows (1,99,100), (2,20,21).
Also verified by direct comparison against an unpatched build: VIRTUAL
column tables, keyless tables, generated-column primary keys, and
dolt_patch() output are corrected; DELETE statements and non-generated
tables are byte-identical.

Notes

Coverage: STORED and VIRTUAL generated columns are both fixed (VIRTUAL
previously leaked into INSERT column lists as NULL values and broke
replay the same way), on primary-keyed and keyless tables — including
when the generated column is itself the primary key. Filters use
IsGenerated() (any non-empty Generated string), consistent with the
existing INSERT path.
Scope of the newly populated fields: nothing on any diff render path
reads Generated/OnUpdate/Virtual except the generated-column
filters this PR relies on. ALTER-DDL statements come from
tds.AlterStmts and the raw SHOW CREATE TABLE text, not from this
parsed schema, so schema-diff output is unchanged (verified empirically
at base and at this branch: ALTER/tabular/JSON outputs byte-identical).

elianddb

A couple of requested changes to rm extra code, and uncovered edge case.

elianddb · 2026-09-03T18:04:57Z

 //
-// TODO(elianddb): Schema isn't recording column's Generated marker
-// correctly, so Column.IsGenerated doesn't filter.
+// Generated columns are skipped, matching InsertStatementPrefix and


No need to mention the other shared helpers, only the the Generated Columns behavior. Would also move to second paragraph.

elianddb · 2026-09-03T18:29:59Z

+				if col.IsGenerated() {
+					continue
+				}
+				updatedCols.Add(col.Name)


Remove no need to duplicate filtering logic here.

elianddb · 2026-09-03T18:31:51Z

 	err := tableSch.GetAllCols().Iter(func(_ uint64, col schema.Column) (stop bool, err error) {
-		if colsToUpdate.Contains(col.Name) {
+		if colsToUpdate.Contains(col.Name) && !col.IsGenerated() {
 			if seenOne {


There's an empty list edge case here !seenOne add an assertion to existing test.

elianddb · 2026-09-03T18:33:08Z

+	stmt, err := sqlfmt.SqlRowAsUpdateStmt(sql.NewEmptyContext(), sql.Row{int64(1), "x", "x!"}, "table_name", sch, colsToUpdate)
+
+	require.NoError(t, err)
+	assert.Equal(t, "UPDATE `table_name` SET `a`='x' WHERE `id`=1;", stmt)


assertion for empty string

coffeegoddd added the contribution label Aug 9, 2026

elianddb self-assigned this Aug 19, 2026

elianddb mentioned this pull request Sep 3, 2026

[no-release-notes] Run #11476 checks #11699

Open

elianddb requested changes Sep 3, 2026

View reviewed changes

Navigation Menu

Uh oh!

Uh oh!

sqlfmt: omit generated columns from diff -r sql - #11476

sqlfmt: omit generated columns from diff -r sql#11476
vishnujayvel wants to merge 1 commit into
dolthub:mainfrom
vishnujayvel:fix/11445-diff-sql-generated-columns

vishnujayvel commented Aug 9, 2026 •

edited by elianddb

Loading

Uh oh!

elianddb commented Aug 19, 2026

Uh oh!

vishnujayvel commented Aug 19, 2026

Got

Want / after this change

Root cause (two parts)

Fix

Test plan

Notes

Uh oh!

elianddb left a comment

Uh oh!

elianddb Sep 3, 2026

Uh oh!

elianddb Sep 3, 2026

Uh oh!

elianddb Sep 3, 2026

Uh oh!

elianddb Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

vishnujayvel commented Aug 9, 2026 • edited by elianddb Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elianddb commented Aug 19, 2026

Uh oh!

vishnujayvel commented Aug 19, 2026

Got

Want / after this change

Root cause (two parts)

Fix

Test plan

Notes

Uh oh!

elianddb left a comment

Choose a reason for hiding this comment

Uh oh!

elianddb Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

elianddb Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

elianddb Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

elianddb Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vishnujayvel commented Aug 9, 2026 •

edited by elianddb

Loading