perf(metadata): avoid re-cloning defensive schema copies - #1990
perf(metadata): avoid re-cloning defensive schema copies#1990fallintoplace wants to merge 5 commits into
Conversation
zeroshade
left a comment
There was a problem hiding this comment.
Behavior is provably equivalent (Schema.Fields() already deep-copies), but the view-side deep-copy removal is pinned by a vacuous test and the PR's new partition/transform regression assertions do not bite under mutation.
Re-review verification: 0 of 1 prior findings confirmed fixed at 5e04984 (each verified by mutating the fix and observing the suite go red, not by taking the claim on trust).
Verification performed
go build ./...; go vet . ./table ./view ./catalog/hive; go test -count=1 . ./table ./view ./catalog/hive (all ok); go test -race -count=1 . ./table ./view ./catalog/hive (ok: iceberg 2.24s, table 19.81s, view 1.72s, hive 2.48s); 5 mutation probes on view/metadata.go:425, table/metadata.go:2407, table/metadata.go:2539, partitions.go:546 (shallow fields) and partitions.go:551 (drop initialize)
This review was drafted by an AI-assisted tool and confirmed by an Apache Iceberg Go maintainer. The findings below are observations, not blockers; an Apache Iceberg Go maintainer — a real person — will take the next look at the PR. If you think a finding is mis-applied, please reply on the PR and a maintainer will weigh in.
More on how Apache Iceberg Go handles maintainer review: CONTRIBUTING.md.
| slices.Clone(schema.IdentifierFieldIDs), | ||
| cloneNestedFields(schema.Fields())..., | ||
| schema.Fields()..., | ||
| ) |
There was a problem hiding this comment.
major — View cloneSchema deep-copy guarantee is entirely unpinned; guarding test is vacuous
This PR removes view's local cloneNestedFields/cloneSchemaType and relies on Schema.Fields() being a deep copy. No view test verifies that. TestCloneSchemaCopiesNestedValues (view/metadata_test.go:712) mutates cloned.Field(i), but Schema.Field (schema.go:279) itself returns cloneField(...), so every mutation lands on a throwaway copy. A future change to view cloneSchema or to Fields() would silently alias view metadata's internal field slice with no test failure. Fix: mirror the table-side test and read the internal slice via FieldsRef(internal.SchemaRef{}) as TestMetadataSchemaGetterCopiesNestedValues already does.
Evidence
Mutated view/metadata.go:425 `schema.Fields()...` -> `schema.FieldsRef(iceint.SchemaRef{})...` (full aliasing); `go test -count=1 ./view` => `ok github.com/apache/iceberg-go/view 0.408s`. Same mutation on table/metadata.go:2407 correctly fails: `--- FAIL: TestMetadataSchemaGetterCopiesNestedValues` with diff `InitialDefault: 01 02 03 -> 63 02 03` and `Name: "list" -> "changed"`. Restored with git checkout; git status --porcelain empty.
| } | ||
|
|
||
| func TestPartitionSpecCloneCopiesFields(t *testing.T) { | ||
| transform := &iceberg.BucketTransform{NumBuckets: 16} |
There was a problem hiding this comment.
minor — TestPartitionSpecCloneCopiesFields does not test that Clone copies fields
The test mutates clone.Field(0), but PartitionSpec.Field (partitions.go:757) already returns clonePartitionField(...), so the mutation never reaches clone.fields. Only the trailing FieldsBySourceID(1) assertion is non-vacuous (it pins initialize()). To actually guard the deep copy, compare spec/clone after mutating through an accessor that returns internal state, or assert on the SourceIDs slice identity of the two specs' FieldsBySourceID results.
| partitionField.SourceIDs[0] = 99 | ||
| partitionField.Name = "mutated" | ||
| partitionField.Transform.(*iceberg.BucketTransform).NumBuckets = 32 | ||
| require.Equal(t, []int{1}, metadata.Specs[0].Field(0).SourceIDs) |
There was a problem hiding this comment.
minor — New transform-mutation assertions in TestMetadataGettersReturnDefensiveCopies are vacuous
The added partitionField.Transform.(*iceberg.BucketTransform).NumBuckets = 32 (line 115) and the sort-order equivalent (line 166) read through PartitionSpec.Field() and SortOrder.Fields(), both of which already deep-copy the transform (partitions.go:757, table/sorting.go:328). The assertions therefore pass regardless of whether cloneSortOrder/PartitionSpec.Clone copy transforms, so they add no regression protection for the behavior the PR description claims they cover. (The cloneSortOrder guarantee itself is still pinned by TestMetadataBuilderFromBaseCopiesBuiltinMetadata, so this is coverage theater rather than a hole.)
| } | ||
|
|
||
| func clonePartitionSpec(spec iceberg.PartitionSpec) iceberg.PartitionSpec { | ||
| fields := make([]iceberg.PartitionField, spec.NumFields()) |
There was a problem hiding this comment.
nit — clonePartitionSpec is now a bare one-line pass-through
After this PR clonePartitionSpec(spec) is just return spec.Clone(). Five call sites could call spec.Clone() directly and drop the wrapper, or keep it if a doc comment explains the indirection.
| fields[i] = iceberg.PartitionField{ | ||
| SourceIDs: []int{i + 1}, FieldID: i + 1000, | ||
| Name: "field", Transform: iceberg.IdentityTransform{}, | ||
| } |
There was a problem hiding this comment.
nit — Benchmark helper builds a spec with duplicate partition field names
partitionSpecCloneBenchmarkFields assigns Name: "field" to all N fields, producing a spec UnmarshalJSON would reject and that no real table can have. Using strconv.Itoa(i) for the name (strconv is already imported) makes the benchmark measure a realistic spec, including distinct url.QueryEscape work in initialize().
Summary
This keeps the defensive copies, but avoids doing the same work more than once.
Schema.Fields().PartitionSpec.Clone()so fields are copied once and the source index is rebuilt.Testing
go test ./table ./viewgo test ./catalog/hive ./table ./viewgo test -race ./table ./viewgo vet ./table ./viewmake lintwithgolangci-lint v2.12.2Benchmark
BenchmarkCloneSchemaWithNestedDefaults: about 3.0-3.8 us/op, 4320 B/op, 63 allocs/op to 1.82-1.84 us/op, 2696 B/op, 38 allocs/op.BenchmarkClonePartitionSpecswith 32 fields: 108 to 75 allocs/op and 12,968 to 10,024 B/op in local runs.