Skip to content

test: backwards compatibility suite and refactor of the test infra - #1137

Open
mihir-datazip wants to merge 23 commits into
stagingfrom
backward-compatibility-tests
Open

test: backwards compatibility suite and refactor of the test infra#1137
mihir-datazip wants to merge 23 commits into
stagingfrom
backward-compatibility-tests

Conversation

@mihir-datazip

@mihir-datazip mihir-datazip commented Aug 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

Adds the backward-compatibility suite -- the same scenarios run twice, once entirely on a baseline image and once handing off to this build after the initial load, then the two destinations are compared -- and splits the test harness so it can host it.

testutils now holds only what every suite shares (TestConfig, docker plumbing, JSON/state helpers); the suites live in integration/, performance/ and compatibility/. TestConfig.Setup resolves the repo root, creates the suite's own working directory, copies the shared fixtures in and lets a driver's own override them by name, so files are addressed by name rather than by a field each.

Also fixes a product bug this surfaced: LoadedStateVersion was initialised from LatestStateVersion by a package-level var initializer, which Go evaluates before init() assigns it -- so every command that does not load a state file ran at state version 0 and silently took the pre-v7 branch of the gated parquet behaviours.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

  • make test.lint -- clean across all nine modules, db2 included
  • make test.discover.<driver> -- passes for postgres, mysql, mongodb, mssql, oracle, kafka (JSON/AVRO) and s3 (all four variants)
  • make test.sync.<driver> -- passes for postgres, mysql, mongodb, mssql, oracle, kafka; s3 passes 3 of 4 variants

Screenshots or Recordings

N/A

Documentation

  • N/A (refactor and test changes only)

Related PR's (If Any):

None

https://claude.ai/code/session_01KoSeYoWQWg4xsEg89NVsRE

@mihir-datazip
mihir-datazip force-pushed the backward-compatibility-tests branch from 9de32a7 to d4795ee Compare August 24, 2026 12:45
@mihir-datazip
mihir-datazip force-pushed the backward-compatibility-tests branch from ad4e9c9 to 7bff5c4 Compare August 27, 2026 14:40
@mihir-datazip
mihir-datazip force-pushed the backward-compatibility-tests branch from 20be56e to 7bff5c4 Compare August 27, 2026 17:09
@mihir-datazip
chore: compatibility test with integration tests
bb9f755
@mihir-datazip
chore: single machine for single compatibility check
3fed000
@mihir-datazip
chore: temp - skip the ghcr image restore to measure the worst case
d62e69b
@mihir-datazip
chore: single machine for single compatibility check
cfc9d93
@mihir-datazip
chore: cache driver docker images
2c42541
@mihir-datazip
chore: artifacts for candidate docker image
bee8d8d
@mihir-datazip
chore: temp
3f858c4
@mihir-datazip
chore: single docker image export job
c4035ec
@mihir-datazip
chore: parallel image restore
f218515
@mihir-datazip
Merge branch 'staging' into backward-compatibility-tests
60d8e60
@mihir-datazip
chore: minor changes
e6447b1
@mihir-datazip
chore: merge staging into backward-compatibility-tests
269611b 
Keeps the workflow consolidation (integration-tests.yml and performance-test.yml
were replaced by tests.yml + test-preflight.yml) and carries staging's dependabot
bumps into the files that inherited those action calls:
  actions/setup-java v3/v4 -> v5
  aws-actions/configure-aws-credentials v4 -> v6
@mihir-datazip
chore: tests UTs
cdf3f0f
@mihir-datazip
Merge branch 'staging' into backward-compatibility-tests
cbc7ea3
@mihir-datazip
chore: minor changes
d3843ef
@mihir-datazip
fix: postgres issue
24b8b12
@mihir-datazip
chore: rename func
44be097
@mihir-datazip
chore(tests): run driver containers as the invoking user in a fixed n…
c7b4571 
…on-UTC zone

--user: TestWorkingDir is a t.TempDir() bind-mounted into the container, and the
image declares no USER. On a Linux bind mount everything the driver writes there
belongs to root. A root-owned file is still removable, but a root-owned directory
is not, and pkg/indexdb creates one (olake-table-index). Go's own t.TempDir
cleanup then fails with permission denied and, reporting through t.Errorf, fails
a test whose sync had already passed.

TZ: several state-version gates only change behavior when the machine timezone is
not UTC. In a UTC container the old and new branches produce identical output, so
the compatibility suite could not tell a working gate from a removed one.

Also drops the temporary pg_class diagnostic added while chasing the concurrent
CREATE TABLE failure.
@mihir-datazip
revert(tests): drop the non-UTC container timezone
f34d07f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mihir-datazip