Sync with github/gh-ost@master - #16
Merged
Merged
Conversation
* add copier test, upgrade testcontainers * vendor * test composite PK
github#1536) * Before the successful renaming, a session accessed the ghost table, which had already unlocked the original table. There is a very small probability that other sessions dml operations on the original table will occur, and this dml operation will appear in the original table after renaming, resulting in data loss. --------- Co-authored-by: dbking <dbking@mexc.com> Co-authored-by: meiji163 <meiji163@github.com>
* Add postpone-cut-over-flag-file interactive command * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix import error --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: meiji163 <meiji163@github.com>
* Fix conversion bug when string data comes from binlog Fixes github#1568 When using gh-ost to migrate a table from latin1 to utf8mb3 character encoding, the initial data copy works correctly, but new data with special characters inserted during the migration via binlog replication fails with "Incorrect string value" errors. The reason for this is that the data is a binary byte array when converted from the binlog, so the character set conversion is not applied. This fix updates the character set conversion logic to apply to both string and []uint8 types when the column has a character set conversion. I added a new test for latin1 input to this method and confirmed that the reproduction from the linked issue is fixed with this change. * Update whitespace formatting
* add sysbench localtest * fix table name * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * ensure cleanup --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: meiji163 <meiji163@github.com>
…ly (github#1594) * Panic if InitiateHeartbeat exhausts retries to avoid looping infinitely. Based on experience, if the writer database fails inbeetween the copy & cutover stages (e.g. during cutover pause), the heartbeat writes will fail and stop, then leading to throttled state and an infinite loop of throttler.shouldThrottle(). Since this state is irrecoverable, make the heartbeat writer panic if retries are exhausted, so that the migration can fail and be restarted later. * Add sysbench localtest (github#1590) * add sysbench localtest * fix table name * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * ensure cleanup --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add toxiproxy option for localtests (github#1591) --------- Co-authored-by: Jan Grodowski <jan.grodowski@shopify.com>
* WIP: add GTID support * Cleanp * Add doc for flag * Rename GTIDSet var * Fix GTID SID parsing * Cleanup * Add to docs * Require enforce_gtid_consistency=ON * Rename validator func * simplify check in validateBinlogsAndGTID() * Only update GTIDSet if there was no err * Simplify GTIDEvent -> GTIDSet * Simplify GTIDEvent -> GTIDSet further, resolve go.uuid dep issue * Fix UUIDSet GNO * Add .ParseGTIDBinlogCoordinates() * Add missing smaller-than/equal logic * Comment-out WIP test * Fail on SID change * Fix import err * Add missing .Equals() check * Fix .SmallerThan()/.SmallerThanEquals() funcs and tests * Fix type change issues * Fix panics * Cleanup .SmallerThan() * Add missing check in .Equals() * Add large GTID sets to test * Simplify if cond * Add localtest/gtid * simplify localtest gtid_mode check * print gtid config * Fix var typo * support enforce_gtid_consistency=1 * fix test * Handle PreviousGTIDsEvent * gofmt * Create `validateGTIDConfig` func * Update copyrights * Update copyrights pt 2 * Copyrights again * Update docs * WIP * WIP * fix BinlogCoordinates interface usage * add binlog stream retry and toxiproxy test * fix max retry * use AddGTID instead of AddSet * remove previous GTID event handling * use AddSet * modify GTID coord tracking * fix last trx coords * re-enable Binlogsyncer retry * fix localtest arg * remove unneccesary coordinate check * rm unused funcs * add back ReplicaTermFor * Update localtests/test.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Tim Vaillancourt <timvaillancourt@github.com> Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: meiji163 <meiji163@github.com>
* add Checkpoint table and read/write funcs * handle no checkpoints returned * store min and max range values in checkpoint * resume from checkpoint * add checkpoint file * fix unique key args * update applier coordinates from _ghc heartbeat * fix test * fix linter * make checkpoint interval configurable * write checkpoint iteration number * store rows copied & dml applied * truncate column name if necessary * drop checkpoint table for final cleanup * add docs * add resume doc
grodowski
force-pushed
the
grodowski/sync-merge-master-2025-11-05
branch
from
November 5, 2025 11:20
11a29b8 to
b7f4e1a
Compare
new file: vendor/github.com/docker/docker/api/types/image/image_inspect.go
grodowski
force-pushed
the
grodowski/sync-merge-master-2025-11-05
branch
from
November 5, 2025 11:23
b7f4e1a to
493d0fd
Compare
Fixup to c25d7dc. The sysbench test does insert and delete data to simulate db load.
coding-chimp
approved these changes
Nov 5, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync upstream at github@1557a95.
Main goals was to keep history clean, which I think is needed as long as we keep features that are still unmerged upstream. Otherwise, we could reset to
github@masterand cherry pick #10Currently unmerged features:
dev.ymland podman setup (Build and local dev workflow #10)I'll document the workflow for this in a follow up PR to
README_Shopify.mdfrom #10Verification