Sync upstream 2026-07-13 - #54
Merged
Merged
Conversation
…uery latency, sleep (github#1701) * metrics: introduce unified Emitter interface and consolidate go_runtime helpers Introduce a single Emitter interface (Gauge, Count, Histogram) on metrics.Client so all metric helpers share one testable contract. The interface replaces the narrow MemStatsGaugeEmitter type used only by the Go runtime reporter. - Add Emitter interface and Histogram method on Client. - Move go_runtime.go and go_runtime_test.go into emit.go and emit_test.go so the metrics package has a single place for helper functions and tests. - Switch MigrationContext.Metrics from *metrics.Client to metrics.Emitter so consumers can be tested with spies. * Refactor status reporting around migrationProgressSnapshot. Sample row-copy, DML, backlog, and lag once per tick into a snapshot passed to printStatus, with reportStatus as the single entry point for status output. Co-authored-by: Cursor <cursoragent@cursor.com> * metrics: emit row-copy progress gauges Add gh_ost.row_copy.rows_copied, gh_ost.row_copy.rows_estimate, and gh_ost.dml.events_applied gauges on each reportStatus tick, sampled from migrationProgressSnapshot. * metrics: emit binlog backlog gauges Add gh_ost.binlog.backlog_size, gh_ost.binlog.backlog_capacity, and gh_ost.binlog.backlog_utilization gauges on each reportStatus tick from the applyEventsQueue depth captured in migrationProgressSnapshot. * metrics: emit replication and heartbeat lag gauges Add gh_ost.lag.replication_seconds and gh_ost.lag.heartbeat_seconds gauges on each reportStatus tick, tagged with throttled:true|false. These are point-in-time readings (not distributions), so gauges are used rather than histograms — DogStatsD histogram aggregation exposes count/max series that do not match the log line lag values in Prometheus/Grafana. * metrics: emit throttle active and throttled-interval metrics Record throttle active state at a debounced cadence (gh_ost.throttle.active) and emit duration plus event metrics when a throttled interval completes (gh_ost.throttle.duration_seconds histogram and gh_ost.throttle.events_total count), each tagged with the throttling reason. * metrics: emit cut-over phase, attempt, and total duration metrics Add cut-over metric helpers and instrument cut-over attempts, phase durations, and terminal duration. Metrics emitted: - gh_ost.cut_over.attempts_total tagged with outcome - gh_ost.cut_over.phase_duration_milliseconds tagged with phase and outcome - gh_ost.cut_over.total_duration_milliseconds tagged with outcome Phase coverage includes the magic lock, original table lock, magic rename, and unlock paths. Durations are reported in milliseconds to preserve sub-second granularity. The atomic rename phase duration is recorded after the rename completes so the histogram reflects the full operation. * metrics: emit query latency histogram Emit gh_ost.query.duration_milliseconds for representative source-side and target-side queries (row count and binlog apply), tagged with side, kind, and outcome (ok|error). Helper validates inputs and is nil-safe. * metrics: emit per-stage sleep histograms Add sleep metric helpers and instrument the main migration sleep/wait paths. Metrics emitted: - gh_ost.sleep.duration_milliseconds tagged with stage - gh_ost.sleep.total_milliseconds tagged with stage Stages covered: - cut_over_postpone - chunk_throttle - retry_backoff - replica_wait Use millisecond units so sub-second waits, such as replica polling and nice-ratio throttling, are not truncated to zero. Skip sub-millisecond chunk-throttle samples to avoid emitting zero-valued sleeps that would distort the histogram. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Jan Grodowski <jan.grodowski@shopify.com>
* Document missing command-line flags. Add sections for command-line flags that were registered in the CLI but missing from command-line-flags.md, including logging, throttling, hooks, StatsD, trigger, revert, and basic connection flags. Update the documented cut-over default from the stale "safe" value to the current "atomic" default.
Rework the replica-test harness to run the localtests suite against MySQL 5.7, 8.0, 8.4 and Percona Server 8.0 from one parameterized setup. - Move docker-compose into script/ and split per-version server config into docker/<flavor>-<major.minor>/ dirs (common.cnf, create_user.sql, start_replication.sql), selected from TEST_MYSQL_IMAGE. Optional flavor.env per dir for knobs (Percona reads /etc/my.cnf.d). - Configure replication from the harness once both servers are up rather than at container init, so that replica connects on the first try. - Keep mysql_native_password for 5.7/8.0/percona, caching_sha2_password for 8.4. - Speed-ups: native mysql client with docker-exec fallback, detect server version once, shorter replication polls, configurable copy-throttle window (THROTTLE_SECONDS, default 2s to still exercise binlog DML replay). - toxiproxy split into its own compose file.
* feat: support MariaDB 11.4 and 11.8 gh-ost mostly works against MariaDB unchanged; this makes the few MariaDB-specific spots correct and runs the replica test suite against MariaDB 11.4 and 11.8 (LTS) alongside the MySQL/Percona flavors. Core: - ReplicaTermFor keeps the legacy slave/master replication terminology on MariaDB. MariaDB reports versions >= 10 (greater than the 8.4 cutoff) but never adopted the new replica/source terms, and its SHOW REPLICA STATUS still emits legacy column names, so the MySQL 8.4 term map doesn't apply. - Accept BINLOG MONITOR as well as REPLICATION CLIENT in the grant check (MariaDB 10.5+ renamed the privilege). Test harness (localtests): - test.sh: pick slave/Master terminology for MariaDB, wait for replica catch-up via MASTER_GTID_WAIT/gtid_binlog_pos, and skip gtid_mode-gated tests where @@gtid_mode is unsupported instead of aborting the run. - Resolve the per-flavor config dir by specificity (docker/<flavor>-<major.minor>, then <flavor>-<major>, then bare <flavor>) so versions that share one config live in a single dir. All MariaDB versions use one docker/mariadb dir (log-bin + ROW + log-slave-updates, native-auth users, MariaDB-GTID replication, MYSQL_CLIENT=mariadb since 11.x dropped the mysql-named client); a version (or major line) needing different config overrides it by adding its own dir. The compose healthcheck honours MYSQL_CLIENT. - Make the zero-date failure tests deterministic across flavors with an explicit sql_mode (NO_ZERO_IN_DATE,NO_ZERO_DATE,...); they relied on MySQL's default sql_mode, which MariaDB doesn't share. - Drop the incidental NOT NULL from the generated-column tests (MariaDB forbids NOT NULL on generated columns) so they run on MariaDB; keep generated-columns-unique skipped (it puts a generated column in the PRIMARY KEY, which MariaDB forbids, Error 1903). - Add mariadb:11.4.12 and mariadb:11.8.8 to the CI matrix. * feat: support MariaDB 10.5, 10.6 and 10.11 Add the MariaDB 10.x LTS lines (10.5, 10.6, 10.11) to the replica test suite. MariaDB versions below 11.1 need one core fix: gh-ost passes transaction_isolation as a DSN param, which the driver applies as `SET transaction_isolation=<v>` on each connection. That system variable does not exist on MariaDB < 11.1 (which has only tx_isolation), so the connection fails with Error 1193 "Unknown system variable 'transaction_isolation'". tx_isolation was in turn removed in MySQL 8.0, so no single variable name is portable across the supported servers. - mysql.OpenDB strips the transaction_isolation param and instead runs the SQL-standard `SET SESSION TRANSACTION ISOLATION LEVEL <level>` on each new connection via a connector wrapper, accepted by every supported MySQL and MariaDB version. GetDBUri still emits the param unchanged, so DSN construction and its tests are unaffected. - Add mariadb:10.5.29, mariadb:10.6.27 and mariadb:10.11.18 to the CI matrix.
Upgrade go-mysql to v1.15.0
* build: standardize release asset naming (github#1728) Rework build.sh so released assets have stable, predictable names that are easy to automate against: - drop the generated timestamp from tarball names - drop the "-binary" infix from tarball names - name macOS tarballs with "darwin" (matches GOOS) instead of "osx" - drop the version from .rpm/.deb filenames; the release download URL already carries it. Package metadata still records version + epoch - build arm64 .rpm and .deb in addition to amd64, using each ecosystem's arch convention (x86_64/aarch64 for rpm, amd64/arm64 for deb) - stop publishing the bare "gh-ost" binary; it was just the last build (darwin/arm64) and was ambiguous. Tarballs and packages cover every OS/arch - emit a SHA256SUMS manifest so downloads can be verified with `sha256sum -c SHA256SUMS` Resulting assets: gh-ost-{linux,darwin}-{amd64,arm64}.tar.gz gh-ost.{x86_64,aarch64}.rpm gh-ost.{amd64,arm64}.deb SHA256SUMS Also refactor build() to take GOOS/GOARCH directly, share common fpm flags via an array, and update the README build.sh description. * build: simplify packaging Dockerfile and dock script (github#1728) Slim down the release packaging tooling: - Dockerfile.packaging: collapse the per-package RUN layers into one, drop packages already provided by the golang:bookworm base (git, tar, curl, gcc, g++, bash) and the unused rsync, and drop the legacy GOPATH src layout (the build is module-mode). Only ruby/ruby-dev/build-essential (for fpm) and rpm (for rpmbuild) are installed now. - Use a multi-stage build with a `scratch` artifacts stage so `docker build --output type=local` exports the assets straight to the host. This drops the intermediate container, bind mount, -it TTY and find|xargs cp dance from `script/dock pkg`. - build.sh: stage the fpm package tree in the system temp dir instead of under $buildpath, so it never lands among the exported release artifacts.
…60713 # Conflicts: # go/logic/migrator.go # go/logic/migrator_test.go # go/logic/throttler_test.go # go/metrics/client.go # go/metrics/emit.go # go/metrics/emit_test.go
coding-chimp
approved these changes
Jul 14, 2026
grodowski
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Sync
masterfrom upstreamgithub/gh-ostmaster.Upstream commits brought in:
c2312729build: standardize release asset naming (build: standardize release asset naming github/gh-ost#1729)64a6e3ddMerge pull request Upgrade go-mysql to v1.15.0 github/gh-ost#1715 from github/upgrade-go-mysql7db8f74cMerge branch 'master' into upgrade-go-mysql653b99e7feat: 🦭 MariaDB support 🦭 (feat: 🦭 MariaDB support 🦭 github/gh-ost#1724)06b3a7e2test(localtests): multi-flavor docker replica-test harness (Multi-flavor docker replica-test harness github/gh-ost#1722)dcda427bDocument missing command-line flags (Document missing command-line flags github/gh-ost#1706)dce77339Merge branch 'master' into upgrade-go-mysql1409f8bcDon’t parse binlog events of unrelated tables (Don’t parse binlog events of unrelated tables github/gh-ost#1699)f50264bbUpgrade go-mysql to v1.15.0aecf39f4StatsD instrumentation: row-copy, backlog, lag, throttle, cut-over, query latency, sleep (StatsD instrumentation: row-copy, backlog, lag, throttle, cut-over, query latency, sleep github/gh-ost#1701) — our upstreamed PRConflict resolutions
go/metrics/emit.go+emit_test.go: accepted upstream's consolidated metrics helper layout and metric names for overlapping StatsD helpers.go/logic/throttler_test.go: accepted upstream'sthrottle.duration_millisecondsexpectation to match upstream metric naming.go/metrics/client.go: accepted upstream's expandedEmitterinterface comment; interface shape remains the same.go/logic/migrator.go+migrator_test.go: accepted upstream's progress metrics comments/test formatting for overlapping code.Verification
go test ./go/...— success