Rollback Prepared Transactions Asynchronously During Binlog Crash Recovery - #711
Rollback Prepared Transactions Asynchronously During Binlog Crash Recovery#711SongLibing wants to merge 1 commit into
Conversation
|
Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA).
To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application. When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated. If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public. |
|
Thank you for signing the OCA. |
e6d0129 to
c85ef66
Compare
c85ef66 to
6d6e94a
Compare
|
Thank you SongLibing for this. We've uncovered that internally in Oracle this same issue and solution(s) have been discovered a couple years ago, discussed and documented, but not prioritized. We've made https://bugs.mysql.com/bug.php?id=114053 public so that it may benefit the review of this PR. The review will take its time, we will probably want both a member from replication team as well as InnoDB team to look at this and this is obviously very critical part of the database, so they will take their time. Other than that everything seems to be in order now, oca signed and tests green. |
6d6e94a to
5be807a
Compare
23d5eee to
23b5313
Compare
Crash Recovery Description =========== Binary log crash recovery resolves internal transactions that were prepared in a storage engine before the server stopped. If the binary log does not contain the transaction's commit decision (Xid_log_event), the transaction must be rolled back. Historically, the server invokes `rollback_by_xid()` in the startup thread and waits for the complete rollback. Undoing a large transaction row-by-row can therefore keep the server unavailable for hours. This change adds an optional recovery-specific storage-engine callback to perform a fast rollback handoff. InnoDB uses it to durably change the state of a prepared DML transaction back to active. The existing InnoDB recovery rollback thread then does the expensive row-by-row undo work in the background while server startup continues. The final transaction outcome does not change. detail: design/server/100-async-rollback-during-binlog-recovery.md
23b5313 to
a2ea313
Compare
jujose-1
left a comment
There was a problem hiding this comment.
Hi @SongLibing,
Thank you for the contribution. I reviewed the design from the replication and transaction-coordinator recovery perspective (the InnoDB-specific aspects would still need review from the InnoDB team).
The overall approach looks reasonable to me and appears to preserve the existing transaction outcome. I’ve left a few inline comments for clarification. Please take a look when you have a chance.
Thank you for working on this.
Regards,
Justin Jose
| rollback thread then does the expensive row-by-row undo work in the background | ||
| while server startup continues. The final transaction outcome does not change. | ||
|
|
||
| ## Functional Requirements |
There was a problem hiding this comment.
Thank you for documenting the requirements. Could you please consider expanding and regrouping this section? The current FR1 appears to describe a performance property, while NFR1 describes functional correctness.
Would it make sense to cover the following requirement areas?
Functional requirements
- Transaction outcome and eligible transaction scope
- Asynchronous handoff and synchronous fallback
- Callback error handling and multi-engine transactions
- Crash, restart, and shutdown behavior
Non-functional requirements
- Startup-time improvement and its measurable boundary
- Replication, GTID, point-in-time recovery (PITR), and upgrade/downgrade compatibility
- Resource usage and observability
| There is no change to external XA recovery: the new callback is selected for | ||
| internal transactions by the existing binlog recovery decision path. There is | ||
| also no change to security checks, SQL interfaces, replication protocols, or | ||
| user-visible configuration. The interaction with the existing | ||
| `innodb_force_recovery` setting is described above. |
There was a problem hiding this comment.
Could you please expand the impact assessment slightly beyond protocol compatibility?
It would be useful to confirm that GTID recovery, the recovered binlog position, and point-in-time recovery preserve the same final transaction outcome. It would also help to describe whether asynchronous replication or Group Replication can start while rollback is still running and what happens when an applier transaction conflicts with locks retained by the recovered transaction.
| There is no change to external XA recovery: the new callback is selected for | ||
| internal transactions by the existing binlog recovery decision path. There is | ||
| also no change to security checks, SQL interfaces, replication protocols, or | ||
| user-visible configuration. The interaction with the existing |
There was a problem hiding this comment.
The design states that there is “no change to … user-visible configuration,” which appears consistent with the “Configuration / Knobs: No change” section in Issue #100.
However, the issue also refers to synchronous fallback “when the feature is disabled,” requests coverage for “disabled-feature behavior,” and shows a recovery path where the “option is ON.”
Could you please clarify the intended behavior and align the issue and design accordingly? If “enabled” refers only to whether a storage engine implements the optional callback, perhaps the option and disabled-feature wording in the issue could be updated. Otherwise, it would be helpful to describe the configuration mechanism in the design.
| - `storage/innobase/trx/trx0trx.cc`: scheduling tables for metadata-lock | ||
| acquisition before background rollback. | ||
|
|
||
| ## Alternatives Considered |
There was a problem hiding this comment.
Since the proposal adds a new storage-engine callback, could you please briefly explain why a recovery-specific callback is preferred over changing the behavior of the existing rollback_by_xid() callback?
It may also be useful to mention that adding a separate rollback worker was not selected because the existing InnoDB recovery rollback thread can already perform this work. A few sentences should be sufficient.
Problem:
Binlog recovery can block server startup for a long time when it must roll back a large prepared internal XA transaction. The rollback runs synchronously before the server becomes available.
Solution:
Add an optional storage engine callback, recover_rollback_by_xid, for recovery-time rollback. During binlog recovery, InnoDB uses the callback to persistently convert a recovered prepared DML transaction back to ACTIVE state. The background recovery rollback thread then performs the expensive undo work asynchronously. DDL transactions still fall back to the normal synchronous rollback_by_xid path.
Add a binlog MTR test that covers prepared DML rollback, background rollback handoff, and repeated-crash safety before the rollback state change is flushed.
Copyright (c) 2026, Oracle and/or its affiliates.
What does this change do?
mysql/mysql-community#100
How was it tested?
mysql-test/scripts/ci/mtr.shpasses locallyContributor checklist
scripts/ci/format.sh)AI assistance
If AI assistance was used, describe the tool(s) and extent of use: codex, only merge code from AliSQL to trunk.
Areas touched
replication, innodb