Repository navigation
EXPORT PARTITION: force-replace and the new Keeper key can duplicate or strand exports #2576
Description
Related: #2548
Two high-severity defects in the plain and replicated EXPORT PARTITION changes on antalya-26.8. Codex already filed both as P1 review comments on commit aaf00db (2026-10-12). The follow-up commit 5705da65 does not fix them. This issue tracks them so they are not left only as review comments.
Defect 1 — Force-replace commits the old export and the new one
Codex: #2548 (comment)
Anchor: MergeTreePartitionExportScheduler::addTask, MergeTreeData::killExportPart, MergeTreePartitionExportScheduler::tryCommit
KILL refuses to cancel a plain MergeTree export while committing is set. export_merge_tree_partition_force_export does not. It persists a new transaction id, replaces the in-memory task, and calls killExportPart. That only sets a cancel flag and drops the part manifest immediately. It does not wait for the part writer to stop, and it does not stop tryCommit if the destination commit has already started.
The old Iceberg commit then publishes a snapshot under the old transaction id. tryCommit discards that result locally because the registry now holds the new id. The replacement export commits the same files again under a different id. Iceberg idempotency is per transaction id, so both snapshots are appended and the current snapshot lists the rows twice. With overwrite, the replacement can also rewrite data files while the first commit is still publishing.
How to reproduce
- Start
EXPORT PARTITIONfrom a plainMergeTreetable to an Iceberg destination with enough data that the part upload or the Iceberg commit is still running. - While that task is in
tryCommit(or while a part write is still running), run the sameEXPORT PARTITIONwithexport_merge_tree_partition_force_export = 1. - Read the destination Iceberg table and its snapshots.
Expected: one snapshot for the replacement export. The previous transaction does not publish.
Actual: two snapshots append the same files. Concurrent writers can also share one part after the old manifest is removed.
Defect 2 — The new Keeper key does not match existing export znodes
Codex: #2548 (comment)
Anchor: ExportPartitionUtils::compositeKey, ExportReplicatedMergeTreePartitionTaskEntry::getCompositeKey, ExportPartitionManifestUpdatingTask::removeStaleEntries, StorageReplicatedMergeTree::exportPartitionToTable
Replicated export task keys changed from {partition_id}_{database}.{table} to {escaped partition}.{escaped database}.{escaped table}. poll still discovers the old Keeper child and inserts its manifest, but the in-memory index is the new key. removeStaleEntries then compares that new key with the old child name, treats the task as deleted, calls killExportPart, and erases it before the model is published. The scheduler never resumes it.
A later EXPORT PARTITION of the same partition checks only the new path, so a completed export stored under the old key does not conflict. The new task gets a new transaction id and appends the data again.
How to reproduce
- On a build from before this key change, run
EXPORT PARTITIONon aReplicatedMergeTreetable so Keeper has anexports/{partition_id}_{database}.{table}znode. Leave onePENDINGand oneCOMPLETED. - Upgrade to this PR.
- Check
system.partition_exportsand whether thePENDINGtask keeps exporting. - Run
EXPORT PARTITIONagain for theCOMPLETEDpartition withoutexport_merge_tree_partition_force_export.
Expected: existing tasks stay visible and keep their old Keeper path. A completed export still blocks a second export unless force is set.
Actual: the PENDING task is dropped from memory on every poll and stalls until timeout. The COMPLETED task is invisible, and a second export appends another snapshot.
Fix direction
- Reject or defer force-replace while
committingis set, and do not schedule the replacement until the previous transaction's part tasks have stopped. - Keep the Keeper child name as the task identity, or migrate old nodes, and resolve the legacy key in the exists-check, scheduler, and stale-entry pass.
Activity
[-]EXPORT PARTITION: force-replace and the new Keeper key can duplicate or strand exports (PR #2548)[/-][+]EXPORT PARTITION: force-replace and the new Keeper key can duplicate or strand exports[/+]