CNDB-18860: Rebase main-5.0 onto cassandra-5.0.9 - #2565
Open
JeremiahDJordan wants to merge 1862 commits into
Open
CNDB-18860: Rebase main-5.0 onto cassandra-5.0.9#2565JeremiahDJordan wants to merge 1862 commits into
JeremiahDJordan wants to merge 1862 commits into
Conversation
JeremiahDJordan
force-pushed
the
cndb-18860-main-5.0.9-rebase
branch
10 times, most recently
from
August 19, 2026 15:52
5693297 to
4faa67a
Compare
… dupes in result set (#2024) (cherry picked from commit ada025c) Copy of #2023, but targeting `main` riptano/cndb#15485 This PR fixes a bug introduced to this branch via #1884. The bug only impacts SAI file format `aa` when the index file was produced via compaction, which is why the modified test simply adds coverage to compact the table and hit the bug. The bug happens when an iterator produces the same partition across two different batch fetches from storage. These keys were not collapsed in the `key.equals(lastKey)` logic because compacted indexes use a row id per row instead of per partition, and the logic in `PrimaryKeyWithSource` considers rows with different row ids to be distinct. However, when we went to materialize a batch from storage, we hit this code: ```java ClusteringIndexFilter clusteringIndexFilter = command.clusteringIndexFilter(firstKey.partitionKey()); if (cfs.metadata().comparator.size() == 0 || firstKey.hasEmptyClustering()) { return clusteringIndexFilter; } else { nextClusterings.clear(); for (PrimaryKey key : keys) nextClusterings.add(key.clustering()); return new ClusteringIndexNamesFilter(nextClusterings, clusteringIndexFilter.isReversed()); } ``` which returned `clusteringIndexFilter` for `aa` because those indexes do not have the clustering information. Therefore, each batch fetched the whole partition (which was subsequently filtered to the proper results), and produced a multiplier effect where we saw `batch` many duplicates. This fix works by comparing partition keys and clustering keys directly, which is a return to the old comparison logic from before #1884. There was actually a discussion about this in the PR to `main`, but unfortunately, we missed this case #1883 (comment). A more proper long term fix might be to remove the logic of creating a `PrimaryKeyWithSource` for AA indexes. However, I preferred this approach because it is essentially a `revert` instead of fixing forward solution. (Rebase of commit fcc246f) (Rebase of commit 3f1b4bb)
) Add defensive checks in SSTableWriter and SSTableReader to handle closed channels and null file handles when creating shallow readers during flush error recovery. SSTableWriter: Catch IllegalStateException on file pointer access SSTableReader: Handle null dfile in bytesOnDisk() (Rebase of commit c688745) (Rebase of commit 8c5b8fb)
…#2004) Port some of the improvements for index hints done by [CASSANDRA-20888](https://issues.apache.org/jira/browse/CASSANDRA-20888), especially the ones in messaging. Also clean up unused methods in index hints. (Rebase of commit 6f77404) (Rebase of commit 7409a4a)
Fixes: riptano/cndb#15448 Bumps jvector version. Commits: datastax/jvector@4.0.0-rc.2...4.0.0-rc.3 diff: ``` jvector % git log 4.0.0-rc.2...4.0.0-rc.3 --oneline 17169513 (tag: 4.0.0-rc.3) chore: update changelog for 4.0.0-rc.3 (#528) 67b2f88d Regression enhancements (#526) baf87e80 chore: update changelog for 4.0.0-rc.3 (#527) f3d235cc Release 4.0.0-rc.3 cfb3004f streamline PR checklist (#525) df4a0688 add checklist template and initial CONTRIBUTIONS.md guide (#523) 63db005a GraphIndexBuilder::addGraphNode must iterate all graph levels to estimate used bytes (#521) 817a25c4 GitHub actions regression test (#499) 8364012f Remove unused construction batch member from OnHeapGraphIndex (#510) 1823b9be Switch from syncronized to concurrent map for pq codebook (#518) 6d590ad7 Enable specifying the benchmarks in the yaml file (#515) 1c298218 Create partial sums for PQ codebook for use during diversity checks (#511) a916a07c PQ ranging bugfix and refactoring (#508) 66399923 Reducing the number of allocations in GraphSearcher (#501) 51d4f0bb SimdOps and NativeSimd ops refactored, VectorUtilSupport simplified (#498) c5c3ff97 Add specific BuildScoreProvider for diversity to avoid extra encoding… (#503) 631515df Start development on 4.0.0-rc.3-SNAPSHOT ``` (Rebase of commit 39b3669) (Rebase of commit df8934a)
Compaction might write SAI differently than flush, thus it's good to have tests running after compaction. Changes to use runThenFlushThenCompact instead of beforeAndAfterFlush in BM25 test. Another test already tests after compaction in addition to memtable and after flush. (Rebase of commit bda4e88) (Rebase of commit 142a233)
…shards and min sstable size are configured (#2031) riptano/cndb#15253 large shard count jump was involved in HCD-130 Replace power-of-two shard progression with factorization-based smooth growth when base shard count is not power of 2. This prevents problematic large jumps. For example, with num_shards 1000 which is 5^3* 2^3: - Before: 1 → 2 → 8 → 1000 (125x jump causes data loss due to hcd-130) - After: 1 → 5 → 25 → 125 → 250 → 500 → 1000 (max 5x jump) * new behavior can be disabled via `-Dunified_compaction.use_factorization_shard_count_growth` (Rebase of commit 9a21657) (Rebase of commit 4274127)
There's a problem impacting all main-5.0 jvm-dtest-upgrade, as the run-tests.sh script it builds dtest-5.0.4.0.jar and then from the apache/cassandra cassandra-5.0 branch builds dtest-5.0.7.jar. Many of the jvm-dtest upgrade tests only perform upgrades to the CURRENT version which is the latest found, and that's the dtest-5.0.7.jar so the dtest-5.0.4.0.jar (and all working code changes) gets ignored. I don't know where/how CC jenkins runs the jvm-dtest-upgrade testsuite, but this happens when doing it like ``` .build/run-tests.sh -a jvm-dtest-upgrade ``` The fix is .build/run-tests.sh:202 needs to remove "cassandra-5.0" (Rebase of commit 38ae506) (Rebase of commit 7eb1ee9)
### What is the issue PaxosCommit lacks sensors ### What does this PR fix and why was it fixed Instrument PaxosCommit, PaxosPrepare, and PaxosPropose to track read, write, and internode bytes. Reuse existing RequestSensors in StorageProxy to prevent sensor data loss during CAS operations. (Rebase of commit 548bfc6) (Rebase of commit f760542)
riptano/cndb#8641 PR in CNDB: riptano/cndb#15306 This pull request introduces new metrics for tracking invalid and other error requests for CQL statements, and integrates these metrics into the query failure notification logic. The main changes are the addition of the `AllRequestsMetrics` class, updates to the `ClientRequestsMetrics` class to include these new metrics, and modifications to the query event notification methods to record errors using the new metrics. Metrics added: * org.apache.cassandra.metrics.ClientRequest.Timeouts.All * org.apache.cassandra.metrics.ClientRequest.Unavailables.All * org.apache.cassandra.metrics.ClientRequest.Failures.All * org.apache.cassandra.metrics.ClientRequest.Invalid.All * org.apache.cassandra.metrics.ClientRequest.OtherErrors.All Note: only requests for which a tenant can be identified are counted. (Rebase of commit 3517b24) (Rebase of commit 909d213)
Very long log lines can be produced during commitlog replay Limits the amount of information printed at debug, but retains full information at trace (Rebase of commit b1833e0) ninjafix – unused imports in test/unit/org/apache/cassandra/db/commitlog/CommitLogReplayerTest.java Rebase notes: - squash into b1833e0 (Rebase of commit e5bebc7) (Rebase of commit bbff159)
The PrimaryKeyWithSource class has been present for two years in the code base as an optimization for hybrid vector workloads, which have to materialize many primary keys in the search-then-sort query path. However, the logic is invalid for version aa (because we have the bug where compacted sstables write per row, not per partition) and it is also invalid for static columns. This commit avoids creation of PrimaryKeyWithSource in those cases. (Rebase of commit c0390b2) (Rebase of commit 1e79c35)
CNDB-15683: Fix incorrect results when querying mixed AA and EC indexes This commit fixes multiple issues with KeyRangeIterator implementations occasionally skipping or emitting duplicate keys when working on a mix of primary keys with empty / non-empty clusterings. This situation is possible while scanning tables with static columns or when some indexes are partition-aware (e.g. version AA) and others have been updated to a row-aware version (e.g. DC or EC). Due to those bugs, users could get incorrect results from SAI queries, e.g. results containing duplicated rows, duplicated partitions or even missing rows. The commit introduces extensive randomized property-based tests for KeyRangeUnionIterator and KeyIntersectionIterator. Previously, the tests did not test for keys with mixed empty/non-empty clusterings. Changes in KeyRangeUnionIterator: KeyRangeUnionIterator merges streams of primary keys in such a way that duplicates are removed. Unfortunately it does not properly account for the fact that if a key with an empty clustering meets a key with a non-empty clustering and the same partition key, we must always return the key with an empty clustering. A key with an empty clustering will always fetch the rows matched by any specific row key for the same partition, but the reverse is not true. The iterator implementation has been modified to always pick the key that matches more rows - a key with empty clustering wins over a key with non-empty clustering. Additionally, once a key with an empty clustering is emitted, no more keys in that partition are emitted. Changes in KeyRangeIntersectionIterator: Due to a very similar problem like in KeyRangeUnionIterator, KeyRangeIntersectionIterator could return either too few or too many keys, when keys with empty clusterings and keys with non-empty clusterings were present in the input key streams. In particular consider 2 input streams A and B with the following keys: A: 0: (1, Clustering.EMPTY) B: 0: (1, 1) 1: (1, 2) Key A.0 matches the whole partition 1. Therefore, the correct result of intersection are both keys of stream B. Unfortunately, the algorithm before this patch would advance both A and B iterators when emitting the first matching key. At the beginning of the second step, the iterator A would be already exhausted and no more keys would be produced. Finally key B.1 would be missing from the results. This patch fixes it by introducing two changes to the intersection algorithm: 1. A key with non-empty clustering wins over a key with empty clustering and same partition. 2. The selected highest key is not consumed while searching for the highest matching key, but that happens only after the search loop finds a match. Then we have more information which iterators would be moved to the next item. Iterators positioned at a key with an empty clustering can be advanced only after we run out of keys with non-empty clustering in the same partition or if there are no other keys with non-empty clustering. This patch also fixes another issue where we could return a less-specific key matching a full partition instead of a key matching one row: A: 0: (1, Clustering.EMPTY) B: 0: (1, 1) In that case the iterator returned a key with empty clustering, which would result in fetching and postfiltering many unnecessary rows. CNDB-15683: Fix incorrect results when querying mixed AA and EC indexes (#2066) When row-aware and non-row-aware indexes are mixed, we now check the clustering index filter for all the keys that have clustering information, i.e. keys coming from the row-aware indexes. Earlier that check was accidentally disabled if at least one non-row-aware index was used by the query. That could cause retrieving rows that do not match the clustering condition of the query. Rebase notes: - includes CNDB-15683 (Rebase of commit b25cba1) (Rebase of commit b722e78)
…raStreamReceiver if CDC is enabled on the node (#2043) Repairs use the local write path for streams on CDC-enabled tables, based on table schema. This interacts poorly with the separation of CNDB services. This commit fixes the issue by only using the CDC write path for a stream if CDC is enabled in the node's configuration (as well as in the schema). This avoids attempting to use the local write path if commitlog-based CDC is not enabled. (Rebase of commit b3bac73) (Rebase of commit 6726c0e)
…earlier than CA (#2071) Creating vector indexes if version is earlier than CA would usually fail in the asynchronous build. This patch makes them fail synchronously at CREATE INDEX depending on the local index version. If the local node has the right version but any of the remotes doesn't, the failure will remain asynchronous. (Rebase of commit de6fda5) (Rebase of commit 2869687)
Port of CASSANDRA-19968
Stop transforming single-partition queries using secondary indexes into range commands,
but use SinglePartitionReadCommand instead, as in not-indexed queries.
Benefits of single-partition commands are:
* They use speculative retries, which aren't supported for range queries.
* They use digest reads, which are benficial for network usage and throughput.
* Metrics and observability are able to distinguish between single-partition and cluster-wide SAI queries.
forward merges c628d7b
(Rebase of commit cab17f1)
Use graceful shutdown for the commitlog segment allocator instead of shutdownNow(), allowing the allocator to run its shutdown path and discard the prepared segment cooperatively. (Rebase of commit 1976b54)
…by simple class name (#2514) ### What is the issue In HCD we have a new custom Index Type com.datastax.opensearch.OpenSearchIndex that indexes data on OpenSearch. We want to make it userfriendly to create the index: `CREATE CUSTOM INDEX .... USING 'OpenSearchIndex' WITH OPTIONS ...` We also want that users do not need to allow all the secondary indexes in cassandra.yaml (in HCD secondary_indexes_enabled is disabled by default) See HCD-459. HCD PR here: riptano/hcd#243 CNDB PR here: riptano/cndb#18508 ### What does this PR fix and why was it fixed Add the `cassandra.trusted_index_implementations ` system property, a comma-separated list of fully qualified custom index class names. Each listed class is registered as an index name alias, so users can create it with `CREATE CUSTOM INDEX ... USING '<SimpleClassName>'` without the package name, like it is already possible for StorageAttachedIndex. Trusted implementations are also exempted from the `secondary_indexes_enabled` guardrail: they can be created even when the creation of secondary indexes is disabled. To keep their number under control, the new `trusted_indexes_per_table` guardrail limits how many indexes of each trusted implementation class can be created on a table (warn/fail thresholds, counted per implementation class, exposed through nodetool and `JMX` like the other index guardrails). It defaults to a failure threshold of 10 under the HCD guardrails profile and is disabled under the other profiles. Also expand index name aliases in CreateIndexStatement.isUnknownCustomIndexCreateStatement(), so that aliased index classes are not misreported as unknown when cassandra.index.unknown_custom_class.ignore is set. (Rebase of commit 99fff3f)
…c-5 only) (#2518) ### What is the issue In order to implement correctly the replication of Logged Batches to OpenSearch we need a mechanism to intercept the "recovery" of batches that have not been completely processed by the Coordinator. See https://datastax.jira.com/browse/HCD-438 ### What does this PR fix and why was it fixed Batchlog recovery (BatchlogManager.replayFailedBatches) re-applies logged batches after a coordinator failure with no way for plugins to observe it: the Mutator interface is coordinator-side only and replay bypasses it on every node. Add a pluggable BatchlogManagerInterceptor, wired like the custom Mutator via -Dcassandra.custom_batchlog_manager_interceptor_class, that is notified on the replaying node once per replayed batch, with the complete list of the batch's mutations (so implementations can process a batch's mutations together), after the batch's mutations have completed — applied locally, acknowledged by remote replicas, or hinted — and before the batchlog entry is deleted. The default implementation is a no-op. The interceptor is resolved eagerly at startup so a misconfigured custom class fails the node immediately (matching custom_mutator_class behaviour) and the installed implementation is visible in the startup log. If the callback throws, the batch is not removed from the batchlog and the callback is retried on later replay cycles — without re-delivering the batch. Retained batches are remembered in memory, mapped to the hosts hinted while replaying them, and retries invoke the callback alone: - The batch is re-read individually (a point read by id) instead of being kept inside the range scan, so lastReplayedUuid advances unconditionally and the scan does not re-walk a growing tombstone range while a batch is retained. Without this, a failing callback would have re-applied the batch's mutations and written a fresh round of hints for every down replica on every 10s replay cycle (batchlog replay hints bypass max_hint_window), accumulating hints until the batch aged past gc_grace_seconds. The retry pass runs after the scan (retries never starve replay of new content) and is throttled by the replay rate limiter. - On callback success the batch's previously hinted hosts rejoin the cycle's hint fsync barrier, and the batch is deleted only after that fsync — preserving the hints-durable-before-batch-deletion invariant even when the original cycle aborted before its own fsync. The in-memory entry is dropped, and the batch counted as replayed, only once its row is actually deleted, so an aborted cycle can neither double-count a batch nor fall back to a full (hint-rewriting) replay; the scan skips rows of batches awaiting their callback for the same reason. - Per-entry failure handling in the retry pass: a row removed by another path drops the pending callback with a warning (the batch still counts as replayed); a row that no longer deserializes (IOException) drops it with an error and deletes the row through the shared fsync barrier; any other error keeps the entry for a later retry, so one bad row cannot wedge batchlog replay node-wide. - The memory is intentionally not persisted: a restart falls back to replaying the whole batch from scratch, which the callback's at-least-once contract already tolerates. Semantics are therefore at-least-once and implementations must be idempotent; the javadoc spells out the carve-outs (truncated/dropped tables, gc_grace expiry, removal by another path, decommission) and that callbacks are not re-delivered in batch order. Repeated callback failures and the number of retained batches are reported through NoSpamLogger at a 1-minute interval. With the default no-op implementation the retained-batches map stays empty and the retry pass exits before issuing any read: the standard replay path performs no additional queries or I/O. Tests cover callback content and ordering relative to mutation application, retry-until-success with a failing callback, hints for a down replica being written exactly once across replay cycles (via StorageMetrics.totalHints, with a registered down replica and an RF=2 keyspace), the scan-skip guard after a simulated aborted cycle, and externally removed batches being forgotten without further callbacks. (Rebase of commit a132f1c)
…ery (#2488) (#2507) Queries with index-based ORDER BY will still use the index, since we don't have a way to run those without the index. Queries in which the indexes changes the semantics of the expression, such as those with analyzers, will preserve the index semantics, even if the index itself is not read. Co-authored-by: Andrés de la Peña <adelapena@users.noreply.github.com> (Rebase of commit 72685b7)
…y in 5.0 (#2521) Several members of PrimaryKeyMap.Factory are initialized in the constructor and not modified outside, however, they are not final allowing to unexpectedly modify them in future changes. Refactors the constructors of PrimaryKeyMap.Factory implementations to have all constructed members final. Reduces the scope of the try block. (Rebase of commit d3dcb74)
(Rebase of commit fcd4d64)
…tEncryptor (#2524) ### What is the issue getEncryptor(...) (introduced by CNDB-15098 to encrypt the Stats metadata component of TDE'd SSTables) opens the COMPRESSION_INFO component just to extract the table's compression/encryption parameters: CompressionMetadata cm = CompressionMetadata.open(compressionFile, 1024, false); ICompressor compressor = cm.parameters.getSstableCompressor(); CompressionMetadata is a ref-counted WrappedSharedCloseable holding the chunk-offset index in off-heap Memory (ChunkOffsetMemory), and the method dropped the instance without ever calling close(). Since getEncryptor runs on BOTH metadata paths -- serialize (every flush/compaction/metadata rewrite of an encrypted SSTable) and deserialize (every metadata read) -- each such operation leaked one unreleased Ref, surfacing at GC time as: ERROR [Ref] LEAK DETECTED: a reference (class org.apache.cassandra. utils.concurrent.WrappedSharedCloseable$Tidy@...:[org.apache.cassandra. io.compress.CompressionMetadata$ChunkOffsetMemory@...]) ... was not released before the reference was garbage collected ### What does this PR fix and why was it fixed Close the CompressionMetadata with try-with-resources once the parameters have been read: only cm.parameters is needed, and the ICompressor obtained from it does not reference the off-heap memory. Same pattern SSTableMetadataViewer already uses for the same call. (Rebase of commit 0162a93)
…re a node finishes leaving (#2533) ### What is the issue There was no way for a plugin to run work when an administrator decommissions a node. The extension points that come close do not fit: addPreShutdownHook fires on any graceful stop and cannot tell a decommission from a restart (and decommission never kills the JVM, so it does not fire then at all), and IEndpointLifecycleSubscriber.onLeaveCluster only reaches the leaving node at the very end of unbootstrap(), after everything has streamed away. This implementation is required to ensure that all the mutations processed locally are sent to OpenSearch before decomissioning the node, see https://datastax.jira.com/browse/HCD-474 ### What does this PR fix and why was it fixed Add a DecommissionHook interface plus StorageService.registerDecommissionHook / unregisterDecommissionHook. Several hooks may be registered and run in registration order. Hooks run in decommission() between unbootstrap() and the shutdown that follows, which is the only window that satisfies all of: - the batch log has completed its final replay and hints are transferred or dropped, and all ranges have streamed to their new owners; - the node has left the ring, so coordinators no longer route mutations to it. This has to be after leaveRing(), not at LEAVING: addLeavingEndpoint only records the endpoint in a set that write routing never consults, so a LEAVING node stays a natural write replica and pending ranges merely add the new owners on top; - messaging, the native transport and the stages are all still up, so a hook can run CQL queries and coordinate against the rest of the cluster. A hook may block indefinitely; decommission does not proceed until every hook returns. That window is also the one place nothing may escape from. leaveRing() has persisted NEEDS_BOOTSTRAP but DECOMMISSIONED is not set yet, so a throw here strands the node for good: a retry is rejected by the ring-membership check at the top of decommission(), since the node is no longer in TokenMetadata, and a restart bootstraps it back into the ring. So runDecommissionHooks() reports rather than throws -- every hook runs, the node finishes its decommission, and decommission() then throws naming the hooks that failed, so `nodetool decommission` still surfaces the error. For the same reason a hook's throwable is not passed to JVMStabilityInspector.inspectThrowable, which would rethrow a wrapped OutOfMemoryError straight out of this window. Interrupt state is consumed rather than propagated, both when a hook throws InterruptedException and when it restores the flag before returning: leaving it set would fail every remaining hook the moment it blocked, break the awaits in the shutdown below, and leak onto the pooled JMX handler thread. (Rebase of commit 46c96f4)
Check the allocation limits before starting a mutation and, once started, allow it to fully progress to completion: - `SubAllocator.allocate()` now only tracks usage; it never consults the limit and never blocks. Cleaner/flush triggering is unchanged. - New `SubAllocator.awaitRoom()` / `MemtableAllocator.awaitRoomToStart()` waits, without reserving memory, until the pool is below its limit. `AbstractAllocatorMemtable.put()` calls it before delegating to `performPut()`, so all subclasses are gated automatically. Groups marked by `Barrier.markBlocking()` skip or are released from this wait, as they were from `allocate()`, so a flush can always drain the ops its barrier awaits. - New `Memtable.putNested()` / `AbstractAllocatorMemtable.putNested()` variant for writes that occur inside an already-started mutation (e.g. legacy 2i writing to its index table's memtable from `indexer.onInserted()` under the base shard write lock). Nested writes skip the room gate — blocking there would hold the enclosing locks and re-create the deadlock. `ColumnFamilyStore.apply()` routes on updateIndexes to dispatch `put()` vs `putNested()`. Beyond the optimization described in the ticket, this fixes the HCD-442 deadlock: blocking mid-mutation under TrieMemtable's shard writeLock strands pre-barrier writes queued on the lock, where `markBlocking()` cannot reach them, and the flush writeBarrier never completes. Adds a regression test reproducing this deterministically on the previous code. (Rebase of commit 3706060)
…v2) (#2525) ### What is the issue The Mutator SPI's only Paxos hook, mutatePaxos, is a commit-phase transport hook of the v1 engine: with paxos_variant=v2 LWTs bypass the Mutator entirely, and even on v1 a custom Mutator cannot observe the begin or the completion of a CAS operation (condition-not-met, unknown-result, CL=ANY commits and pre-commit failures are all invisible), while repairs of other proposers' in-progress rounds fire mutatePaxos indistinguishably from client traffic. CNDB Issue: riptano/cndb#18612 CNDB PR: riptano/cndb#18613 HCD Issue: https://datastax.jira.com/browse/HCD-439 HCD PR: riptano/hcd#262 (not ready yet) ### What does this PR fix and why was it fixed Add an operation-level surface to the SPI, in three default methods so existing implementations stay source- and binary-compatible: - Mutator.mutateCas: called by StorageProxy.cas exactly once per client CAS operation (single conditional statement or conditional batch). The default implementation performs the existing engine dispatch (Paxos.useV2() ? Paxos.cas : legacyCas), so wrapping implementations get begin/completion for both paxos variants, across live variant flips, with the outcome derivable from the return value (null = applied, rows = condition not met) or the thrown exception. The javadoc spells out the v2 caveat that fate-unknown propose failures surface as plain timeout/failure exceptions (only the v1 engine throws CasWriteUnknownResultException). - Mutator.onCasCommit(Commit, ConsistencyLevel, CasCommitOrigin): fired BEFORE every coordinator commit dispatch of both engines and of background PaxosRepair, for every decided value, always through MutatorProvider.notifyCasCommit, which contains (logs and ignores, after JVMStabilityInspector) implementation exceptions so a misbehaving Mutator can never abort a paxos operation, serial read or repair. CLIENT_OPERATION fires at most once per mutateCas, inside that call, on the same thread, before the commit is dispatched, so implementations can correlate the two without ballot inspection. REPAIR_IN_PROGRESS labels the completion of another proposer's accepted round (including via SERIAL reads and background paxos repair, which may run on messaging threads); REFRESH_COMMITTED labels re-transmissions of already-committed values. Empty proposals are never committed and produce no callback. Because it fires before dispatch, a read issued from this callback sees the pre-commit value; use onCasCommitApplied (below) or a deferred SERIAL read to observe the committed value. - Mutator.onCasCommitApplied(Commit, ConsistencyLevel, CasCommitOrigin): fired AFTER a dispatched commit has been acknowledged by a consistencyLevel quorum, i.e. once the value is durably readable at that CL, always after the matching onCasCommit for the same ballot and only on the success path. If the commit times out or fails (the value is decided and will be completed later by a repair, but no quorum ack was received) onCasCommit still fires and this one does not: absence after an onCasCommit means "commit not confirmed", not "not decided". Routed through MutatorProvider.notifyCasCommitApplied with the same exception containment. It is delivered for CLIENT_OPERATION under both engines (on the request thread, inside mutateCas), for v1 REPAIR_IN_PROGRESS, and for all three background PaxosRepair commit sites; it is intentionally NOT delivered (only the dispatched onCasCommit is) for the v1 REFRESH_COMMITTED site (asynchronous fire-and-forget sendCommit with no awaited ack) nor for the v2 engine's begin()-path REFRESH_COMMITTED / REPAIR_IN_PROGRESS sites (where the commit piggybacks on the following prepare via commitAndPrepare and has no separable ack). For those, a deferred LOCAL_SERIAL/SERIAL read issued from onCasCommit is self-correcting and observes the value regardless. The javadoc documents this coverage in full. Call sites: v1 doPaxos and beginAndRepairPaxos (StorageProxy) fire the applied callback after the blocking commitPaxos returns (guarded on consistencyForCommit != ANY, which does not block); v2 Paxos.cas fires it after commit.awaitUntil succeeds (a committedAgreed local carries the value from the propose SUCCESS branch to the await point); the three PaxosRepair commit dispatches thread the committed value and origin into their completion callbacks (CommittingRepair / CommitAndRestart) and fire the applied callback when the commit status is success. MutatorProvider.instance is now public so the paxos package reaches the installed singleton; CommitVerbHandler (currently not registered for any verb in this tree) now uses that singleton instead of constructing a new custom Mutator per invocation. Backward compatibility: existing Mutator implementations -- including those overriding mutatePaxos -- require no changes. mutateCas, onCasCommit and onCasCommitApplied are default methods, so third-party implementations stay source- and binary-compatible, and the default mutateCas performs exactly the dispatch StorageProxy.cas used to perform inline. The v1 commit phase still invokes mutatePaxos on the installed Mutator at the same point as before (commitPaxos is untouched), and the v2 engine did not invoke mutatePaxos before and still does not. mutatePaxos and onAppliedProposal semantics are unchanged. MutatorCasTest covers both variants on a single node (via Paxos.setPaxosVariant -- DatabaseDescriptor.setPaxosVariant alone does not update the engine's own volatile snapshot read by useV2()): applied and non-applying operations (exactly one begin/completion, at most one CLIENT_OPERATION dispatched commit -- asserted to fire on the calling thread -- and none for non-applying), a v1 unavailable operation and a v2 throwing-condition operation completing exceptionally with no commit, and an injected accepted-but-uncommitted round whose completion by a later operation is reported as REPAIR_IN_PROGRESS with the foreign payload. The applied callback is pinned with a monotonic sequence stamp: exactly one CLIENT_OPERATION applied per applied CAS under both variants, on the calling thread, sequenced after the dispatched onCasCommit; none for non-applying or exceptional operations; and, for the injected repair, an applied callback for the v1 completion route (after its blocking commit) but none for the v2 begin()-path completion (dispatched-only by design). Backward compatibility: existing Mutator implementations -- including those overriding mutatePaxos -- **require no changes**. `mutateCas` and `onCasCommit` are default methods, so third-party implementations stay source- and binary-compatible, and the default mutateCas performs exactly the dispatch StorageProxy.cas used to perform inline. The v1 commit phase still invokes mutatePaxos on the installed Mutator at the same point as before (commitPaxos is untouched), and the v2 engine did not invoke mutatePaxos before and still does not. MutatorCasTest pins this: its recording Mutator counts mutatePaxos invocations and asserts exactly one per applied v1 CAS, none for non-applying operations, and none under v2. mutatePaxos and onAppliedProposal semantics are unchanged. MutatorCasTest covers both variants on a single node (via Paxos.setPaxosVariant -- DatabaseDescriptor.setPaxosVariant alone does not update the engine's own volatile snapshot read by useV2()): applied and non-applying operations (exactly one begin/completion, at most one CLIENT_OPERATION commit -- asserted to fire on the calling thread -- and none for non-applying), a v1 unavailable operation and a v2 throwing-condition operation completing exceptionally with no commit (the RF=2 unavailability scenario cannot reproduce under v2, whose consensus quorum is sized from the actual electorate), and an injected accepted-but-uncommitted round whose completion by a later operation is reported as REPAIR_IN_PROGRESS with the foreign payload. Second commit: HCD-439: Replace onCasCommitApplied with terminal onCasCommitCompleted Unify the post-commit callback into a single terminal notification that pairs every onCasCommit with a success/failure outcome, so a Mutator can close out the operation it opened on onCasCommit. - Add Mutator.CasCommitOutcome { APPLIED, CONFIRMED_BY_PREPARE, SUPERSEDED, UNCONFIRMED } and onCasCommitCompleted(commit, cl, origin, outcome), removing onCasCommitApplied. - v2 Paxos.cas CLIENT_OPERATION: APPLIED on commit ack, UNCONFIRMED on commit-await failure before rethrow. - v2 begin() fused commit-and-prepare (REPAIR_IN_PROGRESS / REFRESH_COMMITTED): stash the fused commit and deliver the terminal when the following prepare resolves - CONFIRMED_BY_PREPARE on a promise-quorum, SUPERSEDED if pre-empted, UNCONFIRMED otherwise (including an awaitUntil throw). This is the path that previously delivered no post-commit callback at all. - v1 doPaxos / beginAndRepairPaxos: APPLIED on commit ack, UNCONFIRMED on commitPaxos timeout before rethrow. - Background PaxosRepair: APPLIED on success (retries on failure, as before). Documented residual gaps (no terminal, dispatch-only): v1 fire-and-forget REFRESH sendCommit and any consistencyForCommit == ANY commit. (Rebase of commit 08c7ea0)
### What is the issue 4.1.135.Final is affected by several CVEs, including [CVE-2026-56821](https://www.cve.org/CVERecord?id=CVE-2026-56821) and [CVE-2026-56822](https://www.cve.org/CVERecord?id=CVE-2026-56822). ### What does this PR fix and why was it fixed Upgrades Netty to 4.1.136.Final. (Rebase of commit 5638606)
…red paged reads (#2540) ### What is the issue Custom QueryHandlers cannot fully implement paging HCD side implementation: riptano/hcd#265 (custom QueryHandler that proxies OpenSearch + lookup in Cassandra and handle paging) ### What does this PR fix and why was it fixed Expose a public entry point that runs a SELECT against a caller-supplied ReadQuery instead of the query derived from the statement's own restrictions, returning a single page. This lets a component that resolves which partitions to read (and in what order) out of band — e.g. a custom QueryHandler backed by an external index that assembles a SinglePartitionReadCommand.Group in the desired order — reuse the whole normal read flow: consistency validation, guardrails, read-threshold tracking, selection/projection, LIMIT/OFFSET, aggregation, dynamic-data masking, the single-shot fast path, read metrics, and page continuation via setHasMorePages. A MultiPartitionPager yields partitions in the Group's command order, so the caller controls result order across page boundaries. The method mirrors the public execute(QueryState, QueryOptions, RequestTime) exactly, substituting the supplied ReadQuery for getQuery(...). Adds unit coverage (order preservation across pages including boundaries that fall inside a partition, single page, LIMIT, OFFSET, PER PARTITION LIMIT, clustering order, aggregation, function projection, empty and null query) and a two-node distributed test that pages an externally-ordered read across nodes at CL ALL with the paging state round-tripped through its wire format. (Rebase of commit 205db5b)
### What is the issue Fixes riptano/cndb#17486 ### What does this PR fix and why was it fixed Summary Adds a GitHub workflow, SonarQube Code Quality Scan, to CC `datastax/cassandra` repository. This provides a supplementary scanning tool in GitHub Actions while the primary scanning (with coverage) will be implemented in the Jenkins pipeline. #### New Ad-Hoc Debugging Tool sonarqube-scan.yaml (manual workflow only): Added a new standalone workflow for ad-hoc Sonar scanning without requiring full test suite execution or coverage generation. Useful for: * Quick quality checks during development * Testing Sonar configuration changes * Debugging Sonar issues ##### Features: * Manual dispatch only (not triggered automatically) * Retry logic for transient failures (503, timeouts, connection errors) * Distinguishes quality gate failures (code issues) vs infrastructure failures * Optional debug mode (off by default for clean logs) ##### Changes 1. GitHub Actions Workflow ( .github/workflows/sonarqube-scan.yaml ) * Manual-only workflow for on-demand code quality scans * Runs SonarQube scanner without test coverage (supplementary to Jenkins) * Trigger: Manual dispatch only ( workflow_dispatch ) * Purpose: Testing/debugging tool for quick scans without full test suite 2. SonarQube Analysis Script ( .github/scripts/run_sonar_analysis.sh ) * Orchestrates the SonarQube scan with intelligent error handling and retry logic * Configures scanner arguments dynamically based on branch/PR context * Handles transient infrastructure failures 3. SonarQube Configuration ( sonar-project.properties ) * Project Key: 544478-247111689 (IBM SonarQube format: orgId-repoId) * Server: https://sonarqube-prod.whitewater.ibm.com (Cloudflare endpoint, no truststore needed) * Dashboard: https://sonarqube-prod.apps.wdc-sonarqube-prod.core.cirrus.ibm.com ##### Known Limitations * No code coverage: GitHub Actions workflow doesn't run tests, so coverage metrics are 0% * Primary scanning should be in Jenkins: Where tests and coverage are already integrated * This is a supplementary tool for quick scans and debugging This is the link to GItHub Actions for the new SonarQube workflow: https://github.com/datastax/cassandra/actions/workflows/sonarqube-scan.yaml (Rebase of commit 14b1d4c)
…rror into one status child CASSANDRA-21396 added CassandraXMLJUnitResultFormatterTest, which tests that addFailure() + addError() on the same test case produces exactly one <testcase> element with one merged status child. Our rewritten formatter (from STAR-1248) did not carry forward the CASSANDRA-21396 fix, so the test failed. Two fixes applied to CassandraXMLJUnitResultFormatter: 1. addFailure() + addError() on the same test must merge messages rather than overwrite. Introduce mergeOrPutFailure() that appends the additional throwable message to the existing element's ATTR_MESSAGE attribute instead of replacing the whole element in failedTests. 2. maybeAddClassCaseElement() unconditionally appended a class-level <testcase> to the XML output, giving two testcase elements whenever any real test ran. Only append it to rootElement when no individual test elements have been recorded yet (i.e., all activity happened in @BeforeClass/@afterclass). The element is still created and tracked internally for time-accounting.
…eCompatibilityMode.NONE The upstream test parameterizes over skiplist, skiplist_sharded, and trie memtable implementations. The default test config sets storage_compatibility_mode: HCD_1, which causes MemtableParams.mapCC5KeyToCC4ClassName() to throw a ConfigurationException when attempting to persist 'skiplist_sharded' as schema — because that type does not exist in CC4. Fix: add a @BeforeClass/@afterclass pair that temporarily sets storage_compatibility_mode to NONE before CQLTester initialises, then restores it. NONE is correct for a pure CC5 unit test that exercises all available memtable types without CC4 compatibility constraints.
…ty_mode=NONE AutoRepairTablePropertyTest, AutoRepairSchedulerStatsHelper, and AutoRepairFlagToggleTest start in-JVM clusters that exercise the auto_repair schema column and system_auth.cidr_groups table. Both require StorageCompatibilityMode.NONE: - auto_repair column does not exist in CC4 schema; writing it under HCD_1 compatibility mode triggers a ConfigurationException. - system_auth.cidr_groups is CC5-only; the table is absent under CC4 compatibility, causing a KeyspaceNotDefinedException. Fix: set storage_compatibility_mode=NONE in the cluster config for all three test classes before the in-JVM daemon initialises.
…esSystemViewTest
AutoRepairFlagToggleTest:
- testEnablingAutoRepairFlag: remove nodetool drain before shutdown().
drain shuts down MemtablePostFlush; the subsequent shutdown() tries to
flush again causing RejectedExecutionException. The explicit
flush("system_schema") preceding it is sufficient.
CompactionStatsTest:
- Add missing row(String, int, String, long, long, long) overload that
delegates to row(String, int, int, String, long, long, long). The 6-arg
calls hit CQLTester.row(Object...) returning a 6-element array; the SELECT
returns 12 columns, causing 'expected:<6> but was:<12>' from assertRows.
SSTablesSystemViewTest:
- Update expected values after CNDB-10907 (overlap diagnostics) added
overlap-related columns to the sstables virtual table output.
…rade to 1.1.0 cassandra-5.0.9 introduced commit bccb5ad 'Remove path for unused geomet package in cqlsh' which removed 'geomet-' from the third_parties list in bin/cqlsh.py. Apache Cassandra does not use geo types so it was considered unused, but STAR-254 added pylib/cqlshlib/geotypes.py which does 'from geomet import wkt' unconditionally at import time. With 'geomet-' removed from third_parties, lib/geomet-0.1.0.zip is no longer added to sys.path, so every cqlsh invocation crashes with: ModuleNotFoundError: No module named 'geomet' Fix: restore 'geomet-' to third_parties and upgrade the bundled zip from 0.1.0 to 1.1.0 (matching cassandra-driver 3.29.3's geomet>=1.1 requirement) by fetching it from PyPI in build-resolver.xml.
…efore SSTable import truncateBlocking() schedules SSTable file deletion asynchronously on NonPeriodicTasks. loadNewSSTables() was called immediately after, creating a race where the SSTableLister found files from the just-deleted SSTable and then failed to rename them (NoSuchFileException on TOC.txt). Fix: add LifecycleTransaction.waitForDeletions() after truncateBlocking() in loadTestSStables(). ImportTest already uses this pattern for the same reason.
… for INDEX_BUILD_STARTED flush before inserting data The index build triggered by CREATE INDEX flushes the memtable asynchronously via INDEX_BUILD_STARTED. Using execute() to create the index returns immediately without waiting for that flush to complete. If data is inserted before the flush finishes, the flush itself generates a second SSTable; the tests then fail with "expected exactly one SSTable" assertions. Fix: replace execute() with createIndex() (CQLTester helper) in all three tests. createIndex() blocks until the index is queryable (INDEX_BUILD_STARTED flush has completed), ensuring exactly one SSTable when data is subsequently inserted. Affects: RowAwareSkinnyPrimaryKeyMapTest, RowAwareStaticClusteringPrimaryKeyMapTest, RowAwareWidePrimaryKeyMapTest.
… cassandra-5.0.9 The import org.apache.cassandra.schema.Schema was already unused in ImportTest.java in the cassandra-5.0.9 upstream. This is a pre-existing issue in the Apache code brought in by the rebase onto cassandra-5.0.9.
JeremiahDJordan
force-pushed
the
cndb-18860-main-5.0.9-rebase
branch
from
August 19, 2026 17:13
4faa67a to
38b34ce
Compare
…ore SELECT testCreateAndDropIndex() called createIndexAsync() then later queried the table using the index. Our SecondaryIndexManager.checkQueryability() throws IndexBuildInProgressException when the index status is FULL_REBUILD_STARTED, which is stricter than vanilla 5.0.9 and exposed the pre-existing race. Fix: add waitForIndexQueryable() immediately before the assertRows() SELECT, exactly at the point where queryability is first required. All preceding assertInvalidMessage() calls test schema-level duplicate detection and do not go through checkQueryability(), so they do not need the index to be queryable.
|
❌ Build ds-cassandra-pr-gate/PR-2565 rejected by Butler2 regressions found Found 2 new test failures
Found 3 known test failures |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebases
main-5.0onto thecassandra-5.0.9upstream tag.Summary
cassandra-5.0.9agent_log/rebase-changes-report.mdKey API changes in cassandra-5.0.9 that required fixes
Gossiper.instance.getHostId(InetAddressAndPort)(1-arg) removed → replaced withStorageService.instance.getTokenMetadata().getHostId()+ null-guards (4 sites inAutoRepairUtils,AutoRepair)RepairOptionconstructor gained 2 new boolean params (forceRepair,offlineService) → updated all 3 call sites inAutoRepairStateCompactionParams.classFromName()return type changed; error message updated;CompactionMetricsbyte counters changed fromCountertoMeter(.inc()→.mark())ReverseValueIterator5-arg constructor removed; 6-arg form requiredIndexInputReaderconstructors made package-private;FilterIndexInputadded as base class forTrackingIndexInputSystemDistributedKeyspace.TABLE_NAMESmade private →getTableNames()TableOperation.Progress.TOTAL_COMPRESSEDremoved → local constant restored inCompactionStatsErrorMessage.fromException()→fromExceptionNoStreamId()in transport layer