Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Managed-ledger read scheduling checks every candidate position against the cursor mark-delete position and individual-deletion ranges. Although the check is only a hint before I/O, it currently acquires the cursor read lock for every position. Grouped acknowledgments hold the same cursor's write lock while applying a list, so active consumers and ACK processing contend on the outer cursor lock.
Holding the read lock across a larger scan is not a good alternative because it delays ACK writers. The scheduling check can instead tolerate an acknowledgment racing with it: the existing post-I/O filter checks the authoritative deletion state before entries are returned. A stale-negative scheduling result therefore causes only an extra read.
Modifications
PositionRangeSetwhen ledger membership changes.The client already groups acknowledgments and the broker applies each received list under one cursor writer critical section. This change lets scheduling readers continue without contending on that outer lock while retaining the bitmap's own short read lock.
Performance
A short JMH screen measured the scheduling lookup as follows:
In an end-to-end scenario with 500 producer clients, one topic, and twenty Key_Shared subscriptions with ten isolated consumers each, two interleaved parent/candidate pairs produced these ingress rates:
Each run delivered all 20 million fanout messages with zero duplicates, ordering violations, or invalid messages.
A separate one-subscription/50-consumer Key_Shared profile pair concentrated scheduling and ACK activity on one cursor. Wall-clock sampling was disabled:
Both profile runs delivered all two million messages with zero duplicates, ordering violations, or invalid messages. An unprofiled pair differed by -2.4%, within the scenario's run variance, so the profile result is supporting evidence rather than a precise standalone throughput claim.
Verifying this change
This change added tests and was verified with:
./gradlew :managed-ledger:test --tests org.apache.bookkeeper.mledger.impl.ManagedCursorTest --tests org.apache.bookkeeper.mledger.impl.PositionRangeSetTest./gradlew quickCheck./gradlew spotlessCheck checkstyleMain checkstyleTestReadSkipPredicateBenchmarkDoes this pull request potentially affect one of the following parts: