Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Every consumer write currently copies the dispatched entries into a second
ArrayListsolely so the write-completion listener can release them. The original dispatch list is already owned by the send operation and remains valid until completion, so modern clients pay for an unnecessary list object, backing array, and per-entry copy on every outbound batch. Subscription fanout multiplies this work.Modifications
For clients that support batch messages, release non-null entries directly from the original dispatch list after the channel flush completes. Keep the separate release list for legacy clients, where an incompatible batch entry can be rejected and released before other entries are written.
The change preserves protocol output, write order, flush timing, and the existing rule that entries are released only after Netty completes the flush.
Verifying this change
A saturation scenario used 500 producer connections, one topic, twenty Key_Shared subscriptions, and ten consumers per subscription. Each run produced one million messages and verified every device sequence at every subscription.
The changed run was 8.6% above the arithmetic mean of the surrounding controls. All runs reported zero duplicate, out-of-order, or invalid messages.
An allocation profile confirmed that the sender's
ArrayListallocation and its directly attributedObject[]backing-array allocation disappeared. OtherObject[]allocations from Netty leak tracking and queue resizing were unaffected.Validation:
./gradlew :pulsar-broker:compileJava./gradlew quickCheck