Skip to content

[improve][broker] Recycle entry metadata wrappers - #308

Open
lhotari wants to merge 1 commit into
lh-perfopt-repl-snapshot-timestampfrom
lh-perfopt-entry-metadata-recycler
Open

lhotari wants to merge 1 commit into
lh-perfopt-repl-snapshot-timestampfrom
lh-perfopt-entry-metadata-recycler

Conversation

@lhotari

@lhotari lhotari commented Sep 20, 2026

Copy link
Copy Markdown
Owner

Motivation

Key_Shared dispatch creates an EntryAndMetadata wrapper for each entry and subscription. The wrapper keeps the parsed metadata and computed sticky-key hash local to that dispatcher, but it currently becomes garbage after the entry is sent or moved to replay.

In a five-subscription Key_Shared workload, these short-lived wrappers accounted for 486 MiB of sampled broker allocation in the post-warmup measurement window.

Modifications

Recycle each wrapper after its single owned entry release. Clear the wrapped entry, metadata reference, and cached sticky-key hash before returning it to Netty's recycler.

This preserves independent hash state across subscriptions and does not change entry reference counting or dispatch behavior.

Validation

The targeted sampled allocation fell from 486 MiB to 4 MiB, a 99.2% reduction. No replacement recycler-handle allocation was sampled.

A 500-producer workload with five Key_Shared subscriptions and ten consumers per subscription delivered all 30 million fanout messages with zero duplicates, ordering violations, or invalid messages.

The following checks passed:

  • SharedConsumerAssignorTest
  • PersistentStickyKeyDispatcherMultipleConsumersTest
  • PersistentStickyKeyDispatcherMultipleConsumersClassicTest
  • broker Spotless and Checkstyle
  • ./gradlew quickCheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant