Skip to content

[improve][test] Exclude IoT warmup from measurements - #298

Open
lhotari wants to merge 2 commits into
lh-perfopt-repl-snapshot-timestampfrom
lh-perfopt-iot-warmup-measurement
Open

lhotari wants to merge 2 commits into
lh-perfopt-repl-snapshot-timestampfrom
lh-perfopt-iot-warmup-measurement

Conversation

@lhotari

@lhotari lhotari commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Motivation

Performance recordings currently include workload startup and the first traffic sent through newly initialized clients, connections, producers, and consumers. Periodic managed-ledger rollover and BookKeeper maintenance can also land inside a short measurement interval. These effects make throughput summaries and JFR comparisons sensitive to work outside the broker messaging path being studied.

Modifications

  • Add mutually exclusive warmupSeconds and warmupMessages settings to the IoT workload.
  • Keep the same clients, connections, producers, consumers, and monotonic device sequences across warmup and measurement.
  • Include warmup messages in end-to-end delivery, duplicate, and ordering validation while excluding their completions from measured producer throughput.
  • Record measurementStartEpochMs and measurementEndEpochMs in producer-summary.json so the corresponding interval can be selected from a JFR recording.
  • Configure a time-based warmup for the representative rate-limited scenario and a message-count warmup for the unrestricted scenario.
  • Keep topic and cursor managed-ledger rollover outside ordinary benchmark windows by raising their entry, size, and time limits.
  • Delay BookKeeper ledger garbage collection, disable entry-log compaction, and raise the journal size limit for the ephemeral benchmark cluster. Normal entry-log flushing and disk-space checks remain enabled.

The storage values are benchmark controls rather than production sizing recommendations. Scenarios intended to measure rollover, recovery, deletion, compaction, or long-running storage behavior should override them.

Validation

  • ./gradlew :tests:performance:tools:checkstyleTest :tests:performance:tools:spotlessCheck :tests:performance:tools:test --tests '*IotScenarioTest' quickCheck --no-daemon
  • ./gradlew :tests:performance:tools:installDist :tests:performance:launcher:compileJava --no-daemon
  • ./gradlew quickCheck --no-daemon
  • Container smoke scenario using the storage controls: 100 warmup messages followed by 200 measured messages at 100 msg/s. All 300 messages were received with zero duplicates, ordering violations, or invalid payloads. The resolved configuration and service startup logs confirmed that the broker and BookKeeper settings were applied.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant