[processor/delta_to_cumulative] Fix exponential histogram bucket limit rounding - #50923
Open
SerhiiGoTolstoy wants to merge 2 commits into
Open
Conversation
Contributor
|
Welcome, contributor! Thank you for your contribution to opentelemetry-collector-contrib. Important reminders:
|
Author
|
I will be transparent, i dont know GO and just discovered this repo today to implement OTLP collector for my aws Lambda env. The issue was caught in production under real load of billion of requests. Proper unit test were added to prove the issue. |
Member
|
Please check the Authorship checkbox. |
Member
|
/workflow-approve |
Round the lower bucket index down and the exclusive upper index up so Limit matches Collapse. Add regression coverage for positive and negative offsets, ranges crossing zero, and repeated downscaling. Assisted-by: Codex (GPT-6)
Assisted-by: Codex (GPT-6)
SerhiiGoTolstoy
force-pushed
the
codex/fix-exponential-histogram-bucket-limit
branch
from
September 11, 2026 20:15
0dff1df to
7fa206f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Delta-to-cumulative aggregation can produce exponential histograms with 161 or 162 buckets despite its 160-bucket limit.
Limit()truncates both bucket-range endpoints toward zero, so its predicted size can be smaller than the result ofCollapse()andMerge().For example, the range
[1, 321)contains 320 buckets. The current calculation predicts 160 buckets after one downscale step, but the actual result spans indices 0 through 160 and needs 161 buckets. Ranges crossing index zero can exceed the limit by two buckets.This change rounds the lower index down and the exclusive upper index up, including negative indices. It makes scale selection match the existing bucket-merging algorithm. Observation counts are preserved, and the tests check that ranges which fit do not receive unnecessary downscaling.
I encountered this while using the Collector for AWS Lambda in a production workload handling billions of requests, where oversized histograms were rejected by AWS. The regression tests use small synthetic inputs to isolate the defect; they do not replay the rejected production payloads. The faulty calculation was present in v0.151.0 and is still present on the main branch used for this PR.
Link to tracking issue
No separate issue. This PR includes regression tests that reproduce the defect.
Testing
Eight regression cases exercise
Limit(),Downscale(), andMerge()together. They cover positive and negative offsets, a range crossing zero, repeated downscaling, and ranges that fit the limit. They also check that both observations remain in the merged histogram.make testinprocessor/deltatocumulativeprocessor: all 115 component tests pass with the race detector enabled.make lintin the same directory: passes.Documentation
Added a changelog entry and a code comment explaining the rounding rules. No configuration or public API changes.
Authorship
I do not know Go. I used Codex to investigate the issue and write the implementation, regression tests.