Fix COUNT(DISTINCT ...) collision for multi-column comma values - #3781
Closed
chiliec wants to merge 1 commit into
Closed
Fix COUNT(DISTINCT ...) collision for multi-column comma values#3781chiliec wants to merge 1 commit into
chiliec wants to merge 1 commit into
Conversation
The multi-column path joined each converted value with a comma separator,
so distinct tuples like ('a,','b') and ('a',',b') produced the identical
string 'a,,b,' and hashed to the same bucket. Length-prefix each value so
the encoding is injective.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes a
COUNT(DISTINCT a, b)correctness bug where distinct multi-column tuples are counted as one. Reported downstream at dolthub/dolt#11562.Fix
The multi-column path in
countDistinctBuffer.Updatebuilt its hash key by joining each converted value with a,separator (str += vv + ","). That encoding is not injective:('a,','b')and('a',',b')both produce"a,,b,", so distinct tuples collide into one hash bucket. Length-prefixing each value ("%d:%s") makes the encoding self-delimiting, so different tuples always produce different keys.Tests
Added
TestCountDistinctEvalMultiColumnincount_test.gousing the reporter's exact tuples.Validation
main(returns 1, expected 2), GREEN with the fix.aggregationpackage suite passes.gofmt/goimports -local github.com/dolthub/go-mysql-serverclean.Happy to adjust.