Skip to content

fix: use SUM0 in AggregateCaseToFilter D1 rewrite to avoid NULL on empty windows - #20093

Open
zhang-arvin wants to merge 1 commit into
apache:masterfrom
zhang-arvin:fix/issue-18058-empty-window-null
Open

fix: use SUM0 in AggregateCaseToFilter D1 rewrite to avoid NULL on empty windows#20093
zhang-arvin wants to merge 1 commit into
apache:masterfrom
zhang-arvin:fix/issue-18058-empty-window-null

Conversation

@zhang-arvin

Copy link
Copy Markdown

Description

Fixes #18058 and #17768.

The DruidAggregateCaseToFilterRule D1 rewrite converts SUM(CASE WHEN COND THEN COL1 ELSE 0 END) to SUM(COL1) FILTER(WHERE COND). However, SUM returns NULL when the filter never matches, while the original CASE expression would return 0. This causes incorrect NULL results for queries with aggregations on empty windows.

Root Cause

The D1 rewrite in DruidAggregateCaseToFilterRule.transform() uses call.getAggregation() (which is SUM) when creating the filtered aggregate call. SUM returns NULL for empty groups, but the original SUM(CASE WHEN ... ELSE 0 END) returns 0 when no rows match the condition.

Fix

Changed the D1 rewrite to use SUM0 (via SqlStdOperatorTable.SUM0) instead of SUM. SUM0 returns 0 for empty/null input, which matches the expected behavior of the original CASE expression.

Changes

  • DruidAggregateCaseToFilterRule.java: Changed the D1 case in transform() to use SUM0 instead of SUM for the filtered aggregate call, with non-nullable result type.
  • filtered_sum.iq: Updated expected test results to reflect the corrected behavior (0 instead of NULL for empty window and no-match cases).

Behavior Changes

Scenario Before After
Empty input (no rows) NULL 0
Rows exist, none match filter NULL 0
Rows exist, some match, non-null N N (unchanged)
Rows exist, all null values NULL 0

Key Features

  • Aggregations on empty windows now return 0 instead of NULL
  • Consistent with the documented behavior table in the class javadoc
  • Backward compatible for most use cases

Testing

Updated the filtered_sum.iq quidem test to verify the corrected behavior for all four scenarios (empty input, no match, some match, all null values).

…pty windows

The D1 rewrite in DruidAggregateCaseToFilterRule converts
SUM(CASE WHEN COND THEN COL1 ELSE 0 END) to
SUM(COL1) FILTER(WHERE COND), but SUM returns NULL when the
filter never matches instead of the expected 0.

This fix changes the D1 rewrite to use SUM0 instead of SUM,
which returns 0 for empty/null input, matching the original
CASE expression behavior.

Fixes apache#18058, apache#17768

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity Findings
P0 0
P1 1
P2 1
P3 0
Total 2

Reviewed 2 of 2 changed files.


This is an automated review by Codex GPT-5.6-Luna(max)

RelDataType newType = typeFactory.createTypeWithNullability(call.getType(), false);
return AggregateCall.create(
call.getAggregation(),
SqlStdOperatorTable.SUM0,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] SUM0 still executes as nullable SUM

SumZeroSqlAggregator inherits SumSqlAggregator's nullable native Long/Float/DoubleSumAggregatorFactory. With no matching filter rows, the nullable wrapper returns NULL, so native Druid queries still produce NULL for the empty/no-match cases changed to 0 in filtered_sum.iq.

RelDataType newType = typeFactory.createTypeWithNullability(call.getType(), false);
return AggregateCall.create(
call.getAggregation(),
SqlStdOperatorTable.SUM0,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Rewrite changes all-null semantics

When every row matches the condition and the value is NULL, SUM(CASE WHEN condition THEN value ELSE 0 END) returns NULL, while SUM0(value) FILTER returns 0. The rewrite applies this universally; the rewrite-disabled test still documents NULL for the 7=7,null case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Druid v33 gives null value for each column in aggregation that does not have a value in the query time range

2 participants