fix(sql): add missing DuckDB aggregate names, guard against future drift - #1143
Conversation
…Fixes F-FN-01. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…N-01. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
When I was just working on the function allowlist, That being said, this is probably sufficient here. I mentioned in my original allowlist PR, it might be useful to put function information in json format elsewhere so any package can look at the function introspection. Right now every core and core extension function is enumerated in the go server, split into compute-only and elevated permissions groups. |
Fixes #1144. > **Note:** This fix was created by Claude (an agent team ran a binder-error audit of the SQL layer; the fix went through automated implementation, simplification, and adversarial review passes, plus review notes from @domoritz). It is a draft until @domoritz has reviewed it and marks it ready. ## What changed The literal `s.includes(') over ')` check in `isAggregateExpression` is replaced by a small regex: ```ts // regexp to match window function calls with inline or named definitions const windowRegExp = /\)\s*over(\s*\(|\s+[\w"])/; ``` It tolerates arbitrary whitespace (including none) around `over` and also matches the named-window form (`... over win` / `... over "win"`). Requiring `(` or whitespace-plus-identifier after `over` avoids false positives on `OVERLAPS` / `overlay(...)` — which the old literal check also didn't match. Regression tests in `test/visitors.test.ts`: the three whitespace variants plus a single-space control, each asserting `isAggregateExpression(...) === 0` and binder-validating the generated markQuery-shaped statement via `toBeValidQuery` (no `GROUP BY` emitted), plus a named-window case using a `WINDOW` clause. Out of scope (unchanged behavior): a correctly-classified window expression combined with a real aggregate in the same mark still hits vgplot's `markQuery` limitation tracked in #259. ## Verification - `pnpm --filter @uwdata/mosaic-sql test`: 267/267 pass - `pnpm --filter @uwdata/mosaic-core test`: 63/63 pass (`isAggregateExpression` feeds core preaggregation) - typecheck + lint clean - Revert check: with `src/visit/visitors.ts` reverted, the whitespace-variant tests fail with the binder error above. Heads-up for merging: this PR and #1143 both add tests to `visitors.test.ts` and will conflict textually — whichever lands second should keep this branch's module-level `markStyleQuery` test helper and dedupe. (“F-FN-02” in the commit message is the internal finding ID from the audit.) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
@derekperkins good points. Let's merge this for now and when we have a good inventory of functions, we can refer to that from a test similar to how we use |
# Conflicts: # packages/mosaic/sql/test/visitors.test.ts
Fixes #1142.
What changed
Commit 1 — the fix. Adds the 8 missing names to
aggregateNames(alphabetical placement):arg_max_nulls_last,arg_min_nulls_last,argmax,argmin,count_if,group_concat,listagg,sem. Regression tests intest/visitors.test.ts("Verbatim aggregate detection") build a markQuery-shaped query per name and assert viatoBeValidQuerythat noGROUP BYis emitted over the aggregate alias and the result binds in DuckDB.Commit 2 — the guard. New
test/aggregate-names.test.tsdiffsaggregateNamesin both directions againstduckdb_functions() WHERE function_type = 'aggregate'on the live test-fixture DuckDB, so this drift class (#1051, and now #1142) is caught mechanically whenever the DuckDB dependency is bumped instead of by user bug reports. Two commented exclusion sets carry the intent: window-only functions that DuckDB reports as aggregates (usable only withOVER, whichisAggregateExpressionhandles separately) plus internalsum_no_overflow, and macro-backed names (geomean,geometric_mean,histogram_values,weighted_avg) that still expand to aggregates. Failure messages print the exact ready-to-paste arrays. A minimalqueryFixturehelper was added totest/util/validate.ts.Deliberately not added: the window-only pseudo-aggregates and
sum_no_overflow(see exclusion rationale in the test).Verification
pnpm --filter @uwdata/mosaic-sql test: 274/274 passpnpm --filter @uwdata/mosaic-core test: 63/63 pass (aggregate detection feeds core preaggregation)src/ast/aggregate.tsreverted, all 8 new detection tests fail with the binder error above, and the drift test fails printing exactly the 8 missing names; with a bogus name injected, the drift test fails in the remove direction.(“F-FN-01” in the commit messages is the internal finding ID from the audit.)
🤖 Generated with Claude Code