Skip to content

docs: show the multi-fact AOV measure owned by a cube, not only by a view - #11659

Merged
paveltiunov merged 6 commits into
masterfrom
claude/multi-fact-derived-metric-test-j4uypn
Aug 27, 2026
Merged

docs: show the multi-fact AOV measure owned by a cube, not only by a view#11659
paveltiunov merged 6 commits into
masterfrom
claude/multi-fact-derived-metric-test-j4uypn

Conversation

@paveltiunov

@paveltiunov paveltiunov commented Aug 26, 2026

Copy link
Copy Markdown
Member

Check List

  • Tests have been run in packages where changes have been made if available
  • Linter has been run for changed code
  • Tests for the changes have been added if not covered yet
  • Docs have been added / updated if required

Description of Changes Made

Follow-up to #11656. That PR showed the multi-fact AOV ratio only as a measure of the view, on the premise that a metric spanning two facts has nowhere else to live. That premise is wrong, and this corrects it.

A cube measure may reference another cube's measure. Doing so makes it derived rather than owned by its cube — the same property a view measure has — so AOV can sit on either fact cube:

- name: aov_basket
  type: number
  multi_stage: true
  sql: "{item_location_sales.sales_amount} / NULLIF({CUBE.transactions_without_returns}, 0)"

Both placements plan identically: the same per-fact subqueries, stitched the same way, divided in the same final stage. multi_stage is required either way, and without it both fail with the same Can't find join path. What differs is reuse and coupling — a cube-owned metric is defined once for every view that includes it, at the cost of one cube naming another.

Corrections

Three statements from #11656 overstated the case and are fixed here:

  • the recipe's "Neither cube can define AOV — neither can reference the other's measures"
  • views.mdx: "there is no single cube it could belong to"
  • view.mdx: "which therefore has no single cube to live in"

Docs

  • recipes/data-modeling/average-order-value.mdx — new "Where to put the measure" section with the cube-owned form and a table comparing the two placements on reuse, coupling and query path, plus guidance on picking one.
  • docs/data-modeling/views.mdx, reference/data-modeling/view.mdx — claims corrected, both pointing at the recipe's comparison.

Unit testmulti-fact-derived-measure-in-view.test.ts, now 19 tests

The same ratio added to sales_line_item, alongside the view-owned one so a single model covers both placements. New block pins that the two plan identically (normalising only the emitted column alias), that the cube-owned measure is reachable through a view, that it reaches the other fact even when the query names only its own cube, that it works on the shared date spine, and that multi_stage is required exactly as on the view.

A second view, line_item_analysis, carries only sales_line_item.aov_basket and a shared dimension — no measure of item_location_sales anywhere in it — pinning the recipe's claim that a view can expose AOV without exposing its numerator.

Every YAML sample added here was extracted from the page, compiled, and its SQL checked against the native planner — including grafting the documented aov_basket block verbatim onto the recipe's own step-1 model.

Multi-fact derived measure defined on a view      10 passed
Multi-fact derived measure defined on a cube       7 passed
Derived view measure over a fanned-out join        2 passed

Tests: 19 passed, 19 total

End-to-end testsmoke-multi-fact.test.ts (6 tests), DuckDB birdbox

The unit tests pin the generated SQL; this one pins the numbers that SQL produces, against a running server and a real database. Fixture (birdbox-fixtures/multi-fact/schema/) is inline SELECTs — no tables to load — and the data is chosen so each way of getting it wrong lands on a distinct value:

sales transactions AOV
West 100 2 (T100 spans three lines) 50
East 60 1 (T201 EXCHANGE, T202 ONLINE) 60

Counting lines gives West 25; letting the join multiply the sum gives 200; dropping either of the cube's two measure filters gives East 30. So a failure says which invariant broke rather than just that a number moved.

Runs under the existing DuckDB group in smoke.sh (one extra birdbox start, no extra container). The planner is pinned on in the birdbox env, so it covers both legs of the use_tesseract_sql_planner matrix rather than skipping one.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RLzMjTkAfG9tjDmjVR3EL2


Generated by Claude Code

A metric spanning two facts does not need a view to live in. A cube measure
that references another cube's measure is derived rather than owned by its
cube - the same property a view measure has - so AOV can sit on either fact
cube, and every view including it gets it.

The recipe said "neither cube can define AOV - neither can reference the
other's measures", which is wrong; the views and view-reference pages said
such a metric has no single cube to live in, which overstates it the same
way. All three corrected, and the recipe gains a section comparing the two
placements on reuse, coupling and query path.

Test: the same ratio on `sales_line_item`, pinning that both placements plan
identically (same legs, same stitch, same final division), that it is
reachable through a view and reaches the other fact even when the query names
only its own cube, and that `multi_stage` is required exactly as on the view.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLzMjTkAfG9tjDmjVR3EL2
The schema-compiler tests pin the SQL shape; nothing ran the query. This adds
a birdbox smoke test that asks a real Cube instance for AOV over DuckDB and
checks the number, for both placements - the ratio owned by the view and the
same ratio owned by the line-item cube, the latter both directly and through
a view that includes it.

The fixture is built so each way of getting it wrong lands somewhere
different, and the assertion says which invariant broke:

  West  sales 100, transactions 2 (T100 spans three lines) -> 50
  East  sales  60, transactions 1 (T201 is an EXCHANGE)    -> 60

  counting lines instead of transactions -> West 100/4 = 25
  letting the join multiply the sum      -> West 400/2 = 200
  dropping the cube's measure filters    -> East  60/2 = 30

Runs in the existing integration-smoke job via smoke.sh. Multi-fact is
Tesseract-only, so the suite skips when CUBEJS_TESSERACT_SQL_PLANNER is
explicitly 'false' - not merely unset, since Tesseract is the default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLzMjTkAfG9tjDmjVR3EL2
@paveltiunov
paveltiunov requested a review from a team as a code owner August 27, 2026 00:10
@claude

claude Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Claude finished @paveltiunov's task in 3m 40s —— View job


Review: docs correction + multi-fact AOV coverage — no blockers; 1 medium (untested docs claim), 5 low/nits, posted inline

Full review
  • Read the diff against origin/master (11 files: 3 docs, 1 schema-compiler test, 4 birdbox fixtures, 1 smoke test, package.json, smoke.sh)
  • Verified docs cross-links and anchors resolve
  • Checked birdbox wiring (schemaDir, REQUIRED_ENV_VARS, prepareTestData target folder, DuckDB precedent)
  • Checked the Tesseract skip guard against CI matrix values and env.ts
  • Cross-read the schema-compiler test against the documented claims
  • Posted findings inline

What's good

The correction itself is right and well-argued: a cube measure referencing another cube's measure is derived in the same sense a view measure is, so the earlier "neither cube can define AOV" was too strong. The three overstatements are fixed consistently across recipe, guide and reference, and the new comparison table is the useful part — reuse vs coupling vs query path is the actual decision, not a capability limit.

The fixture data is the standout. Each way of getting AOV wrong lands on a distinct number (25 for counting lines, 200 for a fan-out multiply, 30 for dropped filters), so a smoke failure names the broken invariant instead of just "expected 50". Worth keeping that property in mind for future edits to the fixture.

Wiring checks out: schemaDir: 'multi-fact/schema' matches the DuckDB smoke precedent, REQUIRED_ENV_VARS.duckdb is empty, and getTargetFolder('duckdb') writes driver-test-data into postgresql/schema, so the new fixture directory stays self-contained. The Tesseract-off leg of the matrix skips at describe level, so it costs no birdbox startup there.

Findings

Where What
Medium average-order-value.mdx:404 "a view can expose AOV without exposing sales_amount" is the one new claim no test pins — both models include the other fact's measure in every view
Low SalesLineItem.js:45 the fulfillment_channel_group filter is inert (all rows IN_STORE), so only transaction_type is load-bearing despite the header comment
Low smoke-multi-fact.test.ts:126 unguarded rawData()[0] turns an empty result into a TypeError instead of a readable assertion failure
Nit smoke-multi-fact.test.ts:33 skip guard is === 'false' while the server parses the flag with asBool(); correct for the current CI matrix
Nit RetailAnalysis.js:6 mixes join_path with camelCase props; sibling fixtures use joinPath
Nit smoke.sh:23 DuckDB-backed suite placed outside the DuckDB group; shares the step's 30-minute budget

One more note on the schema-compiler side: plans the same way whichever fact owns it compares full SQL strings after a three-substitution normalize(). That's the strongest possible form of the "both placements plan identically" claim, which is good, but it will also break on any unrelated alias-shape change in the planner. Fine as written — just expect it to be the first test to fail when Tesseract changes aliasing.

Not verified

I could not execute either suite — this checkout has no node_modules and the native planner isn't built, so a full yarn install + native build was out of scope for the review. The findings above are from reading the models and the planner wiring, not from a run. The medium finding in particular is a claim I could not test either way.

· branch `claude/multi-fact-derived-metric-test-j4uypn`

Comment thread docs-mintlify/recipes/data-modeling/average-order-value.mdx
Comment thread packages/cubejs-testing/test/smoke-multi-fact.test.ts Outdated
Comment thread packages/cubejs-testing/test/smoke-multi-fact.test.ts Outdated
Comment thread packages/cubejs-testing/birdbox-fixtures/multi-fact/schema/RetailAnalysis.js Outdated
Comment thread .github/actions/smoke.sh Outdated
Two review findings.

The recipe says a cube-owned measure reaches the other fact whether or not
the view naming it includes that fact, and nothing pinned it - every view in
both models exposed sales_amount alongside the ratio. Adds `line_item_analysis`,
a view carrying only the cube-owned ratio and a shared dimension, and a test
that the ratio still divides the two aggregates through it.

In the birdbox fixture every row was IN_STORE, so the channel filter was
inert: dropping it changed no asserted number, while the header comment
claimed both filters were load-bearing. Adds an ONLINE transaction to East,
which leaves every expected value where it was and makes each filter move
East's denominator on its own - checked by removing each in turn against
DuckDB: 60 as written, 30 with either one gone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLzMjTkAfG9tjDmjVR3EL2
@claude

claude Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Claude finished @paveltiunov's task in 1m 6s —— View job


Re-review of 77cc60b — both blocking-ish findings addressed; 4 prior nits still open, nothing new

Full review
  • Fetched and diffed the new commit against the previously reviewed tree
  • Re-checked the two findings it claims to fix
  • Re-walked the fixture arithmetic with the new row
  • Re-checked the four remaining threads against the current file contents
  • Resolved my own threads that no longer apply

What the new commit does

The untested docs claim is now pinned. line_item_analysis includes only sales_line_item.aov_basket plus locations.region — no measure of item_location_sales anywhere in the view — and the new test asserts both SALES_AMOUNT_AGGREGATE and RATIO_OVER_AGGREGATES on it (multi-fact-derived-measure-in-view.test.ts:399). That is exactly the sentence at average-order-value.mdx:404 ("a view can expose AOV without exposing sales_amount"), and it's the right shape of assertion — reaching the other fact is proved by the numerator subquery appearing in SQL the view never named. Thread resolved.

Both measure filters now bite. Adding select 7 … 202 … 'SALE', 'ONLINE' gives East one row excluded by each predicate separately. Checked the arithmetic against every assertion in the smoke suite:

with both filters drop transaction_type drop channel
East denominator T200 → 1 T200, T201 → 2 T200, T202 → 2
East AOV 60 30 30

West is untouched (2 → 50), ungrouped total is still 3 transactions over 160 dollars, and salesAmount comes from the other cube so East stays 60. Every expected value in the suite is unchanged, which is what makes this a clean strengthening rather than a re-baseline. The updated header comment now names the two drops separately instead of lumping them as "the cube's measure filters" — matching what the data actually distinguishes. Thread resolved.

One note: only the schema-compiler model gained the fact-free view; the birdbox RetailAnalysis still includes salesAmount alongside aovBasketFromCube. That's fine — the planner-level assertion is where the claim lives, and an end-to-end duplicate would cost a query for no extra invariant.

Still open (re-affirmed, no new comments posted)

All four were re-checked against the current file contents and still apply verbatim; the unresolved threads carry the detail.

Where What
Low smoke-multi-fact.test.ts:127 unguarded rawData()[0] turns an empty result into a TypeError
Nit smoke-multi-fact.test.ts:34 skip guard === 'false' vs the server's asBool()
Nit RetailAnalysis.js:6 join_path mixed with camelCase props
Nit smoke.sh:23 DuckDB-backed suite placed outside the DuckDB group

Nothing new surfaced in this delta. No blockers.

Not verified

Same limitation as the previous pass: no node_modules in this checkout and the native planner isn't built, so neither suite was executed. The fixture arithmetic above is worked out by hand from the SQL and the measure filters, not from a run — though the commit message reports each filter was removed in turn against DuckDB and produced 30 both ways, which matches.

· branch `claude/multi-fact-derived-metric-test-j4uypn`

…nits

Four more review findings.

The skip guard compared the raw env string to 'false', while the server parses
it with asBool, which also treats 0/no/False as off. Read it through
getEnv('nativeSqlPlanner') instead, so the skip and the birdbox instance can
never disagree. Checked all three: unset runs, 'false' skips, '0' now skips
too - it did not before.

The ungrouped assertion indexed [0] straight off the result, so an empty
result - a plausible way for multi-fact stitching to fail - would have thrown
a TypeError instead of reporting the number. Asserts the row count first.

The view fixture mixed snake_case join_path into a camelCase file; the
sibling JS fixtures use joinPath. Re-checked the model against DuckDB after
the rename, values unchanged.

Moved smoke:multi-fact inside the DuckDB group in smoke.sh - it is
DuckDB-backed, and the whole script shares one 30 minute budget.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLzMjTkAfG9tjDmjVR3EL2
@claude

claude Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Claude finished @paveltiunov's task in 2m 2s —— View job


Re-review of the two follow-up commits — all 6 prior findings addressed; 1 optional suggestion posted inline

Full review
  • Read the incremental diff (77cc60b, 560c90c) against the previously reviewed state
  • Checked each of the 6 prior findings against the new code
  • Verified the new fixture row and the new line_item_analysis view arithmetic/wiring
  • Verified docs anchors (#where-to-put-the-measure, #keep-shared-logic-in-cubes) still resolve
  • Resolved my 4 stale threads; posted 1 new optional suggestion
  • Posted this summary

Prior findings — status

Finding Status
Medium — average-order-value.mdx:404 "a view can expose AOV without exposing sales_amount" untested Fixed. line_item_analysis view added (only sales_line_item.aov_basket + locations.region), with a test asserting both SALES_AMOUNT_AGGREGATE and RATIO_OVER_AGGREGATES. That's exactly the claim, pinned.
Low — inert fulfillment_channel_group filter in the birdbox fixture Fixed. Row 7 (T202, SALE/ONLINE) added to East. I checked the arithmetic: T201 is excluded only by the type predicate, T202 only by the channel predicate, so East's denominator stays 1 as written and moves to 2 (→ 30) if either filter is dropped. Every existing expectation is unchanged. The header comment now says this accurately.
Low — unguarded rawData()[0] Fixed. expect(rows).toHaveLength(1) precedes the index.
Nit — === 'false' vs asBool() Fixed. Now getEnv('nativeSqlPlanner'); @cubejs-backend/shared is already a dependency of cubejs-testing.
Nit — join_path vs camelCase in RetailAnalysis.js Fixed.
Nit — smoke suite placed outside the DuckDB group Fixed. Moved inside ::group::DuckDB with a comment explaining why.

New comment

One optional suggestion on smoke-multi-fact.test.ts (birdbox env): the skip is now correct — birdbox spreads process.env into the server env in LOCAL/CLI mode (src/birdbox.ts:562), so on the use_tesseract_sql_planner: false leg the instance really does run the legacy planner. But it also means the suite runs on only half the matrix. smoke-shared-calc-group.test.ts:40 pins CUBEJS_TESSERACT_SQL_PLANNER: 'true' in the birdbox env instead, which would give coverage on both legs and remove the skip branch. Not a defect either way.

One stale comment

The module header of multi-fact-derived-measure-in-view.test.ts (lines 17–18) still reads "The ratio of the two is authored as a measure of the view rather than per consumer" — accurate when the file only covered the view placement, slightly behind now that the model carries both. A clause is enough; not worth a commit on its own.

Still not verified

I could not execute either suite in this checkout (no node_modules, native planner not built). The verification above is by reading — including the filter arithmetic on the new fixture row, which I traced by hand rather than by running DuckDB. The commit message states both filters were checked by removing each in turn against DuckDB, and the data supports that.

· branch `claude/multi-fact-derived-metric-test-j4uypn`

Comment thread packages/cubejs-testing/test/smoke-multi-fact.test.ts
The suite was gated on the ambient CUBEJS_TESSERACT_SQL_PLANNER, which is
correct in the sense that birdbox spreads process.env into the server env,
so on the use_tesseract_sql_planner: false leg the server really did run
the legacy planner. But nothing this suite asserts is matrix-dependent, so
that gate just cost the coverage on half the matrix.

Pin the planner on in the birdbox env instead - options.env is spread last
in birdbox.ts, so it overrides the leg's setting - and drop the getEnv /
describe.skip branch, matching smoke-shared-calc-group.test.ts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLzMjTkAfG9tjDmjVR3EL2
@claude

claude Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Claude finished @paveltiunov's task in 1m 25s —— View job


Re-review of 29bcb70 — the last open suggestion is applied correctly; no findings, all 7 prior threads resolved

Full review
  • Listed existing review threads — all 7 already resolved, none to re-affirm or clean up
  • Read the incremental diff (560c90c..29bcb70, one file, +7/−9)
  • Verified the Tesseract pin actually reaches the server in every birdbox mode
  • Checked REQUIRED_ENV_VARS can't clobber it, and that the DuckDB precedent is unchanged
  • Spot-checked that the schema-compiler suite is planner-independent
  • Posted summary

What the commit does

Takes the option from my last comment: describe.skip guard and the getEnv import are gone, and CUBEJS_TESSERACT_SQL_PLANNER: 'true' is pinned in the birdbox env instead, so the suite runs on both legs of the matrix rather than half.

Verified the pin wins in every path the suite could take:

Mode Env assembly Pin wins?
CLI / LOCAL {...process.env, ...defaults, ...options.env} (src/birdbox.ts:561-582) yes — options.env is last
DOCKER dc.withEnvironment(options.env) (src/birdbox.ts:328) yes — process.env never spread
REQUIRED_ENV_VARS copy-back loop (birdbox.ts:735) would overwrite listed keys from process.env no risk — REQUIRED_ENV_VARS.duckdb is [] (src/REQUIRED_ENV_VARS.ts:60)

Placement matters and is right: CUBEJS_TESSERACT_SQL_PLANNER sits after ...DEFAULT_CONFIG, and DEFAULT_CONFIG doesn't carry that key anyway (test/smoke-tests.ts:6-14), so there's no ordering trap either way. The header comment's parenthetical ("birdbox spreads process.env first, so the pin wins over whatever the CI leg exports") describes the CLI/LOCAL path accurately, which is the one the smoke script actually uses.

The trade this makes is worth naming: the legacy leg now runs the suite under Tesseract too, so it's duplicate coverage rather than legacy coverage. That's correct here — the legacy planner cannot build a single join tree over two unrelated facts, so there is no legacy behaviour to assert, and the comment says exactly that instead of leaving it implicit.

smoke.sh placement is also settled — the suite now sits inside the DuckDB group with a line explaining why. Startup cost is the same shape as smoke-duckdb.test.ts, which takes the identical getBirdbox('duckdb', …) path (Postgres container spun up and unused in both).

No new findings. Nothing in this delta touches the docs or the fixture arithmetic verified in the previous pass.

One stale comment, unchanged from last pass

multi-fact-derived-measure-in-view.test.ts:17-18 still reads "The ratio of the two is authored as a measure of the view rather than per consumer" — accurate when the file only covered the view placement, slightly behind now that the model carries both. Purely a comment; not worth a commit on its own, but worth folding in if you touch the file again.

Not verified

Same limitation as both prior passes: no node_modules in this checkout and the native planner isn't built, so neither suite was executed. The env-precedence conclusions above come from reading src/birdbox.ts, not from a run — CI on this push is the actual check that the suite now passes on the use_tesseract_sql_planner: false leg.

· branch `claude/multi-fact-derived-metric-test-j4uypn`

The header described the file from when it only covered the view-owned
placement. The model carries both placements now, so say so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RLzMjTkAfG9tjDmjVR3EL2
@claude

claude Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Claude finished @paveltiunov's task in 36s —— View job


Re-review of a7f1969 — comment-only delta, accurate; no findings, no open threads

Full review
  • Read the incremental diff (29bcb70..a7f1969) — one file, +6/−5, comments only
  • Verified the new header wording against what the model actually contains
  • Listed review threads — all 7 already resolved, nothing to re-affirm or clean up

What the commit does

Folds in the stale-comment note from my last pass. The module header on multi-fact-derived-measure-in-view.test.ts:17-22 no longer says the ratio is "authored as a measure of the view"; it now says it's authored once "in both of the places it can live: on the view, and on the line-item cube itself (from where a view can re-expose it)."

Checked that against the model rather than taking it on the wording:

Claim in the header Where it's actually true
on the view retail_analysis.aov_basket, multi_stage: true (line 171)
on the line-item cube itself sales_line_item.aov_basket, multi_stage: true (line 108)
from where a view can re-expose it retail_analysis aliases it as aov_basket_from_cube (line 153), and line_item_analysis includes it bare (line 187)

The "re-expose" clause is doing real work — both re-exposure shapes exist in the model, and the aliased one is why the normalize() in the plans-identically test has to collapse retail_analysis__aov_basket and sales_line_item__aov_basket. The rest of the paragraph (the multi-fact rationale, the shared-cube join structure) is unchanged and still correct.

Nothing else in the file changed — no test bodies, no model YAML, so nothing verified in the earlier passes is disturbed.

Threads

All 7 of my prior threads are resolved; three of them (smoke-multi-fact.test.ts ×2, RetailAnalysis.js, smoke.sh) are also marked outdated by the rewrites. Nothing left open from my side, and no human threads to leave alone.

Not verified

Same as all three prior passes: no node_modules in this checkout and the native planner isn't built, so the suites weren't executed. Immaterial for this delta — it changes only comment text, which cannot affect a run.

· branch `claude/multi-fact-derived-metric-test-j4uypn`

Copy link
Copy Markdown
Member Author

CI note — Upload merged coverage to Codecov failed, on a superseded commit, from GitHub artifact storage. Not this PR's, and not gating.

The job ends with:

##[error]Unable to download artifact(s): Unable to download and extract artifact:
Artifact download failed after 5 retries.

Two independent reasons this isn't the PR's:

  • Wrong commit. It ran on 77cc60b, which git merge-base --is-ancestor confirms is an ancestor of the current head a7f1969 — two commits behind. The coverage job hasn't been reached yet on the current head; it runs after the driver matrix, which is still in flight.
  • Wrong layer. It downloads ~20 coverage artifacts from Azure blob storage and merges them; it doesn't run any test or compile anything. It got 19 successful downloads with valid SHA256 digests and then exhausted 5 retries on one. Nothing in this diff (docs, one schema-compiler spec, birdbox fixtures, one smoke test, smoke.sh) affects artifact retrieval.

This is the third artifact-download failure of this exact shape across this PR's runs, all transient and all on runs that were superseded by a newer push.

No fix to port — there's no code change that makes GitHub's artifact storage retry successfully. I'm deliberately not re-running this job: it's pinned to an outdated commit, so a green result there would prove nothing about a7f1969. The equivalent job on the current head is the real check, and I'm watching for it.


Generated by Claude Code

@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.15%. Comparing base (2aba3ac) to head (a7f1969).
⚠️ Report is 6 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (2aba3ac) and HEAD (a7f1969). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (2aba3ac) HEAD (a7f1969)
cubesql 1 0
Additional details and impacted files
@@             Coverage Diff             @@
##           master   #11659       +/-   ##
===========================================
- Coverage   80.10%   58.15%   -21.96%     
===========================================
  Files         490      237      -253     
  Lines      103670    18851    -84819     
  Branches     3670     3825      +155     
===========================================
- Hits        83048    10962    -72086     
+ Misses      20070     7335    -12735     
- Partials      552      554        +2     
Flag Coverage Δ
cube-backend 58.15% <ø> (-1.25%) ⬇️
cubesql ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Member Author

CI note — tests (bigquery-export-bucket-gcs, true) failed on a superseded commit, from shared-dataset contention. Partly caused by my own push cadence.

Three tests failed in this job, all the same way:

Cube pre-aggregations build failed: failure: Not found: Table
  cube-open-source:dev_pre_aggregations.ec__t_a_external20200701_tnm3rwmb_lztfixof_1l8v20o
Error: Not found: Table
  cube-open-source:dev_pre_aggregations.ec__t_a_external20200701_tnm3rwmb_lztfixof_1l8v222
Error: Not found: Table
  cube-open-source:dev_pre_aggregations.big_e_commerce__multi_time_dim_for_count_external20200601_03hsvdw4_5naovjwk_1l8v22j

Not this PR's, for two reasons — and I'll name the part that is mine:

  • Wrong commit. It ran on 29bcb70, two commits behind the current head a7f1969.
  • Wrong layer. git diff --name-only origin/master...HEAD matches nothing against bigquery|pre.?agg|driver. The diff is docs, one schema-compiler spec, four birdbox fixtures, one smoke test, and smoke.sh. None of it can affect whether a BigQuery pre-aggregation table exists.

The mechanism, and my part in it. dev_pre_aggregations is a shared BigQuery dataset, so concurrent runs build and drop tables in the same namespace. I pushed a7f1969 at 00:25:08, ~2 minutes after 29bcb70 at 00:23:03, while the older run's driver matrix was still going. Its BigQuery leg then overlapped the new run's from 00:35 to 00:45 and lost tables underneath itself. The two distinct suffixes on the same logical table (…1l8v20o and …1l8v222) are the fingerprint of two runs building it concurrently. That second push was a comment-only fix that could have waited — batching it would have avoided this.

No fix to port: nothing in this repo makes a shared dataset safe against overlapping runs, and the real remedy (per-run dataset isolation) is well outside this PR.

I'm not re-running the failed job — it's pinned to an outdated commit, so green there would say nothing about a7f1969. The same leg on the current head is running now with no competing run, which is the real check; I'll report its result and won't leave this PR until it and the smoke group are settled.


Generated by Claude Code

Copy link
Copy Markdown
Member Author

Correction + update: tests (bigquery-export-bucket-gcs, true) also failed on the current head a7f1969.

In my previous comment I said the current head's BigQuery leg would run "with no competing run". That was wrong. Run 33026661212 (for 29bcb70) was still executing until ~00:46, and this leg ran 00:35:09 → 00:48:37 — so the two overlapped for most of it, exactly like the pair I was describing. The prediction was unfounded and the leg went red.

What the new failure shows — same three tests, same error class, different tables:

Run Missing tables
29bcb70 ec__t_a_external20200701…, …external20200701…, big_e_commerce__multi_time_dim_for_count_external20200601…
a7f1969 ec__t_a_external20200901…, big_e_commerce__count_by_product_external20200801…, big_e_commerce__multi_time_dim_for_count_external20200501…

Different partitions and different hash suffixes each time. A defect in the diff would fail deterministically on the same table; a shared dataset losing tables under concurrent runs would not. That is evidence for contention, but with the overlap still present in this run, it is not yet proof, and I'm not going to claim it is.

Two things still to discriminate, neither of which I'm asserting the outcome of:

  1. tests (bigquery-export-bucket-gcs, false) — the sibling leg on this same commit — is still running. Same code, same head: if it passes while true fails, that points hard at contention rather than the diff.
  2. Once run 33026775638 finishes there will be no competing run, and I'll use my one rerun_failed_jobs on it. That is the clean measurement neither run has had.

For the record, the diff still touches nothing that could plausibly cause this: git diff --name-only origin/master...HEAD matches nothing against bigquery|pre.?agg|driver.

I'll report both results here rather than leaving this hanging.


Generated by Claude Code

Copy link
Copy Markdown
Member Author

CI note — integration-cubestore (24.x) failed: one wall-clock-timing test in a package this PR does not touch.

Different failure from the BigQuery ones, so recording it separately. 37 passed, 1 failed:

● QueryQueueCubeStore Queue Driver › sequence

  expect(received).toEqual(expected)
  - Expected  - 2      + Received  + 2
    Array [ "10", "21",
  -   "32", "43",       +   "33", "42",
    ]
  at test/unit/QueryQueue.abstract.ts:257

Not this PR's — by construction. The failure is in @cubejs-backend/query-orchestrator. This PR's complete file list is 11 files: 3 docs pages, 1 schema-compiler test (comment-only in the last commit), 4 birdbox fixtures, 1 smoke test, packages/cubejs-testing/package.json, and smoke.sh. Nothing in packages/cubejs-query-orchestrator, and nothing that integration:cubestore loads.

Why it's timing-sensitive. QueryQueue.abstract.ts:250-257 stages four background queue submissions on wall-clock offsets and asserts one exact result string each:

offset exec expected got
p1 0 ms 50 ms 10 10
p2 50 ms 50 ms 21 21
p3 75 ms 50 ms 32 33
p4 100 ms 50 ms 43 42

p3 and p4 are 25 ms apart while each query runs for 50 ms, so which one the queue picks up first depends on real scheduling on a loaded shared runner. p3 and p4 swapped their positions — the values aren't wrong, the interleaving is. The two failures are also mutually consistent (33/42 is exactly the p3↔p4 swap), which is what an ordering race looks like and not what a logic error looks like.

No fix to port: the test needs its 25 ms assumption relaxed or an ordering-independent assertion, which is a change to a package outside this PR's scope. I'm not going to widen the diff to touch the query-orchestrator test suite.

Run 33026775626 is still in progress so rerun_failed_jobs is rejected right now. I'll re-run this job once the run completes — the same check-in that re-runs the BigQuery leg — and report the result. If it passes on re-run, that confirms the race; if it fails identically, I'll dig further rather than keep calling it timing.


Generated by Claude Code

Copy link
Copy Markdown
Member Author

Both runs finished on a7f1969. The suite this PR adds passes on both matrix legs. Both promised re-runs are now queued.

The result that matters: smoke:multi-fact ran and passed on both legs

I pulled the full run log archive rather than trusting the job's green tick, because the point of the pin was that the suite must not silently skip:

use_tesseract_sql_planner: false          use_tesseract_sql_planner: true
PASS dist/test/smoke-multi-fact.test.js   PASS dist/test/smoke-multi-fact.test.js
  ✓ each fact is aggregated on its own      ✓ each fact is aggregated on its own
  ✓ a view measure divides the two facts    ✓ a view measure divides the two facts
  ✓ a cube measure divides them the same    ✓ a cube measure divides them the same
  ✓ reachable through a view                ✓ reachable through a view
  ✓ returned next to its components         ✓ returned next to its components
  ✓ ratio over the whole result             ✓ ratio over the whole result
Tests: 6 passed, 6 total                  Tests: 6 passed, 6 total

The false leg is the one that skipped before. Its log also carries Warning. Tesseract planner is a default one, but you are trying to use a legacy planner, from a different suite in the same job — confirming the leg really does export the legacy setting and that the birdbox pin is what makes this suite run there. That's the reviewer's suggestion working exactly as intended.

Drivers tests — 46 pass, 1 fail, 2 cancelled

The discriminator I said I'd report: tests (bigquery-export-bucket-gcs, false) PASSED, on the same commit, against the same shared dataset, while the true leg failed. Same code, opposite outcomes. Combined with the earlier evidence — three runs, three different sets of missing tables, and one test that failed twice and passed once — this is contention on cube-open-source:dev_pre_aggregations, not a defect in this diff.

The 2 cancelled are tests (redshift, true) and tests (redshift-export-bucket-s3, true), from the deliberate redshift-global concurrency group at drivers-tests.yml:255-259 ("redshift Test Serverless Instance is so tiny that it can't serve a few test suites at once"). Worth knowing: because those jobs are cancelled, the whole run reports cancelled even though 46 of 49 jobs passed — the run-level conclusion is misleading here.

Build — 29 pass, 1 fail

Only integration-cubestore (24.x), the p3/p4 interleaving race described above.

Re-runs

rerun_failed_jobs queued on both runs (33026775638, 33026775626) — neither would accept it earlier while in progress. Per what I said last time: if they pass, that confirms contention and the race; if either fails identically, I'll dig further rather than calling it timing again.


Generated by Claude Code

Copy link
Copy Markdown
Member Author

Both re-runs passed. Both diagnoses confirmed. a7f1969 is green and mergeable_state: clean.

Run Attempt 2 Confirms
33026775638 Drivers tests success tests (bigquery-export-bucket-gcs, true) passes with no competing run → shared-dataset contention
33026775626 Build success integration-cubestore (24.x) passes → the p3/p4 queue interleaving race

That was the test I committed to: pass on re-run confirms the diagnosis, identical repeat failure would have meant digging further. Both passed, so both stand — no code change is warranted for either, and none was made.

Summary of the four CI failures on this PR, none of them the diff's:

  1. Upload merged coverage to Codecov — GitHub artifact storage, on a superseded commit. Passed on the current head.
  2. tests (bigquery-export-bucket-gcs) ×3 heads — contention on cube-open-source:dev_pre_aggregations. Three different sets of missing tables, one test that failed twice and passed once, the false leg passing on the same commit the true leg failed, and now a clean pass on re-run.
  3. integration-cubestore (24.x) — the sequence ordering race in @cubejs-backend/query-orchestrator, a package this PR does not touch.
  4. tests (redshift, *) ×2 — cancelled by the deliberate redshift-global concurrency group, not failures.

The contention was avoidable and partly mine. Four heads of this PR had driver runs alive at once because I pushed four times in ~15 minutes, three of them small review fixes. Batching them would have avoided most of these runs and most of this thread.

Two things worth knowing beyond this PR, neither of which I'm acting on here:

  • A cancelled job makes the whole run report cancelled. The Drivers run showed cancelled on attempt 1 with 46 of 49 jobs green — that reads as far worse than it was.
  • The shared dev_pre_aggregations dataset makes concurrent driver runs interfere by construction. Per-run dataset isolation would remove a recurring source of false reds. Happy to file that if it's wanted.

I'm done investigating CI here and won't post further unless something changes. Approved by @keydunov and green — the merge is yours to make.


Generated by Claude Code

@paveltiunov
paveltiunov merged commit 9968f8a into master Aug 27, 2026
170 of 174 checks passed
@paveltiunov
paveltiunov deleted the claude/multi-fact-derived-metric-test-j4uypn branch August 27, 2026 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

javascript Pull requests that update Javascript code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants