Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/actions/smoke.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ echo "::group::DuckDB"
# Should we create a separate job integration-duckdb? I believe not, because it works fast.
yarn lerna run --concurrency 1 --stream --no-prefix integration:duckdb
yarn lerna run --concurrency 1 --stream --no-prefix smoke:duckdb
# Also DuckDB-backed, so it shares this group rather than paying for its own.
yarn lerna run --concurrency 1 --stream --no-prefix smoke:multi-fact
echo "::endgroup::"

echo "::group::View Groups"
Expand Down
15 changes: 10 additions & 5 deletions docs-mintlify/docs/data-modeling/views.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -285,11 +285,10 @@ straightforward.

### Define a metric on a view when it spans cubes

The one exception is a metric whose parts live in different cubes, so there is
no single cube it could belong to. A view can define its own
[measures][ref-view-measures] and [dimensions][ref-view-dimensions] as long as
their `sql` only combines members the view already includes — a member that
reads a column instead is rejected at compile time:
The exception is a metric whose parts live in different cubes. A view can define
its own [measures][ref-view-measures] and [dimensions][ref-view-dimensions] as
long as their `sql` only combines members the view already includes — a member
that reads a column instead is rejected at compile time:

<CodeGroup>

Expand Down Expand Up @@ -328,6 +327,11 @@ inflate the numerator. If the cubes don't join to each other at all, see
[multi-fact views][ref-multi-fact-views]. The full example, with the `cubes`
block, is on the [view reference][ref-view-measures].

A cube can also own such a metric, by referencing the other cube's member
directly — that keeps it defined once for every view that includes it, at the
cost of one cube naming another. The [average order value
recipe][ref-recipe-aov] compares the two placements.

### Control visibility

Not every view should be publicly accessible. Use [`public`][ref-view-public]
Expand Down Expand Up @@ -499,6 +503,7 @@ parameters.
[ref-view-dimensions]: /reference/data-modeling/view#dimensions
[ref-multi-stage]: /reference/data-modeling/measures#multi_stage
[ref-multi-fact-views]: /docs/data-modeling/multi-fact-views
[ref-recipe-aov]: /recipes/data-modeling/average-order-value#where-to-put-the-measure
[ref-view-folders]: /reference/data-modeling/view#folders
[ref-access-policies]: /reference/data-modeling/data-access-policies
[ref-ai-context]: /docs/data-modeling/ai-context
Expand Down
81 changes: 78 additions & 3 deletions docs-mintlify/recipes/data-modeling/average-order-value.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -230,9 +230,9 @@ since one is keyed by day and the other by timestamp.

### 2. Define AOV on the view

Neither cube can define AOV — neither can reference the other's measures. Define
it as a [measure of the view][ref-view-measures] and mark it
[`multi_stage`][ref-multi-stage]:
AOV can live on the view or on either cube — see [where to put
it](#where-to-put-the-measure) below. On the view it is a [measure of the
view][ref-view-measures], marked [`multi_stage`][ref-multi-stage]:

<CodeGroup>

Expand Down Expand Up @@ -331,8 +331,83 @@ measure, looks for a single join tree covering both fact cubes, and fails with

</Note>

## Where to put the measure

A metric spanning two facts does not have to live on a view. A cube measure may
reference another cube's measure, which makes it derived rather than owned by
its cube — the same property a view measure has — so AOV can sit on either fact
cube instead:

<CodeGroup>

```yaml title="YAML"
cubes:
- name: sales_line_item
# …

measures:
- name: transactions_without_returns
sql: transaction_id
type: count_distinct
filters:
- sql: "{CUBE}.transaction_type <> 'EXCHANGE'"
- sql: "{CUBE}.fulfillment_channel_group IN ('IN_STORE', 'SHIP_FROM_STORE')"

- name: aov_basket
type: number
format: currency
multi_stage: true
sql: "{item_location_sales.sales_amount} / NULLIF({CUBE.transactions_without_returns}, 0)"
```

```javascript title="JavaScript"
cube(`sales_line_item`, {
// …

measures: {
transactions_without_returns: {
sql: `transaction_id`,
type: `count_distinct`,
filters: [
{ sql: `${CUBE}.transaction_type <> 'EXCHANGE'` },
{ sql: `${CUBE}.fulfillment_channel_group IN ('IN_STORE', 'SHIP_FROM_STORE')` }
]
},

aov_basket: {
type: `number`,
format: `currency`,
multi_stage: true,
sql: `${item_location_sales.sales_amount} / NULLIF(${CUBE.transactions_without_returns}, 0)`
}
}
})
```

</CodeGroup>

Both placements plan identically — the same per-fact subqueries, stitched the
same way, divided in the same final stage — and `multi_stage` is required either
way. What differs is reuse and coupling:

| | On a cube | On a view |
| --- | --- | --- |
| Reuse | Defined once; every view including it gets it | Redefined in each view that needs it |
| Coupling | The cube names the other cube's measure | The cubes stay unaware of each other |
| Query path | Available as `sales_line_item.aov_basket` too | Only through the view |

Prefer the cube when the metric is part of the model that several views expose —
it keeps [shared logic in cubes][ref-views-shared-logic]. Prefer the view when
the pairing is a presentation choice for one audience, or when the cubes belong
to different domains and you would rather not have one reference the other.

A cube-owned measure reaches the other fact whether or not the view naming it
Comment thread
paveltiunov marked this conversation as resolved.
also includes that fact, so a view can expose AOV without exposing
`sales_amount`.

[ref-multi-fact-views]: /docs/data-modeling/multi-fact-views
[ref-multi-stage]: /reference/data-modeling/measures#multi_stage
[ref-measure-filters]: /reference/data-modeling/measures#filters
[ref-view-measures]: /reference/data-modeling/view#measures
[ref-views-shared-logic]: /docs/data-modeling/views#keep-shared-logic-in-cubes
[link-tesseract]: https://cube.dev/blog/introducing-tesseract
5 changes: 4 additions & 1 deletion docs-mintlify/reference/data-modeling/view.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -571,7 +571,9 @@ If you'd like to override the [metadata][ref-dim-meta] of a member, you can use
The `measures` parameter defines measures on the view itself. A view measure is
always _derived_: its `sql` may only reference members that the view already
includes, never columns of a table. Use it for a metric whose parts come from
different cubes, which therefore has no single cube to live in.
different cubes. (A cube can own such a metric too, by referencing the other
cube's member; the [average order value recipe][ref-recipe-aov] compares the two
placements.)

Reference the included members as `{CUBE.member}` (or `{view_name.member}`); a
bare `{member}` does not resolve inside a view.
Expand Down Expand Up @@ -1112,6 +1114,7 @@ The `access_policy` parameter is used to configure [access policies][ref-ref-dap
[ref-ref-cubes]: /reference/data-modeling/cube
[ref-ref-multi-stage]: /reference/data-modeling/measures#multi_stage
[ref-multi-fact-views]: /docs/data-modeling/multi-fact-views
[ref-recipe-aov]: /recipes/data-modeling/average-order-value#where-to-put-the-measure
[ref-ref-hierarchies]: /reference/data-modeling/hierarchies
[ref-ref-dap]: /reference/data-modeling/data-access-policies
[ref-rest-query-ops]: /reference/core-data-apis/rest-api/query-format#filters-operators
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,12 @@ import { prepareYamlCompiler } from './PrepareCompiler';
// measure - they are never restated in a view. `sales_amount` is a plain sum on
// the day/item/location cube.
//
// The ratio of the two is authored as a measure of the view rather than per
// consumer. The line-item side has to be aggregated to the query grain before
// it can divide a sum coming from the other fact table, which is the multi-fact
// path: both facts join to the shared `items`, `locations` and `dates` cubes,
// but never to each other.
// The ratio of the two is authored once as a measure rather than per consumer,
// in both of the places it can live: on the view, and on the line-item cube
// itself (from where a view can re-expose it). The line-item side has to be
// aggregated to the query grain before it can divide a sum coming from the
// other fact table, which is the multi-fact path: both facts join to the
// shared `items`, `locations` and `dates` cubes, but never to each other.
const model = `
cubes:
- name: items
Expand Down Expand Up @@ -101,6 +102,16 @@ cubes:
filters:
- sql: "{CUBE}.transaction_type <> 'EXCHANGE'"
- sql: "{CUBE}.fulfillment_channel_group IN ('IN_STORE', 'SHIP_FROM_STORE')"
# The same ratio, owned by this cube instead of the view. Referencing the
# other fact's measure is what makes it not owned by this cube, which is
# what a derived member has to be; the \`multi_stage\` rule is unchanged.
- name: aov_basket
type: number
multi_stage: true
sql: "{item_location_sales.sales_amount} / NULLIF({CUBE.transactions_without_returns}, 0)"
- name: aov_basket_single_stage
type: number
sql: "{item_location_sales.sales_amount} / NULLIF({CUBE.transactions_without_returns}, 0)"

- name: item_location_sales
sql: >
Expand Down Expand Up @@ -139,6 +150,8 @@ views:
includes:
- transactions_without_returns
- net_sale_transactions
- name: aov_basket
alias: aov_basket_from_cube
# The shared dimension cubes sit at root-level join paths so their
# dimensions are common to both facts.
- join_path: dates
Expand All @@ -164,6 +177,17 @@ views:
- name: aov_basket_single_stage
type: number
sql: "{retail_analysis.sales_amount} / NULLIF({retail_analysis.transactions_without_returns}, 0)"

# Exposes the cube-owned ratio without exposing the other fact's measure at
# all, which is what the recipe claims a cube-owned metric allows.
- name: line_item_analysis
cubes:
- join_path: sales_line_item
includes:
- aov_basket
- join_path: locations
includes:
- region
`;

let compilers: any;
Expand Down Expand Up @@ -326,6 +350,98 @@ describe('Multi-fact derived measure defined on a view', () => {
});
});

// The same ratio, owned by `sales_line_item` rather than by the view. A cube
// measure that references another cube's measure is not owned by its cube -
// the same property a view measure has - so a metric spanning two facts does
// not need a view to live in. It can sit in the model, and every view that
// includes it gets it.
describe('Multi-fact derived measure defined on a cube', () => {
it('divides the two facts once both have been aggregated to the query grain', () => {
const sql = buildSql({
measures: ['sales_line_item.aov_basket'],
dimensions: ['locations.region'],
});

expect(sql).toMatch(SALES_AMOUNT_AGGREGATE);
expect(sql).toMatch(TRANSACTIONS_AGGREGATE);
expect(sql).toMatch(RATIO_OVER_AGGREGATES);
expect(sql).not.toMatch(/sum\("item_location_sales"\.sales_amount\) \/ NULLIF/);
});

it('plans the same way whichever fact owns it', () => {
// Only the emitted column alias should differ between the two placements,
// so normalising it makes the two plans directly comparable.
const normalize = (sql: string) => sql
.replace(/"(retail_analysis|sales_line_item)__aov_basket"/g, '"__aov"')
.replace(/"retail_analysis__region"/g, '"__region"')
.replace(/"locations__region"/g, '"__region"');

const onView = buildSql({
measures: ['retail_analysis.aov_basket'],
dimensions: ['retail_analysis.region'],
});
const onCube = buildSql({
measures: ['sales_line_item.aov_basket'],
dimensions: ['locations.region'],
});

expect(normalize(onCube)).toEqual(normalize(onView));
});

it('is reachable through a view that includes it', () => {
const sql = buildSql({
measures: ['retail_analysis.aov_basket_from_cube'],
dimensions: ['retail_analysis.region'],
});

expect(sql).toMatch(RATIO_OVER_AGGREGATES);
});

it('is reachable through a view that does not include the other fact', () => {
const sql = buildSql({
measures: ['line_item_analysis.aov_basket'],
dimensions: ['line_item_analysis.region'],
});

// `line_item_analysis` exposes no measure of item_location_sales, yet the
// ratio still reaches it and is divided over the two aggregates.
expect(sql).toMatch(SALES_AMOUNT_AGGREGATE);
expect(sql).toMatch(RATIO_OVER_AGGREGATES);
});

it('reaches the other fact even when the query names only its own cube', () => {
const sql = buildSql({ measures: ['sales_line_item.aov_basket'] });

// No dimensions, so the two legs are aggregated whole and stitched anyway.
expect(sql).toMatch(SALES_AMOUNT_AGGREGATE);
expect(sql).toMatch(TRANSACTIONS_AGGREGATE);
expect(sql).toMatch(RATIO_OVER_AGGREGATES);
});

it('divides the two facts on the shared date spine', () => {
const sql = buildSql({
measures: ['sales_line_item.aov_basket'],
timeDimensions: [{ dimension: 'dates.date', granularity: 'day' }],
});

expect(sql).toContain('DATE_TRUNC(\'day\', "item_location_sales".date) = "dates".date');
expect(sql).toContain('DATE_TRUNC(\'day\', "sales_line_item".sold_at) = "dates".date');
expect(sql).toMatch(RATIO_OVER_AGGREGATES);
});

// Current behaviour, pinned - the same limit as on the view. Owning the
// measure buys the cube nothing here: without `multi_stage` the planner still
// looks for one join tree covering both facts and there is none.
it('cannot plan the ratio when the cube measure is not multi_stage', () => {
// The cubes are listed in the order the planner collected them, which
// differs from the view-owned case, so only their presence is pinned.
expect(() => buildSql({
measures: ['sales_line_item.aov_basket_single_stage'],
dimensions: ['locations.region'],
})).toThrow(/Can't find join path to join (?=.*item_location_sales)(?=.*sales_line_item)/);
});
});

// The other reason a view measure spanning cubes wants `multi_stage`: even when
// the cubes DO join, a plain calculated measure is evaluated inside the single
// joined scan, so a `sum` on the one side is taken over rows the join has
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
// Sales dollars, pre-aggregated to day/item/location in the warehouse. The
// numerator of AOV. West totals 100, East totals 60.
cube(`ItemLocationSales`, {
sql: `
select 1 as id, 1 as location_id, 30 as sales_amount
UNION ALL
select 2 as id, 1 as location_id, 70 as sales_amount
UNION ALL
select 3 as id, 2 as location_id, 60 as sales_amount
`,

joins: {
Locations: {
sql: `${CUBE}.location_id = ${Locations}.id`,
relationship: `many_to_one`,
},
},

dimensions: {
id: {
sql: `id`,
type: `number`,
primaryKey: true,
},
},

measures: {
salesAmount: {
sql: `sales_amount`,
type: `sum`,
},
},
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
// Shared dimension cube. Both facts join to it, which is what gives a
// multi-fact query something to stitch the two aggregates on.
cube(`Locations`, {
sql: `
select 1 as id, 'West' as region
UNION ALL
select 2 as id, 'East' as region
`,

dimensions: {
id: {
sql: `id`,
type: `number`,
primaryKey: true,
},
region: {
sql: `region`,
type: `string`,
},
},
});
Loading
Loading