Design: the xarray data model for multiresolution /2 stores (DataTree vs resolution dimension vs custom multi-res index)

[espg]

🤖 from Claude (filed at espg's direction, 2026-08-05; the model question behind issue #36's mechanism)

Motivation

zagg-pyramid/2 stores are inherently multiresolution (englacial/zagg#381, point (12)): a guaranteed ladder at every order 0…base plus the native resolution, with the finest level already anchored in the root metadata (morton_hive.cell_order, dggs.refinement_level) and the full set in the expanded pyramid.overviews block. Issue #36 covers exposing the ladder; this issue decides what the ladder is in xarray terms — before ad-hoc per-level opens calcify into the de-facto model.

The design questions

1. Container idiom. xarray.DataTree (now upstream) with one dataset per order — the natural multiscales mapping — vs one Dataset with a stacked resolution coordinate, vs the status quo family of independent Datasets. DataTree is the lean; the others deserve a written rejection.
2. Coordinate identity across levels. Morton prefix nesting gives exact parent/child cell relations between any two orders — what does that buy in the API? (sel traversal to a coarser order; a "lift/project" operation between levels; groupby-parent.) What coordinate/index metadata must each level carry for those to be lawful?
3. Cross-resolution operation policy. Explicitly NOT via upstream xdggs — upstream forbids cross-resolution alignment in one index (recorded prior ruling), and that prohibition is the reason this design lives here. Within this package the fixed ladder makes cross-res relations well-defined; the policy should say which ops are exact (exact-class fields: order-to-coarser aggregation is exactly the stored coarser level — the doctrine upgrade), which are approximate-with-provenance (digest fields at recorded merges-from-raw), and which are refused (implicit alignment between arbitrary orders).
4. Index design. One MortonMocIndex per level (status quo) vs a single multiresolution index owning the prefix relations. Interaction with lazy/cubed arrays at each level.
5. Convention mapping. How the zagg-side convention attrs (declare the multiresolution ladder in machine-readable convention metadata (zarr multiscales + DGGS refinement) englacial/zagg#392) map onto the chosen idiom (multiscales entries → DataTree groups), manifest-first as always.
6. Compatibility posture toward xdggs: the register_dggs plugin route needs no upstream PR (recorded prior finding) — per-level xdggs interop as a side-door is plausible; it must not constrain the primary model.

Deliverable

A written model decision on this thread (options + rejection rationale), then implementation lands under issue #36. Refs: #1 (plan), #36 (mechanism), #21 (gridlook feeder), englacial/zagg#381, englacial/zagg#392.

[espg]

🤖 from Claude

espg's lean, recorded from in-session discussion (2026-08-05), plus fit analysis:

• Container idiom lean: xarray.DataTree (question 1) — upstream now, and ecosystem investment in tree-shaped zarr access (e.g. rustytree-xarray, a Rust-backed engine=\"rustytree\" drop-in for engine=\"zarr\" on the same open_datatree entry points; 0.2.0, wheels for py3.12/3.13) keeps making this bet cheaper.
• rustytree fit caveat: it discovers structure from zarr group metadata; a morton-hive store is a forest of independent zarrs with a JSON manifest as truth, plus custom ragged/t-digest decode. So it cannot be the discovery mechanism — moczarr builds the DataTree manifest-first as always; rustytree is a candidate acceleration/compat layer underneath, pinned cautiously (young project).
• Middle-path option for the design space (feeds question 5 and zagg's convention issue declare the multiresolution ladder in machine-readable convention metadata (zarr multiscales + DGGS refinement) englacial/zagg#392): materialize the coarse ladder — small, dense, cap-bounded — as a literal zarr group hierarchy (a multiscales companion group at the store root) while leaves stay hive. That tier opens natively in stock tools (rustytree, generic multiscales viewers, the gridlook feeder) with zero custom reader; the manifest/hive remains normative and the scalable half. Whether the companion group is worth its bytes and its sweep-stage write is part of this issue's written decision.

[espg]

Container idiom. xarray.DataTree (now upstream) with one dataset per order — the natural multiscales mapping — vs one Dataset with a stacked resolution coordinate, vs the status quo family of independent Datasets. DataTree is the lean; the others deserve a written rejection.

one dataset per order. stacked resolution coordinate would be great-- but is a much larger lift (we can document for a future spec)