temporal + located companion channels through the kernel and the pyramid (issue #410)

docs/specification.md

@@ -927,6 +927,25 @@ staged sweep's finisher.
   tolerate entry keys it does not bind. This map is the
   **all-fields** view; the per-overview `zagg_overview.fields` attrs map
   (§4.3) is the materialized subset.
+- **A located ragged field is `approximate`, and its entry carries
+  `location`.** A `location:` declaration (§9) does not exclude a field from
+  the pyramid: the located k-way merge reduces the `{field}_locations` words
+  over the same centroid partition the digest merge produces, so a located
+  digest field folds through every level and the class is `approximate` like
+  any other digest field ([ruling 4 on
+  issue #410](https://github.com/englacial/zagg/issues/410#issuecomment-5310502887)).
+  Its entry carries `location` — the source column the leaves' words were
+  ingested from — keyed **only when set**, so an unlocated field's entry is
+  unchanged. The key is load-bearing for the same reason `weights`/`gain` are:
+  the overview writer reconstructs a level's arrays from this entry alone, so
+  without it the overview template emits no sibling array and the fold has
+  nowhere to write its words. An overview level of a located field therefore
+  carries the `{field}_locations` sibling, its §9 `located` declaration, and
+  the payload's §1.2 `locations` binding, exactly as a leaf does — with the
+  words at the **heterogeneous orders** §9.1 makes normative. A **temporal**
+  companion (§8.3) is a different case and stays `class: "none"`: §8.4's
+  shape-coarsening reduction is not wired through the fold sites, so those
+  fields exist at native resolution only.
 - **`all_time`** — whether the `all.zarr` all-time fold is materialized at
   the declared orders (windowed stores only; a `schedule: none` store's
   single fold is already all-time).
@@ -1068,7 +1087,13 @@ re-invoking the idempotent leaf, never a sweep-side fold from raw cells.
   for every coarser cell (there is no `partial/` grammar; a coarse level
   declared later never rewrites a leaf). Each group holds the `morton`
   coordinate (the node's order-`r` descendant words, ascending) and one
-  array per **composable** field (§4.5 classes; `none` fields are absent).
+  array per **composable** field (§4.5 classes; `none` fields are absent),
+  plus — for a field whose §4.5 entry carries `location` — that field's
+  `{field}_locations` sibling, row-aligned with its payload and carrying the
+  §9 declaration, exactly as an overview level does. The pair is written
+  together or not at all: the words are exact only *given* the centroid
+  partition the payload describes (§9.1), so a group holding a populated
+  payload against an empty sibling is non-conformant, not merely short.
   A group's arrays are **single-chunk and unsharded** — `chunk_shape` equals
   `shape` (`4^(r - node)` cells), no `sharding_indexed` codec — whatever the
   leaf's own `chunk_inner`/sharding: a column group is small by construction,
@@ -1560,6 +1585,17 @@ product is never sharded, §8/#247):
   pinned. `kitchen_sink/`, committed before §9 and unregenerated, is the
   absent-`located` ⇒ §2.2 pin, exactly as `minimal/` is §2.0's.
 
+  Its §4.6 leaf column carries the **folded** companions: every resolution
+  group holds `h_tdigest` with both siblings, each declaring `per-centroid`,
+  row-aligned with the folded payload. That makes it the fixture set's only
+  golden for a companion produced by a *merge* rather than by ingest — both
+  channels reduced over the centroid partition that merge produced, at every
+  level (espg-ruled 2026-08-17, amending
+  [ruling 3](https://github.com/englacial/zagg/issues/410#issuecomment-5310502887)).
+  The dense `observed` array is deliberately absent from those groups: the
+  `"per-cell"` shape's fold law is the grammar's join over a cell group rather
+  than the field's own reducer, so it exists at native resolution only.
+
 `minimal/` and `kitchen_sink/` pin the layout edge cases a reader must
 handle (`column/`'s leaf is `minimal/`'s, so it pins them again): inner chunk
 ordinal 2 is **empty** (absent from the shard index — the §1.5 sentinel, and
@@ -1979,8 +2015,12 @@ satisfy the peer gate above. This revision licenses exactly one — the
 `"per-centroid"` → `"per-cell"` fold that §8.3's closing *"Per-level shapes
 need not match"* clause describes and that
 [ruling 3 on #410](https://github.com/englacial/zagg/issues/410#issuecomment-5310502887)
-requires ("per-cell toc *range* at overview levels, even where leaves are
-per-centroid"). Its terms:
+first called for ("per-cell toc *range* at overview levels, even where leaves
+are per-centroid"). That ruling was **amended on 2026-08-17**: zagg's own digest
+pyramids keep companions `"per-centroid"` at every level, symmetric with the
+located channel, so the reduction below is **licensed but unused by this
+writer** — the terms stand unchanged for any producer that wants it, and a
+reader must still implement the mixed-level case §8.3 permits. Its terms:
 
 - The contributors MUST be peers **of each other**: every one declares
   `shape: "per-centroid"` under the same `grammar`. A reduction over

pyproject.toml

@@ -37,7 +37,13 @@ dependencies = [
     # what PyPI has published (as 0.2.0 once was) fails resolution loudly — the
     # state build_layer.sh's MORTIE_SPEC derivation cites this block for.
     "h5coro-hidefix>=0.3.2",
-    # Floor 0.9.8 for the authalic latitude convention (espg/mortie#186, landed
+    # Floor 0.9.9 for the SEGMENTED toc reduce (espg/mortie#177, released in
+    # 0.9.9): tocs_reduce(words, offsets) is the per-group semilattice join the
+    # spec §8.2/§8.3 temporal companions fold with — one Rust crossing for a
+    # whole cell's centroid partition, where the scalar toc_reduce 0.9.6 ships
+    # would be a Python loop per centroid (issue #410,
+    # zagg.stats.tdigest._centroid_envelopes). Earlier floors: 0.9.8
+    # for the authalic latitude convention (espg/mortie#186, landed
     # in espg/mortie#188): binning is equal-area on the WGS84 ellipsoid by
     # construction — latitude="authalic" is the new mortie default, with
     # "geodetic-spherical" as the legacy escape. Cell ids differ between the
@@ -71,7 +77,7 @@ dependencies = [
     # route through (issue #199) — decimal_repr / to_decimal, the
     # _decimal_to_word parse-back, hive_path, and the MortonIndexScalar decimal
     # display.
-    "mortie>=0.9.8",
+    "mortie>=0.9.9",
     "earthaccess",
     "boto3",
     "fastparquet",

src/zagg/column.py

@@ -141,10 +141,13 @@ def composable_fields(fields: dict) -> dict:
     """The declared fields a column fold may carry: the two composable classes.
 
     The D24 ``class: "none"`` entries — expressions, vector fields,
-    chunk-resolution companions, located ragged, and the derived statistics
-    (:func:`zagg.semantics.field_composability`, recorded by
+    chunk-resolution companions, temporal companions, and the derived
+    statistics (:func:`zagg.semantics.field_composability`, recorded by
     :func:`zagg.pyramid.declared_fields`) — exist at native resolution ONLY,
-    and no coarser fold of them is defined. The fold core filters them here,
+    and no coarser fold of them is defined. A **located** ragged field is not
+    among them: since ruling 4 on issue #410 it is ``approximate`` and folds
+    through the pyramid with its ``{field}_locations`` channel, so it passes
+    this filter and :func:`fold_column` carries the pair. The fold core filters them here,
     the same posture the sweep takes before ``_fold_node``
     (:func:`zagg.sweep_overview.sweep_overviews`), so handing a whole
     declaration's ``fields`` map straight through can neither refuse a leaf
@@ -175,26 +178,36 @@ def leaf_slabs(staged: dict, fields: dict, *, group_path: str, n_cells: int) ->
     ``fields`` is filtered to the composable classes first
     (:func:`composable_fields`), which is what makes the ``(n_cells,)`` extent
     check sound: those two classes admit nothing but cell-resolution scalars
-    and unlocated ragged payloads, so a staged slab of any other extent really
-    is a sink that disagrees with the grid — and folding it would write a
-    wrong column, so it raises.
+    and ragged payloads, both of which are one row per cell, so a staged slab
+    of any other extent really is a sink that disagrees with the grid — and
+    folding it would write a wrong column, so it raises.
+
+    A field's companion siblings — ``{field}_locations`` (§9) and
+    ``{field}_times`` (§8.3) — are picked up under the same rule and the same
+    extent check (issue #410): each is one more cell-extent slab in the same
+    sink, and :func:`fold_column` needs them in one place because the merge
+    produces all of them in one call (spec §9.1/§8.3).
     """
-    from zagg.sweep_overview import _empty_slab
+    from zagg.sweep_overview import _empty_slab, field_companions
+
+    def _slab(key: str, meta: dict):
+        slab = staged.get(f"{group_path}/{key}")
+        if slab is None:
+            return _empty_slab(meta, n_cells)
+        slab = np.asarray(slab)
+        if slab.shape != (int(n_cells),):
+            raise ValueError(
+                f"staged slab for field {key!r} has shape {slab.shape}, not the "
+                f"leaf's ({int(n_cells)},) cell extent — refusing to fold a column "
+                f"from a sink that disagrees with the grid"
+            )
+        return slab
 
     slabs: dict = {}
     for name, meta in composable_fields(fields).items():
-        slab = staged.get(f"{group_path}/{name}")
-        if slab is None:
-            slab = _empty_slab(meta, n_cells)
-        else:
-            slab = np.asarray(slab)
-            if slab.shape != (int(n_cells),):
-                raise ValueError(
-                    f"staged slab for field {name!r} has shape {slab.shape}, not the "
-                    f"leaf's ({int(n_cells)},) cell extent — refusing to fold a column "
-                    f"from a sink that disagrees with the grid"
-                )
-        slabs[name] = slab
+        slabs[name] = _slab(name, meta)
+        for _kwarg, sibling in field_companions(name, meta):
+            slabs[sibling] = _slab(sibling, meta)
     return slabs
 
 
@@ -218,8 +231,25 @@ def fold_column(slabs: dict, fields: dict, *, cell_order: int, resolutions: list
     resolution FINER than ``cell_order`` is refused by name: it would ask for
     a fractional fold factor, which no guard downstream can read as a divisor
     (both classes would surface it as an opaque numpy failure instead).
+
+    A **located** field folds its ``{field}_locations`` sibling in the SAME
+    k-way call as its payload and returns it as its own group member (ruling 4
+    on issue #410, review finding): the §4.6 template
+    (:func:`zagg.sweep_overview._overview_config`) emits the sibling array and
+    the payload's §1.2 binding for every located field, so a fold returning
+    payload slabs only would commit populated payload rows against ``b""``
+    sibling rows under a §9 declaration — §1.1's row-alignment MUST broken, and
+    hashed into the §5 sidecar as content. The sibling slab is required by
+    name rather than defaulted: the words are keyed on the centroid partition
+    the merge produces (spec §9.1), so the pair may never be folded apart.
     """
-    from zagg.sweep_overview import decode_digest, fold_dense, fold_digests, overview_fold_delta
+    from zagg.sweep_overview import (
+        decode_digest,
+        field_companions,
+        fold_dense,
+        fold_digests,
+        overview_fold_delta,
+    )
 
     cell_order = int(cell_order)
     fields = composable_fields(fields)
@@ -241,24 +271,51 @@ def fold_column(slabs: dict, fields: dict, *, cell_order: int, resolutions: list
                 groups[name] = fold_dense(
                     slab, factor, meta.get("method"), meta.get("fill_value", "NaN")
                 )
-            else:
-                dtype = meta.get("dtype") or "float32"
-                inner = tuple(meta.get("inner_shape") or (2,))
-                delta = overview_fold_delta(meta)
-                if slab.shape[0] % factor:
+                continue
+            dtype = meta.get("dtype") or "float32"
+            inner = tuple(meta.get("inner_shape") or (2,))
+            delta = overview_fold_delta(meta)
+            if slab.shape[0] % factor:
+                raise ValueError(f"cannot fold {slab.shape[0]} cells {factor}-to-one for {name!r}")
+            declared = field_companions(name, meta)
+            for kwarg, sibling in declared:
+                if slabs.get(sibling) is None:
                     raise ValueError(
-                        f"cannot fold {slab.shape[0]} cells {factor}-to-one for {name!r}"
+                        f"field {name!r} declares a {kwarg} channel but no {sibling!r} slab "
+                        f"was supplied — the words are keyed on the centroid partition the "
+                        f"merge produces (spec §9.1/§8.3), so the pair cannot be folded apart"
                     )
-                folded = np.full(slab.shape[0] // factor, b"", dtype=object)
-                for j in range(folded.shape[0]):
-                    cell = [
-                        decode_digest(payload, dtype, inner)
-                        for payload in slab[j * factor : (j + 1) * factor]
-                        if payload is not None and len(payload)
-                    ]
-                    if cell:
-                        folded[j] = fold_digests(cell, delta=delta, dtype=dtype)
-                groups[name] = folded
+            folded = np.full(slab.shape[0] // factor, b"", dtype=object)
+            sibling_slabs = {
+                kwarg: np.full(folded.shape[0], b"", dtype=object) for kwarg, _ in declared
+            }
+            for j in range(folded.shape[0]):
+                rows = [
+                    i
+                    for i in range(j * factor, (j + 1) * factor)
+                    if slab[i] is not None and len(slab[i])
+                ]
+                if not rows:
+                    continue
+                cell = [decode_digest(slab[i], dtype, inner) for i in rows]
+                if not declared:
+                    folded[j] = fold_digests(cell, delta=delta, dtype=dtype)
+                    continue
+                payload, *words = fold_digests(
+                    cell,
+                    delta=delta,
+                    dtype=dtype,
+                    channels={
+                        kwarg: [decode_digest(slabs[sibling][i], "uint64", ()) for i in rows]
+                        for kwarg, sibling in declared
+                    },
+                )
+                folded[j] = payload
+                for (kwarg, _), encoded in zip(declared, words, strict=True):
+                    sibling_slabs[kwarg][j] = encoded
+            groups[name] = folded
+            for kwarg, sibling in declared:
+                groups[sibling] = sibling_slabs[kwarg]
         out[res] = groups
     return out