Raster time axis encodes as toc words (spec §8, S2 first)

deployment/aws/lambda_handler.py

@@ -81,7 +81,11 @@
         block for the manifest, sourced from the ShardMap metadata by the
         orchestrator (matching the local dispatcher). Absent on flat runs.
     "times_us": [int, ...] (raster only, issue #264) -- the catalog-derived
-        time coordinate, int64 microseconds since the epoch; the orchestrator
+        time coordinate, in whatever encoding "config" declares: int64
+        microseconds since the Unix epoch by default, or uint64 mortie toc
+        words under output.time_encoding: toc (spec §8, issue #443). Plain
+        ints either way; the worker re-derives the dtype from the config, so
+        the key name is historical and NOT a width claim. The orchestrator
         owns the global timestep index and threads it here so the template
         write needs no S3 access from the dispatcher.
     "run_manifest": dict (optional, issue #327) -- {"run_id", "shards"
@@ -838,10 +842,13 @@ def _handle_setup(event: Dict[str, Any]) -> Dict[str, Any]:
             import numpy as np
 
             from zagg.processing.raster import emit_raster_template
+            from zagg.time_axis import time_axis_dtype, time_encoding
 
             store = open_store(event["store_path"], **_output_store_kwargs(event))
             grid = from_config(config)
-            times_us = np.asarray(event["times_us"], dtype=np.int64)
+            # The wire carries plain ints; the cast is the config's declared
+            # time encoding (spec §8) — a toc word does not fit int64.
+            times_us = np.asarray(event["times_us"], dtype=time_axis_dtype(time_encoding(config)))
             if times_us.size == 0:
                 # A zero-timestep template is degenerate: the arrays get a
                 # 0-length time axis no worker can slab-write into.

docs/hive_layout.md

@@ -905,7 +905,10 @@ cap. The honest options today are:
 Raster (pull-NN) pipelines write the same tree with **windowed `(time, cells)`
 leaves**: one vanilla zarr v3 leaf per **(shard, window)** unit at
 `shard_leaf_path(root, shard, window=label)`, each carrying leaf-local `time`
-(int64 microseconds, CF attrs) and `morton` (packed u64 words) as the sole cell
+(int64 microseconds with CF attrs by default, or `uint64` mortie toc words
+carrying the spec §8 `temporal` declaration and no CF attrs under
+`output.time_encoding: toc` — which the shipped Sentinel-2 config sets, issue
+#443) and `morton` (packed u64 words) as the sole cell
 coordinate — `cell_ids` (NESTED) rides only the `emit_cell_ids` transition hatch
 (issue #304) — plus one
 `(T_leaf, cells_per_shard)` array per configured band, chunked

docs/specification.md

@@ -2,8 +2,8 @@
 
 This page is the **normative record** of zagg's byte-level store conventions:
 the ragged vlen-bytes layout, the t-digest payload bytes, the packed
-composition word, the pyramid/overview declarations, and the O11 content-hash
-recipe. It exists so an independent reader
+composition word, the pyramid/overview declarations, the O11 content-hash
+recipe, and the temporal declaration on a time coordinate. It exists so an independent reader
 ([moczarr](https://github.com/espg/moczarr)) can decode a zagg store from this
 page and the committed conformance fixtures alone — no zagg import, no
 reverse-engineering of `grids/base.py`
@@ -55,7 +55,7 @@ text marked *informative* explains or motivates and binds nothing.
   explicit succession clause; readers add revisions, they never drop them.
 - The committed conformance fixtures (§7) are part of the contract: a reader
   implementation that reproduces the fixtures' expected decoded values and
-  content hashes conforms to §1–§3 and §5. zagg's own test suite asserts the
+  content hashes conforms to §1–§3, §5 and §8. zagg's own test suite asserts the
   same expectations (`tests/test_spec_conformance.py`), so the spec, the
   fixtures, and the shipping reader cannot drift apart silently.
 
@@ -68,6 +68,7 @@ Contents:
 5. [O11 content hashes](#5-o11-content-hashes)
 6. [`zagg-ragged/2` — the typed `vlen-ndarray` revision](#6-zagg-ragged2)
 7. [Conformance fixtures](#7-conformance-fixtures)
+8. [`zagg-toc/1` — the temporal declaration](#8-zagg-toc1)
 
 ---
 
@@ -1397,19 +1398,20 @@ stores may deliberately stay `/1` for vanilla-zarr openability).
 **Status: contract.** The committed stores under
 [`tests/data/spec/`](https://github.com/englacial/zagg/tree/main/tests/data/spec)
 are part of this specification: a reader implementation that reproduces
-their expected decoded values and content hashes conforms to §1–§3 and §5.
-They are generated by
+their expected decoded values and content hashes conforms to §1–§3, §5 and
+§8. They are generated by
 [`tools/generate_spec_fixtures.py`](https://github.com/englacial/zagg/blob/main/tools/generate_spec_fixtures.py)
 through zagg's **production write path** (manifest, sharded leaf template,
 dense + ragged writes, coverage sidecar, commit stamp), so writer↔spec
 drift fails zagg's own suite (`tests/test_spec_conformance.py`) on
 whichever side moved. moczarr vendors the same fixtures for its parity
 gates (espg/moczarr#19/#20).
 
-Four tiny single-shard hive stores plus one manifest-only declaration, all
+Five tiny single-shard hive stores plus one manifest-only declaration, all
 on the same deliberately small geometry — shard order 4, inner-chunk order
 5, cell order 6 (16 cells, K = 4 inner chunks of 4 cells), sharded (the
-hive default):
+hive default; `raster_toc/` is the one exception — a `(time, cells)`
+product is never sharded, §8/#247):
 
 - **`minimal/`** — one *unlocated* digest field (`h_tdigest`) plus `count`.
   The smallest thing that is a conforming store.
@@ -1461,6 +1463,20 @@ hive default):
   arrives with the sweep-side fixtures of
   [#384](https://github.com/englacial/zagg/issues/384).
 
+- **`raster_toc/`** — the §8 temporal declaration surface: one raster
+  `(time, cells)` hive leaf whose `time` coordinate is `uint64` toc words
+  carrying `temporal: {"spec": "zagg-toc/1", "shape": "coordinate", …}` and no CF
+  `units`/`calendar` attrs, beside `morton` and two band arrays. Its axis
+  mixes both word variants deliberately — one single-item timestep encoded
+  as an exact **timestamp** word and two multi-item acquisition groups
+  encoded as **range** words — so a reader that implements only one variant
+  fails a §7 fixture. `raster_toc.expected.json` records the words as
+  decimal strings (JSON numbers cannot carry `uint64` faithfully) together
+  with the `(start, end)` nanoseconds a conforming decode yields and the
+  real acquisition spans they must contain, which is the §8 conservative
+  containment claim pinned on committed bytes. The other four fixtures,
+  which carry no `temporal` key anywhere, are the absent-key ⇒ legacy pin.
+
 `minimal/` and `kitchen_sink/` pin the layout edge cases a reader must
 handle (`column/`'s leaf is `minimal/`'s, so it pins them again): inner chunk
 ordinal 2 is **empty** (absent from the shard index — the §1.5 sentinel, and
@@ -1506,9 +1522,9 @@ itself on both sides, which is also the only mechanism that catches a future
 zagg↔moczarr divergence (neither side's fixture can: espg/moczarr#23).
 
 **Conformance criteria for an external reader**: decode every ragged array
-per §1–§2 and the composition array per §3, reproducing the expected
-decoded values exactly (byte-exact float32/uint64 — no tolerance), and
-reproduce `content_hashes` per §5. zagg's own suite additionally decodes
+per §1–§2, the composition array per §3, and the declared time coordinate
+per §8, reproducing the expected decoded values exactly (byte-exact
+float32/uint64 — no tolerance), and reproduce `content_hashes` per §5. zagg's own suite additionally decodes
 the shard objects with **spec-text-only** decoders (struct + zstd, no zagg
 read path) to prove the byte recipes in §1.4/§1.5 are sufficient on their
 own.
@@ -1521,3 +1537,175 @@ principle as §5). In the committed D20 sidecar
 `cells_with_data` are pinned: `timestamp`, `zagg_version`, `run_id` and the
 run counters are **informative provenance**, they churn on every
 regeneration, and conformance never asserts them.
+
+---
+
+## 8. `zagg-toc/1`
+
+**Status: contract** ([issue #443](https://github.com/englacial/zagg/issues/443)
+— the first shape of the temporal series,
+[#410](https://github.com/englacial/zagg/issues/410)).
+
+An array whose elements are **packed words** rather than self-describing
+scalars carries a **word-typed coordinate declaration**: a spec-owned attrs
+block, keyed by its domain, holding `{spec, shape, grammar}` — the
+convention revision, *where the words sit* relative to the store's cells,
+and *which word grammar* the values follow. The key is spec-owned: the
+writer stamps it, never author-transcribed (§1.2's reserved-key discipline,
+extended to coordinate arrays). The `shape` vocabulary is domain-neutral and
+defined once, here:
+
+- **`"coordinate"`** — the declaring array **is** the coordinate variable of
+  a dimension (CF/xarray sense): one word per index along that dimension,
+  row-aligned with every array sharing it.
+- **`"per-cell"`** — one word per cell of the store's cell grid, aligned
+  with the `morton` coordinate.
+- **`"per-centroid"`** — one word per centroid inside a cell's ragged
+  payload, aligned element-for-element with the §1/§2 digest it accompanies.
+
+This section instantiates that pattern for the **temporal** domain: attrs key
+`temporal`, `spec: "zagg-toc/1"`, `grammar: "mortie-toc/1"`.
+
+**An absent `temporal` key MUST be read as the legacy encoding** — signed
+`int64` microseconds since `1970-01-01T00:00:00` UTC, self-described by the
+CF `units`/`calendar` attrs the writer stamps beside it. Every store written
+before this revision is conformant verbatim, no byte rewritten, and a reader
+MUST NOT refuse a store for lacking the declaration.
+
+```json
+"temporal": {
+  "spec": "zagg-toc/1",
+  "shape": "coordinate",
+  "grammar": "mortie-toc/1"
+}
+```
+
+Three keys, and deliberately no more —
+[ruled on #410](https://github.com/englacial/zagg/issues/410#issuecomment-5310533396):
+the declaration is `{shape, grammar revision}` under the `spec` marker, with
+**no per-store epoch or quantization guards**.
+
+- **`spec`** — the convention revision. Readers MUST strict-check it: an
+  unknown or future revision raises, never half-parses under a guessed
+  layout.
+- **`shape`** — the vocabulary value above. **`"coordinate"`** is the only
+  one this revision defines for `temporal`: the declaring array **is** the
+  time coordinate of a `(time, cells)` product — one word per timestep,
+  row-aligned with the leading axis of every `(time, cells)` array in the
+  same group. A reader MUST refuse a `shape` it does not implement.
+- **`grammar`** — the word grammar the values follow, named as a **grammar
+  revision** in the `{name}/{major}` style this document uses throughout
+  (`zagg-ragged/1`, `morton-hive/2`). It is a fixed token of this revision —
+  not a documentation URL, and not a stamp of the writer's installed mortie
+  — so a store's bytes move neither when a dependency floor moves nor when
+  the documentation that describes the grammar moves. The pointer to that
+  documentation is prose (below), which is where it can be updated freely. A
+  reader MUST refuse a `grammar` it does not implement, and SHOULD record it
+  as what it decoded against.
+
+**Forward instantiations** (informative, both
+[#410](https://github.com/englacial/zagg/issues/410)). The temporal
+companions — one word per cell, one word per centroid — arrive as the
+`"per-cell"` and `"per-centroid"` shapes under *this* `spec` and *this*
+grammar. The **located** (spatial) companion family declares under the same
+pattern with `grammar: "mortie-morton/1"`, landing with the #410 kernel PR.
+Neither re-declares the store's **primary** morton axis: that surface stays
+declared by the `morton-hive/{1,2}` manifest grammar (`morton_hive.json`,
+`docs/hive_layout.md`) and the store's [DGGS-convention](https://github.com/zarr-conventions/dggs)
+`dggs` attrs (`docs/morton_arrow.md`). This pattern is for word-typed
+coordinate and companion arrays *beyond* that primary surface, never a
+second, competing declaration of it.
+
+**The epoch, the timescale, and the range variant's rounding quanta are
+properties of the cited grammar and are deliberately NOT echoed here.** A
+store cannot re-base them — a differing origin would be a different grammar,
+which is precisely what `grammar` (and `spec` above it) already discriminates
+— so a per-store copy could only ever restate a constant, while entering the
+committed conformance bytes and the §5 content hash and forcing a fixture
+regeneration for any upstream constant that changed.
+
+A writer MAY add further **informative** keys to the block (source-time
+lineage and the like). They are non-normative: nothing in §8 is decoded from
+them, and a reader MUST ignore keys it does not recognize rather than refuse
+the store.
+
+### The word grammar is mortie's
+
+A `zagg-toc/1` value is a mortie **toc word** (temporal order coverage): one
+`uint64` that is a tagged union of an exact nanosecond **timestamp** and an
+outward-rounded, conservative **range**, on a continuous, leap-free,
+GPS-aligned timescale with a fixed `1850-01-01T00:00:00` epoch. The bit
+layout, the flag position, the unsigned sort order, and the semilattice
+merge law are normative in mortie's
+[`mortie.toc` reference](https://espg.github.io/mortie/0.9.6/api/toc/) and
+its decision ledger
+([espg/mortie#175](https://github.com/espg/mortie/issues/175)), and are
+**not restated here**. What follows is zagg's half of the contract.
+
+The stored token is the grammar revision `mortie-toc/1`; the **URL above is
+the documentation pointer**, and it lives in this prose precisely so it can
+be re-pointed without moving a store byte. It is **release-pinned
+deliberately**: mortie's documentation is `mike`-versioned and the
+unversioned `/api/toc/` path was never published, so only a versioned URL
+resolves; the pin is `0.9.6`, the earliest release carrying the reference,
+and the page's normative words are unchanged through the current dependency
+floor.
+
+**It is also not yet the same class of citation as §2.2's**, and §8 does not
+claim it is. §2.2 defers the morton word's layout to mortie's
+`docs/specification.md` §1/§4 — sections that document's §10 *"Frozen for
+1.x"* enumerates as immutable within the major version. Mortie's frozen
+specification contains **no toc section**: the grammar above is normative as
+a module reference pinned by mortie's own golden fixtures, which is a weaker
+guarantee. [espg/mortie#193](https://github.com/espg/mortie/issues/193)
+tracks adding the frozen section; when it lands, this pointer swaps to it,
+the stored `grammar` token does not move, and nothing else in §8 changes.
+
+### 8.1 `shape: "coordinate"`
+
+**Contract.**
+
+- The declaring array's element type is `uint64`, one word per timestep.
+- A timestep whose real acquisition is a single instant MUST be encoded as a
+  **timestamp** word, exact to the nanosecond; a timestep covering a real
+  interval MUST be encoded as a **range** word whose envelope conservatively
+  contains that interval. A conforming writer therefore never widens an
+  instant into a range, and never narrows a real interval into an instant.
+- **Row order is the acquisition-group order**: timesteps are ordered by the
+  group's **earliest member observation time** — the order the time axis has
+  always had, the order the `(time, cells)` slabs were indexed by, and
+  identical under both encodings, so a row assignment never drifts with the
+  encoding.
+- **Stored word order is not that key, and a reader MUST NOT assume the
+  stored words ascend.** Unsigned word order is order by the *encoded* start,
+  which is the conservative envelope start — and an envelope may begin before
+  its group's earliest member observation time, by an amount nothing in this
+  section bounds. Where it does, the word leads the row key and the stored
+  axis is materially out of order: `np.sort(words)` yields ascending
+  *envelope-start* order, which is **not guaranteed to equal row order**. In
+  particular a reader MUST NOT bisect the stored axis to resolve a time
+  window; use the overlap predicate below, which is correct regardless of
+  stored order.
+- Decoding to wall time yields **`(start, end)`** per timestep: for a
+  timestamp both bounds are its exact instant; for a range `end` is the
+  envelope's **exclusive** upper bound. A reader that must present one
+  instant per timestep SHOULD present `start`, and MUST NOT present a
+  midpoint as if it were the observation time — the midpoint of a
+  conservative envelope is not an observation.
+- Temporal window selection is the grammar's overlap predicate applied to
+  the stored words directly: it over-reports by at most one quantum at a
+  window edge and **never under-reports**, so a selection is a conservative
+  superset of the timesteps whose real acquisition intersects the window.
+
+**Composition.** Time axes compose only between **matching declarations** —
+`zagg-toc/1` with `zagg-toc/1`, legacy with legacy (an absent key is legacy
+for this rule too). A concatenation across encodings would produce an axis
+whose values mean two different things, so a reader or writer joining two
+stores' time axes MUST refuse a mismatch. This is a *join* rule only:
+reading either store on its own is always legal.
+
+**What this revision does not cover** (informative). Only
+`shape: "coordinate"` has a contract here; the `"per-cell"` and
+`"per-centroid"` temporal companions are the forward instantiations noted
+above and carry none of §8.1's clauses yet. Nothing in §8 constrains the
+`(time, cells)` band arrays themselves, which are unchanged.

src/zagg/config.py

@@ -263,6 +263,13 @@ def validate_config(config: PipelineConfig) -> None:
     # validated before the kind branch, like credentials_provider above.
     _validate_worker(config)
 
+    # The time coordinate's encoding (spec §8, issue #443) is checked on every
+    # pipeline kind, like the two blocks above: the value is only meaningful
+    # where a time coordinate exists, so declaring it elsewhere is a typo
+    # worth catching at submission rather than a knob that silently does
+    # nothing.
+    _validate_time_encoding(config)
+
     ptype = get_pipeline_type(config)
     if ptype != "spatial":
         _validate_temporal_config(config)
@@ -761,6 +768,24 @@ def _validate_temporal_config(config: PipelineConfig) -> None:
     _validate_collection_options(config)
 
 
+def _validate_time_encoding(config: PipelineConfig) -> None:
+    """Validate ``output.time_encoding`` (spec §8, issue #443).
+
+    The knob declares how the ``(time, cells)`` time coordinate is encoded —
+    legacy int64 microseconds (absent/default) or mortie toc words. It is
+    raster-only: the spatial and temporal pipelines write no time coordinate,
+    so a declaration there would be inert.
+    """
+    from zagg.time_axis import DEFAULT_TIME_ENCODING, time_encoding
+
+    encoding = time_encoding(config)
+    if encoding != DEFAULT_TIME_ENCODING and (config.data_source or {}).get("reader") != "raster":
+        raise ValueError(
+            f"output.time_encoding: {encoding!r} applies to raster (time, cells) "
+            f"products only — this pipeline writes no time coordinate (spec §8)"
+        )
+
+
 def _validate_raster_config(config: PipelineConfig) -> None:
     """Validate a raster (pull-NN) pipeline config (issue #218).