Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/validation/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,10 @@ from ngff_zarr import (
`validate_structural(metadata, options=None, version=None)` runs the
image/multiscales rules. When `options` is `None` it uses `ValidateOptions()`,
i.e. `ValidationLevel.STRICT`. `version` is the OME-Zarr version the metadata
declares; the axis rules are inert for the versions that adopt the RFC-3 axis
model (see [[parity]]), so omitting it holds every store to the v0.4 axis
caps. A `ValidationError` carries `.rule` (a `SpecRule`),
declares; the axis count, type and order rules are inert for the versions that
adopt the RFC-3 axis model (see [[parity]]), so omitting it holds every store to
the v0.4 axis caps. `axis-names-unique` is never inert: RFC-3 states it and no
released schema carries it. A `ValidationError` carries `.rule` (a `SpecRule`),
`.message` (str), and `.location` (`str | None`); `str(exc)` is
`Spec rule [<rule>] violated: <message>`.

Expand Down
3 changes: 3 additions & 0 deletions docs/validation/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ finest-to-coarsest dataset ordering, OMERO channel color format, RFC 4
anatomical orientation, and HCS plate/well consistency. Most rules enforce v0.4
MUSTs; two additional rules (`zarr-format`, `ome-namespace`) enforce the v0.5
`ome`-namespace and Zarr v3 store conventions and fire only for v0.5 metadata.
The three RFC 4 orientation rules are normative from OME-Zarr 0.9.dev1: they
are inert when the caller declares an earlier version and stay on, as a
strictness choice, when no version is declared (see [[rule-reference]]).

The rules operate on the already-parsed metadata object (the `Metadata`,
`Plate`, and `Well` dataclasses in Python; their equivalents in TypeScript), so
Expand Down
19 changes: 18 additions & 1 deletion docs/validation/parity.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ for usage, see [[api]].

## The contract

Both ports must agree on five observable dimensions:
Both ports must agree on six observable dimensions:

1. **Rule identifiers** — the same `SpecRule` string values, in the same
canonical declaration/iteration order.
Expand All @@ -42,6 +42,10 @@ Both ports must agree on five observable dimensions:
inert for the versions that adopt the RFC-3 free-form axis model. Both ports
must treat exactly the same version strings as RFC-3, or the same metadata
validates in one language and not the other.
6. **The RFC-4 orientation version set** — the three orientation rules gate the
opposite way: they are normative at exactly the versions that adopt RFC-4
(and when no version is given), and inert below. Both ports must treat the
same version strings as RFC-4 for the same reason.

Because both test suites assert these facts against the **same literal
identifier list**, adding, removing, renaming, or reordering a rule — or
Expand Down Expand Up @@ -85,6 +89,15 @@ at every other. Rule 4, `axis-names-unique`, is never inert: RFC-3 *adds* it,
and ngff-zarr applies it at all versions as a strictness choice (see
[[rule-reference]]).

The versions at which the RFC-4 orientation rules are normative are pinned the
same way, as a `CANONICAL_RFC4_VERSIONS` literal:

1. `0.9.dev1`

Rules 9–11 gate the opposite way from the RFC-3 set: they are enforced at
those versions and when no version is given, and inert at every earlier
version, where RFC-4 has no normative status (see [[rule-reference]]).

## The parity tests

| Language | Test file |
Expand Down Expand Up @@ -116,6 +129,10 @@ Each suite independently locks:
`CANONICAL_RFC3_VERSIONS` and enforced at every other supported version and
when no version is given, asserted through the public orchestrator rather
than through the internal predicate.
- **RFC-4 orientation version set** — the orientation rules are enforced at
exactly the versions in `CANONICAL_RFC4_VERSIONS` (and when no version is
given) and inert at every other supported version, asserted the same way,
once per orientation rule.

Four of the fifteen rules never appear in `EXPECTED_EVALUATION_ORDER`, each for
its own reason. The two v0.5 namespacing rules (`zarr-format`, `ome-namespace`)
Expand Down
30 changes: 18 additions & 12 deletions docs/validation/rule-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,12 @@ stable, lower-kebab-case identifier (the `SpecRule` value) that is identical
across the Python and TypeScript ports — see [[parity]] for the guarantee. Most
rules enforce OME-Zarr v0.4 MUSTs; the two v0.5 namespacing rules
(`zarr-format`, `ome-namespace`) fire only for v0.5 metadata and are inert (a
no-op) for v0.4. For the conceptual background and the two validation levels,
see [[overview]]; for invocation, see [[api]].
no-op) for v0.4. The three RFC 4 orientation rules (9–11) are normative from
OME-Zarr 0.9.dev1, which incorporates RFC-4 through `ome/ngff-spec#190`: they are
inert when the caller declares 0.4, 0.5 or 0.6, where RFC 4 has no normative
Comment thread
coderabbitai[bot] marked this conversation as resolved.
status, and stay on when no version is declared — a strictness choice, like
`axis-names-unique` below 0.9.dev1. For the conceptual background and the two
validation levels, see [[overview]]; for invocation, see [[api]].

Location strings are dotted-segment, JSON-Pointer-style identifiers of the
offending metadata node, emitted byte-for-byte identically in both languages.
Expand All @@ -44,9 +48,9 @@ the `SpecRule` enum declares them and the orchestrators evaluate them.
| 6 | `global-coord-transform-after-per-level` | images/multiscales | Exactly one `scale` per dataset, and a `translation` must follow — not precede — its `scale`. | v0.4: each dataset defines exactly one scale; a translation follows its scale. | `multiscales[0].datasets[1].coordinateTransformations` |
| 7 | `dataset-order-highest-to-lowest` | images/multiscales | Datasets ordered finest → coarsest; the spatial scale must not decrease as the level index rises. | v0.4: multiscale datasets ordered from highest to lowest resolution. | `multiscales[0].datasets[2]` |
| 8 | `omero-channel-color-format` | OMERO | Each OMERO channel `color` is exactly six hexadecimal digits (RGB). | v0.4: OMERO channel color is 6 hex digits. | `multiscales[0].omero.channels[0].color` |
| 9 | `axis-orientation-anatomical-type` | RFC 4 orientation | Every declared spatial-axis `orientation` has `type` `anatomical`. | RFC 4: an orientation's `type` is `anatomical`. | `multiscales[0].axes` |
| 10| `axis-orientation-on-non-space` | RFC 4 orientation | An `orientation` is declared only on `space` axes, never on a non-spatial axis. | RFC 4: orientation applies to spatial axes only. | `multiscales[0].axes[0]` |
| 11| `axis-orientation-unique-axis` | RFC 4 orientation | No two spatial axes declare orientations describing the same anatomical axis. | RFC 4: each spatial axis describes a distinct anatomical axis. | `multiscales[0].axes` |
| 9 | `axis-orientation-anatomical-type` | RFC 4 orientation | Every declared spatial-axis `orientation` has `type` `anatomical`. Inert below 0.9.dev1 when the caller declares a version; kept on when none is given. | RFC 4 (normative from 0.9.dev1): an orientation's `type` is `anatomical`. No released spec below 0.9.dev1 adopts RFC 4, so enforcement without a declared version is a strictness choice, not a spec MUST of those versions. | `multiscales[0].axes` |
| 10| `axis-orientation-on-non-space` | RFC 4 orientation | An `orientation` is declared only on `space` axes, never on a non-spatial axis. Inert below 0.9.dev1 when the caller declares a version; kept on when none is given. | RFC 4 (normative from 0.9.dev1): orientation applies to spatial axes only. Same below-0.9.dev1 status as rule 9. | `multiscales[0].axes[0]` |
| 11| `axis-orientation-unique-axis` | RFC 4 orientation | No two spatial axes declare orientations describing the same anatomical axis. Inert below 0.9.dev1 when the caller declares a version; kept on when none is given. | RFC 4 (normative from 0.9.dev1): each spatial axis describes a distinct anatomical axis. Same below-0.9.dev1 status as rule 9. | `multiscales[0].axes` |
| 12| `zarr-format` | images/multiscales (v0.5) | A v0.5 entry implies a Zarr v3 store; a `zarr_format` value that leaked into the entry must be exactly `3`. Inert for v0.4. | v0.5: metadata is backed by a Zarr v3 store (`zarr_format == 3`). | `multiscales[0]` |
| 13| `ome-namespace` | images/multiscales (v0.5) | A v0.5 entry must not retain a group-level `ome` or `multiscales` wrapper key — the `ome` namespace wraps the group attributes, not each entry. Inert for v0.4. | v0.5: multiscales live under the top-level `ome` namespace, with `version` hoisted to `ome.version`. | `multiscales[0]` |
| 14| `plate-row-index-consistency` | HCS plate | Each well's `path` is `<row>/<column>`, naming declared row/column entries, with `rowIndex`/`columnIndex` equal to those entries' positions. | v0.4: well `rowIndex`/`columnIndex` match the named row/column positions in `plate.rows`/`plate.columns`. | `plate.wells[3]` |
Expand All @@ -65,11 +69,12 @@ rules:
class-ordering then spatial-name ordering for `axis-order`; per-dataset
scale-count then transform-ordering for
`global-coord-transform-after-per-level`. The v0.5 namespacing rules (12 and
13) run last and are inert for v0.4 input. The orientation rules
`axis-orientation-on-non-space` (10) and `axis-orientation-unique-axis` (11)
fire only for specific axis shapes, so the linear fail-fast cascade for a v0.4
metadata is an 11-step sequence ending at
`axis-orientation-anatomical-type`.
13) run last and are inert for v0.4 input, and the orientation rules (9–11)
are inert when the caller declares a version below 0.9.dev1. The orientation
rules `axis-orientation-on-non-space` (10) and `axis-orientation-unique-axis`
(11) fire only for specific axis shapes, so the linear fail-fast cascade for
a v0.4-shaped metadata validated with no declared version is an 11-step
sequence ending at `axis-orientation-anatomical-type`.
- **`validate_plate` / `validatePlate`** evaluates rule **14**
(`plate-row-index-consistency`).
- **`validate_well` / `validateWell`** evaluates rule **15**
Expand All @@ -96,8 +101,9 @@ TypeScript by checking the target version in `axisViews`.
`global-coord-transform-after-per-level`,
`dataset-order-highest-to-lowest`.
- **OMERO** — `omero-channel-color-format`.
- **RFC 4 orientation** — `axis-orientation-anatomical-type`,
`axis-orientation-on-non-space`, `axis-orientation-unique-axis`.
- **RFC 4 orientation** (normative from 0.9.dev1) —
`axis-orientation-anatomical-type`, `axis-orientation-on-non-space`,
`axis-orientation-unique-axis`.
- **v0.5 namespacing** — `zarr-format`, `ome-namespace` (inert for v0.4).
- **HCS plate / well** — `plate-row-index-consistency`,
`well-acquisition-missing`.
44 changes: 37 additions & 7 deletions py/ngff_zarr/structural_validation.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,13 @@
# each OMERO channel color is exactly 6 hex digits
# e.g. multiscales[0].omero.channels[0].color
# axis-orientation-anatomical-type
# each RFC 4 orientation type is "anatomical" (the only value defined)
# each RFC 4 orientation type is "anatomical"; normative from 0.9.dev1
# e.g. multiscales[0].axes
# axis-orientation-on-non-space
# orientation is declared only on spatial axes (never time/channel)
# orientation only on spatial axes (never time/channel); from 0.9.dev1
# e.g. multiscales[0].axes
# axis-orientation-unique-axis
# at most one direction per anatomical axis (antonyms are exclusive)
# one direction per anatomical axis (antonyms exclusive); from 0.9.dev1
# e.g. multiscales[0].axes
# zarr-format
# a v0.5 entry implies a Zarr v3 store (a leaked zarr_format must be 3)
Expand Down Expand Up @@ -367,6 +367,24 @@ def is_rfc3_axis_model_allowed(version: object | None = None) -> bool:
return version is not None and version == NgffVersion.V09dev1


def is_rfc4_orientation_enforced(version: object | None = None) -> bool:
"""Whether ``version`` makes the RFC-4 orientation rules normative.

Only OME-Zarr ``0.9.dev1`` does (``ome/ngff-spec#190`` folds RFC-4 into
it): the released 0.4, 0.5 and 0.6 specs give ``orientation`` no normative
status, so when the caller declares one of those versions the orientation
rules are inert. ``None`` keeps them on: with no declared version,
enforcement is a strictness choice, exactly like ``axis-names-unique``
below 0.9.dev1 (see docs/validation/rule-reference.md).

The gate points the opposite way from :func:`is_rfc3_axis_model_allowed`:
at 0.9.dev1 RFC-3 *lifts* the axis restrictions while RFC-4 *adds* the
orientation requirements, so the rules exit early below 0.9.dev1 rather
than at it.
"""
return version is None or version == NgffVersion.V09dev1


def validate_axis_count(metadata: Metadata, version: object | None = None) -> None:
"""Validate that the axis count is within the v0.4-permitted range.

Expand Down Expand Up @@ -752,7 +770,9 @@ def _axis_to_validation_dict(axis: Axis) -> dict[str, Any]:
return axis_dict


def validate_axis_orientation(metadata: Metadata) -> None:
def validate_axis_orientation(
metadata: Metadata, version: object | None = None
) -> None:
"""Validate RFC 4 anatomical-orientation metadata on the spatial axes.

This rule does not reimplement RFC 4; it wraps the package's existing logic
Expand All @@ -766,6 +786,10 @@ def validate_axis_orientation(metadata: Metadata) -> None:
no-op and the comparatively heavy ``jsonschema`` import inside
:func:`validate_rfc4_orientation` is never triggered.

Inert when ``version`` declares a release below 0.9.dev1, where RFC-4 has
no normative status; ``None`` keeps the checks on (see
:func:`is_rfc4_orientation_enforced`).

Raises
------
ValidationError
Expand All @@ -779,6 +803,8 @@ def validate_axis_orientation(metadata: Metadata) -> None:
Propagated unchanged when an orientation value is outside the RFC 4
vocabulary -- a schema-level concern with no dedicated structural rule.
"""
if not is_rfc4_orientation_enforced(version):
return
from .rfc4_validation import (
has_any_rfc4_orientation,
validate_rfc4_orientation,
Expand Down Expand Up @@ -1050,7 +1076,10 @@ def validate_structural(
version:
The OME-Zarr version the metadata declares. Rules 1-3 are inert for the
versions that adopt the RFC-3 axis model, so omitting it holds every
store to the v0.4 axis caps.
store to the v0.4 axis caps. The RFC 4 orientation checks
(:func:`validate_axis_orientation`) gate the opposite way: they are
normative from 0.9.dev1, inert when an earlier version is declared,
and kept on when the version is omitted.

Raises
------
Expand All @@ -1063,7 +1092,8 @@ def validate_structural(
-----
This orchestrator covers the image/multiscales rules, including the RFC 4
anatomical-orientation checks (:func:`validate_axis_orientation`, a no-op
when no axis declares orientation). The HCS plate/well structural rules
when no axis declares orientation and inert when ``version`` declares a
release below 0.9.dev1). The HCS plate/well structural rules
operate on separate metadata objects and are dispatched by the companion
:func:`validate_plate` and :func:`validate_well` entry points.
"""
Expand All @@ -1082,7 +1112,7 @@ def validate_structural(
validate_transform_order(metadata)
validate_dataset_order(metadata)
validate_omero_color_hex(metadata)
validate_axis_orientation(metadata)
validate_axis_orientation(metadata, version)
validate_zarr_format_for_version(metadata)
validate_ome_namespace(metadata)

Expand Down
34 changes: 12 additions & 22 deletions py/ngff_zarr/v04/zarr_metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -441,11 +441,7 @@ def _from_zarr_attrs(

from .._zarrista_utils import open_lazy_array
from ..ngff_image import NgffImage
from ..parse_metadata import _parse_omero, _raw_axes
from ..rfc4_validation import (
has_any_rfc4_orientation,
validate_rfc4_orientation,
)
from ..parse_metadata import _parse_omero
from ..validate import validate as validate_ngff

# Validate structure before any processing to avoid cryptic KeyError
Expand Down Expand Up @@ -478,17 +474,6 @@ def _from_zarr_attrs(
else:
validate_ngff(root_attrs, version=schema_version)

# RFC 4 validation for anatomical orientation. The axes are read
# through the shared helper, which knows where each version keeps
# them, and a non-dict axis entry is left to the schema check.
axes_dicts = [
axis
for axis in _raw_axes(root_attrs["multiscales"][0])
if isinstance(axis, dict)
]
if axes_dicts and has_any_rfc4_orientation(axes_dicts):
validate_rfc4_orientation(axes_dicts)

omero = _parse_omero(root_attrs.get("omero"))
# OME-Zarr v0.5 hoists the spec ``version`` to the group-level ``ome``
# namespace; v0.4 carries it on each multiscale entry. Capture the
Expand Down Expand Up @@ -619,11 +604,14 @@ def _from_zarr_attrs(

if validate:
# Strict structural validation of the parsed Metadata, layered after
# the schema pass and RFC 4 dict check above. The structural rules
# encode the v0.4+ metadata model (typed axes and per-dataset
# coordinate transformations); the legacy v0.1-0.3 layouts predate
# it, so the rules are scoped to v0.4 and newer. Imported lazily so
# the default validate=False read path incurs no extra import cost.
# the schema pass above. The structural rules encode the v0.4+
# metadata model (typed axes and per-dataset coordinate
# transformations); the legacy v0.1-0.3 layouts predate it, so the
# rules are scoped to v0.4 and newer. Imported lazily so the
# default validate=False read path incurs no extra import cost.
# The declared version is passed through so the version-gated rules
# (the RFC-3 axis rules and the RFC 4 orientation checks) apply
# exactly the requirements of the store's own version.
from ..structural_validation import (
ValidateOptions,
ValidationLevel,
Expand All @@ -633,7 +621,9 @@ def _from_zarr_attrs(
spec_version = packaging.version.parse(str(metadata.version))
if spec_version >= packaging.version.parse("0.4"):
validate_structural(
metadata, ValidateOptions(level=ValidationLevel.STRICT)
metadata,
ValidateOptions(level=ValidationLevel.STRICT),
version=metadata.version,
)

return metadata, images
Expand Down
18 changes: 1 addition & 17 deletions py/ngff_zarr/v06/zarr_metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -731,11 +731,7 @@ def _from_zarr_attrs(

from .._zarrista_utils import open_lazy_array
from ..ngff_image import NgffImage
from ..parse_metadata import _parse_omero, _raw_axes
from ..rfc4_validation import (
has_any_rfc4_orientation,
validate_rfc4_orientation,
)
from ..parse_metadata import _parse_omero
from ..validate import validate as validate_ngff

# make sure root_attrs['ome]['multiscales'] exists
Expand Down Expand Up @@ -778,18 +774,6 @@ def _from_zarr_attrs(
schema_version = V06_ONDISK_VERSION.value
validate_ngff(schema_attrs, version=schema_version)

# RFC 4 validation for anatomical orientation. From v0.6 the axes
# live in the intrinsic coordinate system, so they are read through
# the shared helper rather than from a flat ``axes`` key, which a
# v0.6 entry does not carry.
axes_dicts = [
axis
for axis in _raw_axes(root_attrs["ome"]["multiscales"][0])
if isinstance(axis, dict)
]
if axes_dicts and has_any_rfc4_orientation(axes_dicts):
validate_rfc4_orientation(axes_dicts)

omero = _parse_omero(root_attrs.get("ome", {}).get("omero"))
root_attrs = root_attrs["ome"]["multiscales"][0]

Expand Down
Loading
Loading