Skip to content
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
598d31a
phase 1 of issue #194
espg Aug 24, 2026
4d50dee
phase 2 of issue #194
espg Aug 24, 2026
67d52fa
phase 3 of issue #194
espg Aug 24, 2026
0c8a1a8
phase 4 of issue #194
espg Aug 24, 2026
8f11d8d
fold review: validate to_geometry before the dissolve branch (issue #…
espg Aug 24, 2026
b3a727e
fold review: name both dissolve arms in the geometry register row (is…
espg Aug 24, 2026
b072530
fold review: blame the public parameter name, not the kernel's (issue…
espg Aug 24, 2026
8f38df9
fold review: keep numpy cast errors out of the offsets refusal (issue…
espg Aug 24, 2026
ad570df
fold review: keep split_children's 0-D refusal (issue #194)
espg Aug 24, 2026
4573978
fold review: strict intake at the two array_like arrow seams (issue #…
espg Aug 24, 2026
16d43ac
fold review: extend the pre-change goldens to the toc and convert sea…
espg Aug 24, 2026
7f2c5bf
phase 5 of issue #194
espg Aug 25, 2026
01dbf36
fold review: strict intake at norm2uniq, the UNIQ producer (issue #194)
espg Aug 25, 2026
73839af
fold review: dtype-strict UNIQ intake at orders_of_uniq (issue #194)
espg Aug 25, 2026
40324c8
fold review: run the oversized-int probe only where one can hide (iss…
espg Aug 25, 2026
ca710fc
fold review: document _as_offsets' no-copy pass-through (issue #194)
espg Aug 25, 2026
a7c1154
fold review: pin what the phase-5 valid-path tests name (issue #194)
espg Aug 25, 2026
bdf42b2
fold review: record norm2mort's up-front negative refusal (issue #194)
espg Aug 25, 2026
f21eb65
fold review: capture the phase-5 entry points in the pre-change golde…
espg Aug 25, 2026
8c15e86
fold review: name the folded UNIQ surfaces in the phase-5 test class …
espg Aug 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,40 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

- **BREAKING: strict input validation family-wide — previously-accepted-and-mangled
word/offset arrays now raise** (issue #194, ruled 2026-08-24; lands ahead of
the 1.0 release). The toc module's validators (`_as_offsets` / `_as_u64`)
are hoisted to a shared home and applied at every polymorphic choke point,
retiring the batch family's silent `np.asarray(..., dtype=...)` coercions:
a **float-typed** word or offset array raises `ValueError` instead of
truncating (`2.9` no longer becomes a group boundary at 2 — the issue #185
panic-arc class), a **negative** word raises instead of wrapping into a
different — possibly valid — packed word, and a **uint64 offset ≥ 2⁶³**
raises instead of wrapping negative through the int64 cast (the PR #192
class). Every refusal names the parameter and the first offending value.
Affected entry points, by module:

| module | entry points now strict |
|---|---|
| `_moc` | `compress_moc`, `moc_to_order`, `moc_or`, `moc_and`, `moc_intersects`, `moc_minus`, `moc_xor`, `moc_min`, `moc_not`, `common_ancestor`, `split_base_cells` (both arms where polymorphic) |
| `batch` | `polygons_to_morton_mocs` and every ragged kernel behind the `offsets=` forms |
| `geometry` | `from_wkb(offsets=)`; the `to_geometry` / `to_wkb` / `to_wkt` word intake — validated at the shared seam, so **both** `dissolve` arms refuse alike (the default `dissolve=True` route through `dissolve` included) |
| `orders` | `generate_morton_children`, `clip2order`, `orders_of`, `is_point`, `infer_order_from_morton`, `validate_morton` |
| `convert` | `mort2norm`, `mort2geo`, `mort2bbox`, `mort2polygon` (and `mort2healpix` through them); the UNIQ/normed intakes `unique2parent`, `uniq2geo`, and `norm2mort`'s `normed`/`parent` (phase 5, espg ruling 2026-08-24 — `unique2parent([16.5, 20.9])` used to truncate to UNIQ 16/20 and answer `[0, 1]`; UNIQ ids stay int64-domain, so negatives keep their own `Not a valid UNIQ` refusal downstream) |

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 from Claude (review)

nit — the row's negatives clause under-claims for norm2mort. "UNIQ ids stay int64-domain, so negatives keep their own Not a valid UNIQ refusal downstream" is true of unique2parent / uniq2geo, but norm2mort took _as_u64, so its negatives are now refused up front with a new message — a behaviour change the register does not record.

Before (4900a7e body, replayed):

>>> norm2mort(np.asarray([-3], dtype=np.int64), 0, 4)
thread '<unnamed>' panicked at mortie-core/src/decimal_morton.rs:497:5:
nested index 18446744073709551613 too large for depth 4 (base 72057594037927935 > 11)
ValueError: nested index 18446744073709551613 too large for depth 4 ...

After:

ValueError: normed must be non-negative, got -3

Strictly better (it also stops the stderr panic dump), and it is genuinely part of the #185 panic-arc story this entry opens with — worth half a clause rather than being covered by a sentence that says the opposite. Something like "…negatives keep their own Not a valid UNIQ refusal downstream, while norm2mort's unsigned normed/parent now refuse them by name instead of reaching the kernel's panic".

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 from Claude

Folded in bdf42b2. The clause now reads:

UNIQ ids stay int64-domain, so negatives keep their own Not a valid UNIQ refusal downstream, while the unsigned normed/parent operands refuse them by name up front — normed must be non-negative, got -3 where norm2mort used to wrap to 18446744073709551613 and panic inside the kernel (the #185 arc again, stderr dump included), and where norm2uniq(-3, 0, 4) used to answer UNIQ 1021, a real order-3 cell in base 11, with no error anywhere downstream.

Split by operand rather than left as one sentence, since the two halves now genuinely differ: the int64-domain UNIQ ids keep the downstream domain message, the unsigned normed/parent operands refuse up front. The norm2uniq half comes from the fold at 01dbf36 (thread above), and both are pinned by test_norm2mort_refuses_negative_naming_value / test_norm2uniq_refuses_negative_naming_value.

| `buffer` | `morton_buffer`, `morton_buffer_meters` |
| `moc_object` | `Moc` / `moc` word sources and set-operation operands (float arrays remain *geometry* there, by the documented polymorphism) |

Two deliberate edges: **zero-size input of any dtype passes as a typed
empty** (an untyped `[]` is not numeric, it is empty — the ruling `Toc`
already applied to its source, now uniform; this also *loosens* the toc
functions, which previously refused `[]`), and **`split_children` keeps
accepting the signed `int64` bit-view of packed words** (the trie branches
on the decimal characteristic, whose first column *is* the sign) while
refusing floats like everything else. Valid inputs are unaffected —
byte-identity is pinned against pre-change goldens
(`mortie/tests/data/strict_validation_goldens.json`, captured at
`4900a7e`).

- **BREAKING: one polymorphic function per operation — the plural batch names
are removed** (issue #187, ruled 2026-08-19). Every scalar/batch pair now has
**one** public entry point: the input shape (or the keyword-only `offsets=`)
Expand Down
60 changes: 39 additions & 21 deletions mortie/_moc.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,19 @@
cost — and the array-first consumers keep calling them on plain ndarrays.
:class:`~mortie.moc_object.Moc` is the **object layer** over them, and every
one of its methods is a single delegation to a function on this page.

Input validation is strict family-wide (issue #194): float-typed word or offset
arrays are refused rather than truncated, negative words and past-int64 offsets
are refused rather than wrapped, and the refusal names the argument and the
offending value.
"""

import warnings

import numpy as np

from . import _rustie
from ._validate import _as_u64
from .batch import (
_common_ancestors,
_mocs_and,
Expand Down Expand Up @@ -64,7 +70,7 @@ def compress_moc(morton):
numpy.ndarray
Sorted, compacted morton indices (``uint64``).
"""
morton = np.asarray(morton, dtype=np.uint64).ravel()
morton = _as_u64(morton, "morton").ravel()
return np.asarray(_rustie.rust_moc_normalize(morton))


Expand Down Expand Up @@ -136,15 +142,20 @@ def moc_to_order(morton, order, max_cells=_FLAT_COVER_WARN_THRESHOLD, *,
If ``order`` is outside 0-29, or the estimated densified count exceeds
``max_cells``. In the ragged form, also for offsets that are
non-monotone, out of bounds, or do not exactly cover ``morton``.
Float-typed or negative ``morton`` and float or past-int64
``offsets`` are refused by name (issue #194), never silently cast.

See Also
--------
morton_coverage : flat single-order cover (post-hoc large-cover warning).
mortie.batch._mocs_to_orders : the ragged batch kernel this delegates to.
"""
if offsets is not None:
return _mocs_to_orders(morton, offsets, order, max_cells)
morton = np.asarray(morton, dtype=np.uint64).ravel()
# Name the caller-facing parameter before delegating -- the kernel's
# own pass stays as the backstop and sees uint64 (no rescan).
return _mocs_to_orders(_as_u64(morton, "morton"), offsets, order,
max_cells)
morton = _as_u64(morton, "morton").ravel()
if not 0 <= order <= 29:
raise ValueError(f"Order must be between 0 and 29, got {order}")
if max_cells is not None:
Expand Down Expand Up @@ -184,8 +195,8 @@ def moc_or(a, b):
moc_minus : difference ``a \ b``.
compress_moc : ``moc_or(a, b) == compress_moc(concatenate([a, b]))``.
"""
a = np.asarray(a, dtype=np.uint64).ravel()
b = np.asarray(b, dtype=np.uint64).ravel()
a = _as_u64(a, "a").ravel()
b = _as_u64(b, "b").ravel()
return np.asarray(_rustie.rust_moc_or(a, b))


Expand Down Expand Up @@ -225,9 +236,11 @@ def moc_and(a, b, *, offsets=None):
mortie.batch._mocs_and : the 1 x N broadcast kernel this delegates to.
"""
if offsets is not None:
return _mocs_and(a, b, offsets)
a = np.asarray(a, dtype=np.uint64).ravel()
b = np.asarray(b, dtype=np.uint64).ravel()
# Name the caller-facing parameter before delegating -- the kernel's
# own pass stays as the backstop and sees uint64 (no rescan).
return _mocs_and(_as_u64(a, "a"), _as_u64(b, "b"), offsets)
a = _as_u64(a, "a").ravel()
b = _as_u64(b, "b").ravel()
return np.asarray(_rustie.rust_moc_and(a, b))


Expand Down Expand Up @@ -273,9 +286,11 @@ def moc_intersects(a, b, *, offsets=None):
to.
"""
if offsets is not None:
return _mocs_intersect(a, b, offsets)
a = np.asarray(a, dtype=np.uint64).ravel()
b = np.asarray(b, dtype=np.uint64).ravel()
# Name the caller-facing parameter before delegating -- the kernel's
# own pass stays as the backstop and sees uint64 (no rescan).
return _mocs_intersect(_as_u64(a, "a"), _as_u64(b, "b"), offsets)
a = _as_u64(a, "a").ravel()
b = _as_u64(b, "b").ravel()
return bool(_rustie.rust_moc_intersects(a, b))


Expand All @@ -302,8 +317,8 @@ def moc_minus(a, b):
moc_or : union of two covers.
moc_and : intersection of two covers.
"""
a = np.asarray(a, dtype=np.uint64).ravel()
b = np.asarray(b, dtype=np.uint64).ravel()
a = _as_u64(a, "a").ravel()
b = _as_u64(b, "b").ravel()
return np.asarray(_rustie.rust_moc_minus(a, b))


Expand Down Expand Up @@ -334,8 +349,8 @@ def moc_xor(a, b):
moc_and : intersection of two covers.
moc_minus : difference ``a \ b`` (the directional half of ``xor``).
"""
a = np.asarray(a, dtype=np.uint64).ravel()
b = np.asarray(b, dtype=np.uint64).ravel()
a = _as_u64(a, "a").ravel()
b = _as_u64(b, "b").ravel()
return np.asarray(_rustie.rust_moc_xor(a, b))


Expand Down Expand Up @@ -403,11 +418,11 @@ def moc_not(cover, domain=None):
>>> enumerated = mortie.from_geometry(aoi, moc=True) # doctest: +SKIP
>>> gaps = mortie.moc_not(enumerated, domain=shard) # doctest: +SKIP
"""
cover = np.asarray(cover, dtype=np.uint64).ravel()
cover = _as_u64(cover, "cover").ravel()
if domain is None:
domain = _whole_sphere()
else:
domain = np.asarray(domain, dtype=np.uint64).ravel()
domain = _as_u64(domain, "domain").ravel()

if domain.size == 0:
# The complement within an empty domain is empty for any cover; the
Expand Down Expand Up @@ -471,7 +486,8 @@ def common_ancestor(morton, *, offsets=None):
(non-existent) whole-sphere root. In the ragged form the message names
the lowest-index offending group *within its kind* (layout errors are
screened in their own pass, ahead of the per-group content check), and
bad offsets raise here too.
bad offsets raise here too. Float-typed or negative ``morton`` and
float or past-int64 ``offsets`` are refused by name (issue #194).

See Also
--------
Expand All @@ -491,8 +507,10 @@ def common_ancestor(morton, *, offsets=None):
True
"""
if offsets is not None:
return _common_ancestors(morton, offsets)
morton = np.asarray(morton, dtype=np.uint64).ravel()
# Name the caller-facing parameter before delegating -- the kernel's
# own pass stays as the backstop and sees uint64 (no rescan).
return _common_ancestors(_as_u64(morton, "morton"), offsets)
morton = _as_u64(morton, "morton").ravel()
return np.uint64(_rustie.rust_moc_min(morton))


Expand Down Expand Up @@ -551,7 +569,7 @@ def split_base_cells(words, sort=False):
>>> sorted(int(np.uint64(k) >> np.uint64(60)) - 1 for k in groups)
[2, 5]
"""
words = np.asarray(words, dtype=np.uint64).ravel()
words = _as_u64(words, "words").ravel()
if words.size == 0:
return {}

Expand Down
37 changes: 3 additions & 34 deletions mortie/_toc.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,9 @@

from . import _rustie

# The family's shared strict validators (hoisted from this module, issue #194).
from ._validate import _as_offsets, _as_u64

Q_START_NS = 1 << 31
"""Start quantum: 2^31 ns (~2.15 s); a range's start code floors to this."""

Expand All @@ -71,40 +74,6 @@
(~4 s short of year 2142); the end code must fit its 31-bit field."""


def _as_u64(values, name):
"""Validate non-negative integer input and return it as uint64."""
arr = np.atleast_1d(np.asarray(values))
if arr.dtype.kind not in "iu":
raise ValueError(
f"{name} must be integer-typed, got dtype {arr.dtype}")
if arr.dtype.kind == "i" and arr.size and np.any(arr < 0):
raise ValueError(f"{name} must be non-negative")
return arr.astype(np.uint64)


def _as_offsets(offsets):
"""Validate arrow list offsets and return them as contiguous int64.

Integer-typed by the same rule :func:`_as_u64` applies to words: a float
offset array would otherwise cast silently, truncating ``2.9`` to a group
boundary at 2 rather than saying so. The same standard rules out the
``uint64`` values the cast cannot represent -- at or above ``2**63`` they
would wrap negative, and the Rust validator would then describe the
wrapped copy rather than the offset that was passed. Monotonicity and
bounds stay the Rust validator's job -- it names the offending group.
"""
arr = np.atleast_1d(np.asarray(offsets))
if arr.dtype.kind not in "iu":
raise ValueError(
f"offsets must be integer-typed, got dtype {arr.dtype}")
if arr.dtype.kind == "u" and arr.size:
too_big = arr > np.iinfo(np.int64).max
if too_big.any():
raise ValueError(
f"offsets must fit in int64, got {int(arr[too_big][0])}")
return np.ascontiguousarray(arr.astype(np.int64).ravel())


def _as_scalar_ns(value, name):
"""Validate a scalar ns argument and return it as a plain int."""
value = operator.index(value)
Expand Down
Loading
Loading