-
Notifications
You must be signed in to change notification settings - Fork 1
small fixes 2026-08-17 (2): s2_neon_o9 toc flip, CHANGELOG entry for PR #447 #455
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
ceeb9f3
43fbeed
df6714d
4f02e5c
52b2531
44bc654
c3f9010
a87320c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 | |
|
|
||
| ## [Unreleased] | ||
|
|
||
| - unindexed shard-map builds cover at `parent_order` and intersect before decoding records (#445) ([#447](https://github.com/englacial/zagg/pull/447)) by @espg | ||
| - An unpinned unindexed HEALPix mortie `swath` build now covers at the output | ||
| grid's `parent_order` instead of its chunk order, so `metadata["mortie_order"]` | ||
| in newly built manifests records the shard order (e.g. 9, not 13). Assignment | ||
| is measurably identical at the production order pairs (the | ||
| `bench/neon_order_sweep.py` invariant, verified at 555,867 granules); at | ||
| coarse grids the new default is a documented conservative superset, never a | ||
| subset. Explicit `mortie_order=` pins are unchanged. | ||
| - Unindexed builds cover from the catalog's WKB column and intersect before | ||
| materializing granule records — ~2.5x faster at clone scale (87 s -> 39 s). | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 from Claude (review) The From PR #447's measured table (555,867-granule clone, same machine): Where 2.5 comes from in #447 is its opening paragraph, and it is not a speedup at all:
i.e. old-records-path cost vs index-build-plus-indexed-query cost — a different pair of quantities than the before/after this bullet describes. Two ways out: say
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 from Claude Fixed in
Both numbers come straight from PR #447's measured table ( |
||
| - Disclosed: an explicit `mortie_order=` pin always covers live, indexed or not, | ||
| so a MultiPolygon footprint assigns as a union-of-parts superset; single-part | ||
| CMR granules are unaffected. | ||
| - rename parent_morton event field to shard_key (#24) ([#42](https://github.com/englacial/zagg/pull/42)) by @espg | ||
| - Concurrency-aware Lambda orchestrator: pre-flight probe + FD-exhaustion guard ([#41](https://github.com/englacial/zagg/pull/41)) by @espg | ||
| - drop shapely as an intersection backend (#36) ([#39](https://github.com/englacial/zagg/pull/39)) by @espg | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -43,6 +43,37 @@ def test_raster_targets_manifest_consistent(): | |
| assert get_store_layout(cfg) == "hive" | ||
|
|
||
|
|
||
| def test_s2_neon_o9_tracks_the_shipped_sentinel2_config(): | ||
| # The leg's config claims to be the shipped src/zagg/configs/sentinel2_l2a.yaml | ||
| # with the o9 dispatch shard (issue #451); hold it to that mechanically so a | ||
| # knob added to one and not the other (as time_encoding was, issue #443) | ||
| # cannot drift silently. Divergence budget: parent_order and the harness's | ||
| # pyramid opt-out -- both packaging, so the semantic hash (D19) matches. | ||
| from zagg.config import get_store_layout, load_config | ||
| from zagg.semantics import semantic_hash | ||
|
|
||
| bench = load_config(str(BENCH / "configs" / "s2_neon_o9.yaml")) | ||
| shipped = load_config(str(REPO / "src" / "zagg" / "configs" / "sentinel2_l2a.yaml")) | ||
|
|
||
| assert bench.data_source == shipped.data_source | ||
| assert bench.aggregation == shipped.aggregation | ||
| assert semantic_hash(bench) == semantic_hash(shipped) | ||
|
|
||
| def normalized(cfg): | ||
| # store is a run-local output path (the harness overrides it); grid | ||
| # indexing_scheme is descriptive-only (config.py rejects any other value). | ||
| out = {k: v for k, v in cfg.output.items() if k not in ("store", "grid", "pyramid")} | ||
| out["store_layout"] = get_store_layout(cfg) | ||
| out["grid"] = {k: v for k, v in cfg.output["grid"].items() if k != "indexing_scheme"} | ||
| return out | ||
|
|
||
| b, s = normalized(bench), normalized(shipped) | ||
| assert b["grid"].pop("parent_order") == 9 | ||
| assert s["grid"].pop("parent_order") == 11 | ||
| assert b == s | ||
| assert bench.output["pyramid"] is False | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 from Claude (review) The divergence budget is enforced one-sidedly:
Verified by mutation on this branch — a harness reproducing the test body verbatim ( The hole is not incidental: the one output key exempted from the dict comparison is the same key the docstring names as half the divergence budget ("Divergence budget: parent_order and the harness's pyramid opt-out"). The stated purpose — "a knob added to one and not the other (as time_encoding was, issue #443) cannot drift silently" — holds for every knob except this one. And One line next to 74 closes it:
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 from Claude Fixed in assert bench.output["pyramid"] is False
assert "pyramid" not in shipped.outputThe Separately in |
||
|
|
||
|
|
||
| def test_pinned_s2_catalog_carries_raster_entries(): | ||
| # The pinned catalog is the fixed granule set (offline, no STAC): 2025 | ||
| # Earth Search c1 items over the SERC box, with the raster entry fields | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤖 from Claude (review)
"Explicit
mortie_order=pins are unchanged" is contradicted by this entry's own third bullet and by PR #447's delta 6.What #447 claims unchanged is narrow — "An explicit
mortie_order=is still honored literally and still validated againstparent_order(#92)" — i.e. the resolved order. The assignment for a pinned build did move, measurably, for one population:Pinned in the merged suite at
tests/test_shardmap.py:1470(TestLiveCover::test_a_pinned_indexed_build_covers_live_too).So a reader gets "your pinned build is unaffected" on line 17 and, three lines down, "an explicit
mortie_order=pin always covers live, indexed or not, so a MultiPolygon footprint assigns as a union-of-parts superset". Both cannot be read at face value. Scoping the first to what #447 actually claims — e.g. "The order an explicitmortie_order=pin resolves to is unchanged" — leaves the third bullet to carry the assignment consequence without the entry arguing with itself.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤖 from Claude
Fixed in
4f02e5c— scoped to what #447 actually claims, so the first bullet no longer argues with the third:That leaves the assignment consequence entirely to the third bullet (delta 6 /
TestLiveCover::test_a_pinned_indexed_build_covers_live_too), which is where it belongs.