feat(zarr-metadata): add optional msgspec integration module - #312
Open
d-v-b wants to merge 3 commits into
Open
Conversation
zarr_metadata.msgspec is the msgspec sibling of zarr_metadata.pydantic: field types over the core metadata models plus a dec_hook (and a make_dec_hook composer for applications with hooks of their own) so the models can be used as field types in msgspec Structs and with msgspec.json.decode / msgspec.convert. msgspec consults its hooks only for types it does not support natively, and it supports dataclasses natively — its field-by-field coercion would bypass from_json — so each field type is a runtime marker class msgspec treats as a custom type, with the core model class registered as a virtual subclass; decoded instances ARE the core classes, and static type checkers see the field types as the core classes. Raw documents route through from_json, existing instances pass through unchanged, and MetadataValidationError surfaces as msgspec.ValidationError with the loc-annotated problem messages. Serialization cannot be delegated (msgspec's value-driven encoders never consult enc_hook for dataclasses), so the canonical path stays an explicit to_json(); the module documents this and a test pins the msgspec behavior. Assisted-by: ClaudeCode:claude-fable-5
Findings from a multi-angle review of the new module, applied: - Import msgspec at module scope so the integration fails fast where msgspec is absent, mirroring zarr_metadata.pydantic; the docstring's "requires msgspec" claim was previously unenforced. - Treat unhashable annotation objects as ordinary lookup misses: they previously raised TypeError past the KeyError guard, breaking the NotImplementedError contract and preventing a composed hook from ever delegating to the wrapped hook. - Make the runtime markers non-instantiable (shared _FieldType base): a hollow marker instance would have passed msgspec's result check while having none of the core class's fields. The module docstring now also states the exported names are for annotations only. - Derive the marker registrations from the decode table and add a parity test, so a marker cannot pair with one core class in the registration and another in the decoder, and the table provably covers the exported surface. - Document the third msgspec limit (no JSON Schema for custom types without a schema_hook) and show the explicit to_json serialization path in the docs index snippet. - Share the canonical test documents between the pydantic and msgspec integration tests via tests/model/_cases.py instead of a verbatim copy, correct the no-mutable-state test to claim only what from_json guarantees, and pin the native-encode test to msgspec's message rather than the sentinel type name. Assisted-by: ClaudeCode:claude-fable-5
ci/check_changelog_entries.py requires an integer PR-number stem, which the towncrier orphan name used until the PR existed. Assisted-by: ClaudeCode:claude-fable-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Adds
zarr_metadata.msgspec, the msgspec sibling ofzarr_metadata.pydantic: an optional integration module (msgspec stays out of the core package's dependencies) so the frozen-dataclass metadata models can be used as field types inmsgspec.Structclasses and withmsgspec.json.decode/msgspec.convert.Design
msgspec consults its extension hooks only for types it does not support natively — and it supports dataclasses natively, so annotating a field with a core model class directly would engage msgspec's own field-by-field coercion and bypass
from_json. The module therefore exports:dec_hookreturns pass msgspec's result check and ARE the core classes (equality, isinstance, nesting all interoperate). Static type checkers see each field type as its core model class.dec_hook/make_dec_hook(wrapped=...): raw documents route throughfrom_json(the single source of truth for structural validation and normalization), existing model instances pass through unchanged by identity, andMetadataValidationErrorsurfaces asmsgspec.ValidationErrorcarrying the loc-annotated problem messages plus msgspec's field path.make_dec_hookcomposes with a decode hook an application already has, including custom types with unhashable annotation objects.enc_hook, deliberately: msgspec's value-driven encoders never consultenc_hookfor dataclasses, so no hook can route a model throughto_json— encoding a model directly either raises (theUNSETsentinel) or silently emits a non-canonical field dump. Serialization stays explicit viato_json(); the module docstring documents this (plus the one-custom-type-per-union and no-JSON-schema limits), and pinning tests fail loudly if msgspec's behavior ever changes. This mirrors the TypeScript port's Standard Schema interop, which is likewise validation-only.Contents
src/zarr_metadata/msgspec.py— the module, with usage docstests/model/test_msgspec_module.py— round-trips across all seven field types, instance pass-through, wrapped-hook composition, loc-annotated error surfacing, registry parity, marker non-instantiability, and pins on the msgspec behaviors the design depends on; canonical test documents are shared with the pydantic tests viatests/model/_cases.py>=0.19, matching the zarr package; suite verified against both 0.19 and 0.21)docs/api/msgspec.md, nav + index mentions, and a towncrier fragment🤖 Generated with Claude Code