feat(assets): records travel with what needs them, open locations through adapters, the tier per asset, the guard, the AR4 out of the library

.github/workflows/ci.yml

@@ -781,6 +781,27 @@ jobs:
           rm src/mime/_planted_type_error.py
       - name: Build the wheel
         run: uv build --wheel
+      # The guard (mime assets audit): neither distribution, nor the tree a release's source archive and Zenodo's
+      # snapshot of a tag hold, carries a file whose terms let no copy be hosted (the AR4's meshes among them, by the
+      # manifest's hashes), a mesh baked from one, or a mesh with no record. A plugin's CI runs the same command.
+      - name: No file that may not be redistributed in the wheel, the sdist or the committed tree
+        run: |
+          uv build --sdist
+          git archive --format=tar -o "$RUNNER_TEMP/tree.tar" HEAD
+          uv run --no-sync mime assets audit dist/*.whl dist/*.tar.gz "$RUNNER_TEMP/tree.tar" \
+            --manifest examples/descriptions/ar4/visuals.json | tee -a "$GITHUB_STEP_SUMMARY"
+      - name: Control, a made mesh planted in a copy of the wheel fails the guard
+        run: |
+          cp dist/*.whl "$RUNNER_TEMP/planted.whl"
+          uv run --no-sync python -c "
+          import struct, sys, zipfile
+          with zipfile.ZipFile(sys.argv[1], 'a') as wheel:
+              wheel.writestr('mime/data/planted.STL', b'planted'.ljust(80, b'\0') + struct.pack('<I', 1) + bytes(50))
+          " "$RUNNER_TEMP/planted.whl"
+          if uv run --no-sync mime assets audit "$RUNNER_TEMP/planted.whl"; then
+            echo "::error::the guard passed a wheel holding a planted mesh, so it cannot fail"
+            exit 1
+          fi
       - name: A consumer's code type-checks against the installed wheel
         run: |
           uv venv "$RUNNER_TEMP/consumer" --python 3.12

.github/workflows/publish.yml

@@ -45,6 +45,15 @@ jobs:
           python -m pip install --upgrade twine
           python -m twine check dist/*
 
+      # The release check: no distribution, and not the tag's tree (the GitHub release's source archive, Zenodo's
+      # snapshot), holds a file whose terms let no copy be hosted, or a mesh with no record (mime assets audit).
+      - name: No file that may not be redistributed in the distributions or the tag's tree
+        run: |
+          python -m pip install dist/*.whl
+          git archive --format=tar -o "$RUNNER_TEMP/tree.tar" HEAD
+          mime assets audit dist/*.whl dist/*.tar.gz "$RUNNER_TEMP/tree.tar" \
+            --manifest examples/descriptions/ar4/visuals.json
+
       - uses: actions/upload-artifact@v4
         with:
           name: dist

CHANGELOG.md

@@ -8,6 +8,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Changed
+- **Asset records travel with what needs them, and the library knows how to fetch a file, never what or from where.**
+  One record type (`mime.assets.AssetRef`) serves every asset: beside its sha256, size, licence and locations it states,
+  where known, its credit, role, kind (with a derived asset's derivation record by hash), notice, custom licences,
+  version, citation and the packages of the adapters its locations need. Its tier is the verdict of its licence, never
+  written by hand. A robot's meshes are a manifest of such records beside its description (`visuals.json`), found from
+  the description in use; a lab's records come through a package's `mime.assets` entry point; MIME's own data holds its
+  own assets only. A loaded experiment keeps its pins (the lock's records, the saved stage's meshes) whatever a source
+  holds now, and its build report says where one differs; two shared sources that disagree are refused
+  (`asset.conflict`). Locations are open: `https`, `doi:`, `package:` (bytes inside an installed package), `file://`
+  mirrors, and any scheme a package's adapter adds (`mime.asset_adapters`), with the hash, the size cap, redirects, a
+  credential bound to its host, retries with backoff and resuming kept in the resolver; `mime.testing.verify_adapter`
+  is an adapter's conformance harness. A machine's locations file (`MIME_ASSET_LOCATIONS`) adds a lab's mirrors,
+  tried first, and never travels; `MIME_CA_BUNDLE` trusts an institution's CA; `mime credentials set host:<host>` keeps
+  a host's own token. `mime assets sources` shows where each asset can be fetched from now; `mime assets audit` finds,
+  in a wheel, an sdist, an archive or a tree, any file whose terms let no copy be hosted, a mesh baked from one, or a
+  mesh with no record, and runs in CI on MIME's distributions and tree. An experiment declares a one-off file by its
+  whole record, `mime.scene.asset(AssetRef(...))`.
+- **The saved form is version 6, and the bundle format 3.** The lock holds each asset's record whole and the manifests
+  of the meshes its stage draws (`visuals`), both outside the input hash; a lock of version 5 or earlier loads as it
+  did, and one that holds a key version 6 added is refused. Saving a folder again under version 6 moves its `document`
+  and `input` hashes (the schema string is inside what is hashed); its `derived` and `observations` hashes are
+  unchanged. A bundle's manifest holds each record whole; formats 2 and 1 still unpack.
+- **The AR4 left the library.** Its URDF (unchanged, byte for byte) and its meshes' manifest are in
+  `examples/descriptions/ar4/`, shipped in the sdist with the URDF's MIT licence; the manifest's notice states the
+  settled terms (the meshes are Annin Robotics' design files, whose licence forbids redistributing them in any form,
+  derived ones included). `mime visuals fetch` takes the description's folder, a manifest, or a saved folder.
 - **`exploratory` waives the inertia tensor's products, and the stamp is short.** A body whose parts give
   an inertia tensor with products of inertia (a screw's, 0.11% of its largest moment) is no longer refused in
   `exploratory`: the diagonal is used with the products dropped, and the build stamps `umr.inertia` pending with the

docs/ratchets/refusal_names.txt

@@ -6,20 +6,29 @@
 access.embargo  # a path in sharing.yaml, said by a refusal of its shape
 bath.density             # a typed path in a family's rig: the bath's density
 bulk.density             # an OpenMaterica record field a value is read from
+ca.key  # a test certificate authority's key file (mime.testing's loopback stores)
+ca.pem  # a test certificate authority's certificate file
 coils.magnetic           # a typed path in a family's rig: the coils' magnetic field
 commands.jsonl           # the command log a live session writes into its run folder (mime.live)
 doi.org  # a host doi.org answers to, compared with an address's
 dx.doi.org  # a host doi.org answers to, compared with an address's
 files.enabled  # the records API's field a draft is saved naming while its files are still to come
 fluid.dynamic_viscosity  # an OpenMaterica record field a value is read from
+locations.toml  # the user's or lab's locations file
 lubrication.epsilon      # a setting group's setting, as an entry reads it in SI
 magnetic.remanence       # an OpenMaterica record field a value is read from
+mime.asset_adapters  # the entry-point group an asset adapter is registered through
 mime.entries             # the entry-point group a load finds namespaces by
+no-locations-file.toml  # the empty locations file the adapter harness points MIME at
 pyproject.toml           # the project's metadata, whose pins the number rule reads releases from
 pytest.xfail             # the type JUnit gives an expected failure's skip, which the evidence resolver reads
 scipy.spatial            # SciPy's module, imported by its name (mime.viz._meshes)
 screw.inertia            # a value the surrogate generator's experiment declares: its body's inertia
 screw.mass               # a value the surrogate generator's experiment declares: its body's mass
+server.csr  # a test server certificate's request file
+server.ext  # a test server certificate's extensions file
+server.key  # a test server certificate's key file
+server.pem  # a test server certificate file
 source.push              # a node's field in a toy experiment
 stand.magnetic           # a typed path in a family's rig: the stand's magnetic field
 stand.magnetic_moment    # a value in a family's rig: the stand's magnet's moment

docs/recordings.md

@@ -51,7 +51,7 @@ experiment's declarations alone, the same bytes for the same declarations. For a
 - **A prim for every field a law reads at a body** (a flux density), below its apparatus: an arrow, hidden until a
   recording or a client gives it a value.
 - **What each frame draws** below it: its description's visuals (`visuals/`), a box, cylinder or sphere as its USD
-  primitive, a mesh as a prim naming its visual set and the sha256 of its source; and the simple shapes that stand in
+  primitive, a mesh as a prim naming its manifest and the sha256 of its source; and the simple shapes that stand in
   for meshes the stage does not hold (`fallback/`): a bone from the frame's origin to each child's, and a link's
   collision primitives.
 - **Drawn proxies within a triangle budget**: a geometry kind with no USD primitive (the screw) is drawn at fewer facets
@@ -66,34 +66,45 @@ current writer's stage.
 
 ## Visuals MIME does not ship
 
-Some descriptions' meshes cannot be redistributed. The AR4's are Annin Robotics' design: their licence forbids
-redistributing or hosting them in any form, and allows educational, research and personal non-commercial use. MIME
-ships a **recipe** for them, the visual set `ar4_mk5`, and never their bytes:
+Some descriptions' meshes cannot be redistributed. The AR4's are Annin Robotics' design files, under its
+non-commercial licence, which forbids redistributing or hosting them in any form, converted or decimated copies
+included, and allows educational, research and personal non-commercial use. The description's kinematics and inertial
+values are MIT. Neither distribution of MIME holds the AR4: its description folder, `examples/descriptions/ar4/` in the
+sdist and the repository, holds the URDF with its MIT licence and a **manifest** of its meshes (`visuals.json`), never
+their bytes. An experiment that reads the URDF finds the manifest beside it, so choosing the robot brings its meshes'
+records:
 
 ```sh
-mime visuals list              # each set, fetched or not, with its licence, tier and role
-mime visuals fetch ar4_mk5     # prints the credit and the terms, then downloads and bakes
-mime visuals view a0_run       # the layer that opens the run's recording with the meshes drawn
+mime visuals fetch examples/descriptions/ar4/   # prints the credit and the terms, then downloads and bakes
+mime visuals fetch a0_saved/                    # the same, from the records a saved folder's lock holds
+mime visuals list                               # each set an installed package ships, and each baked one
+mime visuals view a0_run                        # the layer that opens the run's recording with the meshes drawn
 ```
 
-- **The recipe** names the upstream repository at a pinned commit and each file by the path a description writes it
-  with, its sha256 and its git blob, with the set's licence (an SPDX `LicenseRef-`), its tier (no redistribution: the
-  bytes are referenced, never uploaded) and its role (presentation: no result depends on it).
+- **The manifest** is a named group of ordinary asset records (`mime.assets`): each file by the path a description
+  writes it with, its sha256 and its git blob, at the upstream repository at a pinned commit, with its licence (an
+  SPDX `LicenseRef-` whose custom licence says redistribution is not permitted) and its role (presentation: no result
+  depends on it). Its tier is the licence's verdict, `referenced`, never a value written by hand. A saved folder's lock
+  holds the manifests its stage draws (`visuals`), so the records travel with the folder and its bytes never do.
 - **Fetching** downloads each file from that commit through the asset cache's one resolver, held to its sha256,
   decimates it to its share of the set's triangle budget (vertex clustering; every vertex moves at most a cell's
   diagonal), measures its distance to the original from every vertex of each to the other's surface, both ways (so
   that no denser sample of vertices finds more), and writes it as a USD layer in the asset cache, under `visuals/`,
-  with a record of its source, parameters, checks and output. The record keeps both directions: a baked vertex lies
+  with a record of its source, parameters, checks and output, and its tier: a baked mesh is a derived file, and takes
+  the strictest terms of its source, so the AR4's are referenced too. The record keeps both directions: a baked vertex lies
   within the clustering cell's diagonal of the original, but an original vertex need not lie within it of the bake,
   where a feature thinner than a cell collapsed and its triangles were dropped, so that direction is measured, not
   bounded.
-- **The stage refers to a mesh by its set and its source's sha256**, never by a path into the cache, and holds none of
-  its bytes. When it is opened through `mime visuals view` or `mime.viz.open_stage`, or by a client from the overlay
+- **The stage refers to a mesh by its manifest's name and its source's sha256**, never by a path into the cache, and
+  holds none of its bytes. A saved stage's names are its pins: loaded, the experiment's stage names each mesh as it was
+  saved, whatever manifest is beside its description now, and the build report says where one names another file. When it is opened through `mime visuals view` or `mime.viz.open_stage`, or by a client from the overlay
   `hello` names, an overlay in the cache references each baked mesh into its prim and hides the bones. Without it, the
   links are drawn as bones, and `mime visuals view` and `hello` say which sets are missing and how to fetch them, with
   each set's notice (its credit and terms) whether it is drawn or not.
 - **Nothing of it is ever written into a saved folder or a run folder**, so `mime pack` cannot carry it: a test packs
   A0's saved folder and a live session's run folder with a stand-in baked in the cache, and finds none of its bytes.
+  `mime assets audit` finds a baked mesh copied anywhere (a tree, a wheel), by its hash, and `mime pack` and `mime push`
+  refuse one copied into an experiment as its author's own file (`bundle.referenced-copy`, `push.referenced-copy`).
 
 The triangle budget is a presentation setting with its basis (`mime.viz.TriangleBudget`): the earlier line's AR4
 meshes at CAD density shimmered in the viewport, their triangles smaller than its pixels, and drew slowly on a

docs/reference/mime_usd.md

@@ -54,8 +54,9 @@ plans the fixtures, which needs their assets, so a run without them does not exe
 ## What the stage draws
 
 - `mime:visual:file`: on a mesh visual of a chain's frame, the file its description names, never its bytes.
-- `mime:visual:asset`: the visual set that bakes the mesh, where one lists it, for the asset cache's overlay.
-- `mime:visual:sha256`: the hash of the mesh file's bytes, beside the visual set.
+- `mime:visual:asset`: the manifest that lists the mesh (beside the description, or in an installed package), where
+  one does, for the asset cache's overlay.
+- `mime:visual:sha256`: the hash of the mesh file's bytes, beside the manifest's name.
 - `mime:visual:role`: what the shape stands for on its frame: `visual`, `collision`, or `frames`.
 - `mime:vector:name`: on the prim of a field a law reads at a body, the field's interface name.
 - `mime:vector:value`: the field's value at the body, set by a recording or a live client, in `mime:vector:units`.

docs/specification/saved_form.md

@@ -119,6 +119,30 @@ refusal table's (`mime/build/refusal_table.txt`).
   `asset.missing`, with the command that fetches and places it; one that is there is held to its pin as every file is.
   *Cases:* `tests/sharing/test_referenced.py::test_a_folder_missing_a_referenced_file_is_refused_with_the_command_that_fetches_and_places_it`.
 
+- **[lock.records]** Each asset record of the lock (`assets`) is whole: beside the five keys every record has, each
+  key the record states (its credit, role, kind and derivation, notice, custom licences, version, citation, and the
+  package of each adapter its locations need), written only where stated, outside the input hash; its tier is never
+  written, but read from its licence. A file the experiment declares by its record is held so, and the lock names only
+  the record's own locations, never those a machine's locations file adds.
+  *Cases:* `tests/assets/test_records.py::test_a_full_record_round_trips_and_keeps_its_kind_role_and_identity`,
+  `tests/assets/test_records.py::test_control_a_record_that_writes_its_own_tier_is_refused`,
+  `tests/sharing/test_records_travel.py::test_the_lock_holds_the_declared_record_whole_and_never_this_machines_locations`.
+- **[lock.visuals]** The lock's `visuals` holds the manifest of each set of meshes the stage draws, by its name: their
+  records, never their bytes, outside the input hash.
+  *Cases:* `tests/viz/test_visual_pins.py::test_the_lock_holds_the_manifest_and_the_folder_holds_none`.
+- **[lock.previous-records]** A lock of a form before the one that added them that holds `visuals` or a key of an asset record
+  beyond the five is refused as `load.lock-shape`, never read as stating it.
+  *Cases:* `tests/saved/test_form6.py::test_an_earlier_lock_holding_a_key_version_six_added_is_refused`,
+  `tests/saved/test_form6.py::test_control_an_upgrade_that_accepts_the_keys_in_an_earlier_lock_is_caught_every_run`.
+- **[lock.pins]** The lock's records are the experiment's pins: a load reads each asset under the hash its lock names,
+  whatever a shared source holds under that name now, and the build report names each source that holds another
+  version; one not stored is refused as `asset.missing`. A saved stage's meshes keep the manifest and sha256 it names
+  likewise.
+  *Cases:* `tests/saved/test_lock_assets.py::test_the_pin_wins_a_lock_naming_the_table_under_another_hash_reads_that_hash`,
+  `tests/saved/test_lock_assets.py::test_a_shared_source_holding_another_version_of_the_table_is_reported_and_the_pin_kept`,
+  `tests/viz/test_visual_pins.py::test_the_saved_arm_loads_with_its_stage_rebuilt_from_its_pins`,
+  `tests/viz/test_visual_pins.py::test_control_a_rebuild_that_ignores_the_saved_stages_pins_is_refused_every_run`.
+
 ## The files the lock pins
 
 - **[files.reference]** Reference outputs that cannot be read are refused as `load.reference`.
@@ -136,10 +160,11 @@ refusal table's (`mime/build/refusal_table.txt`).
   `load.record-changed`.
   *Cases:* `tests/materials/test_pins.py::test_a_saved_record_file_edited_by_one_byte_is_refused_at_load`,
   `tests/materials/test_pins.py::test_a_record_file_missing_from_the_folder_is_refused_at_load`.
-- **[files.remote-asset]** A remote asset the build read must be named in the lock under the contents it was read
-  with; a lock without it, or naming it under another hash, is refused as `load.assets`.
+- **[files.remote-asset]** A remote asset the build read must be named in the lock; a lock without it is refused as
+  `load.assets`. A lock naming it under another hash is the experiment's pin, which the rebuild reads
+  (`lock.pins`).
   *Cases:* `tests/saved/test_lock_assets.py::test_control_a_lock_without_the_tables_name_is_refused_when_loaded_naming_it`,
-  `tests/saved/test_lock_assets.py::test_control_a_lock_naming_the_table_under_another_hash_is_refused_when_loaded`.
+  `tests/saved/test_lock_assets.py::test_the_pin_wins_a_lock_naming_the_table_under_another_hash_reads_that_hash`.
 
 ## Rebuilding the experiment