docs: split the purity contract out of file semantics; digest CompletedProcess

[pmrv]

Split out of #828 per review. Answers the four inline review threads on #797; #828 keeps the SshCache × Path work. Independent of #828 — no shared hunks, either can land first.

"feels almost more like it should go into a more generic section rather than file specific"

You were right, and it's stronger than "feels like": I checked, and the rules aren't path-specific at all. A mutated list argument behaves identically —

@fleche
def append_and_report(xs, n):
    xs.append(n)
    return len(xs)

a = [1, 2]; append_and_report(a, 9)   # 3 — body runs, a is now [1, 2, 9]
b = [1, 2]; append_and_report(b, 9)   # 3 — cache hit, b is still [1, 2]

— and a mutated argument that is returned comes back post-mutation. So there's now a usage/purity page carrying the general contract (keyed as passed, mutation not replayed, other side effects cold-call-only, plus the None-not-cached tie-in), and file_semantics keeps only the path instance and links out. Wired into the "Using Fleche" toctree right after tldr.

"cross ref destructuring here, since that's the key"

Done, and I reframed the section around it rather than appending a link. Only destructured children reach the path machinery, so _DESTRUCTURERS is the list of places a nested path gets content treatment — one fact that explains the container list, the exact-types caveat, and the "opaque containers store paths by location" section below it. Links :ref:extending-destructurer``.

"move the completedprocess digester into fleche itself"

Now a match arm in digest.py — args, returncode, stdout, stderr. run()'s result is neither a dataclass nor iterable, so it was Indigestible and every shell-wrapping function needed the same hand-rolled hook.

Purely additive: those values raised before, so no stored digest changes and no hash_version bump. The type name is already mixed in upstream of the match, so it salts like every other arm.

"save executed version"

Files.ipynb committed executed — same command rendernb.yml runs — and its add_hook cell is gone now that the digest is built in, so the pipeline section demonstrates the built-in path (second run fully cached, no [shell] prints).

While in there: PathsInContainers.ipynb had been added with no docs/notebooks/ symlink, no toctree entry, and no slot in test_notebooks.py, so it wasn't in the docs or under test at all. Wired up all three and committed it executed too.

Review round 2 (62a2c3c)

• Argument mutation framing. Leading with "treat received paths as read-only" made returning-what-you-wrote read as the fallback, when it's the normal shape — and the one the shell pipeline in Files.ipynb uses. Flipped: returning leads and is stated as intended; read-only now scopes to paths that are pure input.
• "any depth" vs remaining_depth. Verified rather than asserted, and the docs now say why: a Path matches no destructurer, so _intern_rec leaves its depth at float("inf"), inf < remaining_depth is never true, and it is therefore always written out as its own entry — which is exactly where PathValueMixin intercepts. remaining_depth only tunes inline-vs-separate for destructurable nodes. Checked across remaining_depth ∈ {0, 1, 3, 10} × nesting depth ∈ {0, 1, 2, 3, 5, 8}, original deleted before load: 24/24 by content.
• Files.ipynb query cell was generic introspection, not path-specific. Now prints the value store's own view of a stored path — content bytes under their digest plus the FileBlob/DirectoryBlob naming records — landing right after the dedup cell, where left.txt and right.txt visibly share one content digest.
• PathsInContainers.ipynb framing. The six "Edge N" sections read as a defect list awaiting fixes. Three aren't defects — location changes, aliasing loss, per-hit copies follow from caching a pure function by value — and are now Beware: sections that say so. The genuinely sharp ones (Path dict keys, opaque containers, container subclasses) are Caveat:. Intro and summary draw the line.
• Dropped the explanatory comment on the CompletedProcess digest arm.

Tests

tests/unit/digest/test_digest.py — CompletedProcess field-by-field discrimination (args / returncode / stdout / stderr), not-digest-equal to its own field tuple, text-mode str and uncaptured None streams, real subprocess.run results, and nesting in a tuple/dict. tests/integration/test_notebooks.py gains PathsInContainers.ipynb.

1763 passed / 11 skipped. Docs build succeeds; only non-autoapi warning is the pre-existing fleche.D duplicate (I fixed a short title underline in usage/helpers that surfaced once the build got that far).

ty check src/ reports 3 diagnostics — all of them already on temppath, none added here. (An earlier revision of this description said 4; that was an unpinned local ty, see the comment.) ty is red on the base branch and green on main; that thread has the diagnosis and a verified fix, pending your call on where it should land.

[pmrv]

ty went red here, but it's not this PR — ty is red on temppath itself, and this PR adds zero diagnostics.

Reproduced locally with CI's pinned ty == 0.0.62 in a .[tests,ty] env, ty check src/:

ref	result
main	All checks passed!
temppath (base)	3 diagnostics
this PR (97d23e8)	the same 3, same lines
#828 (c047dd3)	the same 3

The three, and where they came from — both are your commits on temppath, neither is on main:

• storage/destructuring.py:133 and :146 — too-many-positional-arguments. type(self.items)(...) / type(value)(zip(...)): ty narrows the constructor to type[Mapping], and Mapping being an ABC with no __init__, it resolves to object.__init__ and objects to the argument. From a37ef21 "feat(storage): generalize DigestedDict to DigestedMapping".
• storage/paths.py:12 — unsupported-base on class TempPath(type(Path())); ty can't resolve an MRO through a dynamic base. From d917df3 "feat(storage): cache filesystem paths by content".

So every PR into temppath fails ty until it's fixed at the base — #828 will fail identically when its turn in the queue comes up.

I checked that suppressions clear it, anchored on the exact diagnostic lines (ty binds the comment to the line it reports, so on the generator line at :133, not the return type(self.items)( line above it) — with those three comments, ty check src/ on temppath gives All checks passed!:

            (self.get(storage, k), self.get(storage, v))  # ty: ignore[too-many-positional-arguments]
        return type(value)(zip(children[:n], children[n:]))  # ty: ignore[too-many-positional-arguments]
class TempPath(type(Path())):  # ty: ignore[unsupported-base]

The TempPath one looks unavoidable while 3.11 is supported — type(Path()) is there because bare Path only became subclassable in 3.12 — so a suppression seems right rather than a rewrite. The two in destructuring.py could instead be a cast on the constructor if you'd prefer typing over silencing.

I've not pushed any of that: it's your in-flight code on temppath, and fixing it from a docs PR that doesn't otherwise touch those files seemed like the wrong place. Happy to push it wherever you want — temppath/#797 fixes it once for everything downstream, or I can put it on #828, which already touches both files. Just say which.

One correction while I'm here: both PR bodies say "4 diagnostics" — that was me running an unpinned newer ty locally, which adds an unused-suppression warning. Against CI's pinned 0.0.62 it's 3. The substance was right (unchanged from base, none introduced by either PR); the count wasn't.

---

Generated by Claude Code

[pmrv]

revise

[pmrv]

any depth might actually only apply when remaining_depth is 0 :[=].

[pmrv]

Checked this rather than trusting the wording — it holds at any remaining_depth, and the reason is worth stating, so 62a2c3c now says why instead of just asserting "any depth".

A Path matches no destructurer, so in _intern_rec it falls through the _DESTRUCTURERS loop with depth still at float("inf"). The inline test is if depth < self.remaining_depth, and inf is never less than anything — so a path is always written out via super().save(...), which is precisely where PathValueMixin intercepts it. remaining_depth only chooses inline-vs-separate-entry for nodes that are destructurable; it has no lever on a path.

(It also propagates: a container holding a path gets 1 + inf, so the parents above it are written out separately too.)

Verified empirically — remaining_depth ∈ {0, 1, 3, 10} × nesting depth ∈ {0, 1, 2, 3, 5, 8}, alternating list/dict/dataclass wrappers, deleting the original file before loading so only content addressing can survive. All 24 combinations materialize with the right content.

Added to the docs:

"Any depth" is not a figure of speech, and no storage setting narrows it. A Path matches no destructurer, so it is always written out as its own stored entry rather than inlined into the container above it — and being written out is exactly what hands it to the content machinery. The remaining_depth knob only decides how eagerly destructurable nodes are split into separate entries, so it cannot put a path out of reach however it is set.

---

Generated by Claude Code

[pmrv]

second clause is correct and intended behavior, but the first one makes it read as it if was some edge behavior.

[pmrv]

Fixed in 62a2c3c. You're right that the ordering did the damage — leading with "treat received paths as read-only" implies the returning case is the exception you fall back to, when it's the normal shape, and the one the shell pipeline in Files.ipynb actually uses.

Flipped it: returning what you wrote to now leads and is stated as intended, and the read-only advice is scoped to paths that are pure input.

Returning what you wrote to is the normal shape, not a workaround for this: a function that writes into a directory and hands that directory back is recorded with it in its final, post-mutation state, and that is the intended way to produce files. What is not replayed is a write to something you never return — so a path that is pure input should be treated as read-only, with new files written somewhere you do return (tempfile.mkdtemp).

---

Generated by Claude Code

[pmrv]

Cell 9 is not path specific, delete or make it applicable to the path/file caching.

[pmrv]

Agreed — write.query().table() was generic query introspection that happened to be sitting in a file-caching notebook. Made it path-specific rather than deleting it, since the slot is useful: 62a2c3c replaces it with the value store's own view of what a stored path actually is.

# What a `Path` actually becomes in storage: its content as plain `bytes` under
# its own digest, plus a small record pairing that content with a name.  Note
# that left.txt and right.txt reference the *same* content digest.
for blob in c.values.storage.values():
    if type(blob).__name__ in ("FileBlob", "DirectoryBlob"):
        print(blob)

FileBlob('out.txt', '785d68f8426805e292630852bdedb46dd56ac44dcb7047740d30704f3d84d4fa')
FileBlob('sentence.txt', '0f61b76af53fa2dc41528c3866206d22ceb3b7f560e7f42aacc006fc5ace228c')
DirectoryBlob({'leaf.bin': 'c70f6db1a5371bc6046fb5a040fd13bd5220c78908eac8126c1361daad854904'})
DirectoryBlob({'sub': '7aa02a10cd58fe7b0f16ef0e06b255d7842ce362687e489e941281d27761d95a', 'top.txt': '8eb42147b1727df4b082ebc0bdfc5fbaea064308411a4d801cae67c908ce4287'})
FileBlob('left.txt', '32cbd77d1dbff488cd42dc84ea72ebd47358fbf321412b35a5c1084e36f5b775')
FileBlob('right.txt', '32cbd77d1dbff488cd42dc84ea72ebd47358fbf321412b35a5c1084e36f5b775')

It follows the content-addressing section directly, so it lands well: the cell above asserts the shared body is stored once, and this one shows why — left.txt and right.txt are two small name records pointing at one content digest, and the directory's root name is nowhere to be seen while its children's names are. The same split the dev/path_storage page describes, made concrete.

---

Generated by Claude Code

[pmrv]

edges 3,5,6 are real, the others are from struct pure function perspective always been wrong, so rephrase these more in a 'beware that ...' kind of way. Currently it reads as if this is a defect that might get fixed.

[pmrv]

That's a real distinction I'd flattened — calling all six "Edge N" made them read as one list of things that are wrong, with the implication that someone will get around to them. Fixed in 62a2c3c by splitting the vocabulary.

Beware (1, 2, 4 — consequences of the model, not going to change). Each now says why it isn't a defect:

• a hit gives you a copy, somewhere else — "the function's value is the file it returned, and a location is not part of a file's content, so this is content addressing working exactly as advertised, not a fidelity gap." The sibling footgun follows: the sibling was never part of the value.
• aliasing is not part of a value — "object identity is a property of one process's memory, not of the value being cached — nothing about 'these two files are the same object' survives a round trip through storage, and nothing could."
• each hit is its own copy, with temp-file lifetime — "the cache owns the content and hands you a copy; it cannot know when you're finished with it except by watching the reference you were given."

Caveat (3, 5, 6 — the real ones), unchanged in substance: Path dict keys, opaque containers, container subclasses.

The intro now draws the line explicitly rather than promising "the edge cases where a cache hit is not a faithful replay":

• Beware — consequences of what caching a pure function by content means. A cached call is replayed by its value, so a file comes back as a copy: its location, its identity, and whatever sat next to it on disk were never part of that value. Code that leaned on them was relying on something the cache never promised. These are not defects and they are not going to change.
• Caveat — real limits of the mending machinery, where a hit hands back something less faithful than it could: paths used as dict keys, and paths hidden inside containers fleche does not destructure.

Summary rewritten to match. The code cells are untouched — the demonstrations were fine, it was the framing around them that was wrong.

---

Generated by Claude Code

[pmrv]

Suggested change

	# Wrapping command-line tools is a first-class use case (see
	# notebooks/Files.ipynb), and `run()`'s return value is neither a
	# dataclass nor iterable, so without this arm every such function
	# is Indigestible. The four public fields are the whole result:
	# what was run, how it exited, and what it wrote. Purely
	# additive — these values raised before, so no stored digest
	# changes and no `hash_version` bump is implied.

[pmrv]

Dropped in 62a2c3c — the arm is (value.args, value.returncode, value.stdout, value.stderr), which says it itself.

The one non-obvious bit was that it's purely additive (those values raised Indigestible before, so no stored digest changes and no hash_version bump); that's recorded in the DEVELOPING design-theme entry instead, where the other digest-arm additions are listed.

---

Generated by Claude Code

[pmrv]

lgtm, merge