feat(snapshot): add cow memory and resident pause lifecycle

.github/actions/cache-libkrunfw-kernel/action.yml

@@ -18,6 +18,10 @@ runs:
             # Published in kernel.org's signed v6.x sha256sums.asc index.
             kernel_sha256=194eef900ade82df74ed1d695daa45d03ee4bb415cae4f936a3dbaab2dbbb951
             ;;
+          linux-6.12.108:'tarballs/$(KERNEL_VERSION).tar.gz')
+            # https://cdn.kernel.org/pub/linux/kernel/v6.x/sha256sums.asc
+            kernel_sha256=c4127aa9614a6a829c537cff96a58da634a5f8cfd1aed9d1ba076d3b3a80891a
+            ;;
           *)
             echo "::error::Add the checksum for ${kernel_version} (${kernel_tarball}) to ${GITHUB_ACTION_PATH}/action.yml"
             exit 1

.github/workflows/check.yml

@@ -576,13 +576,13 @@ jobs:
           Set-MsvcEnvironment -Architecture ${{ matrix.vs_arch }} -HostArchitecture ${{ matrix.vs_host_arch }}
           cargo +stable test --no-default-features --features local,net -p microsandbox --lib --target ${{ matrix.rust_target }} sandbox::patch::tests::bind_patch_
 
-      - name: Test Windows stdio inheritance cleanup
+      - name: Test Windows lifecycle handoff and stdio cleanup
         shell: pwsh
         run: |
           $ErrorActionPreference = "Stop"
           . "$env:GITHUB_WORKSPACE\vendor\libkrunfw\scripts\msvc-env.ps1"
           Set-MsvcEnvironment -Architecture ${{ matrix.vs_arch }} -HostArchitecture ${{ matrix.vs_host_arch }}
-          cargo +stable test --no-default-features --features local,net -p microsandbox --lib --target ${{ matrix.rust_target }} runtime::spawn::tests::windows_stdio_guard_
+          cargo +stable test --no-default-features --features local,net -p microsandbox --lib --target ${{ matrix.rust_target }} runtime::spawn::tests::windows_
 
       - name: Test Windows DNS resolver
         shell: pwsh
@@ -1457,6 +1457,40 @@ jobs:
           scripts/smoke/cli/image-archive.sh
           scripts/smoke/cli/split-irqchip-bind-net.sh
 
+      - name: Snapshot smoke runner unit tests
+        run: python3 -m unittest discover -s scripts/smoke/cli -p test_snapshot_branch.py
+
+      - name: Snapshot and branch live smoke
+        # The short operation deadline excludes image setup and bounded cleanup.
+        # Leave both layouts enough failure-path time to stop VMs and save evidence.
+        timeout-minutes: 15
+        env:
+          MSB_LIBKRUNFW_PATH: ${{ github.workspace }}/build/libkrunfw.so.${{ env.LIBKRUNFW_VERSION }}
+          MSB_AGENTD_PATH: ${{ github.workspace }}/build/agentd
+          LD_LIBRARY_PATH: ${{ github.workspace }}/build
+        run: |
+          status=0
+          # Short isolated homes avoid Unix socket limits and cross-layout reuse.
+          # Run both even if one fails so the artifact reports retain both outcomes.
+          for layout in managed flat; do
+            python3 scripts/smoke/cli/snapshot-branch.py \
+              --binary "${{ github.workspace }}/build/msb" \
+              --output "/tmp/msb-smoke-${{ github.run_id }}-${{ github.run_attempt }}-$layout" \
+              --layout "$layout" || status=$?
+          done
+          exit "$status"
+
+      - name: Upload snapshot smoke reports and logs
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: snapshot-branch-smoke-linux-x86_64
+          # Keep text evidence, not the potentially large guest RAM/disk artifacts.
+          path: |
+            /tmp/msb-smoke-${{ github.run_id }}-${{ github.run_attempt }}-*/report.json
+            /tmp/msb-smoke-${{ github.run_id }}-${{ github.run_attempt }}-*/logs/*.log
+          if-no-files-found: warn
+
       - name: Disk usage
         if: always()
         run: scripts/ci/clean-runner-disk.sh

COMPATIBILITY.md

@@ -132,6 +132,12 @@ Sources: [`crates/runtime/lib/runner/control.rs`](crates/runtime/lib/runner/cont
 
 Add operations and optional fields rather than redefining existing ones. Capability-gate behavior whose absence cannot be interpreted safely by older clients.
 
+Live disk-only snapshots use the distinct `disk_checkpoint_create` operation and capability. An absent capability is false: callers refuse before capture rather than silently capturing RAM or copying a writable disk. The runtime serializes the disk rollover with other control mutations and preserves a user's pause. This does not change the agent protocol or snapshot format; the result uses the existing file-state layer descriptor. Stopped disk capture retains its lifecycle lock and existing behavior.
+
+Resident pause/resume sends one authoritative mutation rather than first observing pause state and querying capabilities. The runtime checks support before mutation, including idempotent requests, and the client requires the expected state in the response; unknown operations and incomplete replies fail. Ordinary get/list pause projection remains unchanged. Guest freezing retains one `cgroup.events` descriptor and waits for notifications with a fixed deadline; each poll timeout is capped at 1 ms so rate-limited kernel notifications cannot delay the next authoritative state check. Clock correction still precedes workload thaw, and no control or agent wire format changes.
+
+The subsequent unreleased generation-9 transport repair routes internal freeze/thaw directly from the coordinator to the existing relay through a bounded in-process queue. Ordinary control/bulk input is gated at complete frames; admitted input remains guest-owned until consumed, while unadmitted input stays source-owned and ordered. The guest keeps stdin/TCP delivery nonblocking with respect to its control loop and preserves accepted input through restore. Private replies and cumulative credit updates never become SDK responses. The immutable frame header, released generation-8 schema, and public sockets are unchanged. Full capture requires an acknowledged bidirectional frame boundary; failure or timeout does not authorize a partial capture.
+
 ## 5. Launcher-to-Runtime Process Protocol
 
 Starting a sandbox crosses a private process boundary. On Unix, launch JSON is passed through inherited descriptor 96, the parent watchdog uses descriptor 97, startup JSON uses descriptor 98, and the lifecycle lock uses descriptor 99. Windows uses a short-lived launch-config file and platform-specific startup plumbing. Detach acknowledgement bytes and graceful-shutdown signals are also part of this contract.
@@ -140,7 +146,9 @@ Compatibility-sensitive elements include descriptor numbers, ownership and close
 
 Sources: [`crates/runtime/lib/client/launch.rs`](crates/runtime/lib/client/launch.rs), [`crates/runtime/lib/runner/vm.rs`](crates/runtime/lib/runner/vm.rs), [`sdk/rust/lib/runtime/spawn.rs`](sdk/rust/lib/runtime/spawn.rs), and [`crates/cli/lib/sandbox_cmd.rs`](crates/cli/lib/sandbox_cmd.rs).
 
-This protocol has no explicit version envelope. Treat additions as optional and consider adding explicit version or capability negotiation before allowing independently versioned launchers and runtimes.
+Launch JSON requires an explicit `execution` intent (`boot` or `restore`) and rejects unknown fields. Restores also pass the internal `msb sandbox --restore` argument: a runtime predating this contract rejects the unknown argument rather than ignoring a JSON restore source and cold-booting. The argument, intent, and complete strictly validated `checkpoint_restore` source must agree before VM construction. Unsupported restore behavior is an error, never a fresh-boot fallback. These #8 development contracts replace superseded unreleased forms without shims; they do not change portable snapshot bytes.
+
+The child database config retains `checkpoint_restore` while construction is incomplete. Only successful restore activation and creation finalization remove it. A failed or interrupted attempt retains its child-owned staging and rejects start, auto-start through exec, modification, compaction, and snapshot creation; remove and recreate it from the intact input snapshot. This replaces the earlier unreleased #8 behavior that discarded restore intent before success. Do not reopen these development rows with older #8 binaries that skip that field. Successful restores retain the ordinary later stop/start lifecycle; no portable snapshot format or schema version changes.
 
 ## 6. Database, Configuration, and Migration History
 
@@ -180,10 +188,28 @@ Parsers and mutators must validate the complete supported feature set before the
 
 ## 9. Snapshots, Manifests, and Portable Archives
 
-Snapshot descriptor bytes are identity-bearing: their canonical bytes determine the snapshot ID. Compatibility-sensitive elements include field order, required `null` values, map ordering, duplicate-key handling, tag spellings, schema and integrity identifiers, payload names, parent identities, state/scope/format variants, extension requirements, and translation-graph behavior.
+Snapshot descriptors carry a stable random `snap_...` ID; their canonical bytes determine the descriptor digest, not that ID. Compatibility-sensitive elements include field order, required `null` values, map ordering, duplicate-key handling, tag spellings, schema and integrity identifiers, payload names, parent identities, state/scope/format variants, extension requirements, and translation-graph behavior.
 
 Archive compatibility includes compression detection, `archive.json`, canonical inventory order, transport digests, accepted path grammar, legacy paths, cache-closure entries, and rejection of duplicate, missing, or escaping paths.
 
+Installed snapshots now live under `snapshots/<group>/<snapshot_id>/`. `group.json` selects a head; `group-member.json` stores a local friendly name without changing descriptor identity. Bare selectors mean a group head, and `group:member` selects an exact member. Existing flat artifacts remain readable by explicit path; this change does not silently move their directories. The index keys local artifact paths rather than globally unique portable IDs/digests, so importing the same snapshot into two groups preserves both copies. Downgrade refuses grouped state before rewriting artifacts or rolling back the index.
+
+Capture records the actual source snapshot lineage in the existing descriptor `parent` field. Per-sandbox cursor publication serializes captures without holding a VM pause; group head publication is locked separately. Automatic head advancement requires known ancestry, not capture timestamps, export dependency bases, or import order. An explicit head selection may rewind or choose a sibling. Missing ancestry may prevent advancement but is not a missing payload dependency. Archives optionally carry friendly names in `msb-snapshot-member-names`; their snapshot IDs, payload paths and descriptor schema are unchanged.
+
+Capture publication and source removal/replacement share a stable lock in `run_dir/locks/<lifecycle-hash>.snapshot-lineage.lock`, outside the removable sandbox directory. A caller needing multiple ownership guards acquires transition, then lineage, then runtime lifecycle ownership. The cursor remains in the sandbox directory; its schema and portable snapshot identities are unchanged. This replaces the unreleased directory-local lock, not a shipped artifact format.
+
+`snapshot load` accepts multiple archive paths; the former positional destination is now `--dest DIR`. Single-archive SDK methods and their return types remain; batch methods return one handle per input archive head in input order. The batch resolves exact disk-layer and RAM-object dependencies from supplied archives, the explicitly selected destination group, and an optional external base. No archive encoding changes or global snapshot search are involved. Borrowed payloads belong to destination staging and use the existing integrity codecs before publication. A compatible source may contain more layers than the omitted prefix; dependency identities still must match. Direct archive restore retains its explicit-base contract.
+
+Batch head selection is independent of input order: one proven lineage tip uses existing fast-forward rules; ambiguous tips preserve an existing head or leave a new group headless. `--set-head` refuses an ambiguous batch. IDs, aliases, duplicate labels, and payloads are checked before member publication. An I/O failure during final publication can still leave complete additional members, as with single-archive publication, but never a head pointing at an incomplete member.
+
+Unreleased #8 incremental exports use `completeness: "dependent"` and the must-understand `msb-snapshot-dependencies-v1` extension. `--since` records omitted physical disk-prefix layers and reusable RAM-object identities; `--last-layers` only omits disk layers. The complete target memory manifest and CPU/device state remain included. Loading and direct archive restore resolve the explicitly supplied base into owned staging before opening the complete target. This replaces the unreleased disk-only dependency encoding without a compatibility shim or snapshot descriptor change. Readers that do not understand this requirement refuse it; ordinary standalone archives are unchanged.
+
+Full checkpoints and local branches now retain `transport_host_input`, `transport_input_credit`, and `transport_guest_bulk_bytes` in the existing `guest:agentd` resource binding. These are complete-frame cumulative positions and absolute grants, including credit still owned by pending captured input. Restore validates and seeds them before guest activation; resetting them would incorrectly grant capacity twice. Older unreleased development full snapshots missing this state are refused, and new full captures require their matching host/guest implementation. This is an approved replacement of unreleased state, not a snapshot schema bump or migration; released disk-only snapshots are unaffected.
+
+The finalized private transport-credit contract charges stdin, inline filesystem/TCP payloads, and ordered EOF to the existing logical data (`bulk_*`) counters on either physical port. Command/control counters remain available when captured input is still awaiting consumption. Ready advertises barrier contract `2`; the superseded development contract `1` is not translated or restored. The outer frame, generation-8 data format, snapshot descriptor schema, and public SDK requests are unchanged. The ordinary writer retains bounded admission permits until physical delivery, permits unrelated metadata to pass credit-blocked payloads, and preserves per-correlation and client-disconnect ordering. Guest input processing also yields to the runtime after bounded actual reads, including partial records; this does not shrink wire records or change snapshot boundaries.
+
+Routine host clock maintenance is independent of unrelated correlation input, but stays ordered with other clocks and true global lifecycle fences. Its timestamp is sampled at console admission, not when queued; disconnect cleanup signals fence their own session only. Maintenance remains subject to the pause gate. This bounds host-queue timestamp age, not subsequent aging of already-admitted bytes during arbitrary host suspension or the kernel-only pause fallback when the workload freezer is unavailable.
+
 Evolution rules:
 
 - Do not make semantically harmless serialization changes to identity-bearing bytes without treating them as an identity format change.
@@ -194,6 +220,10 @@ Evolution rules:
 
 Sources: [`crates/image/lib/snapshot/manifest.rs`](crates/image/lib/snapshot/manifest.rs), [`crates/image/lib/snapshot/migration.rs`](crates/image/lib/snapshot/migration.rs), and [`sdk/rust/lib/snapshot/archive.rs`](sdk/rust/lib/snapshot/archive.rs).
 
+Runtime restore admits disk payloads before activation. Journal creation may reuse the verified root only for that same unchanged immutable file. Its in-process cache retains at most 32 file handles, preferring larger physical files; every layer still receives full admission, and uncached, copied or rewritten layers receive a fresh hash. Detected mutation of a retained admitted file fails. Candidates are opened once per lookup, with comparisons bounded by the cache size. This reuse is not a persistent "verified" flag or a path-only cache, and the cache bound does not reduce supported chain depth.
+
+Incremental capture retains immutable object receipts only within the owning runtime's store lifetime. Reopened stores and unadmitted objects still verify bytes. Receipts retain no file descriptors; active operations open, check and temporarily pin the exact file. Capture uses two writers and three recycled 32 MiB packs, then synchronizes new directory entries before the existing root-last publication. Eager restore and cold memory-cache construction use at most four reusable 32 MiB read/hash buffers. Errors join workers before cleanup. These changes preserve the snapshot format and restored bytes, dirty-baseline rollover, pause/publication ordering and durability barriers; summed worker timings must not be interpreted as additive wall time.
+
 ## 10. OCI Cache and Materializer ABI
 
 The cache is rebuildable, but cache entries and closures can cross releases through `MSB_HOME` and snapshot archives. OCI semantics are externally defined: compressed descriptor digests, uncompressed diff IDs, ordered layers, whiteouts, opaque directories, hardlinks, extended attributes, non-UTF-8 paths, special files, and permissions must retain their meaning.
@@ -271,6 +301,8 @@ Compatibility-sensitive ordering includes:
 
 Sources: [`crates/runtime/lib/client/ipc.rs`](crates/runtime/lib/client/ipc.rs), [`sdk/rust/lib/backend/local/mod.rs`](sdk/rust/lib/backend/local/mod.rs), [`sdk/rust/lib/runtime/handle.rs`](sdk/rust/lib/runtime/handle.rs), and artifact-specific migration and publication modules.
 
+TCP completion follows both ordered half-closes; the first EOF alone keeps the opposite direction usable. In combined-port mode, validated guest-to-host TCP credit may pass queued host-to-guest raw data and its finish marker: it services the opposite direction without reordering input data or EOF. Opening, cancellation, ownership, and global lifecycle fences still constrain it. Raw TCP output may still be draining on the dedicated lane after its producer finishes. Decoded credit updates for a finished producer or absent TCP session are therefore no-ops, not cancellation: they cannot enable further output, and must not discard the queued tail or create a second terminal response. Active producers retain credit validation; data and finish messages retain their existing validation.
+
 Review concurrency and crash points explicitly. A same-version happy-path test does not establish cross-version or crash compatibility.
 
 ## Review Triggers in Diffs