snosi

A bootable container image build system using mkosi for creating Debian-based bootable containers and system extensions (sysexts).

Downloads

• Firn installer — all image families, bootc and A/B (latest x86-64)
• Installer checksums and OpenPGP signature
• Older bootc installer: Bootc live installation media See the supported installation guide to choose an image, verify the native installer, create boot media, install safely, and recover or update the installed system.

Community

Contributions are welcome. Read the contributing guide and follow the Code of Conduct. Report vulnerabilities only through the private channels in the security policy, never in a public issue.

What This Project Does

snosi builds immutable, bootable OCI container images based on Debian Trixie. These images are designed for use with bootc / systemd-boot and can be deployed as atomic, updateable operating system images.

The project produces:

Image	Description	Output Format
snow	GNOME desktop with backports kernel	directory → OCI (buildah/chunkah)
snowfield	snow with linux-surface kernel for Surface devices	directory → OCI (buildah/chunkah)
cayo	Headless server with podman + backports kernel	directory → OCI (buildah/chunkah)
cayo-ab-raw	Experimental native A/B server image (dev fixture, never published)	GPT disk (EROFS + dm-verity)
cayo-ab	Production native A/B server image (Secure Boot + TPM/LUKS /var)	GPT disk (EROFS + dm-verity)
snow-ab	Production native A/B GNOME desktop, backports kernel	GPT disk (EROFS + dm-verity)
snowfield-ab	Production native A/B GNOME desktop, linux-surface kernel	GPT disk (EROFS + dm-verity)
1password	1Password desktop application	sysext
1password-cli	1Password CLI tool	sysext
azurevpn	Microsoft Azure VPN client	sysext
bitwarden	Bitwarden password manager desktop application	sysext
chatgpt	ChatGPT desktop application with Codex	sysext
claude-desktop	Claude desktop application	sysext
code-server	code-server (VS Code in the browser)	sysext
coder	Coder self-hosted development workspaces server	sysext
debdev	Debian development tools (debootstrap, distro-info)	sysext
dev	Build essentials, Python, cmake, valgrind, gdb	sysext
docker	Docker CE container runtime	sysext
edge	Microsoft Edge browser	sysext
github-copilot	GitHub Copilot agent-native desktop application	sysext
incus	Incus container/VM manager	sysext
k3s	k3s lightweight Kubernetes node (server or agent)	sysext
lemonade	Lemonade local LLM server (GPU/NPU accelerated)	sysext
nix	Nix package manager	sysext
paseo	Paseo coding agent workspace desktop application	sysext
pilothouse	Pilothouse web administration with capability-gated Updex/container backends	sysext
podman	Podman + Distrobox	sysext
sunshine	Sunshine self-hosted game streaming host for Moonlight	sysext
tailscale	Tailscale VPN client	sysext
voxtype	Voxtype AI voice dictation for Wayland	sysext
vscode	Visual Studio Code desktop application	sysext

Protected bootc publication validates each candidate with host Podman while running the candidate image's pinned bootc for its composefs storage digest. The publisher does not depend on a separate host bootc installation, avoiding version drift at that compatibility boundary. The protected packager passes its secure assembly flag and credential paths explicitly through sudo. The private bytes stay in mode-0600 runner files; only their paths cross the privilege boundary. For protected secure images, it chunks the pristine candidate before the candidate's pinned bootc obtains the authoritative storage digest. The final image inherits those chunked layers and overlays only /boot; its bootc must return the same digest in the second of exactly two digest probes. Protected assembly never runs a post-assembly chunk pass.

Architecture

Naming, path, and policy contracts for the production native A/B products (cayo-ab, snow-ab, snowfield-ab) are frozen in docs/native-ab-contracts.md and validated statically by test/native-ab-contracts-test.sh; see docs/plans/2026-07-14-bootc-native-ab-coexistence-plan.md for the coexistence plan that freeze unblocks.

The isolated cayo-ab-raw profile (renamed from cayo-ab in Phase 1; cayo-ab now names the production secure posture — see below — and check-native-publication-guard.sh hard-fails if cayo-ab-raw ever picks up a publication marker) is an experimental development spike for native systemd-repart/systemd-sysupdate A/B roots. Its raw image and installer boot in QEMU, and test/native-ab-update-test.sh validates signed three-hop updates, inactive-slot reuse, dm-verity boot, /var plus /etc persistence, rejected incomplete/tampered update sets, explicit rollback, and boot-count fallback. test/native-ab-components-test.sh validates the Phase 1 exit criterion: no failed legacy updaters, independently versioned sysext components updating without touching OS partitions or each other, an OS update succeeding with those components enabled, and native /etc drift reporting. Production key management (the offline ceremony, in docs/native-ab-publication.md) and hardware soak remain; the publication/signing pipeline itself (shared/native-ab/publish/{publish-candidate,verify-remote,promote, withdraw}.sh) is built and rehearsed end to end locally — see that same runbook doc for the candidate/verify/promote/withdraw procedure and test/native-ab-publication-test.sh for the local rehearsal. It does not replace the supported bootc images. See docs/plans/2026-07-13-mkosi-native-ab-root-design.md; the destructive raw-disk installer spike lives at test/cayo-ab-install-spike.sh. Its UKI uses the profile's dracut archive rather than mkosi's independently generated default initrd so the pre-pivot persistent /etc service is present at boot.

The shared shared/native-ab-secure/mkosi.conf fragment extends the raw spike with standard Secure Boot through Debian's Microsoft-signed shim and MOK-signed systemd-boot; it is Include=d by the three production profiles (cayo-ab, snow-ab, snowfield-ab — the former standalone cayo-ab-secure spike profile was retired once this fragment generalized its content). Snosi UKIs are locally signed, so their certificate must be enrolled once through shim's MokManager; this does not require UEFI setup mode or custom firmware keys. The installer can replace the image's disposable ext4 /var with per-machine LUKS2, retain an external recovery passphrase, and enroll TPM unlock against a signed PCR 11 policy. Raw PCR 7 cannot be bound by the installer because its Debian boot authority differs from the installed MOK-signed UKI. Signed PCR 11 lets newly authorized A/B UKIs unlock the same /var without binding to one immutable PCR value. LUKS2 and TPM/recovery unlock are validated. Secure Boot profile construction is also validated, including signed shim and MOK-signed systemd-boot on the ESP and a signed UKI carrying .pcrpkey/.pcrsig. MOK boot, virtual-TPM unlock, and PCR signing-key rotation, signed updates, rollback, and boot-count fallback are validated in Incus. Systemd 261's optional NvPCR measurements are disabled in these profiles: their anchor credential cannot migrate between PCR signing keys, and they are not used by the signed-PCR LUKS policy.

PCR signing-key overlap must use a transition UKI signed by both old and new PCR keys. Two independent TPM tokens are not sufficient: systemd 261.1 falls through after a raw-PCR mismatch, but a signed-policy key mismatch stops at the lower-numbered token. Place the old private key in .snosi-private/history/, make the new key active, and opt into a transition build with PCR_SIGNING_KEY_PREVIOUS=<old-key-filename>. The UKI keeps the new key in .pcrpkey and carries each PCR 11 policy signed by both keys. Retain the old token until every supported rollback UKI contains the new signature. Each production native profile carries a coherent Forky 261+ systemd family through the shared fragment's isolated, low-priority APT source; normal Trixie profiles remain unchanged. Validate a transition artifact with (OUTPUT_NAME selects cayo-ab/snow-ab/ snowfield-ab; defaults to cayo-ab):

OUTPUT_NAME=cayo-ab test/native-ab-secure-artifact-test.sh \
  output/cayo-ab.manifest output/cayo-ab.efi \
  .snosi-private/history/<old-key-certificate> \
  .snosi-private/pcr-signing.pub

test/native-ab-secure-artifact-negative-test.sh also proves that a missing signature and an old published .pcrpkey are rejected. To exercise runtime rotation on an already MOK-enrolled disposable VM, retain its external recovery key and run:

test/native-ab-secure-rotation-test.sh --yes \
  output/cayo-ab \
  .snosi-private/history/<old-key-certificate> \
  .snosi-private/pcr-signing.pub /path/to/recovery.key \
  root@<vm-address> /path/to/ssh-key <expected-machine-id>

The destructive harness verifies the recovery key and exact machine ID before making changes. It installs the transition through guest-local systemd-sysupdate using a verified ephemeral signed manifest, boots with only the old TPM token, enrolls the new token, retires the old token, and reboots the identical UKI with only the new token. It requires MOK enrollment and VM provisioning to have been completed first.

Keep durable signing keys outside .mkosi-private: that directory belongs to mkosi and mkosi clean -ff removes it. The gitignored .snosi-private directory is the repository convention for active and archived PCR signing keys.

After validating rotation, exercise the complete secure rollback window on the same disposable Incus VM with three preserved builds:

test/native-ab-secure-update-test.sh --yes \
  /path/to/N+1/cayo-ab \
  /path/to/N+2/cayo-ab \
  /path/to/N+3/cayo-ab \
  .snosi-private/history/<old-key-certificate> \
  .snosi-private/pcr-signing.pub /path/to/recovery.key \
  root@<vm-address> /path/to/ssh-key <expected-machine-id> <incus-instance>

The harness requires N+1 and N+2 to be dual-signed and N+3 to be new-only. It checks signed transfer, alternating slots, new-key-only TPM unlock, explicit rollback, a return to N+3, three failed dm-verity boots, the exhausted +0-3 entry, and automatic N+2 fallback.

To run the full spike, preserve the N raw image and three consecutive builds, then pass each update artifact prefix:

sudo test/native-ab-update-test.sh \
  /path/to/cayo-ab-raw-N.raw \
  /path/to/N+1/cayo-ab-raw \
  /path/to/N+2/cayo-ab-raw \
  /path/to/N+3/cayo-ab-raw

                              base                ← Debian Trixie + bootc foundation
                                │
                ┌───────────────┴───────────────┐
                │                               │
             sysexts                         profiles
    ┌────┬────┬────┬────┬────┬────┬────┬────┬────┬────┐  │
    │    │    │    │    │    │    │    │    │    │    │  ┌──┴──────┐
  1password 1password-cli azurevpn bitwarden chatgpt claude-desktop code-server coder debdev dev docker edge github-copilot incus k3s lemonade localsend moonlight nix obsidian paseo pilothouse podman sunshine tailscale voxtype vscode
                                     snow            cayo
                                      │
                                  snowfield

Base Image

The base image (mkosi.images/base/mkosi.conf) provides the foundation for all derivatives:

• Debian Trixie (testing) with main, contrib, non-free, and non-free-firmware repositories
• systemd, systemd-boot, and boot infrastructure
• Network management (NetworkManager, wpasupplicant)
• Container tooling prerequisites (erofs-utils, skopeo)
• Firmware packages for common hardware
• Core utilities (fish, zsh, vim, git) and host hardware diagnostics (lspci, lsusb)

System Extensions (sysexts)

Sysexts are overlay images that extend the base system without modifying it. They're built with Format=sysext and Overlay=yes:

Sysext	Contents	Config
1password	1Password desktop app	mkosi.images/1password/mkosi.conf
1password-cli	1Password CLI tool	mkosi.images/1password-cli/mkosi.conf
azurevpn	Microsoft Azure VPN client	mkosi.images/azurevpn/mkosi.conf
bitwarden	Bitwarden password manager desktop app	mkosi.images/bitwarden/mkosi.conf
chatgpt	ChatGPT desktop application with Codex	mkosi.images/chatgpt/mkosi.conf
claude-desktop	Claude desktop application	mkosi.images/claude-desktop/mkosi.conf
code-server	code-server (VS Code in the browser)	mkosi.images/code-server/mkosi.conf
coder	Coder workspaces server + workspace proxy	mkosi.images/coder/mkosi.conf
debdev	debootstrap, distro-info, archive keyrings	mkosi.images/debdev/mkosi.conf
dev	build-essential, cmake, Python, valgrind, gdb	mkosi.images/dev/mkosi.conf
docker	Docker CE, containerd, buildx, compose	mkosi.images/docker/mkosi.conf
edge	Microsoft Edge browser	mkosi.images/edge/mkosi.conf
github-copilot	GitHub Copilot agent-native desktop application	mkosi.images/github-copilot/mkosi.conf
incus	Incus, QEMU/KVM, OVMF, virt-viewer	mkosi.images/incus/mkosi.conf
lemonade	Lemonade local LLM server (lemond)	mkosi.images/lemonade/mkosi.conf
nix	Nix package manager, systemd integration	mkosi.images/nix/mkosi.conf
paseo	Paseo desktop app (Electron)	mkosi.images/paseo/mkosi.conf
pilothouse	Pilothouse web administration with capability-gated Updex/container backends	mkosi.images/pilothouse/mkosi.conf
podman	Podman, Distrobox, buildah, crun	mkosi.images/podman/mkosi.conf
sunshine	Sunshine self-hosted game streaming host for Moonlight	mkosi.images/sunshine/mkosi.conf
tailscale	Tailscale VPN client	mkosi.images/tailscale/mkosi.conf
voxtype	Voxtype AI voice dictation for Wayland	mkosi.images/voxtype/mkosi.conf
vscode	Visual Studio Code desktop application	mkosi.images/vscode/mkosi.conf

The Pilothouse sysext explicitly configures Updex, Podman, Docker, and Incus. Pilothouse advertises each backend only after its executable or socket probe succeeds, so unavailable endpoints do not prevent pilothoused from starting.

How Profiles Work

Profiles in mkosi.profiles/ define complete image variants by composing shared components. Each profile's mkosi.conf uses Include= directives to pull in reusable configuration fragments from the shared/ directory.

Profile Structure

mkosi.profiles/
├── cayo/           ← Headless server + podman
├── snow/           ← GNOME desktop + backports kernel
└── snowfield/      ← GNOME desktop + Surface kernel

Shared Components

The shared/ directory contains reusable configuration fragments that profiles include:

shared/
├── kernel/
│   ├── backports/mkosi.conf   ← Trixie backports kernel + firmware
│   ├── surface/mkosi.conf     ← linux-surface kernel + iptsd
│   └── scripts/               ← dracut postinst scripts
├── download/
│   ├── sysext-checksums.json ← Pinned direct downloads consumed by sysext builds
│   ├── image-checksums.json  ← Pinned direct downloads consumed by OCI profile builds
│   ├── package-versions.json ← External APT package version sentinels for sysexts
│   └── verified-download.sh  ← verified_download() helper
├── kernel/
│   ├── backports/mkosi.conf   ← Trixie backports kernel + firmware
│   ├── surface/mkosi.conf     ← linux-surface kernel + iptsd
│   ├── stock/mkosi.conf       ← Stock Trixie kernel
│   └── scripts/               ← dracut postinst scripts
├── manifest/postoutput/       ← Manifest annotation postoutput script
├── outformat/
│   └── image/
│       ├── mkosi.conf         ← Sets Format=directory
│       ├── finalize/          ← Image finalization scripts
│       ├── buildah-package.sh ← Packages rootfs dir into an OCI image
│       └── chunkah-package.sh ← Chunks OCI candidates for efficient updates
├── packages/
│   ├── cayo/mkosi.conf        ← Server packages + podman
│   ├── snow/mkosi.conf        ← GNOME desktop packages
│   ├── edge/mkosi.conf        ← Microsoft Edge browser
│   ├── azurevpn/mkosi.conf    ← Azure VPN Client
│   ├── vscode/mkosi.conf      ← Visual Studio Code
│   ├── bitwarden/mkosi.conf   ← Bitwarden password manager
│   ├── paseo/mkosi.conf       ← Paseo desktop app
│   ├── docker-onimage/        ← Docker CE for baked-in images
│   ├── virt-base/mkosi.conf   ← Headless Incus virtualization
│   └── virt/mkosi.conf        ← Incus virtualization
├── scripts/
│   ├── build/                 ← Shared build-time scripts (brew.chroot)
│   └── common-postinst.sh     ← Shared postinstall logic (os-release, manifest)
├── sysext/postoutput/         ← Shared sysext versioning/naming postoutput
├── cayo/
│   ├── tree/                  ← Extra files overlaid into cayo image
│   └── scripts/
│       └── postinstall/       ← Post-installation customizations
├── snow/
│   ├── tree/                  ← Extra files overlaid into image
│   └── scripts/
│       ├── build/             ← Build-time scripts (hotedge, logomenu, bazaar, surface-cert)
│       └── postinstall/       ← Post-installation customizations

Example: snow Profile

The snow profile composes a GNOME desktop image:

[Output]
ImageId=snow
Output=snow
ManifestFormat=json

[Content]
# Overlay additional files into the image
ExtraTrees=%D/shared/snow/tree

# Build-time scripts
BuildScripts=%D/shared/scripts/build/brew.chroot
BuildScripts=%D/shared/snow/scripts/build/hotedge.chroot
BuildScripts=%D/shared/snow/scripts/build/logomenu.chroot
BuildScripts=%D/shared/snow/scripts/build/bazaar.chroot
BuildScripts=%D/shared/snow/scripts/build/surface-cert.chroot

# Post-installation scripts (run after packages installed)
PostInstallationScripts=%D/shared/kernel/scripts/postinst/mkosi.postinst.chroot
PostInstallationScripts=%D/shared/snow/scripts/postinstall/snow.postinst.chroot

# Finalization (prepare for boot)
FinalizeScripts=%D/shared/outformat/image/finalize/mkosi.finalize.chroot

# Post-output (process manifest)
PostOutputScripts=%D/shared/manifest/postoutput/mkosi.postoutput

[Include]
# Package sets
Include=%D/shared/packages/snow/mkosi.conf    # GNOME desktop
Include=%D/shared/kernel/backports/mkosi.conf # Backports kernel
Include=%D/shared/outformat/image/mkosi.conf    # OCI output format

Profile Comparison

Profile	Kernel	Extra Packages	Include Path
snow	backports	—	kernel/backports, packages/snow, outformat/image
snowfield	surface	—	kernel/surface, packages/snow, outformat/image
cayo	backports	—	kernel/backports, packages/cayo, outformat/image

Building Images

Prerequisites

• just task runner
• git and python3
• Root/sudo access (mkosi requires privileges for chroot operations)

mkosi does not need to be installed: the Justfile automatically fetches it into a repo-local .mkosi/ checkout at the same commit pinned in the CI workflows, so local builds always match CI. Delete .mkosi/ to remove it, or run just mkosi=/usr/bin/mkosi <target> to use a system-installed mkosi instead.

Build Commands

# List available build targets
just

# Build base + all 22 system extensions, including Sunshine
just sysexts

# Build snow desktop image
just snow

# Build snowfield (Surface devices)
just snowfield

# Build flurry (Hyprland/Omarchy-replica desktop)
just flurry

# Build cayo server image
just cayo

# Clean build artifacts
just clean

# Run the bootc installation test in QEMU/KVM
just test-install

# Boot a built image in QEMU
just run-qemu

Optional bcvk Testing

bcvk is an optional local convenience for testing the OCI bootc images. It does not replace the supported installation and test path, just test-install, or its CI workflow. bcvk 0.18 does not pull images itself, so pull the image into the same user's Podman storage first:

podman pull ghcr.io/frostyard/cayo:latest
bcvk libvirt run --name test-cayo \
  --detach --ssh-wait \
  --firmware uefi-insecure \
  --composefs-backend --filesystem btrfs \
  ghcr.io/frostyard/cayo:latest
bcvk libvirt ssh test-cayo

The OCI profiles ship bubblewrap for bcvk and already ship virtiofsd. The host still needs bcvk, Podman, libvirt, QEMU/KVM, and UEFI firmware. Native A/B profiles intentionally do not inherit this OCI-only bcvk dependency.

Secure Boot status: OCI bootc profiles include a bootc-only secure composition and a Task 5 two-pass UKI assembly adapter. Protected builds set SNOSI_BOOTC_SECURE=1; Buildah chunks the pristine candidate before that candidate's bootc computes the authoritative OCI composefs digest, constructs a MOK-signed Type #2 UKI plus MOK-signed systemd-boot, retains the signed second stage under /usr/lib/snosi/bootc/ for installed-ESP reconciliation, then derives the final image from the chunked candidate with only /boot overlaid. The final candidate's bootc is the second of exactly two digest probes and must return the same digest; protected assembly never chunks after that overlay. The static reconciler activates after local filesystems without writing /etc, verifies the MOK signer before atomically replacing only shim's grubx64.efi, and allows valid rollback deployments to restore their own stage. It never remounts an already-mounted read-only ESP. The real cayo proof validates immutable-source assembly only; FAT-ESP reconciler execution is deferred to Task 9 secure-install runtime coverage. Secure and insecure images carry explicit io.snosi.bootc.secureboot-capable=true|false labels. This is a maintained, fail-closed compatibility contract for Frostyard bootc 1.16.8, including its hidden storage-digest command and direct two-pass ukify behavior, not an upstream-stable API. See docs/bootc-secure-assembly-compatibility.md. The protected packager checks its pinned bootc through the built rootfs with a temporary procfs bind, then removes that bind before assembly. It does not require a matching host bootc or host libostree ABI; candidate-image bootc remains the storage-digest authority. Direct ukify similarly runs as /usr/bin/ukify inside the pristine first-pass candidate, with networking disabled, individual read-only credential mounts, and a public-only writable work mount. Protected run 30579247524 exposed that the unprivileged candidate cannot read a mode-restricted in-image initramfs. The assembler therefore canonicalizes in-root kernel/initrd sources, stages byte-identical mode-0644 public copies at fixed work paths, compares them with the protected originals after ukify, and validates final UKI sections against those originals. Its pinned systemd-ukify 261.1-3 and dependencies therefore come from the candidate, never the host; the disposable container and mounts do not enter a layer. Host .linux and .initrd byte checks remain valid because first-pass packaging is a byte-identical cp -a snapshot. The candidate runs with all Linux capabilities dropped as the common numeric credential owner, retaining mode-0600 credentials without capabilities. Protected active and optional previous PCR public identities remain outside the writable mount; the copied public inputs are checked against those identities after execution. This remains not production Secure Boot support. Firn is the sole supported secure bootc installer and its enforced-Secure-Boot E2E plus the lab run-firn-install-tests matrix own fresh-install proof. Snosi's former Dakota/Fisherman Task 9 adapter lane is retired. Installed-image update, recovery, rotation, and bootloader-reconciliation evidence still require a Firn-native Snosi lifecycle lane and authorized signed OCI fixtures. Use --firmware uefi-insecure only for bcvk mechanics testing, not Secure Boot validation. Native A/B Secure Boot validation and the established bootc install/test workflow are unaffected.

The secure bootc operations runbook is the normative entry point for the blocked-path status, recovery, rotation, and incident procedures; it does not make the secure fresh-install path supported.

Secure profiles also ship snosi-kargs, a root CLI for persistent machine-specific kernel arguments. It builds a MOK-signed systemd-stub command-line addon at <ESP>/loader/addons/50-snosi-cmdline-local.addon.efi, so one global artifact follows native A/B UKI changes and bootc deployments. Addon arguments are append-only, measured into PCR 12, and deliberately excluded from Snosi's signed-PCR-11 LUKS policy. Dangerous root, verity, LUKS, and emergency arguments are refused unless an operator completes the interactive --force confirmation. Native Secure Boot/TPM/update behavior has QEMU coverage; bootc persistence remains unproven pending the Firn-native lifecycle lane and authorized artifacts.

The image-facing schema-1 requirements are defined by /usr/lib/snosi/bootc-secure.json and validated by test/bootc-secure-install-contract-test.sh. Firn consumes that contract and owns secure fresh installation. The former docs/bootc-secure-install-contract.md Task 9 adapter protocol is retained only as frozen compatibility and fixture history; no workflow clones Dakota or invokes those adapters.

test/bootc-secure-install-test.sh --fixtures validates Task 9's install harness contract without privileged artifacts. This is retained fail-closed fixture coverage, not the supported installer path or E2E evidence. Firn's lab matrix supplies fresh-install evidence. Snowfield additionally requires the existing representative Surface-hardware gate.

test/bootc-secure-update-test.sh --fixtures validates the paired update handoff. Live mode consumes the mode-0600, path-only install-state manifest, immutable N+1/N+2 references with distinct 14-digit image versions, and marked external publisher/negative runners. Every negative case must create a fresh failed update record after its prior runtime state is cleared. The retained manifest's exact TPM state/socket paths are reused; runtime LUKS checks derive the backing /dev path from cryptsetup status root, and MOK verification compares public host/guest certificate fingerprints without passing a host path to the guest. Live update, rollback, and recovery remain unproven until a Snosi-owned Firn-native lifecycle lane supplies an installed-state handoff and authorized secure artifacts.

CI keeps these boundaries explicit. Pull requests build only local bootc mechanics images labelled io.snosi.bootc.secureboot-capable=false; they do not receive publication credentials or write a registry. Protected builds publish an immutable version tag, validate its digest, labels, signature, restrictive policy copy, and artifact before moving latest; a failed candidate cannot move latest. GHCR authentication uses only the run-scoped GITHUB_TOKEN (packages: write for secure publication and packages: read for release discovery); command-line logins receive it through stdin, never process arguments. Shell login steps pass github.actor through a quoted GHCR_USER environment variable instead of interpolating context data into shell source. Native pull requests likewise use disposable RSA-4096 MOK and RSA-2048 PCR credentials and cannot publish. Fixture success proves runner and publication-contract behavior only, not a live secure installation, update, rotation, full rollback window, or Snowfield hardware result.

test/bootc-secure-spike-test.sh separately proves the sealed-UKI bootc chain in a disposable QEMU/OVMF/swtpm environment, including MOK Secure Boot, a measured Type #2 UKI, encrypted DPS-root recovery, signed-PCR-11 TPM auto-unlock after a distinct reboot, and recovery-key retention. It is a feasibility gate only: it validates the compatibility adapter's prerequisite behavior but is not installer proof. Both PR/push contract workflows run its non-root --fixtures mode, the privileged disposable-LUKS recovery-key byte regression, and the Task 3 console-pump socket fixture. They do not run the live QEMU/OVMF feasibility gate.

Build Process

1. Base Build: The base image is built first and cached in output/base/
2. Profile Application: Selected profile's mkosi.conf clears the root sysext dependency list, depends only on base, and includes shared components
3. Package Installation: Packages from all included configs are installed
4. Script Execution: Build → PostInstall → Finalize → PostOutput scripts run in order
5. Output Generation: Final image written to output/ in the configured format

Output Artifacts

output/
├── base/                    # Base image directory (build cache)
├── snow/                    # OCI image directory
├── snow.manifest            # Package manifest (JSON)
├── snow.vmlinuz             # Extracted kernel for boot
├── docker.raw               # Docker sysext (erofs)
├── docker.manifest          # Package manifest
├── incus.raw                # Incus sysext
├── podman.raw               # Podman sysext
└── ...

Repository Configuration

External repositories are configured in mkosi.sandbox/etc/apt/ for packages not in Debian:

• Docker: docker.com official repository
• Incus: Debian trixie (no external repo)
• linux-surface: Surface kernel packages
• Frostyard: Custom packages (nbc, chairlift, updex)

Legacy/archival files under saved-unused/ are kept for historical reference and are not part of active build inputs.

CI/CD Pipeline

The project uses GitHub Actions for automated builds and publishing: Where feasible, third-party workflow actions are pinned to specific commit SHAs to improve reproducibility and supply-chain safety.

build.yml - System Extensions

Triggered on push/PR to main, this workflow:

1. Builds the base image and all 27 sysexts (1password, 1password-cli, azurevpn, bitwarden, chatgpt, claude-desktop, code-server, coder, debdev, dev, docker, edge, github-copilot, incus, k3s, lemonade, localsend, moonlight, nix, obsidian, paseo, pilothouse, podman, sunshine, tailscale, voxtype, vscode). The PR-facing root mkosi build holds only contents: read and no package, OIDC, or attestation write scope.
2. Outside pull requests, publishes sysexts to the Frostyard repository (Cloudflare R2) via the frostyard/repogen action
3. Outside pull requests, uploads package manifests for version tracking

build-images.yml - OCI Images

Triggered on push/PR to main or via repository dispatch, this workflow:

1. Runs an insecure, local-only mechanics matrix for pull requests and a protected native-build matrix for main-branch publication
2. Resets mkosi dependencies to base for each profile build so sysexts are not rebuilt in every matrix job
3. On protected publication, validates the locally assembled secure image, pushes and signs only its immutable version digest, then re-pulls it through the production policy for remote label, signature, and artifact validation
4. Moves latest with a registry-to-registry digest copy only after those checks pass
5. Generates SBOMs (Syft), attaches them via ORAS, signs both images and SBOM artifacts with Cosign, and uploads manifests to R2 after promotion
6. Records the Snow release artifact only after SBOM signing, provenance, and manifest publication all succeed, then on main-branch pushes creates a GitHub Release by diffing against the newest eligible prior Snow marker. Eligibility requires a GitHub Release <!-- snow-tag: ... --> marker and an immutable older image with an exact Syft (application/vnd.syft+json) SBOM referrer; it never falls back to arbitrary registry tags and safely skips when no eligible predecessor exists — see Releases

Verifying image signatures

All published images are signed with a fixed keypair. The public key is committed at cosign.pub; verify any image with cosign (v2.6.x is the tested release — cosign v3 currently trips over the co-published GitHub provenance attestations when doing key verification):

cosign verify --key cosign.pub ghcr.io/frostyard/snow:latest

Images also carry GitHub build-provenance attestations, independently verifiable with:

gh attestation verify oci://ghcr.io/frostyard/snow:latest --owner frostyard

The test-install.yml workflow verifies the signature before every installation test. Secure bootc OCI images additionally enforce this key at pull/install/update time through containers/image policy. The only accepted repositories are ghcr.io/frostyard/cayo, ghcr.io/frostyard/snow, and ghcr.io/frostyard/snowfield; other images, keys, and repository identities are rejected. Cosign v2.6.1 signs repository identities, so the policy uses repository matching rather than tag matching and enables GHCR Sigstore attachments explicitly.

The policy retains global rejection and signed docker registry enforcement. Its only local exception is containers-storage:: after Podman has verified a permitted registry image, bootc may consume that already-local image for its storage-transport update path. This does not accept an additional registry.

This global-reject policy is intentional: Podman and Distrobox cannot pull arbitrary desktop containers by default. Users who need additional images must create ~/.config/containers/policy.json with their own narrowly scoped trust rules; this per-user override does not weaken the system policy used by bootc installation or the root update service.

Other workflows

Workflow	Trigger	Purpose
ai-fix-requested.yml	ai-fix-requested issue label / manual	Validates the request and assigns the issue to the Copilot coding agent
claude.yml	Manual	ACMM-recognized GitHub Actions AI integration marker that documents the active AI issue handoff
triage.yml	Issue open/edit/reopen	Adds a missing classification label from explicit title signals without replacing human labels
build-native-images.yml	Push/PR/repository dispatch/manual	Builds, verifies, and publishes the native A/B product images
build-installer-iso.yml	Relevant main pushes/repository dispatch/manual	Independently builds, boot-verifies, and publishes the Firn installer ISO
check-dependencies.yml	Weekly	Checks pinned direct downloads and inline image-tool pins, opens target-specific PRs
check-packages.yml	Daily	Checks external APT package versions for sysexts, updates package-versions.json, opens PRs
nightly-compliance.yml	Nightly / manual	Re-runs secretless security and publication policy contracts to detect compliance drift
validate.yml	PR/push	shellcheck (all shebang-discovered scripts, -S warning) + guard fixture suites + mkosi summary validation for every profile
test-install.yml	Manual	Signature-verified bootc installation test in QEMU/KVM
scorecard.yml	Weekly	OpenSSF supply-chain security analysis

The Copilot issue handoff workflow uses the fleet-wide COPILOT_ASSIGNMENT_TOKEN organization secret. The operator runbook defines its minimum scope, selected repositories, migration order, validation, failure behavior, and rotation.

Delivery metrics for the change process itself — PR acceptance rate, review iterations, time to merge, and CI first-pass rate, with the exact gh/jq queries that collect each one — are defined in docs/metrics/README.md.

Frostyard Custom Packages

The Frostyard repository provides custom packages for Snow Linux:

• nbc (Not BootC): CLI tool for installing, updating bootc-compatible container based Operating Systems
• chairlift: System extension manager with GUI integration
• updex: Update executor service for applying staged updates
• intuneme: Intune management agent
• snow-first-setup: First-boot setup wizard

Immutable OS Filesystem Layout

The images produced by snosi are immutable atomic systems. Understanding the filesystem layout is essential for packaging decisions:

/                   ← Read-only root filesystem (erofs/squashfs)
├── usr/            ← Read-only, contains all OS binaries and libraries
├── etc/            ← Overlay: base layer from /usr/etc, writes go to persistent storage
├── var/            ← Persistent, writable (logs, caches, container storage, databases)
├── home/           ← Persistent, writable (user data)
├── opt/            ← Bind mount to /var/opt (writable, persistent)
└── run/            ← tmpfs, ephemeral

Key Constraints

Path	Behavior	Implication
/usr/*	Read-only after boot	All binaries, libraries, icons must live here
/etc/*	Overlay on /usr/etc	Base configs in image, user changes persist
/opt/*	Bind mount to /var/opt	Writable, but problematic for sysexts (see below)
/var/*	Persistent, writable	Container storage, logs, state - but not binaries

Why /opt Is Problematic

Many third-party packages (Chrome, Edge, VS Code, Slack, etc.) install to /opt because they expect a traditional mutable filesystem.

On the base bootc image, /opt is a bind mount to /var/opt, making it writable and persistent. This works fine for packages baked into the main image—you relocate them to /usr/lib at build time, and /opt remains available for user-installed software.

However, sysexts change the equation. System extensions use overlay filesystems to merge their contents with the base system. If a sysext contains files in /opt:

1. The sysext merge makes /opt read-only - the overlay takes precedence over the bind mount
2. Applications expecting writable /opt break - they can no longer write configs, caches, or updates
3. The bind mount to /var/opt is shadowed - user data in /var/opt becomes inaccessible

This is why we always relocate /opt contents to /usr/lib during build, for both main images and sysexts. It keeps /opt available as a writable bind mount for runtime use while ensuring package binaries are in the read-only, atomically-updated /usr tree.

Extending the Build

Adding a New Package Set

Most packages "just work" - you add them to a mkosi.conf and they install correctly to /usr. However, some packages require post-installation scripts to relocate files or fix paths.

Simple Package (No Scripts Needed)

For packages that install to standard locations (/usr/bin, /usr/lib, /usr/share):

1. Create shared/packages/mypackages/mkosi.conf:

   [Content]
   Packages=package1
            package2

2. Include it in a profile:

   [Include]
   Include=%D/shared/packages/mypackages/mkosi.conf

Complex Package Example: Microsoft Edge

Microsoft Edge installs to /opt/microsoft/msedge/, which won't work on an immutable OS. The edge package includes a post-installation script to fix this:

Directory structure:

shared/packages/edge/
├── mkosi.conf                 # Package definition
└── mkosi.postinst.d/
    └── edge.chroot            # Post-installation script

mkosi.conf:

[Content]
Packages=microsoft-edge-stable

edge.chroot: (runs inside the build chroot)

#!/bin/bash
set -euo pipefail

# Move Edge from /opt to /usr/lib (read-only safe location)
mv /opt/microsoft/msedge /usr/lib/microsoft-edge
rm -rf /opt/microsoft

# Create symlink for the binary
ln -sf /usr/lib/microsoft-edge/microsoft-edge /usr/bin/microsoft-edge-stable

# Fix icon paths (Edge expects /opt paths)
mkdir -p /usr/share/icons/hicolor/{16x16,24x24,32x32,48x48,64x64,128x128,256x256}/apps
for size in 16 24 32 48 64 128 256; do
    ln -sf /usr/lib/microsoft-edge/product_logo_${size}.png \
           /usr/share/icons/hicolor/${size}x${size}/apps/microsoft-edge.png
done

# Fix GNOME Control Center default apps XML
sed -i 's|/opt/microsoft/msedge/microsoft-edge|/usr/lib/microsoft-edge/microsoft-edge|g' \
    /usr/share/gnome-control-center/default-apps/microsoft-edge.xml

Sysext usage (mkosi.images/edge/mkosi.conf):

[Content]
PostInstallationScripts=%D/shared/packages/edge/mkosi.postinst.d/edge.chroot

[Include]
Include=%D/shared/packages/edge/mkosi.conf

When You Need Post-Installation Scripts

You need a mkosi.postinst.chroot script when a package:

Issue	Solution
Installs binaries to /opt	Move to /usr/lib/<package>, symlink binary to /usr/bin
Has hardcoded /opt paths in configs	Use sed to rewrite paths
Expects to write to /etc at install time	Move default configs to /usr/share/factory/etc
Creates state directories in wrong locations	Ensure state goes to /var
Relies on update-alternatives	Create symlinks manually

Adding a New Profile

1. Create mkosi.profiles/myprofile/mkosi.conf
2. Set output name and include required components
3. Add post-installation scripts for any packages that need relocation
4. Add a just target:

   myprofile: clean
       mkosi --profile myprofile build

Adding a New Sysext

System extensions have additional constraints beyond regular packages because they overlay onto an already-running immutable system.

Sysext Filesystem Constraints

mysysext.raw (erofs image)
└── usr/                    ← ONLY /usr is merged into the base system
    ├── bin/
    ├── lib/
    └── share/

Sysexts can only provide files under /usr. They cannot:

• Add files to /etc (the overlay is already mounted)
• Add files to /var (it's persistent state, not part of the image)
• Run post-installation scripts on the target system (no dpkg triggers)

Sysext Script Types

Script	When It Runs	Purpose
mkosi.postinst.chroot	Build time, in chroot	Relocate files, fix paths
mkosi.finalize	Build time, outside chroot	Capture needed /etc paths to factory defaults
shared postoutput (PostOutputScripts=)	After image creation	Versioned naming + manifest processing — every sysext points at shared/sysext/postoutput/sysext-postoutput.sh and sets Environment=KEYPACKAGE=; there is no per-sysext postoutput script

Example: Incus Sysext

The incus sysext needs special handling because:

1. Incus packages install configs to /etc - but sysexts can't modify /etc at runtime
2. The sysext needs versioned filenames - for update management

mkosi.finalize: (captures the needed /etc paths for tmpfiles.d)

#!/bin/bash
set -euo pipefail

# Capture ONLY the /etc paths referenced by tmpfiles.d C directives.
# Never capture all of /etc: the buildroot /etc is the merged base view,
# so a full capture ships the base image's /etc/shadow and SSH host keys
# in the published sysext.
FACTORY="$BUILDROOT/usr/share/factory/etc"
mkdir -p "$FACTORY"

# Paths relative to /etc, matching the C directives in usr/lib/tmpfiles.d/incus.conf
FACTORY_PATHS=(
    libnl-3
    default/incus
    logrotate.d/incus
    needrestart/conf.d/incus.conf
    libvirt
    profile.d/vte-2.91.sh
    profile.d/vte.csh
    qemu-ifdown
    qemu-ifup
)

cd "$BUILDROOT/etc"
for path in "${FACTORY_PATHS[@]}"; do
    if [ -e "$path" ]; then
        cp --archive --parents --update=none "$path" "$FACTORY/"
    else
        echo "incus finalize: /etc/$path not present in buildroot; factory capture skipped" >&2
    fi
done

This pattern allows configs to be "injected" into /etc via systemd-tmpfiles rules when the sysext is activated.

Versioned naming is handled by the shared postoutput script — the sysext's mkosi.conf wires PostOutputScripts=%D/shared/sysext/postoutput/sysext-postoutput.sh and sets Environment=KEYPACKAGE=incus; the script reads the key package's version from the manifest and renames the output to incus_<version>_<os>_<arch>.raw. Do not create per-sysext postoutput scripts.

Sysext Checklist

When creating a new sysext, verify:

• All binaries are under /usr/bin or /usr/lib
• No files in /opt (relocate during build)
• Configs captured to /usr/share/factory/etc if needed
• No runtime dependencies on post-install scripts
• Symlinks/alternatives created manually (no update-alternatives)
• State directories expected in /var (not baked into image)
• Use tmpfiles.d, sysusers.d and systemd presets first, as a last resort add a one-shot systemd unit for any preconfiguration that usually would happen in the debian package's postinst scripts
• If the sysext ships a systemd service: add usr/lib/systemd/system/multi-user.target.d/10-<name>.conf with [Unit]\nUpholds=<name>.service — do NOT rely on WantedBy= + preset alone (the sysext isn't merged when PID 1 first scans units)

Basic Sysext Template

# mkosi.images/mysysext/mkosi.conf
[Config]
Dependencies=base

[Output]
ImageId=mysysext
Output=mysysext
Overlay=yes
ManifestFormat=json
Format=sysext

[Content]
Bootable=no
BaseTrees=%O/base
PostOutputScripts=%D/shared/sysext/postoutput/sysext-postoutput.sh

Packages=mypackage

[Build]
Environment=KEYPACKAGE=mypackage

Then register it: add mysysext to the root mkosi.conf Dependencies= list, and create mysysext.transfer + mysysext.feature in their own component directory, mkosi.images/base/mkosi.extra/usr/lib/sysupdate.mysysext.d/ (copy an existing component directory as a template).

If the package needs relocation, add:

# mkosi.images/mysysext/mkosi.postinst.chroot
#!/bin/bash
set -euo pipefail

# Move from /opt to /usr/lib
mv /opt/mypackage /usr/lib/mypackage
ln -sf /usr/lib/mypackage/bin/mybin /usr/bin/mybin

# mkosi.images/mysysext/mkosi.finalize
#!/bin/bash
set -e

# Capture /etc for systemd-tmpfiles
mkdir -p "$BUILDROOT/usr/share/factory/"
cp --archive --no-target-directory --update=none \
   "$BUILDROOT/etc" "$BUILDROOT/usr/share/factory/etc"

Adding External Downloads with Checksum Verification

Some build scripts download files directly from external URLs instead of resolving them through APT. These downloads use SHA256 checksum verification for security, reproducibility, and reviewable update PRs.

Files Involved

shared/download/
├── verified-download.sh      # Helper function for verified downloads
├── sysext-checksums.json     # Direct downloads consumed by mkosi.images/* sysexts
├── image-checksums.json      # Direct downloads consumed by OCI profile builds
├── package-versions.json     # APT package version sentinels for sysexts
└── update-checksums.sh       # Manual helper to update an existing checksum key

The checksum files have the same schema. They are split by the build artifact that must be rebuilt when a dependency changes:

{
  "bitwarden": {
    "url": "https://github.com/bitwarden/clients/releases/download/desktop-v2026.6.1/Bitwarden-2026.6.1-amd64.deb",
    "sha256": "421bfc6d787d842406909d393c2a9044791e957aa234718ae5670e87ae4deba7",
    "version": "2026.6.1"
  }
}

Use the target-specific file:

Dependency kind	Metadata file	Rebuild workflow
Direct download used by a sysext (mkosi.images/<name>/... or shared/packages/<app>/... consumed only by sysexts)	shared/download/sysext-checksums.json	build.yml
Direct download used by profile/image build scripts (shared/scripts/build/, shared/snow/scripts/build/, shared/cayo/...)	shared/download/image-checksums.json	build-images.yml
External APT package installed by a sysext through Packages=	shared/download/package-versions.json	build.yml

This routing keeps sysext-only dependency updates from spending the larger OCI image matrix. A dependency consumed by both sysexts and OCI profile builds needs an explicit design choice: split the consumers so each target has its own metadata entry, or update the workflow filters in the same PR. Do not hide a shared build input in one target-specific checksum file and assume both build workflows will run.

Using Verified Downloads in Build Scripts

In any .chroot build script:

#!/bin/bash
set -euo pipefail

source "$SRCDIR/shared/download/verified-download.sh"
verified_download "mykey" "/path/to/output"

The verified_download function:

1. Searches sysext-checksums.json and image-checksums.json for the key
2. Downloads the file with retries
3. Verifies the SHA256 checksum matches
4. Fails the build with a clear error if verification fails

Set CHECKSUMS_FILE=/path/to/file.json before sourcing the helper only when a script must intentionally restrict lookup to one explicit metadata file.

Adding a New External Download

1. Choose the metadata file by consumer.

   • Sysext-only direct download: shared/download/sysext-checksums.json
   • OCI profile direct download: shared/download/image-checksums.json
   • Sysext APT package, no direct URL/checksum: shared/download/package-versions.json

2. Add the checksum entry.

   curl -fsSL -o /tmp/myfile "https://example.com/myfile.tar.gz"
   sha256sum /tmp/myfile

   Then add to the selected checksum file:

   {
     "mykey": {
       "url": "https://example.com/myfile.tar.gz",
       "sha256": "<computed_sha256>",
       "version": "1.2.3"
     }
   }

   For an existing key, the helper updates whichever split checksum file already contains the key:

   ./shared/download/update-checksums.sh mykey "https://example.com/myfile.tar.gz" "1.2.3"

3. Use the key in the build script.

   source "$SRCDIR/shared/download/verified-download.sh"
   verified_download "mykey" "/tmp/myfile.tar.gz"

4. Add the update check.

   • Sysext direct downloads go in the check-sysext-updates job in .github/workflows/check-dependencies.yml.
   • OCI profile direct downloads go in the check-image-updates job in .github/workflows/check-dependencies.yml.
   • Sysext APT package sentinels go in .github/workflows/check-packages.yml and shared/download/package-versions.json.

Pinning Strategy

• GitHub releases: Use the direct release asset URL with version in path (not latest redirects)
• Raw files from repos: Pin to a specific commit SHA, not HEAD or branch names
• Version field: Store the version/commit for tracking; the GitHub Action uses this to detect updates

Automated Update Checking

The .github/workflows/check-dependencies.yml workflow runs weekly and has two independent jobs:

1. check-sysext-updates compares sysext direct-download pins and opens auto-update-sysext-checksums PRs against sysext-checksums.json.
2. check-image-updates compares OCI profile direct-download pins plus the inline Syft, Cosign v2, and chunkah pins, then opens auto-update-image-checksums PRs against the affected files.

Syft and Cosign updates modify .github/workflows/build-images.yml, so GitHub requires the WORKFLOW_PAT repository secret to have workflow-write access before those edits can be pushed. Without that secret, the job warns and skips only those workflow-file pins; checksum and chunkah updates still produce PRs using GITHUB_TOKEN.

The .github/workflows/check-packages.yml workflow runs daily for external APT packages consumed by sysexts (code, docker-ce, 1password-cli, claude-desktop, chatgpt). It updates package-versions.json, which is only a change-detection sentinel; mkosi still resolves the package from APT during the sysext build. The job has a 15-minute timeout so a stalled external APT request cannot retain repository write permissions for GitHub Actions' six-hour default.

To check manually or trigger an update PR, use the "Run workflow" button in GitHub Actions.

License

See individual package licenses. This build system configuration is provided as-is.