fix(release): restore installs below macOS 26.2 with a baseline metallib, and stop misreporting the failure

.github/workflows/ci.yml

@@ -15,6 +15,8 @@ jobs:
         run: |
           ./scripts/check-release-version.sh
           ./scripts/sync-install-embed.sh check
+          ./scripts/check-macos-floor.sh
+          ./scripts/test-install-preflight.sh
           ./scripts/test-prod-env-refresh.sh
 
   test-coordinator:
@@ -284,6 +286,11 @@ jobs:
         # green under.
         timeout-minutes: 10
         run: ../../scripts/run-nested-suite.sh CBv2KVSharingParityTests
+      # The floor gate ships to macOS /bin/bash 3.2; Release Integrity only ever
+      # runs it under Ubuntu bash 5.x.
+      - name: Run installer preflight tests (macOS bash 3.2)
+        if: ${{ !cancelled() }}
+        run: ./scripts/test-install-preflight.sh
       - name: Run atomic installer artifact tests
         timeout-minutes: 2
         run: ./scripts/test-install-atomic.sh

CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## Unreleased
+
+### Provider (Swift)
+
+#### Fixes
+
+- **Restore installs on macOS below 26.2** — Since v0.8.2 the release workflow has built the shipped `mlx.metallib` itself with `MLX_METALLIB_DEPLOYMENT_TARGET=26.2` (required for the M5 `_nax` kernels). A metallib linked `-mmacosx-version-min=26.2` is rejected by every older Metal runtime, so MLX's `Device()` constructor threw while loading its default library and every Mac below macOS 26.2 failed to install or self-update — even though `install.sh` advertised macOS 14+ and `hardware-support.md` claimed macOS 15+. Releases now also build a NAX-free baseline library for the floor and stage it at `Contents/MacOS/Resources/mlx.metallib`, MLX's own second colocated probe, which is reached only when the primary fails to load. macOS 26.2+ hosts keep the NAX kernels; older hosts land on the baseline, which they never ask NAX kernels of because `is_nax_available()` is gated on macOS 26.2. Because MLX resolves those paths against the running executable's directory, the installer and self-update mirror both libraries into `~/.darkbloom/bin` (and retire the baseline mirror on a downgrade), or a `$PATH` invocation would still have no fallback. A signed `baseline-metallib-v1` capability marker couples library and marker in installer and self-update artifact verification, and the release workflow runs the full packaged smoke twice more: once with the baseline in the primary slot, and once with a corrupted primary beside a good baseline so MLX's probe order itself is pinned.
+- **Attest the kernel library that actually runs** — `template_hashes["mlx_metallib"]` reports `Contents/MacOS/mlx.metallib`, which below macOS 26.2 is precisely the library the process cannot load. Providers now also report `mlx_metallib_baseline`, the release registers it through the existing comma-separated `template_hashes` field (no schema change), and the coordinator scopes routing verification to both keys. The baseline expectation is enforced only when the provider reports it, so registering it cannot deroute the pre-baseline fleet or a macOS 26.2+ provider with nothing to attest. Without this, a swapped baseline would have been invisible to every runtime check on the entire population the fix serves.
+- **Stop misreporting an unusable Metal runtime as a safe-R1 latch failure** — `mlx_metal_gemma4_expert_qmm_diagnostics_snapshot` catches every exception out of `metal::device(gpu)` and returns an all-zero struct, which is indistinguishable from a route that was never requested. Because the packaged smoke has already proved `MLX_GATHER_QMM_EXPERT_SLICES=1` is live in the process, it now probes Metal directly on an unrequested route and reports either "no Metal device" or the host OS plus the real `NSError` from each packaged kernel library it tried. `install.sh` prints the child's diagnosis instead of a bare "smoke failed" line.
+- **Refuse to install below the supported macOS floor** — `install.sh` checks `sw_vers -productVersion` against `MIN_MACOS` before downloading, rather than failing after a multi-gigabyte download and a staged-app verification error, and `darkbloom update` pauses on sub-floor hosts instead of re-downloading a candidate that can never pass staged verification. `LSMinimumSystemVersion` and `Package.swift` move from macOS 14 to the same floor, as do the landing page, console setup page, and provider hardware doc. `scripts/check-macos-floor.sh` (new, wired into CI) pins the floor and both metallib deployment targets across the installers, the release workflow, `scripts/fetch-metallib.sh`, `Package.swift`, `PackagedMetallib.swift`, and every surface that advertises a floor.
+
 ## v0.8.9 (2026-08-21)
 
 ### Provider (Swift)

console-ui/src/app/providers/setup/page.tsx

@@ -93,8 +93,8 @@ const REQUIREMENTS = [
   },
   {
     icon: Monitor,
-    title: "macOS 14.0+",
-    description: "Sonoma or later required for Secure Enclave attestation and hardware security features.",
+    title: "macOS 15.0+",
+    description: "Sequoia or later required for the bundled Metal GPU kernels, Secure Enclave attestation, and hardware security features.",
   },
   {
     icon: Wifi,

coordinator/api/install.sh

@@ -14,17 +14,22 @@ set -euo pipefail
 #   5. Optionally enrolls in MDM (device attestation)
 #   6. Optionally downloads a starter model
 #
-# Zero prerequisites — just macOS 14+ on Apple Silicon. The Swift CLI
-# links mlx-swift directly and ships a colocated mlx.metallib for Metal
-# kernels; there is no Python interpreter to install and no inference
-# subprocess to spawn.
+# Zero prerequisites — just macOS 15+ on Apple Silicon. The Swift CLI
+# links mlx-swift directly and ships colocated Metal kernel libraries;
+# there is no Python interpreter to install and no inference subprocess
+# to spawn.
 
 # This source intentionally retains the placeholder used by the coordinator.
 # End users fetch /install.sh from a coordinator, which substitutes its own base
 # URL. Direct source execution must set COORD_URL explicitly.
 COORD_URL="${COORD_URL:-__DARKBLOOM_COORD_URL__}"
 INSTALL_DIR="$HOME/.darkbloom"
 BIN_DIR="$INSTALL_DIR/bin"
+# Oldest macOS the packaged app can serve. The Swift binary itself runs older,
+# but MLX cannot start without a loadable Metal kernel library, and the baseline
+# library shipped in the bundle is built for this floor. Pinned against
+# PackagedMetallib.swift and the release workflow by scripts/check-macos-floor.sh.
+MIN_MACOS="15.0"
 DARKBLOOM_DESIGNATED_REQUIREMENT='anchor apple generic and identifier "io.darkbloom.provider" and certificate leaf[subject.OU] = "SLDQ2GJ6TL"'
 DARKBLOOM_FAN_HELPER_REQUIREMENT='anchor apple generic and identifier "io.darkbloom.fan-helper" and certificate leaf[subject.OU] = "SLDQ2GJ6TL"'
 FAN_HELPER_REQUIREMENT="$DARKBLOOM_FAN_HELPER_REQUIREMENT"
@@ -35,6 +40,27 @@ fail_install() {
     return 1
 }
 
+# Dotted numeric compare, base-macOS only (no sort -V, no developer tools).
+version_at_least() {
+    local have=$1
+    local want=$2
+    local index have_part want_part
+    local -a have_parts want_parts
+    IFS=. read -r -a have_parts <<< "$have"
+    IFS=. read -r -a want_parts <<< "$want"
+    for index in 0 1 2; do
+        have_part=${have_parts[index]:-0}
+        want_part=${want_parts[index]:-0}
+        have_part=${have_part//[!0-9]/}
+        want_part=${want_part//[!0-9]/}
+        have_part=${have_part:-0}
+        want_part=${want_part:-0}
+        if [ "$((10#$have_part))" -gt "$((10#$want_part))" ]; then return 0; fi
+        if [ "$((10#$have_part))" -lt "$((10#$want_part))" ]; then return 1; fi
+    done
+    return 0
+}
+
 verify_file_hash() {
     local file=$1
     local expected=$2
@@ -115,6 +141,38 @@ verify_fan_helper_capability() {
     }
 }
 
+# The primary mlx.metallib is built for macOS 26.2 so it can carry the M5 _nax
+# kernels, which no older Metal runtime will load. Releases therefore also ship
+# a NAX-free baseline at Contents/MacOS/Resources/mlx.metallib — MLX's own
+# second colocated probe, reached only when the primary fails to load. Marker
+# and file must appear together; pre-baseline releases have neither and stay
+# installable.
+verify_baseline_metallib_capability() {
+    local app=$1
+    local baseline="$app/Contents/MacOS/Resources/mlx.metallib"
+    local marker="$app/Contents/Resources/darkbloom-runtime-capabilities/baseline-metallib-v1"
+    local baseline_present=0
+    local marker_present=0
+
+    if [ -e "$baseline" ] || [ -L "$baseline" ]; then baseline_present=1; fi
+    if [ -e "$marker" ] || [ -L "$marker" ]; then marker_present=1; fi
+    [ "$baseline_present" -eq "$marker_present" ] || {
+        fail_install "Baseline Metal kernel library and its signed marker must be present together."
+        return 1
+    }
+    [ "$baseline_present" -eq 1 ] || return 0
+
+    [ -f "$baseline" ] && [ ! -L "$baseline" ] && [ -s "$baseline" ] || {
+        fail_install "Baseline Metal kernel library must be a non-empty regular file."
+        return 1
+    }
+    [ -f "$marker" ] && [ ! -L "$marker" ] \
+        && [ "$(tr -d '[:space:]' < "$marker")" = "1" ] || {
+        fail_install "Baseline metallib capability marker is invalid."
+        return 1
+    }
+}
+
 verify_staged_app() {
     local app=$1
     local executable="$app/Contents/MacOS/darkbloom"
@@ -129,6 +187,7 @@ verify_staged_app() {
         verify_staged_app_signature "$app" || return 1
     fi
     verify_fan_helper_capability "$app" || return 1
+    verify_baseline_metallib_capability "$app" || return 1
 
     local code_has_paged=0
     local marker_present=0
@@ -161,9 +220,15 @@ verify_staged_app() {
             return 1
         }
 
-    DARKBLOOM_NO_UPDATE_CHECK=1 "$executable" runtime-smoke >/dev/null \
+    # Keep the child's own diagnosis: it is the only place that can tell
+    # "this Mac cannot load the packaged Metal kernels" apart from a genuine
+    # kernel fault, and a bare "smoke failed" line sends every reader hunting
+    # the wrong bug.
+    local smoke_output=""
+    smoke_output=$(DARKBLOOM_NO_UPDATE_CHECK=1 "$executable" runtime-smoke 2>&1) \
         || {
-            fail_install "Packaged paged-kernel runtime smoke failed."
+            printf '%s\n' "$smoke_output" | tail -n 5 | sed 's/^/    /' >&2
+            fail_install "Packaged runtime smoke failed."
             return 1
         }
 }
@@ -181,6 +246,22 @@ verify_staged_app_payload() {
         && verify_file_hash "$app_bin/mlx.metallib" "$metallib_hash" "App metallib"
 }
 
+# MLX resolves its metallib probes against the directory of the RUNNING
+# executable, which for a $PATH invocation through bin/darkbloom is bin/, not
+# the app bundle. That is why bin/mlx.metallib exists at all, and the baseline
+# library needs the same mirror or a macOS 15 host loses the fallback for every
+# foreground command. Retired when the installed app has no baseline, so a
+# downgrade cannot leave a dangling probe behind.
+link_baseline_metallib() {
+    local install_dir=$1
+    local app_bin=$2
+    rm -rf "$install_dir/bin/Resources" || return 1
+    [ -f "$app_bin/Resources/mlx.metallib" ] || return 0
+    mkdir -p "$install_dir/bin/Resources" || return 1
+    ln -sfn "../../Darkbloom.app/Contents/MacOS/Resources/mlx.metallib" \
+        "$install_dir/bin/Resources/mlx.metallib"
+}
+
 commit_staged_app() {
     local staged_app=$1
     local install_dir=$2
@@ -207,7 +288,8 @@ commit_staged_app() {
     if ! ln -sfn "../Darkbloom.app/Contents/MacOS/darkbloom" "$install_dir/bin/darkbloom" \
         || ! ln -sfn "../Darkbloom.app/Contents/MacOS/darkbloom-enclave" "$install_dir/bin/darkbloom-enclave" \
         || ! ln -sfn "../Darkbloom.app/Contents/MacOS/mlx.metallib" "$install_dir/bin/mlx.metallib" \
-        || ! ln -sfn "darkbloom-enclave" "$install_dir/bin/eigeninference-enclave"
+        || ! ln -sfn "darkbloom-enclave" "$install_dir/bin/eigeninference-enclave" \
+        || ! link_baseline_metallib "$install_dir" "$app_bin"
     then
         rm -rf "$destination"
         [ "$had_previous" -eq 1 ] \
@@ -311,6 +393,20 @@ install_bundle_atomically() {
     rm -rf "$stage"
 }
 
+if [ "${1:-}" = "--version-gate-test" ]; then
+    [ "$#" -eq 3 ] || {
+        echo "usage: $0 --version-gate-test <have> <want>" >&2
+        exit 64
+    }
+    version_at_least "$2" "$3"
+    exit $?
+fi
+
+if [ "${1:-}" = "--min-macos-test" ]; then
+    printf '%s\n' "$MIN_MACOS"
+    exit 0
+fi
+
 if [ "${1:-}" = "--verify-staged-app-signature-test" ]; then
     [ "$#" -eq 3 ] || {
         echo "usage: $0 --verify-staged-app-signature-test <app> <requirement>" >&2
@@ -356,10 +452,25 @@ fi
 CHIP=$(sysctl -n machdep.cpu.brand_string 2>/dev/null || echo "Apple Silicon")
 MEM=$(sysctl -n hw.memsize 2>/dev/null | awk '{printf "%.0f", $1/1073741824}')
 SERIAL=$(ioreg -c IOPlatformExpertDevice -d 2 | awk -F'"' '/IOPlatformSerialNumber/{print $4}')
-MACOS=$(sw_vers -productVersion 2>/dev/null || echo "?")
+# env -u SYSTEM_VERSION_COMPAT: with that variable set (compatibility shims,
+# some CI images) sw_vers reports 10.16 on every modern macOS, which would
+# refuse the install on a perfectly good host.
+MACOS=$(env -u SYSTEM_VERSION_COMPAT sw_vers -productVersion 2>/dev/null || echo "?")
+[ -n "$MACOS" ] || MACOS="?"
 echo "  $CHIP · ${MEM}GB · macOS $MACOS"
 echo ""
 
+# Below the floor the bundled Metal kernel libraries cannot be loaded at all, so
+# MLX never starts. Say so here rather than after a multi-gigabyte download and
+# a staged-app verification failure. An unreadable version is not a refusal:
+# the staged-app smoke still fails closed if the kernels really cannot load.
+if [ "$MACOS" != "?" ] && ! version_at_least "$MACOS" "$MIN_MACOS"; then
+    echo "Error: Darkbloom requires macOS $MIN_MACOS or later (found $MACOS)."
+    echo "       The bundled Metal GPU kernels cannot be loaded on this version."
+    echo "       Update in System Settings > General > Software Update."
+    exit 1
+fi
+
 # ─── Step 1: Fetch latest release ────────────────────────────
 echo "→ [1/5] Fetching latest release from $COORD_URL ..."
 

coordinator/api/runtime_manifest_test.go

@@ -48,6 +48,110 @@ func TestSyncRuntimeManifestIncludesSwiftMetallibHash(t *testing.T) {
 	}
 }
 
+// Providers below macOS 26.2 execute the BASELINE kernel library, not the
+// primary, so `mlx_metallib` alone attests a library that host never runs.
+func TestVerifyRuntimeHashesChecksBaselineMetallibWhenReported(t *testing.T) {
+	metallibHash := strings.Repeat("a", 64)
+	baselineHash := strings.Repeat("d", 64)
+	tamperedBaseline := strings.Repeat("e", 64)
+
+	newServer := func(t *testing.T) *Server {
+		t.Helper()
+		srv, _ := runtimeManifestTestServer(t)
+		srv.SetRuntimeManifest(&RuntimeManifest{
+			PythonHashes:  map[string]bool{},
+			RuntimeHashes: map[string]bool{},
+			TemplateHashes: map[string]string{
+				metallibTemplateKey:         metallibHash,
+				baselineMetallibTemplateKey: baselineHash,
+			},
+		})
+		return srv
+	}
+
+	t.Run("matching baseline passes", func(t *testing.T) {
+		ok, mismatches := newServer(t).verifyRuntimeHashesForBackend(
+			"mlx-swift", "", "", map[string]string{
+				metallibTemplateKey:         metallibHash,
+				baselineMetallibTemplateKey: baselineHash,
+			})
+		if !ok {
+			t.Fatalf("expected pass, got mismatches: %+v", mismatches)
+		}
+	})
+
+	t.Run("swapped baseline is caught", func(t *testing.T) {
+		ok, mismatches := newServer(t).verifyRuntimeHashesForBackend(
+			"mlx-swift", "", "", map[string]string{
+				metallibTemplateKey:         metallibHash,
+				baselineMetallibTemplateKey: tamperedBaseline,
+			})
+		if ok {
+			t.Fatal("a swapped baseline metallib was accepted")
+		}
+		if len(mismatches) != 1 ||
+			mismatches[0].Component != "template:"+baselineMetallibTemplateKey ||
+			mismatches[0].Got != tamperedBaseline {
+			t.Fatalf("unexpected mismatches: %+v", mismatches)
+		}
+	})
+
+	// Registering the expectation must not deroute the fleet that predates the
+	// two-library layout, nor a macOS 26.2+ provider with nothing to attest.
+	t.Run("a provider that reports no baseline still passes", func(t *testing.T) {
+		ok, mismatches := newServer(t).verifyRuntimeHashesForBackend(
+			"mlx-swift", "", "", map[string]string{
+				metallibTemplateKey: metallibHash,
+			})
+		if !ok {
+			t.Fatalf("pre-baseline provider was fenced: %+v", mismatches)
+		}
+	})
+
+	// And the primary is still required, baseline or not.
+	t.Run("a swapped primary is still caught", func(t *testing.T) {
+		ok, _ := newServer(t).verifyRuntimeHashesForBackend(
+			"mlx-swift", "", "", map[string]string{
+				metallibTemplateKey:         strings.Repeat("f", 64),
+				baselineMetallibTemplateKey: baselineHash,
+			})
+		if ok {
+			t.Fatal("a swapped primary metallib was accepted")
+		}
+	})
+}
+
+func TestSyncRuntimeManifestCarriesBaselineMetallibFromTemplateHashes(t *testing.T) {
+	srv, st := runtimeManifestTestServer(t)
+	baselineHash := strings.Repeat("d", 64)
+
+	// The release workflow ships the baseline expectation through the existing
+	// comma-separated template_hashes field, so no schema change is needed.
+	if err := st.SetRelease(&store.Release{
+		Version:        "0.9.0",
+		Platform:       "macos-arm64",
+		Backend:        "mlx-swift",
+		BinaryHash:     strings.Repeat("b", 64),
+		BundleHash:     strings.Repeat("c", 64),
+		MetallibHash:   strings.Repeat("a", 64),
+		TemplateHashes: baselineMetallibTemplateKey + "=" + baselineHash,
+		URL:            "https://example.com/swift.tar.gz",
+		Active:         true,
+	}); err != nil {
+		t.Fatalf("SetRelease: %v", err)
+	}
+
+	srv.SyncRuntimeManifest()
+
+	if srv.knownRuntimeManifest == nil {
+		t.Fatal("knownRuntimeManifest = nil")
+	}
+	got := srv.knownRuntimeManifest.TemplateHashes[baselineMetallibTemplateKey]
+	if got != baselineHash {
+		t.Fatalf("%s = %q, want %q", baselineMetallibTemplateKey, got, baselineHash)
+	}
+}
+
 func TestVerifyRuntimeHashesForSwiftRequiresMetallibButNotLegacyRuntime(t *testing.T) {
 	srv, _ := runtimeManifestTestServer(t)
 	metallibHash := strings.Repeat("a", 64)