omacom · 0xSero · Sep 7, 2026
@@ -58,6 +58,7 @@ GROUP_DESCRIPTIONS[hw]="Hardware detection and controls"
 GROUP_DESCRIPTIONS[hyprland]="Hyprland window, monitor, and toggle controls"
 GROUP_DESCRIPTIONS[install]="Optional software installers"
 GROUP_DESCRIPTIONS[installed]="Installed optional service checks"
+GROUP_DESCRIPTIONS[local]="Local AI models on your GPU"
 GROUP_DESCRIPTIONS[launch]="Application launchers"
 GROUP_DESCRIPTIONS[menu]="Omarchy menu commands"
 GROUP_DESCRIPTIONS[migrate]="Migration runner"

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+# omarchy:summary=Run the validated local model for this GPU and open coding agents on it
+# omarchy:args=<snapshot|load|unload|open-agent [name]|share [--key value]|gpu [auto|key]|agent-dir <path>|agent-args <name> [-- flags]>
+# omarchy:examples=omarchy local ai load | omarchy local ai open-agent claude | omarchy local ai share
+
+exec "${OMARCHY_PATH:-$(cd -- "$(dirname -- "$0")/.." && pwd)}/shell/plugins/panels/local-ai/bin/omarchy-local-ai" "$@"
@@ -54,6 +54,12 @@ Hermes Desktop is the one to know about, because there is only ever one Hermes o
 
 OpenClaw's desktop experience is its Control UI, which opens as a web app backed by its own local gateway. OpenClaw updates arrive through Omarchy's package updates, so skip the Control UI's own "Update Gateway" button: it would try to write into the package-managed install and fail. Removing OpenClaw under _Remove > AI_ takes the gateway service and the app with it and then asks whether `~/.openclaw` should go too, since that holds your chats and credentials alongside the plugin runtimes OpenClaw downloads for itself; the default keeps it.
 
+### The Local AI panel
+
+Machines with a supported GPU and Docker can run a local model as one button on the bar. Enable it with `omarchy plugin enable omarchy.local-ai`. The card shows the one model validated for your card (the tier map picks the biggest model that fits, from a 2.6B on 8 GB cards to a 27B on 32 GB and up), what GPUs were detected, and which one is in use; more than one card is a picker. Start downloads the weights, with the size shown before anything lands on disk, then serves the model and proves it works before calling it ready. Stop removes it and keeps the download. Open agent starts any installed coding agent on the running model, with the endpoint and key in that agent's own environment and nothing written to its config. Share on Tailscale publishes the same endpoint, keyed, on your tailnet.
+
+Everything runs as two Docker containers only this plugin touches: the engine, never reachable from the host, and a gateway on `127.0.0.1:12434` that speaks the OpenAI chat, Anthropic Messages, and OpenAI Responses dialects. Recipes are vendored from the [local-ai-registry](https://github.com/0xSero/local-ai-registry), pinned by image digest and model revision, and validated on the card they name. The same verbs are available as `omarchy local ai load`, `unload`, `open-agent`, `share`, and `gpu`; every step is written to `~/.local/state/omarchy/local-ai/log`.
+
 ### Local LLMs
 
 Omarchy recommends two ways of running local LLM models: LM Studio and Ollama. LM Studio provides a GUI interface for finding open-weight models, installing them, and running them. It's a great way to get going easily. Ollama offers a CLI for doing so similarly. But if you're new to local models, I'd start with LM Studio. You can install either under _Install > AI_ in the Omarchy Menu.

@@ -28,6 +28,7 @@ User-installed plugins live alongside these conceptually but on disk under
 | Power         | `omarchy.power`           | `bar-widget`            | `panels/power/Panel.qml`              |
 | Tailscale     | `omarchy.tailscale`       | `bar-widget`            | `panels/tailscale/Panel.qml`          |
 | Agents   | `omarchy.agents`     | `bar-widget`            | `agents/Panel.qml`               |
+| Local AI      | `omarchy.local-ai`        | `bar-widget`            | `panels/local-ai/Panel.qml`           |
 | Weather       | `omarchy.weather`         | `bar-widget`            | `panels/weather/BarWidget.qml`        |
 | Media         | `omarchy.media`           | `service`, `bar-widget` | `services/media/Service.qml`, `services/media/BarWidget.qml` |
 | Battery       | `omarchy.battery`         | `service`               | `services/battery/Service.qml`        |

@@ -0,0 +1,59 @@
+# Local AI Omarchy Widget
+
+Native Omarchy bar widget that runs the validated local model for your GPU
+and hands it to your coding agents.
+
+## Features
+
+- Shows the one model validated for the detected GPU; Start downloads and
+  serves it, Stop removes it, and every refusal says why on the card
+- Lists every detected GPU with its VRAM; the largest card with a validated
+  recipe is the default, and any other can be pinned
+- Opens any installed coding agent (pi, omp, opencode, claude, codex, crush,
+  copilot, grok, and more) on the running model with the endpoint and key in
+  the agent's own environment; no user config is written
+- Shares the endpoint on your tailnet, keyed, with one click
+- Bar icon: hollow while idle, filling with progress, full when ready;
+  right-click opens the selected agent
+
+## How it works
+
+Recipes come from the [local-ai-registry](https://github.com/0xSero/local-ai-registry)
+and are vendored in `recipes.json`, one per hardware id, each validated on
+that card. A recipe runs as two labeled Docker containers on a private
+network: the engine (image pinned by digest, model pinned by revision) and a
+gateway that speaks OpenAI chat, Anthropic Messages, and OpenAI Responses on
+`127.0.0.1:12434` and requires a key on every request. Acceptance proves the
+served model, the key, a chat reply, decode speed, every dialect, and a tool
+call before the model is called ready; a failure rolls back to the previous
+model. The gate refuses any recipe that is not digest-pinned, asks for host
+IPC, extra capabilities, a weakened security profile, or a mount outside the
+plugin's own cache roots.
+
+## Commands
+
+```
+omarchy local ai snapshot               refresh and print the state the panel renders
+omarchy local ai load                   download if needed, then start the model
+omarchy local ai unload                 stop the model; keep downloads
+omarchy local ai open-agent [name]      open an installed coding agent on it
+omarchy local ai share [--key <value>]  toggle tailnet sharing, or replace the key
+omarchy local ai gpu [auto|<key>]       which detected card to use
+omarchy local ai agent-dir <path>       directory agents open in
+omarchy local ai agent-args <name> [-- flags]  extra flags for one agent
+```
+
+## Requirements
+
+- Docker, with the user in the `docker` group
+- An NVIDIA GPU with the container toolkit, or an Intel Arc Pro B70
+- `jq`, `curl`; `tailscale` for sharing
+
+State lives in `~/.local/state/omarchy/local-ai/` (0700; `log` records every
+step) and weights in `~/.cache/omarchy/local-ai/` or the Hugging Face cache.
+
+## Add to the bar
+
+This widget ships as first-party plugin `omarchy.local-ai`. Add it with
+`omarchy plugin enable omarchy.local-ai`, then place it with
+`omarchy bar move omarchy.local-ai` if desired.
@@ -0,0 +1,119 @@
+#!/usr/bin/env bash
+# omarchy:summary=Run the validated local model for this GPU and open coding agents on it
+# omarchy:args=<snapshot|load|unload|open-agent [name]|share [--key value]|agent-dir <path>>
+#
+# Five verbs. `snapshot` is a pure read that rewrites the state file the panel watches. `load`
+# and `unload` spawn a worker that holds the op lock; everything they do is logged. `open-agent`
+# launches an agent with the endpoint in its environment. `share` toggles the tailnet route.
+set -euo pipefail
+HERE=$(cd -- "$(dirname -- "$(readlink -f "${BASH_SOURCE[0]}" 2>/dev/null || printf '%s' "${BASH_SOURCE[0]}")")" && pwd)
+SELF="$HERE/$(basename "${BASH_SOURCE[0]}")"
+# shellcheck source=../lib/common.sh
+source "$HERE/../lib/common.sh"
+# shellcheck source=../lib/hardware.sh
+source "$HERE/../lib/hardware.sh"
+# shellcheck source=../lib/recipes.sh
+source "$HERE/../lib/recipes.sh"
+# shellcheck source=../lib/weights.sh
+source "$HERE/../lib/weights.sh"
+# shellcheck source=../lib/share.sh
+source "$HERE/../lib/share.sh"
+# shellcheck source=../lib/runtime.sh
+source "$HERE/../lib/runtime.sh"
+# shellcheck source=../lib/agents.sh
+source "$HERE/../lib/agents.sh"
+# shellcheck source=../lib/snapshot.sh
+source "$HERE/../lib/snapshot.sh"
+
+recipes_ok || fail "recipes.json is missing or not the expected schema" || exit 1
+
+spawn() { # spawn <op-name> <worker-verb>: detach a worker; it takes the lock itself and reports through the ledger
+  local name=$1; shift
+  if [[ ${OMARCHY_AI_FOREGROUND:-0} == 1 ]]; then "$SELF" "$@" >/dev/null 2>>"$LOGFILE" || true; return; fi
+  # the worker is its own session so closing the panel's process never kills it; its pid goes into
+  # the ledger right away so the panel turns busy on the next snapshot, not a second later
+  if command -v setsid >/dev/null 2>&1; then setsid "$SELF" "$@" >/dev/null 2>>"$LOGFILE" </dev/null & disown
+  else "$SELF" "$@" >/dev/null 2>>"$LOGFILE" </dev/null & disown; fi
+  op_pending "$name" "$!"
+}
+
+current_recipe() { # the one recipe for this machine, gated; prints json or fails out loud
+  local m id r gate
+  m=$(match_hardware); id=$(jq -r .hardwareId <<<"$m")
+  [[ -n $id ]] || { fail "$(jq -r .reason <<<"$m")"; return 1; }
+  r=$(recipe_for "$id") || return 1
+  r=$(jq -c --argjson m "$m" '. + {gpuIndex:$m.gpu.index, match:{backend:$m.gpu.backend}}' <<<"$r")
+  gate=$(gate_reason "$r"); [[ -z $gate ]] || { fail "recipe refused: $gate"; return 1; }
+  driver_ok "$(hardware_json | jq -r .driver)" "$(jq -r '.minDriver // ""' <<<"$r")" \
+    || { fail "needs NVIDIA driver $(jq -r .minDriver <<<"$r") or newer"; return 1; }
+  printf '%s\n' "$r"
+}
+
+w_load() {
+  guard || oops "another operation is running"
+  trap worker_exit EXIT   # a step that dies unexpectedly still leaves a reason on the card, never a silent idle
+  local r id t0
+  op download "" "checking the GPU and recipe" 0
+  r=$(current_recipe 2>"$STATE/gate.err") || oops "$(sed -n '$s/^local-ai: //p' "$STATE/gate.err")"
+  id=$(jq -r .id <<<"$r")
+  op download "$id" "checking" 0
+  ensure_image "$(jq -r .launch.image <<<"$r")" "$id"
+  ensure_image "$(gateway_image)" "$id"
+  weights_present "$r" || download_weights "$r"
+  t0=$SECONDS
+  op starting "$id" "setting aside the previous model" 0
+  set_aside || oops "could not set aside the running containers"
+  op starting "$id" "starting" 0
+  if ! start_pair "$r" || ! accept "$r" 2>"$STATE/accept.err"; then
+    local why; why=$(sed -n '$s/^local-ai: //p' "$STATE/accept.err" 2>/dev/null); [[ -n $why ]] || why="engine failed to start (see $LOGFILE)"
+    log "rollback: $why"; op starting "$id" "rolling back" 0
+    restore_previous
+    oops "$why"
+  fi
+  drop_previous
+  # how long Start took from the first container step to accepted: the next Start's progress bar
+  lwrite '.lastStartSeconds=($s|tonumber)' --arg s "$((SECONDS - t0))"
+  op_done; log "ready $id"
+}
+
+w_unload() {
+  guard || oops "another operation is running"
+  trap worker_exit EXIT
+  op unload "" "stopping" 0
+  share_forget
+  stop_all
+  lwrite '.accepted={recipeId:"",servedModel:"",registry:"",apis:[]}'
+  op_done; log "unloaded"
+}
+
+usage() {
+  echo 'Usage: omarchy-local-ai <command>
+  snapshot               refresh and print the state the panel renders
+  load                   download if needed, then start the model for this GPU
+  unload                 stop the model; keep downloads
+  open-agent [name]      open an installed coding agent on the running model
+  share [--key <value>]  toggle sharing on your tailnet, or replace the key
+  agent-dir <path>       directory agents open in (default: where the shell started)
+  agent-args <name> [-- flags]  extra flags for one agent when opened from the panel (none to clear)
+  gpu [auto|<backend:index>]  which detected card to use (snapshot lists them); auto = largest card with a recipe'
+}
+
+cmd=${1:-help}; shift || true
+case $cmd in
+  snapshot) snapshot_write; cat "$SNAPSHOT" ;;
+  load) [[ -z $(busy_pid) ]] || fail "another operation is running" || exit 1; spawn download _worker-load ;;
+  unload) [[ -z $(busy_pid) ]] || fail "another operation is running" || exit 1; spawn unload _worker-unload ;;
+  open-agent) snapshot_write; open_agent "${1:-}" 2>"$STATE/agent.err" || { m=$(sed -n '$s/^local-ai: //p' "$STATE/agent.err"); refuse "${m:-agent launch failed (see $LOGFILE)}"; exit 1; } ;;
+  share) share_toggle "$@" ;;
+  agent-args) [[ -n ${1:-} ]] || { fail "agent-args <name> [-- flags...]"; exit 1; }; n=$1; shift; [[ ${1:-} == -- ]] && shift; mkdir -p "$STATE/agents/args"; if (( $# )); then printf '%s\0' "$@" >"$STATE/agents/args/$n"; else rm -f "$STATE/agents/args/$n"; fi; log "agent-args $n: $*" ;;
+  gpu) case ${1:-} in
+         ""|auto) rm -f "$STATE/gpu"; log "gpu auto" ;;
+         *) [[ $1 =~ ^[a-z-]+:[0-9]+$ ]] || { fail "gpu <backend:index>, as listed in the snapshot"; exit 1; }
+            state_dir; printf '%s' "$1" >"$STATE/gpu"; log "gpu $1" ;;
+       esac; snapshot_write; jq -c '.gpus' "$SNAPSHOT" ;;
+  agent-dir) [[ -d ${1:-} ]] && { (cd "$1" && pwd) >"$STATE/agent-dir"; log "agent-dir $1"; } || { fail "agent-dir: not a directory: ${1:-}"; exit 1; } ;;
+  _worker-load) w_load ;;
+  _worker-unload) w_unload ;;
+  help|-h|--help) usage ;;
+  *) usage >&2; exit 2 ;;
+esac