fix(cli): report inference health from a served request

.agents/skills/_shared/controlled-words.md

@@ -184,7 +184,7 @@ other literal identifiers.
 | `custom endpoint` | Technical noun | A user-supplied inference endpoint that is not one of NemoClaw's named provider choices. | compatible endpoint before compatibility is validated |
 | `hosted inference` | Technical noun | Inference served by a remote provider-operated service. | local inference, cloud model |
 | `inference` | Technical noun | Model execution that produces a response from an input. | AI, generation when model execution is intended |
-| `inference health` | Technical noun | The complete classification from the named `inference.local` `/v1/models` status probe: `reachable` for HTTP `200` through `499`, `unhealthy` for HTTP `500` through `599`, or `unreachable` when no qualifying HTTP response arrives. | model health, successful inference, validation request |
+| `inference health` | Technical noun | The classification reported for a sandbox's inference route: the `inference.local` `/v1/models` probe result, and, when that route is reachable and the sandbox records a provider and a model, the result of one inference request sent over the same route. Values are `healthy`, `unauthorized`, `reachable`, `unhealthy`, `unreachable`, and `not probed`. | model health, successful inference |
 | `inference profile` | Technical noun | A blueprint selection that defines an inference provider type, provider name, endpoint, model, credential input, and route settings. | provider profile, model profile |
 | `inference request` | Technical noun | One request sent through an inference route to a model. | API call when the inference purpose matters |
 | `inference route reachability` | Technical noun | The `reachable` inference-health result produced when `https://inference.local/v1/models` returns HTTP `200` through `499`. It does not establish valid credentials, successful model invocation, readiness, compatibility, or support. | inference health, successful inference, validation request |
@@ -391,7 +391,7 @@ A result can support more than one claim only when its evidence meets each defin
 | Class | Claim | Establishes | Does not establish |
 |---|---|---|---|
 | Operational | `inference route reachability` | The named `/v1/models` route returned HTTP `200` through `499`. | Valid credentials, successful model invocation, readiness, compatibility, or support. |
-| Operational | `inference health` | The named `/v1/models` probe produced a `reachable`, `unhealthy`, or `unreachable` classification. | Valid credentials, successful model invocation, readiness, compatibility, or support. |
+| Operational | `inference health` | The named `/v1/models` probe classification, plus the result of one inference request over the same route when NemoClaw sent one. | Broader API conformance, other requests or models, readiness, compatibility, or support. |
 | Operational | `readiness check` | A service or resource meets named criteria to begin its intended work. | Broader reliability, compatibility, or support. |
 | Operational | `validation request` | One authenticated request succeeded for the named endpoint, API family, model, and request shape. | Broader API conformance, other requests or models, reliability, or support. |
 | Evidence | `verification` | Evidence confirms the stated result for the named revision and environment. | Compatibility or support unless the evidence and decision establish them. |

ci/source-architecture-budget.json

@@ -40,7 +40,7 @@
       "src/lib/actions/sandbox/connect.ts": 38,
       "src/lib/actions/sandbox/destroy.ts": 29,
       "src/lib/actions/sandbox/doctor.ts": 30,
-      "src/lib/actions/sandbox/status-snapshot.ts": 21,
+      "src/lib/actions/sandbox/status-snapshot.ts": 20,
       "src/lib/actions/sandbox/policy-channel.ts": 29,
       "src/lib/actions/sandbox/process-recovery.ts": 21,
       "src/lib/actions/sandbox/rebuild-pipeline.ts": 28,

docs/deployment/deploy-to-headless-server.mdx

@@ -235,8 +235,8 @@ $$nemoclaw headless-agent connect --probe-only
 ```
 
 `$$nemoclaw headless-agent status` exits nonzero when the sandbox, gateway, local container, or authoritative inference route is not verified.
-Its main `Inference` line probes `https://inference.local/v1/models` from inside the sandbox.
-HTTP `200` through `499` reports `reachable`, while HTTP `500` through `599` reports `unhealthy`.
+Its main `Inference` line probes `https://inference.local/v1/models` from inside the sandbox, then sends one inference request over the same route when that probe reports the route reachable.
+The line reports `healthy` when the route served the request, `unauthorized` when the route rejected it with HTTP `401` or `403`, and `unhealthy` when the route returned HTTP `500` through `599`.
 <AgentOnly variant="openclaw,hermes">
 
 During an SSH session, `status` points to `$$nemoclaw headless-agent dashboard-url` only when the agent gateway is running and loopback dashboard access needs a port forward.
@@ -251,7 +251,7 @@ It does not restart or replace the shared host OpenShell gateway.
 Readiness requires all of these results:
 
 - The exact OpenShell sandbox row is `Ready` or `Running`.
-- `$$nemoclaw headless-agent status` exits with status `0` and reports the inference route as `reachable`.
+- `$$nemoclaw headless-agent status` exits with status `0` and reports the inference route as `healthy`.
 - `$$nemoclaw headless-agent connect --probe-only` exits with status `0`.
 
 ## Access the Dashboard and API

docs/inference/choose-local-inference-server.mdx

@@ -127,8 +127,8 @@ $$nemoclaw my-assistant status
 $$nemoclaw my-assistant doctor
 ```
 
-Accept the result when `status` reports the inference route as `reachable` and `doctor` exits with status `0`.
-Route reachability does not by itself establish successful model invocation.
+Accept the result when `status` reports the inference route as `healthy` and `doctor` exits with status `0`.
+The `healthy` state means the route served one inference request; it does not establish results for other requests or models.
 
 Verify the bounded host publication:
 

docs/inference/set-up-llama-cpp.mdx

@@ -158,7 +158,7 @@ $$nemoclaw my-assistant doctor
 docker inspect --format '{{json .HostConfig.PortBindings}}' nemoclaw-llama-cpp
 ```
 
-Accept the result when `status` reports `Managed llama.cpp: running`, the inference route is `reachable`, and `doctor` exits with status `0`.
+Accept the result when `status` reports `Managed llama.cpp: running`, the inference route is `healthy`, and `doctor` exits with status `0`.
 The Docker inspection output must be `{}`, confirming that the container has no Docker-published port.
 The host-owned private bridge still listens on loopback.
 These checks do not establish agent and model qualification.

docs/inference/set-up-ollama.mdx

@@ -179,7 +179,8 @@ The wizard manages the proxy lifecycle:
 
 All proxy endpoints require the token, including `GET /api/tags`.
 The host-side proxy liveness check treats any HTTP response, including `401`, as evidence that the proxy answered.
-The authoritative `status` and `doctor` route probe runs inside the sandbox and reports HTTP `200` through `499` as `healthy`, HTTP `500` through `599` as `unhealthy`, and transport failures or unavailable probes as `unreachable` or `not probed`.
+The `status` and `doctor` route probe runs inside the sandbox and reports HTTP `200` through `499` as reachable, HTTP `500` through `599` as `unhealthy`, and transport failures or unavailable probes as `unreachable` or `not probed`.
+`status` then sends one inference request over the reachable route and reports `healthy` when the route served it, or `unauthorized` when the proxy token is no longer accepted.
 
 If Ollama is already bound to a non-loopback address, onboarding restarts it on `127.0.0.1:11434` so the proxy becomes the only network path.
 

docs/inference/set-up-vllm-on-two-dgx-stations.mdx

@@ -82,8 +82,8 @@ $$nemoclaw my-assistant status
 ```
 
 Confirm that `inference get` reports the `vllm-local` provider and the `nemotron-ultra` model.
-Continue only when the `Inference` row in the status output reports `reachable`.
-This result confirms that the sandbox route returned HTTP `200` through `499`; it does not prove a successful model invocation.
+Continue only when the `Inference` row in the status output reports `healthy`.
+This result confirms that the sandbox route served one inference request; it does not establish results for other requests or models.
 
 ## Review Reboot and Resume Behavior
 

docs/inference/verify-inference-route.mdx

@@ -30,9 +30,14 @@ Run the named sandbox status command.
 $$nemoclaw <name> status
 ```
 
-The `Inference` row checks the sandbox's `inference.local` path and reports it as `reachable` when the route returns HTTP `200` through `499`.
-This includes authentication responses such as HTTP `401` and `403`: they confirm route access, not valid provider credentials or a successful model invocation.
-Use the status command's upstream provider diagnostic or send a short agent request to verify the configured model.
+The `Inference` row first checks the sandbox's `inference.local` path.
+When that route responds, `status` sends one inference request through the same path.
+The row reports `healthy` only when the route returns a structurally valid result for the recorded API family.
+An empty body, malformed JSON, provider-error envelope, or wrong response shape reports `unhealthy`, even with a 2xx status.
+Status diagnostics do not include the response body.
+An HTTP `401` or `403` response reports `unauthorized`.
+Correct the stored provider credential.
+The route-reachability and upstream provider subprobes remain available to identify the failing hop.
 The provider, model, and endpoint appear with the rest of the sandbox state.
 This path includes the OpenShell proxy and its authentication rewrite.
 When onboarding prints a dashboard summary, use it to verify that NemoClaw ran the same route-reachability probe from inside the sandbox.

docs/monitoring/monitor-sandbox-activity.mdx

@@ -122,7 +122,7 @@ If the request fails, check these items.
 
 1. Run `$$nemoclaw <name> status` to confirm the active provider and endpoint.
    Check the main `Inference` line first.
-   If it shows `unhealthy`, `unreachable`, or `not probed`, inspect the labeled upstream, local-backend, and auth-proxy diagnostics to identify the failing hop.
+   If it shows `unauthorized`, `unhealthy`, `unreachable`, or `not probed`, inspect the labeled upstream, local-backend, and auth-proxy diagnostics to identify the failing hop.
    Restart a local backend only when its own diagnostic fails.
 2. Run `$$nemoclaw <name> logs --follow` to view error messages from the blueprint runner.
 3. Verify that the host can reach the inference endpoint.

docs/reference/commands.mdx

@@ -1633,6 +1633,13 @@ Starting an already-running sandbox skips the container start and still runs the
 A paused container is unpaused.
 If the container was removed entirely, `start` fails and points you to `$$nemoclaw <name> rebuild`.
 
+After the gateway and forward checks pass, `start` sends one inference request through `https://inference.local` using the sandbox's recorded provider and model.
+A gateway that answers the `/v1/models` probe can still reject an inference request or return an invalid result, so the command exits non-zero in either case.
+It prints the probe result, including the HTTP status when the route returned one, and points you to the sandbox doctor command.
+Each run sends one 16-token request through the stored provider credential, so `start` waits up to 30 seconds for it and consumes provider tokens on a hosted route.
+When the sandbox records no provider or no model, `start` skips the request and exits `0`.
+`doctor` still classifies an HTTP `401` or `403` route response as reachable, so correct the provider credential when `start` reports one of those statuses.
+
 ### `$$nemoclaw <name> status`
 
 Show sandbox-scoped status, health, and inference configuration for one registered sandbox.
@@ -1659,6 +1666,24 @@ Refer to [Use Shared Gateway Routes](../inference/manage-inference/use-shared-ga
 `openshellDriver` and `openshellVersion` are always strings (falling back to `"unknown"` when the registry has no value), so consumers can rely on `typeof` checks.
 `agent` is always a string and reports `openclaw` when the registry records no agent for the sandbox.
 `failureLayer` is `null` when no preflight failure was detected and otherwise one of `docker_unreachable`, `sandbox_container_stopped`, or `sandbox_dashboard_port_conflict`; when set, `inferenceHealth` is suppressed to `null` so automation does not see a stale remote-provider healthy status during a local outage.
+`inferenceHealth.ok` reports whether the inference route returned a structurally valid result for one request sent from inside the sandbox.
+The result must match Chat Completions, Responses, or Anthropic Messages for the selected route.
+An empty body, malformed JSON, provider-error envelope, or wrong response shape reports `unhealthy`, even with a 2xx status.
+The probe captures at most 64 KiB and does not include the response body in diagnostics.
+The route probe treats any final HTTP status from `200` through `499` as reachable, so a route with an invalidated provider credential answers HTTP `401` while the route is up.
+The request uses the live gateway route's provider and model, and falls back to the recorded values when the live route is unreadable.
+When the live provider and model match the recorded route, the request also uses the sandbox's recorded API family, including `openai-responses`.
+During route drift, NemoClaw does not carry the sandbox's recorded API family to the live provider and model.
+Each run sends one 16-token request through the stored provider credential, so `status` waits up to 30 seconds for it and consumes provider tokens on a hosted route.
+When NemoClaw sends an inference request, `inferenceHealth.subprobes` reports the route probe result as the `route reachability` hop, so a failing verdict still shows that the route itself answered.
+`inferenceHealth.failureLabel` reports why the inference request failed:
+
+- `unauthorized` when the route rejected it with HTTP `401` or `403`.
+- `unhealthy` when the route returned another failing HTTP status or an invalid 2xx response body.
+- `unreachable` when the request returned no HTTP status, including a probe that could not run.
+
+A host-side upstream probe under `inferenceHealth.subprobes` stays a diagnostic and does not change `inferenceHealth.ok`, because the sandbox route is the one the agent uses.
+When the route probe failed, or the sandbox records no provider or no model, NemoClaw skips the inference request and `inferenceHealth` reports the route probe result alone.
 `dockerPaused` is `true` when NemoClaw detects that the Docker-driver sandbox container is paused.
 In that case, text output keeps OpenShell's authoritative phase but prints a `docker unpause <container>` recovery hint instead of sending you directly to rebuild.
 For terminal runtime sandboxes, the command also checks cgroup OOM kill counters.
@@ -1702,19 +1727,22 @@ $$nemoclaw my-assistant status --json
 $$nemoclaw sandbox status my-assistant --json
 ```
 
-The command probes `https://inference.local/v1/models` from inside the sandbox as the authoritative inference health check.
-This check exercises the same route that agent traffic uses.
+The command probes `https://inference.local/v1/models` from inside the sandbox, and when that probe reports the route reachable it sends one inference request over the same route.
+That inference request is the authoritative inference health check, and both checks exercise the route that agent traffic uses.
 The main `Inference` line reports one of these states:
 
 | State | Meaning |
 |-------|---------|
-| `reachable` | The route returned an HTTP status from `200` through `499`. Authentication responses such as `401` and `403` confirm route reachability. |
-| `unhealthy` | The route returned an HTTP status from `500` through `599`. |
-| `unreachable` | The route had a transport failure, returned no final HTTP status (`000` or an interim `100` through `199`), or returned an invalid status outside `100` through `599`. |
+| `healthy` | The route returned a structurally valid result for the inference request. |
+| `unauthorized` | The route rejected the inference request with HTTP `401` or `403`. |
+| `reachable` | The route returned an HTTP status from `200` through `499` and NemoClaw did not send an inference request. |
+| `unhealthy` | The route returned HTTP `500` through `599`, another failing status, or an invalid 2xx response body. |
+| `unreachable` | The route had a transport failure, returned no final HTTP status (`000` or an interim `100` through `199`), returned an invalid status outside `100` through `599`, or the inference request returned no HTTP status. |
 | `not probed` | NemoClaw could not run the authoritative route probe from a reachable sandbox. |
 | `not verified` | NemoClaw could not verify the sandbox or gateway state, so it skips inference probing. |
 
-An authentication response confirms that the route is reachable, not that provider credentials are valid.
+An authentication response on the route probe alone confirms that the route is reachable, not that provider credentials are valid.
+`$$nemoclaw <name> doctor` sends no inference request, so it reports an HTTP `401` or `403` route response as reachable and exits `0` where `status` reports `unauthorized`.
 The command can also print direct host-side provider checks such as `Inference (upstream)` and provider-specific subprobes.
 For supported remote providers, this diagnostic sends an authenticated request to the configured model and accepts only a recognized Chat Completions, streaming Chat Completions, or Anthropic Messages response.
 It uses a 3-second connection timeout, a 5-second total timeout, and an 8-token output limit.

docs/reference/troubleshooting.mdx

@@ -1894,8 +1894,9 @@ Check the active provider and endpoint:
 $$nemoclaw <name> status
 ```
 
-The main `Inference` line probes `https://inference.local/v1/models` from inside the sandbox, so it reflects the route the agent actually uses.
-If that line shows `unhealthy`, `unreachable`, or `not probed`, inspect the labeled diagnostic lines to identify the failing hop.
+The main `Inference` line probes `https://inference.local/v1/models` from inside the sandbox and then sends one inference request over the same route, so it reflects the route the agent uses.
+If that line shows `unauthorized`, `unhealthy`, `unreachable`, or `not probed`, inspect the labeled diagnostic lines to identify the failing hop.
+An `unauthorized` line means the route answered but rejected the request, so refresh the provider credential rather than the route.
 For local Ollama and local vLLM, `Inference (ollama backend)` or the corresponding local-backend line reports the host-side service separately.
 For Local Ollama, current releases can also print `Inference (auth proxy)` when a proxy token is available.
 If a local backend or auth-proxy diagnostic fails, start the backend or re-run onboarding so NemoClaw can recreate the proxy token, restart the proxy, and refresh the route.

docs/security/credential-rotation.mdx

@@ -221,7 +221,8 @@ Plan for recreation downtime when automating messaging or web search rotation.
 ## Verify the Replacement
 
 `$$nemoclaw credentials list` confirms that a provider exists, but it does not reveal or validate the stored value.
-`$$nemoclaw status` performs reachability checks without sending cloud API keys, so even an HTTP `401` or `403` can count as reachable.
+`$$nemoclaw status` performs host-side reachability checks without sending cloud API keys, so even an HTTP `401` or `403` can count as reachable there.
+`$$nemoclaw <name> status` also sends one inference request through the stored provider credential and reports `unauthorized` when the route rejects it with HTTP `401` or `403`.
 `$$nemoclaw inference get` reports the active route and does not authenticate a model request.
 
 Complete a real request through the rotated integration before declaring the rotation successful.