feat(cloudxr): supervised background runtime (systemd user service) - #927
Draft
jiwenc-nv wants to merge 6 commits into
Draft
feat(cloudxr): supervised background runtime (systemd user service)#927jiwenc-nv wants to merge 6 commits into
jiwenc-nv wants to merge 6 commits into
Conversation
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
A second launch treated the existence of `ipc_cloudxr` as proof of a stale runtime and `fuser -k -TERM`ed whoever held it — including a healthy runtime mid-session, which the first session saw as a broken pipe. When `fuser` was missing the error was swallowed and the socket unlinked anyway, leaving the old process running and the operator to clean `~/.cloudxr/run/` by hand. Liveness is now decided by connecting to the socket rather than by stat-ing it. A live runtime makes the launcher refuse, pointing at the env file and `--no-launch-cloudxr-runtime`; replacing it means stopping it yourself. A dead socket is cleaned with a warning. Ambiguous probe errors count as live, since refusing is recoverable and clobbering a session is not. `-35` is thrown after `xrCreateInstance` succeeds, so the runtime was found and only the headset is missing, but library defaults are fail-fast and a direct consumer got a bare `Failed to get OpenXR system: -35`. Both throw sites now name `XR_ERROR_FORM_FACTOR_UNAVAILABLE` and point at the docs. The device profile is the usual culprit, so the startup banner prints the resolved value and the launcher/env-config defaults agree on one constant — the docs claimed `auto-webrtc` where `Quest3` always won. Overriding it stays an env-file or `CloudXRLauncher(device_profile=...)` job; Apple Vision Pro needs `auto-native`. Closes #908 Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
jiwenc-nv
force-pushed
the
jiwenc-nv/cloudxr-user-service
branch
from
August 8, 2026 03:45
411218e to
e3b1282
Compare
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
CloudXRLauncher both owned the runtime process and served as the API for reaching one, which left ownership invisible at the call site: whether a teleop script's runtime outlived it depended on how the host was set up, not on anything in the script. CloudXRService now holds the runtime subprocess, the WSS proxy thread and their teardown. CloudXRLauncher keeps the CLI plumbing and delegates lifecycle to a service it constructs, so behaviour is unchanged here; a later commit turns it into an attach-only client. Lifecycle tests move to test_service.py with it, and their doubles move to conftest.py so both suites share them. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
jiwenc-nv
force-pushed
the
jiwenc-nv/cloudxr-user-service
branch
from
August 8, 2026 03:54
e3b1282 to
1c2b928
Compare
python -m isaacteleop.cloudxr.service run is what a supervisor executes; install/uninstall/status/logs manage the systemd user service around it. python -m isaacteleop.cloudxr keeps working as a deprecated alias for run. service becomes a package so the CLI lives in its own __main__: as a flat module, python -m re-executed it after the parent package had already imported it, leaving two CloudXRService classes and a RuntimeWarning. The unit is Restart=on-failure rather than always — a rejected EULA and the --setup-oob adb preflights both exit non-zero with no headset attached, which is the normal state at boot. Install refuses unless the EULA marker exists or --accept-eula is passed, and records acceptance as a marker in the operator's own process: a service started by systemd has no stdin. Where there is no user systemd, every unit command fails with instructions naming both escapes (run the service yourself, or run_embedded). Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
| parts.append( | ||
| f"stderr: {stderr_tail.decode(errors='replace').strip()}" | ||
| ) | ||
| except Exception: |
| if len(content) > _MAX_LOG_BYTES: | ||
| content = "...\n" + content[-_MAX_LOG_BYTES:] | ||
| parts.append(f"{log_path.name}:\n{content}") | ||
| except Exception: |
| return | ||
| try: | ||
| proc.wait(timeout=RUNTIME_TERMINATE_TIMEOUT_SEC) | ||
| except subprocess.TimeoutExpired: |
| return | ||
| try: | ||
| proc.wait(timeout=RUNTIME_TERMINATE_TIMEOUT_SEC) | ||
| except subprocess.TimeoutExpired: |
jiwenc-nv
force-pushed
the
jiwenc-nv/cloudxr-user-service
branch
from
August 8, 2026 05:59
63dce94 to
5676b31
Compare
systemctl --user needs a per-user manager, which containers and CI runners do not have — PID 1 is the entrypoint, so the manager never starts. That left every unit command dead on exactly the hosts this is most useful on, with no fallback but a wall of instructions. service start now spawns the detached run with start_new_session=True, which is setsid(2): the child leads its own session, so neither a hangup nor a signal aimed at the shell's process group reaches it. Works on a host, in a container, and under CI, with no dependency beyond util-linux semantics Python already exposes. status asks the IPC socket rather than a pid file, since pids are reused and a socket cannot lie. stop is SIGTERM only: the runtime leads its own session, so killing the service outright would orphan a process holding the GPU. What this gives up, relative to the systemd unit it replaces: no restart after a crash, and no start at boot. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
jiwenc-nv
force-pushed
the
jiwenc-nv/cloudxr-user-service
branch
from
August 8, 2026 06:01
5676b31 to
a76d946
Compare
start printed three lines and status three more, so neither told you the device profile, the log paths, or the web client URL — the things you actually need to connect a headset. Both now print the run banner. Two values the banner took from live state had to be reachable from another process: the WSS log path, now found by latest_wss_log(), and the device profile, now read out of the cloudxr.env the service wrote. Neither goes through EnvConfig, because resolving it here would rewrite that file underneath the running service — which is also why the log finders take a logs_dir instead of the singleton. status reconstructs the running session's flags from the service's own command line, so it reports the session that exists rather than this command's defaults. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
/proc/<pid>/cmdline NUL-terminates every argument, so splitting on NUL leaves a trailing empty string. read_run_flags passed it to argparse, which rejected it, and status exited 2 with 'unrecognized arguments' against any detached service. The tests missed it because they mocked read_run_flags with a clean list, which cannot reproduce a terminator; the regression test reads a real process's cmdline instead. status also no longer dies when those flags do not parse. They come from another process, possibly a build that knows options this one does not, and a status command should still report what it can. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #919.
Draft — opening early for design review. One finding reshaped the approach: a systemd unit alone doesn't fix what #919 complains about.
CloudXRLauncher.__init__runsfuser -k -TERMon~/.cloudxr/run/ipc_cloudxrwhether or not a live runtime owns it, and--launch-cloudxr-runtimedefaults to true. With a supervised runtime up, any example run with default flags kills it, systemd restarts it, and the headset reconnects anyway — worse than today's two-terminal workflow.So this lands two halves together: the launcher becomes idempotent (detect → attach → tear down only what it started), and a user unit runs
python -m isaacteleop.cloudxrso runtime + WSS proxy + OOB hub outlive the calling script.Sequence (one commit each, first is pushed):
runtime_is_live()— detect a live runtime without touching its socketCloudXRLauncher; gate both stomping sites--no-launch-cloudxr-runtime; registrar stays a deprecated no-op__main__ensure_headset_session()+CONTROL_TOKENto the run dirservice.py, unit template, docsType of change
Testing
pytest src/core/cloudxr_tests/python/— 37 intest_runtime.py, 6 new. The/proc/net/unixparser was also checked against a real listener on a live host, since fixtures can't prove the column indices match the kernel's format.Checklist
SKIP=check-copyright-year pre-commit run --all-filesgit commit -s) per the DCO