Skip to content

feat(computer-use): install the Wayland desktop-control toolchain #146

Description

@cernoh

What is the problem?

An agent session on this workstation can capture the screen with grim. It cannot drive the keyboard or the pointer.

Those tools exist in nixpkgs, but no host installs them. The current desktop-control skill fetches wtype and wlrctl through nix shell on each call.

That fetch needs the network. It also adds seconds of latency to every keystroke or click.

What do you expect?

A new computer-use feature installs the desktop-control toolchain. attrs/desktop imports the feature, so NIXPC and ASAHI both get it.

A managed skill documents the toolchain, the capture loop, and the safety rules.

Acceptance criteria

  • modules/features/computer-use/default.nix exports flake.nixosModules.computerUse.

  • The feature installs grim, slurp, wayland-utils, wlr-randr, wlrctl, and wtype.

  • attrs/desktop imports computerUse.

  • ~/.omp/agent/managed-skills/computer-use/SKILL.md exists and matches the installed set.

  • The NIXPC toplevel evaluates with no errors.

Other context

Probed on NIXPC on 2026-09-10. grim captures both outputs as 3840x1080.

wtype "" exits 0, so mango exposes zwp_virtual_keyboard_manager_v1. wlrctl pointer move 2 2 exits 0, so mango exposes wlr-virtual-pointer.

ydotool stays out. It needs write access to /dev/uinput, which is root only.

wl-clipboard stays out. NIXPC gets wl-clipboard-rs from mango and ASAHI gets wl-clipboard from the clipboard feature. A third copy would collide in the system profile.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions