RFC: An MCP server for Ray on Kubernetes (KubeRay)

[risjai]

RFC: An MCP server for KubeRay

TL;DR

AI agents are starting to operate infrastructure, but there is no Ray-aware control surface an agent can connect to. Generic Kubernetes MCP servers can CRUD the KubeRay CRDs yet are blind to Ray's runtime. The KubeRay API Server is a CRUD layer over those same CRDs — it does not reach Ray's dashboard, and by default it does not authenticate the caller. The hard things an agent actually needs — why is this job stuck, live logs, follow-a-job-to-completion, and Ray-aware safety around destructive ops — require a custom, Ray-aware layer either way. This RFC proposes building that layer as an MCP server, and asks whether it should live under ray-project.

What's in this PR

• The case — this description. Why the gap is real and why existing surfaces don't close it.
• reps/2026-06-13-ray-mcp-usecases.md — concrete agent use cases, each mapped against what a generic K8s MCP server and the KubeRay API Server can and cannot do. (Start here if you only read one file.)
• reps/2026-06-13-ray-mcp-design.md — a full design spec (architecture, tool surface, safety model) so the discussion has something concrete to react to.

The gap

I went looking across OSS and internally for an MCP server an agent could use to create/inspect RayClusters, submit and follow RayJobs, and manage RayServices with Ray-aware semantics, and couldn't find one that fits. What exists is adjacent: the ray-kubectl-plugin REP (an explicitly human-facing CLI), the kuberay-authentication REP (dashboard auth, relevant but not an agent surface), and generic Kubernetes MCP servers (CRD CRUD with zero Ray awareness). The "let an agent operate Ray" angle seems absent. Is it missing on purpose, rejected, on a roadmap, or just hasn't come up?

Why the KubeRay API Server isn't enough

The KubeRay API Server is genuinely useful — but it solves a different problem. It is a CRUD surface over the CRDs (the control plane). An agent's hard problems live in the data plane (Ray's dashboard/job API on the head node) and in cross-plane correlation — and a CRUD proxy addresses neither:

• It doesn't reach Ray's runtime. Live job status, the granular reason a job is wedged (e.g. unschedulable placement group / no GPU nodes), and job logs live behind the Ray dashboard / Job Submission API (port 8265) — not in RayJob.status. The API Server never touches 8265, so an agent gets coarse CRD lifecycle and nothing more.
• By default it doesn't authenticate the caller. API Server V1 is deprecated and ships unauthenticated over a NodePort (with an optional single shared static token). V2 (apiserversdk, alpha) is a reverse proxy that mirrors the Kubernetes API — but as shipped it forwards every request under one shared credential; it does not pass the caller's identity through to Kubernetes RBAC unless the deployer writes custom middleware. So "put the agent behind the API Server" does not, by itself, give you per-agent authorization.
• It has no Ray-aware safety. Nothing distills runtime status for an LLM's bounded context, nothing guards Ray-specific footguns (scale-to-zero, deleting a RayService that's serving traffic), nothing predicts CRD field pruning before applying.

Per-surface, blow-by-blow comparison is in reps/2026-06-13-ray-mcp-usecases.md.

Security: you'd write a custom layer regardless

This is the part I most want maintainers to check. The Ray dashboard / Job API is unauthenticated by default — Ray's own guidance treats network isolation as the primary control, and opt-in token auth only arrived in Ray 2.52.0. It is the surface behind the disputed-but-actively-exploited ShadowRay reports (CVE-2023-48022; Anyscale's position is that this is intended behavior for a tool meant to run inside a trusted network). An agent that wants live status and logs has to reach that surface — and a thin CRUD proxy in front of the CRDs does nothing to make that safe.

So whatever you put in front, an agent control surface needs a layer that:

1. is read-only by construction toward the dashboard — exposes no Ray-side write verb, so the unauthenticated/RCE-capable surface is never a mutation vector through the tool (the tool's own contract; it does not make 8265 itself safe), and
2. routes every mutation through the guarded, RBAC-gated CRD path, with per-call dry-run + diff and tiered guards.

The destructive-op guards are agent-safety / anti-footgun, explicitly not a security control — RBAC is the real boundary. The point stands: even with the KubeRay API Server in front, you still have to build this Ray-aware + safety layer on top. That layer is the proposal.

Work in progress - https://github.com/risjai/ray-mcp

Corrections to anything I've gotten wrong are very welcome — I'd rather be corrected now than build on a wrong assumption. Slack thread: https://ray.slack.com/archives/C02GFQ82JPM/p1781359267855649

[Future-Outlier]

sounds interesting, thanks for opening this!

[Future-Outlier]

can you include the usecase in the design doc?

[risjai]

can you include the usecase in the design doc?

Done. Added reps/2026-06-13-ray-mcp-usecases.md

[Future-Outlier]

can kubectl ray plugin meets your need?
https://docs.ray.io/en/latest/cluster/kubernetes/user-guides/kubectl-plugin.html

[risjai]

Great question! kubectl ray is the closest existing tool, so it's worth being precise about the overlap.

It covers more than the generic-K8s-MCP / API-Server columns suggest: create/get/scale cluster, job submit that follows a job to completion, session that port-forwards the dashboard to localhost:8265, and log for head/worker logs. For a human at a terminal, that's most of the workflow.

The gap this RFC targets is an agent consumer, and that's a different shape than a CLI:

1. session opens the dashboard; it doesn't consume it. Port-forwarding 8265 to localhost still leaves an agent to drive raw HTTP against the unauthenticated, RCE-capable Job/dashboard API itself - the "loaded gun" this doc is trying to keep out of the agent's hands. The proposed server reads that surface for the agent, read-only by construction, and never exposes a Ray-side write verb.
2. No "why is this job stuck" distillation. The plugin emits status tables; an agent needs a bounded reason string ("Pending: no GPU nodes") that fits a finite context window, not a firehose to re-parse.
3. No agent-safety layer - no dry-run/diff, no tiered destructive guards (scale-to-zero, deleting a RayService that's serving), and no readOnlyHint/destructiveHint annotations an MCP client can gate on.
4. It's a CLI, not a tool surface. An agent wrapping it shells out and screen-scrapes human-formatted tables; there's no typed contract.

So I see them as complementary rather than overlapping: kubectl ray is the paved human path; this is the agent path. An agent built on top of kubectl ray would still have to add the distillation + safety layer which is the proposal.

I just updated the table for including this in comparison as well. Please check that now.