cloudparts

KubeRay

The Kubernetes operator for Ray: it runs each Ray cluster as a head pod plus worker pods, and adds custom resources for long-running clusters, batch jobs and Ray Serve applications.

CNCF · Sandbox
What it is
Actor model,Container orchestration
Part of
Tool for Ray
Type
Open source
Made by
KubeRay project, a CNCF sandbox project; started in the Ray community
Foundation
Cloud Native Computing Foundation, sandbox. Voted into the sandbox in September 2026; onboarding began on 29 September. Part of the Linux Foundation
License
Apache 2.0
Facts checked
23 October 2026
Website Source code

What it is

KubeRay is a Kubernetes operator written in Go that turns Ray clusters into Kubernetes objects. Each Ray node becomes a pod: one head pod and one or more groups of worker pods, which can have different shapes, for example a CPU group and a GPU group. Several Ray clusters, even on different Ray versions, can share one Kubernetes cluster. The Ray documentation calls the operator the recommended way to run Ray on Kubernetes.

It adds three custom resources. A RayCluster is a cluster you manage yourself; the operator creates and deletes its pods, restarts failed ones and, with enableInTreeAutoscaling, adds the Ray autoscaler as a sidecar in the head pod. That autoscaler watches the CPUs, GPUs and other resources that waiting tasks and actors ask for and changes the worker group’s replicas; idle workers go after 60 seconds by default. A RayJob creates a cluster, submits a job once it is ready and can delete the cluster when the job finishes. A RayService runs a Ray Serve application: changes to the Serve config are applied in place, while changes to the cluster itself start a new RayCluster, move traffic to it once it is ready, and remove the old one.

KubeRay works with Kubernetes batch schedulers such as Kueue, Volcano and Apache YuniKorn for queueing and gang scheduling, and there is a kubectl ray plugin (beta), an API server (alpha) and a dashboard (experimental). It was co-developed by contributors from Anyscale, Google, Microsoft, ByteDance, AWS and others. In September 2026 the CNCF accepted it as a sandbox project, and it moved from the ray-project GitHub organization to its own kuberay-project organization; v1.7.1 came out the same month.

Why teams pick it

  • Ray clusters become ordinary Kubernetes objects, so GitOps, RBAC, quotas, node pools and existing monitoring apply to them
  • RayJob gives one-off training or batch jobs a cluster that exists only while the job runs
  • RayService upgrades a Ray Serve application without downtime, including changes to the Ray version or cluster shape
  • Scales on what Ray work is waiting for, not on CPU load, and leaves adding nodes to the Kubernetes cluster autoscaler

Watch out for

  • You run two autoscalers: Ray’s adds pods, but if there is no room the Kubernetes cluster autoscaler must add nodes, and you configure that yourself
  • A RayService cluster upgrade runs old and new clusters side by side by default, so you need spare capacity, often GPUs, for both
  • The head pod is still a single point of failure; surviving its loss needs GCS fault tolerance with an external Redis
  • Two layers to debug: a problem can sit in the operator and pod specs or inside Ray’s own scheduler and object store

How it fits with other technologies

A RayJob: create a cluster, run one script, delete the cluster · yaml

apiVersion: ray.io/v1
kind: RayJob
metadata:
  name: train-once
spec:
  entrypoint: python train.py
  shutdownAfterJobFinishes: true   # delete the cluster when the job ends
  rayClusterSpec:
    rayVersion: "2.59.0"
    headGroupSpec:
      rayStartParams: {}
      template:
        spec:
          containers:
            - name: ray-head
              image: rayproject/ray:2.59.0
    workerGroupSpecs:
      - groupName: gpu
        replicas: 2
        minReplicas: 0
        maxReplicas: 4
        rayStartParams: {}
        template:
          spec:
            containers:
              - name: ray-worker
                image: rayproject/ray:2.59.0
                resources:
                  limits:
                    nvidia.com/gpu: 1

See it in a real stack

No architecture uses KubeRay yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2020–2026

Ray first ran on Kubernetes through an operator that lived in the Ray repository. In November 2020 a separate Go operator was started in what became the ray-project/kuberay repository, with early code merged from a branch at Alipay. Engineers from several companies, including Anyscale, Google, Microsoft, ByteDance and AWS, developed it from there. Ray 2.0 (August 2022) named KubeRay the preferred way to deploy on Kubernetes, and Ray 2.2.0 removed the old operator.

Version 0.1.0 came out in October 2021 and 1.0.0 in November 2023. From September 2023 the user documentation moved into the Ray docs, while the repository kept only developer material. Later versions added the kubectl ray plugin (from v1.3.0) and an experimental dashboard (from v1.4.0).

In 2026 the maintainers applied to the CNCF for vendor-neutral governance. The sandbox application passed in September 2026, and the project moved to the new kuberay-project GitHub organization, with Helm charts to follow from v1.8.0. Ray itself had gone to the PyTorch Foundation a year earlier, so Ray and its Kubernetes operator now sit in two different Linux Foundation groups.

Why it’s built this way

An operator in Go, outside the Ray repository
A controller released on its own schedule, separate from Ray, fits how Kubernetes users run operators, and lets one operator manage clusters on several Ray versions in the same Kubernetes cluster.
Ray’s autoscaler as a sidecar, not a Kubernetes autoscaler
Only Ray knows which tasks and actors are waiting and what they asked for, so the sidecar decides how many worker pods to run and leaves node provisioning to Kubernetes.
Separate resources for clusters, jobs and services
A notebook cluster, a nightly training job and a production model server have different lifecycles: kept running, deleted when done, and upgraded without downtime.

Timeline

  1. 2020

    Go operator started in November

  2. 2021

    v0.1.0 released in October

  3. 2022

    Ray 2.0 makes KubeRay the preferred deployment; Ray 2.2.0 drops the old operator

  4. 2023

    User docs move into the Ray documentation; v1.0.0 released in November

  5. 2026

    Accepted as a CNCF sandbox project in September; moves to the kuberay-project organization

Sources8 · facts checked 23 October 2026

Improve this page on GitHub