Ray

A Python framework that spreads functions (tasks) and stateful objects (actors) across a cluster, with libraries on top for data processing, training, tuning and model serving.

PyTorch Foundation project

What it is: Actor model
Used for: Model serving and MLOps
Type: Open source
Made by: PyTorch Foundation, started at UC Berkeley and developed mainly by Anyscale
Foundation: PyTorch Foundation, joined October 2025. Part of the Linux Foundation
License: Apache 2.0
Facts checked: 23 October 2026

What it is

Ray turns ordinary Python into distributed code with one decorator. A function marked @ray.remote becomes a task: calling it with .remote() returns at once with an object reference, a future, while the function runs in a worker process somewhere in the cluster. A class marked the same way becomes an actor, a stateful worker that lives in its own process and runs its methods one call at a time. Each task or actor declares what it needs, such as CPUs, GPUs or a custom resource, and the scheduler places it on a node that has them. ray.get waits for results.

Results live in a shared-memory object store, one per node, and are passed between tasks by reference, so a large array or model is not copied for every call. When a store fills up, Ray spills objects to local disk. A cluster is one head node, which holds the cluster metadata in its Global Control Store (GCS), plus worker nodes; it can run on a laptop, on VMs in AWS, GCP or Azure, or on Kubernetes, YARN or Slurm, and an autoscaler adds and removes nodes as tasks and actors wait for resources.

On top of this core sit the AI libraries: Ray Data for streaming data loading and batch inference, Ray Train for distributed training with PyTorch, Hugging Face, XGBoost and others, Ray Tune for hyperparameter search, Ray Serve for online model serving and RLlib for reinforcement learning. Ray started at UC Berkeley; its creators founded Anyscale in 2019 to build a managed service around it, and in October 2025 Anyscale contributed Ray to the PyTorch Foundation.

Why teams pick it

Small API: @ray.remote, .remote() and ray.get scale plain Python functions and classes from a laptop to thousands of cores
Tasks and actors in one system, so stateless fan-out and long-lived stateful workers (a loaded model, a simulator, a parameter server) share the same scheduler and object store
GPUs and other accelerators are first-class resources, and Ray Data streams CPU preprocessing straight into GPU inference or training to keep them busy
One cluster covers data preparation, training, tuning and serving, instead of a separate system for each step

Watch out for

The head node is a single point of failure by default: if the GCS goes down, the whole cluster fails, unless you back it with an external Redis
Scheduling and autoscaling follow the resources each task or actor declares, not what it actually uses, so wrong declarations oversubscribe or waste nodes
Memory needs watching: a full object store spills to disk, and above 95% of node memory Ray kills tasks or actors to free space
Debugging spans many processes and machines; failures show up as remote exceptions, lost objects or killed workers rather than a local stack trace

The Ray family

1 technologies built from or on Ray

Tools

KubeRayOpen sourceThe Kubernetes operator for Ray: it runs each Ray cluster as a head pod plus worker pods, and adds custom resources for long-running clusters, batch jobs and Ray Serve applications.

Alternatives

What all actor model have in common →Compare side by side →

AkkaCommercialThe JVM toolkit that brought Erlang-style actors, clustering and sharding to Java and Scala; source-available since 2022, with a paid licence for most production use.

Apache PekkoOpen sourceThe Apache-licensed fork of Akka 2.6: actors, cluster sharding, persistence, streams and HTTP for Java and Scala, run by a foundation instead of a company.

celldBetaOpen sourceA self-hosted, distributed implementation of Cloudflare Durable Objects that runs an unchanged Workers app across your own machines, with its durable state in an object storage bucket you own.

Durable ObjectsCommercialSingle-threaded JavaScript objects on Cloudflare’s network, each with a globally unique name and its own private SQLite database, that any Worker can call to coordinate state.+1 Cloudflare Agents SDK

How it fits with other technologies

What Ray uses

What uses Ray

A task fanned out four times, and an actor that keeps state between calls · python

import ray
ray.init()  # starts a local cluster, or connects to an existing one

@ray.remote
def square(x):
    return x * x

futures = [square.remote(i) for i in range(4)]  # returns object refs at once
print(ray.get(futures))  # [0, 1, 4, 9]

@ray.remote(num_cpus=1)
class Counter:
    def __init__(self):
        self.i = 0

    def incr(self, value):
        self.i += value

    def get(self):
        return self.i

counter = Counter.remote()  # a process of its own, somewhere in the cluster
for _ in range(10):
    counter.incr.remote(1)
print(ray.get(counter.get.remote()))  # 10

The libraries on top of Ray Core#

Each library is a Python package built from tasks and actors, so they share one cluster and can pass data to each other through the object store.

Ray Data

data

Loads and transforms datasets from Parquet, CSV, JSON, images, audio, video and object storage, using Arrow underneath. Its streaming engine feeds batches from CPU preprocessing into GPU workers, which suits batch inference and feeding training jobs.

Ray Train

training

Runs one training script across many workers and GPUs, with fault tolerance, for PyTorch, Lightning, Hugging Face Transformers and Accelerate, DeepSpeed, TensorFlow, XGBoost, LightGBM and JAX.

Ray Tune

tuning

Runs many training trials with different hyperparameters in parallel. It plugs in search libraries such as Optuna and HyperOpt, and schedulers such as ASHA stop poor trials early.

Ray Serve

serving

Serves models as HTTP endpoints. Each deployment is a set of actor replicas that scales up and down; several models and plain Python logic can be chained into one application, with FastAPI routes, request batching and streaming.

RLlib

reinforcement learning

Distributed reinforcement learning, the workload Ray was first built for: many simulator actors gather experience while learners update the policy.

See it in a real stack

No architecture uses Ray yet. Drop it into the playground and wire it up yourself.Open playground →

Where it came from2016–2026

Ray began at UC Berkeley in 2016, the work of Robert Nishihara, Philipp Moritz, Ion Stoica, Michael I. Jordan and colleagues. They were aiming at what they called the next generation of AI applications: programs that keep interacting with an environment and learning from it, as in reinforcement learning. Such programs mix many small parallel computations with long-lived stateful ones, so the team built a single engine that could express both task-parallel and actor-based computation.

The first release, 0.1, appeared in May 2017. The paper “Ray: A Distributed Framework for Emerging AI Applications”, posted in December 2017 and presented at OSDI 2018, described a distributed scheduler and a fault-tolerant store for control state, and reported more than 1.8 million tasks per second. By Ray 1.0 it shipped its own libraries for reinforcement learning (RLlib), hyperparameter tuning (Tune), serving (Ray Serve) and training (RaySGD).

Nishihara, Moritz and Stoica founded Anyscale in 2019 to build a managed Ray service. Ray 1.0 came out on September 30, 2020, at the first Ray Summit, and Ray 2.0 followed in August 2022. Usage grew with large-model training and inference, and on October 22, 2025, Anyscale contributed Ray to the PyTorch Foundation, part of the Linux Foundation, next to PyTorch and vLLM. Releases continue roughly monthly; 2.59.0 came out in October 2026.

Why it’s built this way

Tasks and actors in one engine: Reinforcement learning needs both stateless work (simulations) and stateful work (a model held in GPU memory). Putting both behind one API and one scheduler avoids gluing a batch system to a separate service framework.
Futures everywhere: Every call returns an object reference at once and the work runs in the background, so a program can launch more work as results arrive. The paper calls this a dynamic execution engine, as opposed to a job graph fixed before it starts.
A shared-memory object store on each node: Tasks on the same node read large results from shared memory instead of each receiving its own copy, which matters when the objects are arrays, tensors or model weights.
Libraries instead of separate systems: Data, Train, Tune, Serve and RLlib are built from the same tasks and actors, so one cluster and one set of operations cover a whole ML workflow.

Timeline

2016
Repository created at UC Berkeley
2017
Ray 0.1 released in May; the Ray paper posted in December
2018
Paper presented at USENIX OSDI
2019
Ray’s creators found Anyscale
2020
Ray 1.0 released on September 30 at the first Ray Summit
2022
Ray 2.0 released in August
2025
Contributed to the PyTorch Foundation on October 22
2026
Ray 2.59.0 released in October

Sources14 · facts checked 23 October 2026

Improve this page on GitHub