Showing the first 18000 characters. Load the complete file to read the remainder.
# Ray
> Ray is an open-source unified compute framework for scaling AI and Python workloads, including data processing, model training, model serving, hyperparameter tuning, and reinforcement learning. The full documentation lives at https://docs.ray.io/.
Full page text, grouped by section, is in [llms-full.txt](https://docs.ray.io/en/latest/llms-full.txt).
Request the links that follow with the HTTP header `Accept: text/markdown` to retrieve a Markdown rendering of the page.
## Overview
- [Overview](https://docs.ray.io/en/latest/ray-overview/index.html): Introduction to Ray, the open-source framework for scaling AI and Python workloads — its core primitives, AI libraries, and cluster integrations.
- [Ray Examples](https://docs.ray.io/en/latest/ray-overview/examples.html): Gallery of end-to-end Ray examples spanning data processing, model training, serving, and hyperparameter tuning workloads.
## Getting Started
- [Getting Started](https://docs.ray.io/en/latest/ray-overview/getting-started.html): Quickstart paths for Ray: scale ML workloads with the AI libraries, parallelize Python with Ray Core, or deploy to the cloud with Ray Clusters.
## Installation
- [Installing Ray](https://docs.ray.io/en/latest/ray-overview/installation.html): Install Ray and its AI libraries with pip across Linux, Windows (beta), and macOS on x86_64, ARM, and Apple silicon hardware.
## Use Cases
- [Ray Use Cases](https://docs.ray.io/en/latest/ray-overview/use-cases.html): Common Ray use cases: LLMs and generative AI, batch inference, distributed training, hyperparameter tuning, reinforcement learning, and ML platforms.
- [Ray for ML Infrastructure](https://docs.ray.io/en/latest/ray-air/getting-started.html): Overview of using Ray as ML infrastructure, covering how Ray Train, Ray Data, and Ray Serve compose end-to-end ML workflows on a unified compute runtime.
## Examples
- [Examples](https://docs.ray.io/en/latest/ray-overview/examples/index.html): End-to-end Ray examples: multimodal AI, audio and timeseries, XGBoost, object detection, RAG, and serving LLM agents with MCP and LangChain.
- [Multi-modal AI pipeline](https://docs.ray.io/en/latest/_collections/ray-overview/examples/e2e-multimodal-ai-workloads/README.html): 💻 Run this entire tutorial on Anyscale for free: https://console.anyscale.com/template-preview/image-search-and-classification or access the repository here.
- [LLM training and inference](https://docs.ray.io/en/latest/_collections/ray-overview/examples/entity-recognition-with-llms/README.html): This end-to-end tutorial fine-tunes an LLM to perform batch inference and online serving at scale. While entity recognition (NER) is the main task in this tutorial, you can easily extend these end-to-end workflows to any use case.
- [Audio batch inference](https://docs.ray.io/en/latest/_collections/ray-overview/examples/e2e-audio/README.html): This tutorial demonstrates a batch inference pipeline that converts raw audio files into a curated subset using two different ML models. The pipeline chains together four stages that each run on the same Ray cluster using heterogeneous resources:
- [Distributed XGBoost pipeline](https://docs.ray.io/en/latest/_collections/ray-overview/examples/e2e-xgboost/README.html): These tutorials implement an end-to-end XGBoost application including:
- [Time-series forecasting](https://docs.ray.io/en/latest/_collections/ray-overview/examples/e2e-timeseries/README.html): These tutorials implement an end-to-end time-series application including:
- [Scalable video processing](https://docs.ray.io/en/latest/_collections/ray-overview/examples/object-detection/README.html): This tutorial builds an end-to-end face mask detection pipeline that leverages distributed fine-tuning, large-scale batch inference, video analytics, and scalable serving:
- [Distributed RAG pipeline](https://docs.ray.io/en/latest/_collections/ray-overview/examples/e2e-rag/README.html): This tutorial covers end-to-end Retrieval-Augmented Generation (RAG) pipelines using Ray, from data ingestion and LLM deployment to prompt engineering, evaluation and scaling out all workloads in the application.
- [Deploy MCP servers](https://docs.ray.io/en/latest/_collections/ray-overview/examples/mcp-ray-serve/README.html): This repository provides end-to-end examples for deploying and scaling Model Context Protocol (MCP) servers using Ray Serve and Anyscale Service, covering both streamable HTTP and stdio transport types:
- [Build a tool-using agent](https://docs.ray.io/en/latest/_collections/ray-overview/examples/langchain_agent_ray_serve/content/README.html): This tutorial guides you through building and deploying a sophisticated, tool-using agent using LangChain, LangGraph, and Ray Serve on Anyscale.
- [Build a multi-agent system with the A2A protocol](https://docs.ray.io/en/latest/_collections/ray-overview/examples/multi_agent_a2a/README.html): This tutorial guides you through building and deploying a multi-agent system where agents communicate using the A2A (Agent-to-Agent) protocol. The system is built on Ray Serve for scalable deployment, LangChain for agent orchestration, and MCP for…
## Ecosystem
- [The Ray Ecosystem](https://docs.ray.io/en/latest/ray-overview/ray-libraries.html): Third-party libraries and tools that integrate with Ray for distributed execution, plus an ecosystem map showing component maturity.
## Ray Core
- [What’s Ray Core?](https://docs.ray.io/en/latest/ray-core/walkthrough.html): Ray's distributed computing primitives — tasks, actors, and objects — with examples for turning Python functions and classes into distributed apps.
- [Key Concepts](https://docs.ray.io/en/latest/ray-core/key-concepts.html): Ray Core primitives: tasks (remote functions), actors (stateful remote classes), objects in the distributed object store, and placement groups.
- [User Guides](https://docs.ray.io/en/latest/ray-core/user-guide.html): Ray Core how-to guides: tasks, actors, objects, dependencies, scheduling, fault tolerance, design patterns, and compiled graphs.
- [Tasks](https://docs.ray.io/en/latest/ray-core/tasks.html): Run Python functions asynchronously as Ray tasks: request resources, pass ObjectRefs, wait for partial results, and cancel tasks.
- [Nested Remote Functions](https://docs.ray.io/en/latest/ray-core/tasks/nested-tasks.html): Call remote functions from within remote functions, and how nested tasks yield their resources while blocked.
- [Actors](https://docs.ray.io/en/latest/ray-core/actors.html): Turn Python classes into stateful Ray actors: declare resources, call methods, pass handles around, use generators, and cancel actor tasks.
- [Named Actors](https://docs.ray.io/en/latest/ray-core/actors/named-actors.html): Give an actor a unique name in its namespace so any job in the cluster can retrieve it, with get-or-create and actor lifetime options.
- [Terminating Actors](https://docs.ray.io/en/latest/ray-core/actors/terminating-actors.html): Terminate Ray actors from a handle or from inside the actor, and how actor cleanup runs when ray.shutdown is called.
- [AsyncIO / Concurrency for Actors](https://docs.ray.io/en/latest/ray-core/actors/async_api.html): Concurrency for Ray actors: async actors built on asyncio, threaded actors, ObjectRefs as futures, and setting max concurrency.
- [Limiting Concurrency Per-Method with Concurrency Groups](https://docs.ray.io/en/latest/ray-core/actors/concurrency_group_api.html): Limit concurrency per actor method with concurrency groups, including the default group and choosing a group at call time.
- [Utility Classes](https://docs.ray.io/en/latest/ray-core/actors/actor-utils.html): Utility classes for actors: ActorPool for load-balanced method calls and Ray Queue for message passing between tasks and actors.
- [Out-of-band Communication](https://docs.ray.io/en/latest/ray-core/actors/out-of-band-communication.html): Communicate between actors outside method calls: wrapping library processes, Ray Collective, and running an HTTP server in an actor.
- [Actor Task Execution Order](https://docs.ray.io/en/latest/ray-core/actors/task-orders.html): How Ray orders actor method execution, and how that differs between synchronous single-threaded actors and async or threaded actors.
- [Objects](https://docs.ray.io/en/latest/ray-core/objects.html): Ray remote objects and ObjectRefs: fetching data, passing objects as arguments, closure capture, nested objects, and fault tolerance.
- [Serialization](https://docs.ray.io/en/latest/ray-core/objects/serialization.html): How Ray serializes data for the Plasma object store, with NumPy zero-copy reads, read-only array pitfalls, and custom serializers.
- [Object Spilling](https://docs.ray.io/en/latest/ray-core/objects/object-spilling.html): Configure where Ray spills objects once the object store fills up, including custom spill directories and spill statistics.
- [Environment Dependencies](https://docs.ray.io/en/latest/ray-core/handling-dependencies.html): Ship Python and system dependencies to a Ray cluster with runtime environments, per job or per task, using local files, conda, or pip.
- [Scheduling](https://docs.ray.io/en/latest/ray-core/scheduling/index.html): How Ray schedules tasks and actors onto nodes: labels, resources, and the DEFAULT, SPREAD, placement group, and node affinity strategies.
- [Use labels to control scheduling](https://docs.ray.io/en/latest/ray-core/scheduling/labels.html): Learn about using labels to control how Ray schedules tasks, actors, and placement groups to nodes in your Kubernetes cluster.
- [Resources](https://docs.ray.io/en/latest/ray-core/scheduling/resources.html): Ray physical and logical resources: define custom resources, set node capacity, and request fractional resources per task or actor.
- [Accelerator Support](https://docs.ray.io/en/latest/ray-core/scheduling/accelerators.html): Use accelerators in Ray tasks and actors: start nodes with GPUs, request fractional accelerators, and pin to accelerator types.
- [Placement Groups](https://docs.ray.io/en/latest/ray-core/scheduling/placement-group.html): Atomically reserve resources across nodes with placement groups, Ray's gang scheduling primitive, using bundles and placement strategies.
- [Memory Management](https://docs.ray.io/en/latest/ray-core/scheduling/memory-management.html): How Ray manages memory: ObjectRef reference counting, inspecting usage with ray memory, and memory-aware scheduling.
- [Out-Of-Memory Prevention](https://docs.ray.io/en/latest/ray-core/scheduling/ray-oom-prevention.html): Ray's memory monitor, which prevents node OOM by killing workers under memory pressure, with its retry and worker-killing policies.
- [Fault tolerance](https://docs.ray.io/en/latest/ray-core/fault-tolerance.html): How Ray classifies application-level and system-level failures, and where to find the per-component fault tolerance guarantees.
- [Task Fault Tolerance](https://docs.ray.io/en/latest/ray-core/fault_tolerance/tasks.html): Task fault tolerance in Ray: catch application-level errors, configure retries for failed tasks, and cancel misbehaving tasks.
- [Actor Fault Tolerance](https://docs.ray.io/en/latest/ray-core/fault_tolerance/actors.html): Actor fault tolerance in Ray: process and creator failure, restarts and checkpointing, force-killing, and unavailable-actor errors.
- [Object Fault Tolerance](https://docs.ray.io/en/latest/ray-core/fault_tolerance/objects.html): Object fault tolerance in Ray: lineage-based recovery from data loss, recovery from owner failure, and understanding ObjectLostError.
- [Node Fault Tolerance](https://docs.ray.io/en/latest/ray-core/fault_tolerance/nodes.html): Node fault tolerance in Ray: what happens when a worker node, the head node, or an individual raylet fails.
- [GCS Fault Tolerance](https://docs.ray.io/en/latest/ray-core/fault_tolerance/gcs.html): Make the GCS fault tolerant by backing cluster metadata with Redis or the alpha embedded RocksDB, with tuning guidance.
- [Design Patterns & Anti-patterns](https://docs.ray.io/en/latest/ray-core/patterns/index.html): Catalog of Ray Core design patterns and anti-patterns for structuring tasks, actors, and object passing.
- [Pattern: Using nested tasks to achieve nested parallelism](https://docs.ray.io/en/latest/ray-core/patterns/nested-tasks.html): Pattern: call remote functions from inside remote functions to express nested parallelism such as divide-and-conquer.
- [Pattern: Using generators to reduce heap memory usage](https://docs.ray.io/en/latest/ray-core/patterns/generators.html): Pattern: yield results from a generator task instead of returning them all at once, to cap heap memory usage.
- [Pattern: Using ray.wait to limit the number of pending tasks](https://docs.ray.io/en/latest/ray-core/patterns/limit-pending-tasks.html): Pattern: use ray.wait to bound the number of in-flight tasks so the submission loop doesn't outrun the cluster.
- [Pattern: Using resources to limit the number of concurrently running tasks](https://docs.ray.io/en/latest/ray-core/patterns/limit-running-tasks.html): Pattern: use custom resource requirements to cap how many tasks or actors run concurrently, limiting memory pressure.
- [Pattern: Using asyncio to run actor methods concurrently](https://docs.ray.io/en/latest/ray-core/patterns/concurrent-operations-async-actor.html): Pattern: use an async actor so its methods run concurrently on one worker, overlapping I/O-bound operations.
- [Pattern: Using an actor to synchronize other tasks and actors](https://docs.ray.io/en/latest/ray-core/patterns/actor-sync.html): Pattern: use a single actor as a synchronization point to coordinate other tasks and actors.
- [Pattern: Using a supervisor actor to manage a tree of actors](https://docs.ray.io/en/latest/ray-core/patterns/tree-of-actors.html): Pattern: use a supervisor actor to create and manage a tree of worker actors, centralizing lifecycle and failure handling.
- [Pattern: Using pipelining to increase throughput](https://docs.ray.io/en/latest/ray-core/patterns/pipelining.html): Pattern: pipeline task submission and result processing so compute overlaps with data transfer and throughput rises.
- [Anti-pattern: Returning ray.put() ObjectRefs from a task harms performance and fault tolerance](https://docs.ray.io/en/latest/ray-core/patterns/return-ray-put.html): Anti-pattern: returning ray.put ObjectRefs from a task adds a copy and weakens fault tolerance; return the value instead.
- [Anti-pattern: Calling ray.get on task arguments harms performance](https://docs.ray.io/en/latest/ray-core/patterns/nested-ray-get.html): Anti-pattern: calling ray.get on task arguments serializes execution; pass ObjectRefs and let Ray resolve them.
- [Anti-pattern: Calling ray.get in a loop harms parallelism](https://docs.ray.io/en/latest/ray-core/patterns/ray-get-loop.html): Anti-pattern: calling ray.get inside a loop blocks on each task in turn and destroys parallelism; collect the refs first.
- [Anti-pattern: Calling ray.get unnecessarily harms performance](https://docs.ray.io/en/latest/ray-core/patterns/unnecessary-ray-get.html): Anti-pattern: calling ray.get before the value is needed blocks the driver and forfeits overlap between tasks.
- [Anti-pattern: Processing results in submission order using ray.get increases runtime](https://docs.ray.io/en/latest/ray-core/patterns/ray-get-submission-order.html): Anti-pattern: processing results in submission order idles on slow tasks; use ray.wait to handle them as they complete.
- [Anti-pattern: Fetching too many objects at once with ray.get causes failure](https://docs.ray.io/en/latest/ray-core/patterns/ray-get-too-many-objects.html): Anti-pattern: fetching too many objects at once with ray.get can exhaust the object store or heap and fail the job.
- [Anti-pattern: Over-parallelizing with too fine-grained tasks harms speedup](https://docs.ray.io/en/latest/ray-core/patterns/too-fine-grained-tasks.html): Anti-pattern: splitting work into very small tasks lets per-task overhead dominate; batch work into coarser tasks.
- [Anti-pattern: Redefining the same remote function or class harms performance](https://docs.ray.io/en/latest/ray-core/patterns/redefine-task-actor-loop.html): Anti-pattern: redefining the same remote function or actor class in a loop re-exports it each time, adding overhead.
- [Anti-pattern: Passing the same large argument by value repeatedly harms performance](https://docs.ray.io/en/latest/ray-core/patterns/pass-large-arg-by-value.html): Anti-pattern: passing the same large argument by value to many tasks re-serializes it each time; ray.put it once instead.
- [Anti-pattern: Closure capturing large objects harms performance](https://docs.ray.io/en/latest/ray-core/patterns/closure-capture-large-objects.html): Anti-pattern: capturing large objects in a remote function's closure copies them to every worker; put them in the object store instead.
- [Anti-pattern: Using global variables to share state between tasks and actors](https://docs.ray.io/en/latest/ray-core/patterns/global-variables.html): Anti-pattern: global variables don't propagate between Ray workers; pass state explicitly or hold it in an actor.
- [Anti-pattern: Serialize ray.ObjectRef out of band](https://docs.ray.io/en/latest/ray-core/patterns/out-of-band-object-ref-serialization.html): Anti-pattern: serializing an ObjectRef out of band escapes reference counting and can let its value be reclaimed early.
- [Anti-pattern: Forking new processes in application code](https://docs.ray.io/en/latest/ray-core/patterns/fork-new-processes.html): Anti-pattern: forking processes inside a Ray task or actor breaks Ray's process management and resource accounting.
- [Ray Direct Transport (RDT)](https://docs.ray.io/en/latest/ray-core/direct-transport/direct-transport.html): Ray Direct Transport moves tensors between actors without going through the object store, using collective groups over Gloo or NCCL.
- [Implementing a custom tensor transport (Advanced)](https://docs.ra
Measured from the saved file
Linked document
UTF-8 text size
125.8 KB128810 bytes
Token count
29008o200k_base encoding
Markdown links
534534 distinct URLs
Fenced code blocks
0600 lines in the file
Size measures the captured UTF-8 text, not compressed transfer size. Tokens are counted with a named tokenizer; other models can count differently. Neither measure establishes context compatibility or answer quality.
Project H1
Ray
Opening blockquote
Ray is an open-source unified compute framework for scaling AI and Python workloads, including data processing, model training, model serving, hyperparameter tuning, and reinforcement learning. The full documentation lives at https://docs.ray.io/.
Context architecture
How the content is divided
20 H2 sections organize 534 parsed Markdown links. 0 links point to .md or .markdown paths; 0 link occurrences use another hostname. Linked destinations have not been recursively validated.
H2 section
Links
UTF-8 bytes
Tokens
OverviewLine 9
2
418
88
Getting StartedLine 14
1
232
55
InstallationLine 18
1
210
53
Use CasesLine 22
2
475
109
ExamplesLine 27
11
3273
718
EcosystemLine 41
1
221
47
Ray CoreLine 45
90
20787
4627
Ray DataLine 138
38
8592
1898
Ray TrainLine 179
45
10863
2387
Ray TuneLine 227
29
6742
1528
Ray ServeLine 259
94
23202
5236
Ray RLlibLine 356
27
6135
1371
More LibrariesLine 386
18
3728
951
APIsLine 407
3
592
143
Ray ClustersLine 413
123
29503
6800
Monitoring and DebuggingLine 539
23
5584
1243
Show 4 more sections
H2 section
Links
UTF-8 bytes
Tokens
Developer GuidesLine 565
19
6226
1319
GlossaryLine 587
2
372
85
SecurityLine 592
2
434
92
Project GovernanceLine 597
2
440
95
Link counts exclude parsed fenced and inline code and resolve relative destinations against the final endpoint. Section sizes cover the text below each H2 until the next H2.
Embedded material
Code, installation and API examples
Fenced blocks
0 closed
Declared languages
No fenced blocks detected
Package-install commands
0 detected in fenced or inline code
Authentication examples
0 fenced lines match API-key, Authorization, Bearer, OAuth or access-token patterns
HTTP request examples
0 fenced lines match request-method patterns
Authentication links
0 links have authentication-related labels
Embedded example text and links to further documentation are counted separately. A code block does not establish that a sample runs or that an authentication flow succeeds.
Publisher prioritization
The Optional section
No H2 section named “Optional” was detected in this file.
The community proposal uses Optional by convention for secondary links that can be skipped for a shorter context. This label does not establish that everything outside it is critical, or that the material inside it is deprecated.
Machine-readable routes
MCP references and configuration text
MCP-related text was detected in the captured file. 1 links have MCP-related labels or paths. 0 fenced lines match mcpServers, mcp_servers or an mcp add command.
A verified text response establishes a companion at the checked URL. It does not prove the bundle contains all product documentation, that generation is automated, or that it fits an assistant’s context window.
Change evidence
What can be said about freshness
This capture establishes a baseline. Update frequency cannot be determined from one observation.
Linked release and migration material
1 links have release, changelog, deprecation or migration terms in their labels or paths. These provide places to investigate product changes separately.
File changes can reveal documentation edits or navigation changes. They do not by themselves measure product release velocity, accuracy, agent adoption or AI visibility.
Counts use a deterministic parser, not a spec-compliance grade. Unusual Markdown and indented code can fall outside its recognized patterns. See the llms.txt proposal and tokenizer implementation.