Modal llms.txt

The actual file

Source Markdown

The saved llms.txt text, displayed as source. The file belongs to its publisher; the live endpoint may change.

Showing the first 18000 characters. Load the complete file to read the remainder.

# Modal Documentation

> Modal is a platform for running AI workloads in the cloud with minimal
> configuration. Key use cases include AI model training and inference,
> high-performance batch jobs, and sandboxed code execution. It offers
> fast prototyping, serverless APIs, scheduled jobs, GPU inference,
> distributed storage volumes, and highly scalable secure sandboxes.

Important notes:

- Modal's platform is optimized for AI use cases, but it can support
  general-purpose cloud workflows.
- Modal is a serverless platform, meaning you are only billed for resources used.
  Containers scale to zero when idle and boot in seconds when needed.
- Modal offers official SDKs in Python, Go, and JavaScript/TypeScript, along with
  a fully-featured CLI for interacting with the platform.

You can sign up for free at [https://modal.com] and get $30/month of credits.

The docs are organized into three main sections:
- _Guide_ pages explain Modal's features, primitives, and workflows
- _Examples_ pages contain didactic examples of many different AI applications
- _Reference_ pages provide detailed information about the SDKs and CLI

## Guide

- Guide
  - [Introduction to Modal](https://modal.com/docs/guide/intro.md)
  - [Getting started](https://modal.com/docs/guide/getting-started.md)
- Functions
  - [Functions](https://modal.com/docs/guide/functions.md)
  - [Invocation methods](https://modal.com/docs/guide/function-invocation-methods.md)
  - [Function lookups](https://modal.com/docs/guide/trigger-deployed-functions.md)
  - [Container lifecycle management](https://modal.com/docs/guide/lifecycle-functions.md)
  - [Parametrized Functions](https://modal.com/docs/guide/parametrized-functions.md)
  - [Dynamic configuration](https://modal.com/docs/guide/dynamic-function-config.md)
- Sandboxes
  - [Sandboxes](https://modal.com/docs/guide/sandboxes.md)
  - [Running commands](https://modal.com/docs/guide/sandbox-spawn.md)
  - [Networking and security](https://modal.com/docs/guide/sandbox-networking.md)
  - [File access](https://modal.com/docs/guide/sandbox-files.md)
  - [Snapshots](https://modal.com/docs/guide/sandbox-snapshots.md)
  - [Pricing and resources](https://modal.com/docs/guide/sandbox-resources.md)
  - [Sidecars (Beta)](https://modal.com/docs/guide/sandbox-sidecars.md)
  - [Secret injection (Beta)](https://modal.com/docs/guide/sandbox-secret-injection.md)
- Endpoints
  - [Endpoints](https://modal.com/docs/guide/endpoints.md)
  - [Shared Endpoints](https://modal.com/docs/guide/shared-endpoints.md)
  - [Dedicated Endpoints](https://modal.com/docs/guide/dedicated-endpoints.md)
  - [Endpoint candidates](https://modal.com/docs/guide/endpoint-candidates.md)
  - [Endpoint integrations](https://modal.com/docs/guide/endpoint-integrations.md)
- Training
  - [Quickstart](https://modal.com/docs/guide/training-quickstart.md)
  - [Tinker-compatible API (Beta)](https://modal.com/docs/guide/spindle.md)
  - [Multi-node training](https://modal.com/docs/guide/multi-node-training.md)
- Images
  - [Defining Images](https://modal.com/docs/guide/images.md)
  - [Using existing container images](https://modal.com/docs/guide/existing-images.md)
  - [Named images](https://modal.com/docs/guide/named-images.md)
  - [Fast pull from registry](https://modal.com/docs/guide/fast-pull-from-registry.md)
- GPUs and other resources
  - [GPU acceleration](https://modal.com/docs/guide/gpu.md)
  - [Using CUDA on Modal](https://modal.com/docs/guide/cuda.md)
  - [Configuring CPU, memory, and disk](https://modal.com/docs/guide/resources.md)
- Scaling out
  - [Scaling out](https://modal.com/docs/guide/scale.md)
  - [Input concurrency](https://modal.com/docs/guide/concurrent-inputs.md)
  - [Batch processing](https://modal.com/docs/guide/batch-processing.md)
  - [Job queues](https://modal.com/docs/guide/job-queue.md)
  - [Dynamic batching](https://modal.com/docs/guide/dynamic-batching.md)
  - [Multi-node Clusters](https://modal.com/docs/guide/multi-node-clusters.md)
- Deployment
  - [Apps, Functions, and entrypoints](https://modal.com/docs/guide/apps.md)
  - [Managing deployments](https://modal.com/docs/guide/managing-deployments.md)
  - [Continuous deployment](https://modal.com/docs/guide/continuous-deployment.md)
  - [Scheduled Functions](https://modal.com/docs/guide/cron.md)
- HTTP Applications
  - [Servers](https://modal.com/docs/guide/servers.md)
  - [Sticky Sessions](https://modal.com/docs/guide/sticky-sessions.md)
  - [Web Functions](https://modal.com/docs/guide/webhooks.md)
  - [Streaming endpoints](https://modal.com/docs/guide/streaming-endpoints.md)
  - [Web Function URLs](https://modal.com/docs/guide/webhook-urls.md)
  - [Request timeouts](https://modal.com/docs/guide/webhook-timeouts.md)
  - [Proxy Tokens](https://modal.com/docs/guide/webhook-proxy-auth.md)
- Networking
  - [Tunnels](https://modal.com/docs/guide/tunnels.md)
  - [Proxies (Beta)](https://modal.com/docs/guide/proxy-ips.md)
  - [Cluster networking](https://modal.com/docs/guide/private-networking.md)
- Data sharing and storage
  - [Passing local data](https://modal.com/docs/guide/local-data.md)
  - [Volumes](https://modal.com/docs/guide/volumes.md)
  - [Storing model weights](https://modal.com/docs/guide/model-weights.md)
  - [Cloud bucket mounts](https://modal.com/docs/guide/cloud-bucket-mounts.md)
  - [Dicts](https://modal.com/docs/guide/dicts.md)
  - [Queues](https://modal.com/docs/guide/queues.md)
  - [Dataset ingestion](https://modal.com/docs/guide/dataset-ingestion.md)
- Secrets and environment variables
  - [Secrets](https://modal.com/docs/guide/secrets.md)
  - [Environment variables](https://modal.com/docs/guide/environment_variables.md)
- Performance
  - [Cold start performance](https://modal.com/docs/guide/cold-start.md)
  - [Memory Snapshots](https://modal.com/docs/guide/memory-snapshots.md)
  - [High-performance LLM inference](https://modal.com/docs/guide/high-performance-llm-inference.md)
- Reliability and robustness
  - [Failures and retries](https://modal.com/docs/guide/retries.md)
  - [Preemption](https://modal.com/docs/guide/preemption.md)
  - [Timeouts](https://modal.com/docs/guide/timeouts.md)
  - [GPU health](https://modal.com/docs/guide/gpu-health.md)
  - [Troubleshooting](https://modal.com/docs/guide/troubleshooting.md)
- Security and privacy
  - [Security and privacy](https://modal.com/docs/guide/security.md)
  - [Data residency](https://modal.com/docs/guide/data-residency.md)
  - [Customer-supplied encryption keys (Alpha)](https://modal.com/docs/guide/customer-supplied-encryption-keys.md)
  - [Audit logs](https://modal.com/docs/guide/audit-logs.md)
- Integrations
  - [Using OIDC to authenticate with external services](https://modal.com/docs/guide/oidc-integration.md)
  - [Connecting Modal to your Datadog account](https://modal.com/docs/guide/datadog-integration.md)
  - [Connecting Modal to your OpenTelemetry provider](https://modal.com/docs/guide/otel-integration.md)
  - [Okta SSO](https://modal.com/docs/guide/okta-sso.md)
  - [Microsoft Entra SSO](https://modal.com/docs/guide/entra-sso.md)
  - [Custom SAML SSO](https://modal.com/docs/guide/saml-sso.md)
  - [SCIM Integration](https://modal.com/docs/guide/scim.md)
  - [Slack notifications (Beta)](https://modal.com/docs/guide/slack-notifications.md)
- Workspace & account settings
  - [Workspaces](https://modal.com/docs/guide/workspaces.md)
  - [Environments](https://modal.com/docs/guide/environments.md)
  - [Service users](https://modal.com/docs/guide/service-users.md)
  - [User groups](https://modal.com/docs/guide/user-groups.md)
  - [Role-Based Access Control (RBAC)](https://modal.com/docs/guide/rbac.md)
  - [Billing](https://modal.com/docs/guide/billing.md)
  - [Network egress billing](https://modal.com/docs/guide/network-egress-billing.md)
  - [Budgets](https://modal.com/docs/guide/budgets.md)
- Other topics
  - [Feature maturity](https://modal.com/docs/guide/feature-maturity.md)
  - [Notebooks](https://modal.com/docs/guide/notebooks.md)
  - [Restricted Functions](https://modal.com/docs/guide/restricted-access.md)
  - [JavaScript/Go SDKs (Beta)](https://modal.com/docs/guide/sdk-javascript-go.md)
  - [Modal 1.0 migration guide](https://modal.com/docs/guide/modal-1-0-migration.md)
  - [File and project structure](https://modal.com/docs/guide/project-structure.md)
  - [Developing and debugging](https://modal.com/docs/guide/developing-debugging.md)
  - [Jupyter notebooks](https://modal.com/docs/guide/jupyter-notebooks.md)
  - [Asynchronous API usage](https://modal.com/docs/guide/async.md)
  - [Global variables](https://modal.com/docs/guide/global-variables.md)
  - [Region selection](https://modal.com/docs/guide/region-selection.md)
  - [GPU Metrics](https://modal.com/docs/guide/gpu-metrics.md)

## Examples

- [Featured](https://modal.com/docs/examples)
- Getting started
  - [Hello, world](https://modal.com/docs/examples/hello_world.md)
  - [Simple web scraper](https://modal.com/docs/examples/webscraper.md)
  - [Serving Web Functions](https://modal.com/docs/examples/basic_web.md)
- Large language models (LLMs)
  - [Deploy an OpenAI-compatible LLM service with vLLM](https://modal.com/docs/examples/llm_inference.md)
  - [Cut Ministral 3 cold start times by 10x with snapshots](https://modal.com/docs/examples/ministral3_inference.md)
  - [Maximize tokens per second in batch processing with vLLM](https://modal.com/docs/examples/vllm_throughput.md)
  - [Serve an ultra-low-latency chatbot with SGLang](https://modal.com/docs/examples/sglang_low_latency.md)
  - [Deploy Nemotron 3](https://modal.com/docs/examples/nemotron_inference.md)
  - [Serve Inkling-Small with SGLang](https://modal.com/docs/examples/inkling_small.md)
  - [Serve DeepSeek-V4-Flash](https://modal.com/docs/examples/deepseek_v4_flash.md)
  - [Efficient LLM Finetuning with Unsloth](https://modal.com/docs/examples/unsloth_finetune.md)
  - [Run a multimodal RAG chatbot to answer questions about PDFs](https://modal.com/docs/examples/chat_with_pdf_vision.md)
  - [Fine-tune an LLM to replace your CEO](https://modal.com/docs/examples/llm-finetuning.md)
  - [Deploy a stateless MCP with FastMCP](https://modal.com/docs/examples/mcp_server_stateless.md)
- Images, video, & 3D
  - [Edit images with Flux Kontext](https://modal.com/docs/examples/image_to_image.md)
  - [Fine-tune Wan2.1 video models on your face](https://modal.com/docs/examples/music-video-gen.md)
  - [Run Flux fast with torch.compile](https://modal.com/docs/examples/flux.md)
  - [Fine-tune Flux with LoRA](https://modal.com/docs/examples/diffusers_lora_finetune.md)
  - [Animate images with LTX-Video](https://modal.com/docs/examples/image_to_video.md)
  - [Generate video clips with LTX-Video](https://modal.com/docs/examples/ltx.md)
  - [Run Stable Diffusion with a CLI, API, and web UI](https://modal.com/docs/examples/text_to_image.md)
- Audio
  - [Deploy a Moshi voice chatbot](https://modal.com/docs/examples/llm-voice-chat.md)
  - [Stream transcripts at the speed of speech using Kyutai STT](https://modal.com/docs/examples/streaming_kyutai_stt.md)
  - [Make music with ACE-Step](https://modal.com/docs/examples/generate_music.md)
  - [Generate speech with Chatterbox](https://modal.com/docs/examples/chatterbox_tts.md)
  - [Run high throughput batched transcription with Whisper](https://modal.com/docs/examples/batched_whisper.md)
  - [Fine-tune Whisper to recognize new words](https://modal.com/docs/examples/fine_tune_asr.md)
- Real-time communication (WebRTC)
  - [Serverless WebRTC](https://modal.com/docs/examples/webrtc_yolo.md)
  - [WebRTC quickstart with FastRTC](https://modal.com/docs/examples/fastrtc_flip_webcam.md)
- Computational biology
  - [Design protein binders with ESMFold2](https://modal.com/docs/examples/esmfold2_binder_design.md)
  - [Fold proteins with Chai-1](https://modal.com/docs/examples/chai1.md)
  - [Build a protein-folding dashboard](https://modal.com/docs/examples/esm3.md)
  - [Fold proteins with Boltz-2](https://modal.com/docs/examples/boltz_predict.md)
- Modal Sandboxes
  - [Run Cursor Cloud Agents on Modal](https://modal.com/docs/examples/cursor.md)
  - [Run a background coding agent with OpenCode](https://modal.com/docs/examples/opencode_server.md)
  - [Build a scalable AI coding platform](https://modal.com/docs/examples/modal-vibe.md)
  - [Create GIFs from Slack using the Claude Agent SDK](https://modal.com/docs/examples/claude-slack-gif-creator.md)
  - [Run a LangGraph agent's code in a secure GPU sandbox](https://modal.com/docs/examples/agent.md)
  - [Watch a computer-use agent work in real-time over VNC](https://modal.com/docs/examples/computer_use_vnc.md)
  - [Run massively parallel evals on Modal Sandboxes](https://modal.com/docs/examples/harbor_evals.md)
  - [Build a stateful, sandboxed code interpreter](https://modal.com/docs/examples/simple_code_interpreter.md)
  - [Run Node.js, Ruby, and more in a Sandbox](https://modal.com/docs/examples/safe_code_execution.md)
  - [Speed up Sandbox starts with warm pools](https://modal.com/docs/examples/sandbox_pool.md)
  - [Inject secrets into outbound requests](https://modal.com/docs/examples/sandbox_secret_injection.md)
- Reinforcement Learning
  - [Paint flowers with code](https://modal.com/docs/examples/paint_flowers.md)
  - [Train coding agents](https://modal.com/docs/examples/swe_gym.md)
  - [Make LLMs better at math](https://modal.com/docs/examples/miles_grpo.md)
- Embeddings
  - [Serve Liquid AI embeddings with Modal Servers](https://modal.com/docs/examples/liquidai_embeddings_server.md)
  - [Embed millions of documents with TEI](https://modal.com/docs/examples/amazon_embeddings.md)
  - [Turn satellite images into vectors and store them in MongoDB](https://modal.com/docs/examples/mongodb-search.md)
- Parallel processing and job scheduling
  - [Deploy a Hacker News Slackbot](https://modal.com/docs/examples/hackernews_alerts.md)
  - [Run a Document OCR job queue](https://modal.com/docs/examples/doc_ocr_jobs.md)
  - [Serve a Document OCR web app](https://modal.com/docs/examples/doc_ocr_webapp.md)
- Training models from scratch
  - [Train an SLM with early-stopping grid search over hyperparameters](https://modal.com/docs/examples/hp_sweep_gpt.md)
  - [Run long, resumable training jobs](https://modal.com/docs/examples/long-training.md)
- Hosting popular libraries
  - [YOLO: Fine-tune and serve computer vision models](https://modal.com/docs/examples/finetune_yolo.md)
  - [Blender: Build a 3D render farm](https://modal.com/docs/examples/blender_video.md)
  - [Streamlit: Run and deploy Streamlit apps](https://modal.com/docs/examples/serve_streamlit.md)
  - [SQLite: Publish explorable data with Datasette](https://modal.com/docs/examples/cron_datasette.md)
  - [Algolia: Build docsearch with a crawler](https://modal.com/docs/examples/algolia_indexer.md)
- Connecting to other APIs
  - [Discord: Deploy and run a Discord Bot](https://modal.com/docs/examples/discord_bot.md)
  - [Google Sheets: Sync databases and APIs to a Google Sheet](https://modal.com/docs/examples/db_to_sheet.md)
  - [OpenAI: Run a RAG Q&A chatbot](https://modal.com/docs/examples/potus_speech_qanda.md)
  - [Tailscale: Add Modal Apps to your VPN](https://modal.com/docs/examples/modal_tailscale.md)
  - [Prometheus: Publish custom metrics with Pushgateway](https://modal.com/docs/examples/pushgateway.md)
- Managing data
  - [Mount S3 buckets in Modal Apps](https://modal.com/docs/examples/s3_bucket_mount.md)
  - [Build your own data warehouse with DuckDB, dbt, and Modal](https://modal.com/docs/examples/dbt_duckdb.md)
- [Miscellaneous](https://modal.com/docs/examples/miscellaneous)

## Reference

- [Python SDK](https://modal.com/docs/sdk/py/latest/intro.md)
  - [Python SDK & CLI Release Notes](https://modal.com/docs/sdk/py/releases.md)
  - [`App`](https://modal.com/docs/sdk/py/latest/App.md)
  - [`Client`](https://modal.com/docs/sdk/py/latest/Client.md)
  - [`CloudBucketMount`](https://modal.com/docs/sdk/py/latest/CloudBucketMount.md)
  - [`Cls`](https://modal.com/docs/sdk/py/latest/Cls.md)
  - [`Cluster`](https://modal.com/docs/sdk/py/latest/Cluster.md)
  - [`Cron`](https://modal.com/docs/sdk/py/latest/Cron.md)
  - [`Dict`](https://modal.com/docs/sdk/py/latest/Dict.md)
  - [`Environment`](https://modal.com/docs/sdk/py/latest/Environment.md)
  - [`Error`](https://modal.com/docs/sdk/py/latest/Error.md)
  - [`FilePatternMatcher`](https://modal.com/docs/sdk/py/latest/FilePatternMatcher.md)
  - [`Function`](https://modal.com/docs/sdk/py/latest/Function.md)
  - [`FunctionCall`](https://modal.com/docs/sdk/py/latest/FunctionCall.md)
  - [`Image`](https://modal.com/docs/sdk/py/latest/Image.md)
  - [`Period`](https://modal.com/docs/sdk/py/latest/Period.md)
  - [`Probe`](https://modal.com/docs/sdk/py/latest/Probe.md)
  - [`Proxy`](https://modal.com/docs/sdk/py/latest/Proxy.md)
  - [`Queue`](https://modal.com/docs/sdk/py/latest/Queue.md)
  - [`Retries`](https://modal.com/docs/sdk/py/latest/Retries.md)
  - [`Sandbox`](https://modal.com/docs/sdk/py/latest/Sandbox.md)
  - [`SandboxSnapshot`](https://modal.com/docs/sdk/py/latest/SandboxSnapshot.md)
  - [`Secret`](https://modal.com/docs/sdk/py/latest/Secret.md)
  - [`Server`](https://modal.com/docs/sdk/py/latest/Server.md)
  - [`Tunnel`](https://modal.com/docs/sdk/py/latest/Tunnel.md)
  - [`Volume`](https://modal.com/docs/sdk/py/latest/Volume.md)
  - [`Workspace`](https://modal.com/docs/sdk/py/latest/Workspace.md)
  - [`asgi_app`](https://modal.com/docs/sdk/py/latest/asgi_app.md)
  - [`batched`](https://modal.com/docs/sdk/py/latest/batched.md)
  - [`billing`](https://modal.com/docs/sdk/py/latest/billing.md)
  - [`clustered`](https://modal.com/docs/sdk/py/latest/clustered.md)
  - [`concurrent`](https://modal.com/docs/sdk/py/latest/concurrent.md)
  - [`config`](https://modal.com/docs/sdk/py/latest/config.md)
  - [`container_process`](https://modal.com/docs/sdk/py/latest/container_process.md)
  - [`current_function_call_id`](https://modal.com/docs/sdk/py/latest/current_function_call_id.md)
  - [`cur