# Together AI docs

> Documentation for the Together AI platform for inference and training.

- [Overview](https://docs.together.ai/intro.md): Run, train, and serve open-source AI models on Together AI.
- [Quickstart](https://docs.together.ai/docs/quickstart.md): Make your first request to Together AI in a few minutes.
- [Coding agent setup](https://docs.together.ai/docs/agent-skills.md): Make your AI coding agent Together-AI-aware with a ready-made skill for code generation and an MCP server for live docs lookup.
- [Configure Claude Code, Codex, OpenCode & Pi Code with OSS models](https://docs.together.ai/docs/togetherlink.md): Use Together Link to run Claude Code, Codex, OpenCode, Pi Code, Claude Desktop and Cowork, and ChatGPT Desktop with models hosted by Together AI.
- [Recommended models](https://docs.together.ai/docs/inference/recommended-models.md): Our picks for common inference use cases.
- [Overview](https://docs.together.ai/docs/serverless/overview.md): Call 100+ open-source models with per-token pricing and no provisioning latency.
- [Available models](https://docs.together.ai/docs/serverless/models.md): Browse the catalog of available models for instant inference.
- [Rate limits](https://docs.together.ai/docs/serverless/rate-limits.md): Understand serverless performance, how to handle rate limits during high demand, and options for throughput guarantees.
- [Provisioned throughput](https://docs.together.ai/docs/inference/provisioned-throughput.md): Reserved inference capacity for production workloads.
- [Overview](https://docs.together.ai/docs/inference/overview.md): Run inference on 100+ open-source models.
- [OpenAI compatibility](https://docs.together.ai/docs/inference/openai-compatibility.md): Point your OpenAI Python or TypeScript client at Together AI to call open-source models without rewriting your app.
- [Third-party integrations](https://docs.together.ai/docs/inference/sdk-integrations.md): Use Together AI models through partner SDKs and integrations.
- [Send chat completions](https://docs.together.ai/docs/inference/chat/overview.md): Query chat models with single prompts, multi-turn conversations, and system prompts.
- [Parameters](https://docs.together.ai/docs/inference/chat/parameters.md): The full list of parameters you can pass to the chat completions endpoint.
- [Structured outputs](https://docs.together.ai/docs/inference/chat/structured-outputs.md): Use JSON mode to get structured outputs from supported chat models.
- [Reasoning](https://docs.together.ai/docs/inference/chat/reasoning.md): Use reasoning models that think step-by-step before answering.
- [Prompt caching](https://docs.together.ai/docs/inference/chat/prompt-caching.md): Together reuses previously processed prompt prefixes to lower input cost and time to first token.
- [Log probabilities](https://docs.together.ai/docs/inference/chat/logprobs.md): Return per-token log probabilities to measure model confidence and route low-confidence outputs to a stronger model.
- [Function calling patterns](https://docs.together.ai/docs/inference/function-calling/overview.md): Function calling lets LLMs respond with structured function names and arguments your application can execute.
- [Call functions](https://docs.together.ai/docs/inference/function-calling/single-call.md): One tool call per response, covering simple and multiple-tool patterns.
- [Call functions in parallel](https://docs.together.ai/docs/inference/function-calling/parallel.md): Multiple tool calls in one response, covering parallel calls to the same tool and to different tools.
- [Agentic function calling patterns](https://docs.together.ai/docs/inference/function-calling/agentic.md): Tool use across multiple steps or conversation turns, covering multi-step and multi-turn agent loops.
- [Function calling best practices](https://docs.together.ai/docs/inference/function-calling/best-practices.md): Design tools, write descriptions, and control tool selection so Together models call functions reliably.
- [Text-to-image generation](https://docs.together.ai/docs/inference/images/overview.md): Generate images from text prompts.
- [Image-to-image generation](https://docs.together.ai/docs/inference/images/reference-images.md): Edit or transform an existing image by passing image_url (Kontext) or reference_images (FLUX.2 and Google models).
- [Image generation parameters](https://docs.together.ai/docs/inference/images/parameters.md): Parameter reference for the images API: dimensions, quality control, base64 responses, safety checker, and troubleshooting.
- [Generate videos](https://docs.together.ai/docs/inference/videos/overview.md): Generate videos from text and image prompts.
- [Reference images and keyframes](https://docs.together.ai/docs/inference/videos/reference-and-keyframes.md): Guide visual style with reference images and control specific frames in your video.
- [Audio input for videos](https://docs.together.ai/docs/inference/videos/audio-input.md): Drive video generation with an audio file for lip sync, beat-matched motion, or narration.
- [Video generation parameters](https://docs.together.ai/docs/inference/videos/parameters.md): Reference for video generation parameters, including guidance scale and quality control.
- [Use image inputs](https://docs.together.ai/docs/inference/vision/overview.md): Run vision-language models on Together: pass images alongside text and get structured replies, transcripts, comparisons, or extracted data.
- [Vision input modes](https://docs.together.ai/docs/inference/vision/inputs.md): Send local images, video URLs, or multiple images to a vision model in a single request.
- [Structured extraction with vision](https://docs.together.ai/docs/inference/vision/structured-extraction.md): Combine image input with a JSON schema to extract typed data from screenshots, documents, and photos, or run OCR to pull text out of images.
- [Vision-language function calling](https://docs.together.ai/docs/inference/vision/function-calling.md): Combine image understanding with tool use on Together AI vision-language models.
- [Transcribe audio](https://docs.together.ai/docs/inference/transcription/overview.md): Transcribe and translate audio into text.
- [Streaming transcription](https://docs.together.ai/docs/inference/transcription/streaming.md): Use the real-time WebSocket API for low-latency, incremental speech-to-text.
- [Audio translation](https://docs.together.ai/docs/inference/transcription/translation.md): Translate speech in any language into English text.
- [Voice activity detection](https://docs.together.ai/docs/inference/transcription/voice-activity-detection.md): Configure voice activity detection to control how speech segments are detected in real-time transcription.
- [Advanced transcription options](https://docs.together.ai/docs/inference/transcription/features.md): Speaker diarization, word-level timestamps, response formats, async support, and best practices.
- [Generate speech](https://docs.together.ai/docs/inference/text-to-speech/overview.md): Generate speech audio from text with Together AI text-to-speech models.
- [Text-to-speech streaming](https://docs.together.ai/docs/inference/text-to-speech/streaming.md): Stream audio over HTTP for low time-to-first-byte and access raw PCM bytes.
- [WebSocket API](https://docs.together.ai/docs/inference/text-to-speech/websocket.md): Stream text in and audio out over a single WebSocket connection for the lowest interactive latency.
- [Rerank](https://docs.together.ai/docs/inference/embeddings/rerank.md): Reorder retrieved documents by relevance to a query for sharper search and RAG results.
- [Evaluations](https://docs.together.ai/docs/ai-evaluations.md): Use LLMs to classify, score, and compare model outputs on Together AI.
- [Run an evaluation](https://docs.together.ai/docs/run-an-evaluation.md): Prepare a dataset, launch an evaluation job, and download results with the Together CLI or API.
- [Parameters and result formats](https://docs.together.ai/docs/evaluations-reference.md): Parameters, result formats, and template syntax for the evaluations API.
- [Supported models](https://docs.together.ai/docs/evaluations-supported-models.md): Serverless models and external provider shortcuts supported by the evaluations API.
- [Overview](https://docs.together.ai/docs/inference/batch/overview.md): Run asynchronous batch workloads at up to 50% lower cost.
- [Run a batch job](https://docs.together.ai/docs/inference/batch/tutorial.md): Prepare a JSONL file, upload it, start a batch job, poll until it finishes, and retrieve results.
- [Manage batch jobs](https://docs.together.ai/docs/inference/batch/manage.md): Status, results, errors, and operational reference for the Together Batch API.
- [Overview](https://docs.together.ai/docs/dedicated-endpoints/overview.md): Deploy a model for inference on dedicated GPUs.
- [Quickstart](https://docs.together.ai/docs/dedicated-endpoints/quickstart.md): Deploy a model on dedicated hardware in a few minutes.
- [Concepts](https://docs.together.ai/docs/dedicated-endpoints/concepts.md): Understand the resource model and development workflow for dedicated model inference.
- [Pricing](https://docs.together.ai/docs/dedicated-endpoints/pricing.md): Billing and pricing details for dedicated model inference.
- [Supported models](https://docs.together.ai/docs/dedicated-endpoints/models.md): View the supported models you can deploy or fine-tune for dedicated model inference.
- [Upload a fine-tuned model](https://docs.together.ai/docs/dedicated-endpoints/custom-models.md): Serve a fine-tuned model uploaded from your machine, Hugging Face, or S3.
- [Upload a LoRA adapter](https://docs.together.ai/docs/dedicated-endpoints/adapter.md): Serve a custom LoRA adapter uploaded from Hugging Face or S3.
- [Choose a deployment profile](https://docs.together.ai/docs/dedicated-endpoints/configs.md): Pick the hardware deployment profile that your model runs on.
- [Manage endpoints and deployments](https://docs.together.ai/docs/dedicated-endpoints/manage.md): Create, update, and delete resources for dedicated model inference.
- [Configure autoscaling](https://docs.together.ai/docs/dedicated-endpoints/scaling.md): Autoscale a deployment between replica bounds, pick the right scaling metric, and stop idle deployments automatically.
- [Migrate from v1](https://docs.together.ai/docs/dedicated-endpoints/migrate-from-v1.md): Move a dedicated endpoint from the v1 API to the v2 dedicated model inference resource model.
- [Overview](https://docs.together.ai/docs/dedicated-endpoints/route-traffic.md): Route traffic to a deployment, and explore strategies that change how traffic moves over time.
- [Split traffic across deployments](https://docs.together.ai/docs/dedicated-endpoints/split-traffic.md): Run multiple deployments on one endpoint and split requests between them by weight.
- [Start a rollout](https://docs.together.ai/docs/dedicated-endpoints/rollouts.md): Shift traffic to a new deployment on the same endpoint.
- [Gate rollouts with metrics](https://docs.together.ai/docs/dedicated-endpoints/rollout-metric-gates.md): Pause a canary rollout automatically when the new deployment regresses.
- [Run an A/B test](https://docs.together.ai/docs/dedicated-endpoints/ab-tests.md): Compare a candidate deployment against a baseline on live traffic.
- [Run a shadow experiment](https://docs.together.ai/docs/dedicated-endpoints/shadow-experiments.md): Mirror a sampled fraction of endpoint traffic to a target deployment without affecting the client response.
- [Send requests](https://docs.together.ai/docs/dedicated-endpoints/requests.md): After deploying a model, send requests using the shared inference API.
- [Monitor endpoints and deployments](https://docs.together.ai/docs/dedicated-endpoints/monitoring.md): Monitor endpoint and deployment metrics with built-in dashboards and a Prometheus-compatible metrics endpoint.
- [Visualize endpoint metrics in Grafana](https://docs.together.ai/docs/dedicated-endpoints/grafana.md): Scrape the metrics endpoint with Prometheus and import the example Grafana dashboard for dedicated endpoints.
- [Overview](https://docs.together.ai/docs/dedicated-container-inference.md): Deploy your own containers on Together's managed GPU infrastructure with automatic scaling, job queues, and built-in observability.
- [Quickstart](https://docs.together.ai/docs/containers-quickstart.md): Deploy your first container in 20 minutes.
- [Image generation with Flux2](https://docs.together.ai/docs/dedicated-containers-image.md): Deploy a Flux2 image generation model on Together's managed GPU infrastructure using dedicated containers.
- [Video generation with Wan 2.1](https://docs.together.ai/docs/dedicated-containers-video.md): Deploy a multi-GPU video generation model on Together's managed GPU infrastructure using dedicated containers.
- [Serve an OpenAI-compatible endpoint](https://docs.together.ai/docs/dedicated-containers-openai.md): Deploy a custom model behind an OpenAI-compatible endpoint using dedicated containers.
- [Architecture](https://docs.together.ai/docs/together-deployments.md): Architecture, deployment lifecycle, and core concepts for dedicated container inference.
- [Jig CLI](https://docs.together.ai/docs/deployments-jig.md): Build, push, and deploy containers to Together's managed GPU infrastructure.
- [Sprocket SDK](https://docs.together.ai/docs/deployments-sprocket.md): A Python SDK for building inference workers that support both synchronous and asynchronous requests via Together's platform.
- [Queue API](https://docs.together.ai/docs/deployments-queue.md): Submit, monitor, and manage asynchronous jobs for your dedicated container inference deployments.
- [Sprocket SDK reference](https://docs.together.ai/reference/dci-reference-sprocket.md): API reference for Sprocket classes, functions, and configuration.
- [Overview](https://docs.together.ai/docs/post-training/overview.md): Train models from your own training loop on Together AI GPUs.
- [Supervised fine-tuning quickstart](https://docs.together.ai/docs/post-training/quickstart-sft.md): Run your first post-training session with supervised fine-tuning on instruction-following data, then download the trained weights.
- [Reinforcement learning with GRPO quickstart](https://docs.together.ai/docs/post-training/quickstart-rl.md): Train a model to solve grade-school math with GRPO, monitor the run, and retrieve the trained model.
- [Concepts](https://docs.together.ai/docs/post-training/concepts.md): See how a training run divides work between your code and a model resource on Together AI GPUs, how resources, sessions, and checkpoints relate, and how you deploy or download a trained model.
- [Pricing](https://docs.together.ai/docs/post-training/pricing.md): How post-training bills for GPU time, how to estimate the cost of a resource or a training run, and how refunds and the minimum credit balance work.
- [Supported models](https://docs.together.ai/docs/post-training/supported-models.md): Every base model available for post-training, with training modes, context lengths, GPU types, and adapter capacity.
- [Overview](https://docs.together.ai/docs/post-training/training-methods.md): Match a post-training method to your data, choose between adapter and full-weight training, and combine methods.
- [Supervised fine-tuning](https://docs.together.ai/docs/post-training/sft.md): Train a model on example conversations with supervised fine-tuning.
- [Direct preference optimization](https://docs.together.ai/docs/post-training/dpo.md): Train on preference pairs with a custom loss function that runs in your own process.
- [Reinforcement learning with GRPO](https://docs.together.ai/docs/post-training/grpo.md): Train a model against a reward function with group relative policy optimization.
- [Write a reward function](https://docs.together.ai/docs/post-training/reward-functions.md): How to write a reward function for GRPO, register it with your dataset, and test it before you start training.
- [Train in multi-turn environments](https://docs.together.ai/docs/post-training/multi-turn-environments.md): Train tool-use agents with GRPO over whole episodes, where the model acts, an environment responds, and a reward scores the outcome.
- [Write a custom loss function](https://docs.together.ai/docs/post-training/custom-loss-functions.md): Train with any objective you compute in PyTorch from the model's log-probabilities, for SFT, DPO, or RL.
- [Manage model resources](https://docs.together.ai/docs/post-training/model-resources.md): Configure, create, attach to, inspect, and stop the GPU resource that post-training sessions run on, and learn its lifecycle states, timers, error codes, and limits.
- [Manage checkpoints](https://docs.together.ai/docs/post-training/checkpoints.md): Save inference and training checkpoints, download trained weights, resume or fork a run, and find a session's checkpoints after it ends.
- [Train multiple adapters on one base model](https://docs.together.ai/docs/post-training/multi-lora.md): Share one provisioned model resource across several adapter sessions to run hyperparameter sweeps at the cost of a single base model.
- [Monitor training runs](https://docs.together.ai/docs/post-training/monitoring.md): Track sessions and model resources in the console, and read the run artifacts and Weights & Biases metrics that show whether a training run is healthy.
- [Cookbook reference](https://docs.together.ai/docs/post-training/cookbook.md): The post-training cookbook's runnable recipes, and every field on its GRPO, SFT, and DPO config classes.
- [Migrate from Tinker](https://docs.together.ai/docs/post-training/migrate-from-tinker.md): Run a Tinker training script on Together AI post-training by changing its import line.
- [Deploy a checkpoint](https://docs.together.ai/docs/post-training/deployment.md): Deploy a trained model or LoRA adapter from an inference checkpoint to dedicated model inference.
- [Multi-LoRA deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md): Attach the adapters from post-training inference checkpoints to one dedicated deployment of their base model and choose an adapter per request.
- [Prepare for FP4 inference](https://docs.together.ai/docs/post-training/prepare-for-fp4-inference.md): Merge a trained LoRA adapter into its base model and quantize the result to FP4 for dedicated model inference.
- [Overview](https://docs.together.ai/docs/fine-tuning/overview.md): Adapt a base model to a task by training it on your data.
- [Fine-tuning quickstart](https://docs.together.ai/docs/fine-tuning/quickstart.md): Prepare a conversational dataset, launch a LoRA job on Qwen3.5 9B, and evaluate the fine-tuned model.
- [Pricing](https://docs.together.ai/docs/fine-tuning/pricing.md): Fine-tuning is billed per token processed, scaled by model size, training method, and training type.
- [Data preparation](https://docs.together.ai/docs/fine-tuning/data-preparation.md): Format your training file as JSONL or Parquet to match your task, then validate and upload.
- [Supported models](https://docs.together.ai/docs/fine-tuning/supported-models.md): Every base model available for fine-tuning, with context length and batch size limits.
- [Bring your own model](https://docs.together.ai/docs/fine-tuning/byom.md): Fine-tune a Hugging Face model that isn't in the Together catalog.
- [Supervised fine-tuning](https://docs.together.ai/docs/fine-tuning/supervised.md): Train a model on demonstration data with supervised fine-tuning (SFT).
- [Preference fine-tuning](https://docs.together.ai/docs/fine-tuning/preference-tuning.md): Align a model with paired preferred and dispreferred responses using DPO.
- [LoRA vs. full fine-tuning](https://docs.together.ai/docs/fine-tuning/lora-vs-full.md): Choose between LoRA and full fine-tuning, then tune LoRA's rank and target modules.
- [Vision fine-tuning](https://docs.together.ai/docs/fine-tuning/vision.md): Fine-tune vision-language models on image and text data with Together AI.
- [Function-calling fine-tuning](https://docs.together.ai/docs/fine-tuning/function-calling.md): Train a model to invoke tools and structured functions reliably.
- [Reasoning fine-tuning](https://docs.together.ai/docs/fine-tuning/reasoning.md): Train a reasoning model on chain-of-thought data.
- [Monitor a fine-tuning job](https://docs.together.ai/docs/fine-tuning/monitoring.md): Poll job status, retrieve per-step loss and evaluation metrics, and stream training progress to Weights & Biases.
- [Early stopping](https://docs.together.ai/docs/fine-tuning/early-stopping.md): Halt a fine-tuning job when validation loss stops improving.
- [Troubleshooting fine-tuning jobs](https://docs.together.ai/docs/fine-tuning/troubleshooting.md): Diagnose failed or cancelled fine-tuning jobs, understand job timing, and resolve common errors.
- [Deploy a fine-tuned model](https://docs.together.ai/docs/fine-tuning/deployment.md): Serve your fine-tuned model on a dedicated endpoint or download it for local inference.
- [Overview](https://docs.together.ai/docs/gpu-clusters-overview.md): High-performance GPU clusters for training, fine-tuning, and large-scale AI workloads.
- [Quickstart](https://docs.together.ai/docs/gpu-clusters-quickstart.md): Create a reserved or on-demand GPU cluster and connect to it.
- [Cluster management](https://docs.together.ai/docs/gpu-clusters-management.md): Manage, scale, and operate your GPU clusters.
- [Preemptible compute](https://docs.together.ai/docs/preemptible-compute.md): Run interruptible workloads on discounted preemptible GPU nodes in your cluster.
- [Health checks](https://docs.together.ai/docs/health-checks.md): Monitor GPU node health with active diagnostic tests and continuous passive monitoring.
- [Node repair](https://docs.together.ai/docs/node-repair.md): Restore unhealthy GPU nodes through automated recommendations or manual repair actions.
- [Cluster storage](https://docs.together.ai/docs/cluster-storage.md): Understand storage types, persistence, and best practices for GPU clusters.
- [Billing & pricing](https://docs.together.ai/docs/gpu-clusters-billing.md): Understand billing, pricing, and lifecycle policies for GPU clusters.
- [Set up OIDC authentication](https://docs.together.ai/docs/cluster-oidc.md): Authenticate team members to a GPU cluster's Kubernetes API using your organization's identity provider.
- [Slurm management system](https://docs.together.ai/docs/slurm.md): Run HPC-style batch workloads on GPU clusters with Slurm job scheduling, partitions, and job arrays.
- [Slurm configuration](https://docs.together.ai/docs/slurm-configuration.md): Customize Slurm cluster settings to match your workload requirements.
- [Slurm startup scripts](https://docs.together.ai/docs/slurm-startup-scripts.md): Configure lifecycle hook scripts that run automatically at node startup, job start, and job completion.
- [Run nanochat on GPU clusters](https://docs.together.ai/docs/nanochat-on-gpu-clusters.md): Train Andrej Karpathy's end-to-end ChatGPT clone on Together's on-demand GPU clusters.
- [Gang-schedule GPU jobs with Volcano](https://docs.together.ai/docs/volcano-on-gpu-clusters.md): Install the Volcano scheduler and run gang-scheduled GPU jobs on a Together Kubernetes cluster.
- [Queue GPU jobs with Kueue](https://docs.together.ai/docs/kueue-on-gpu-clusters.md): Install Kueue and gate GPU jobs on quota so a shared cluster admits work as capacity frees up.
- [API & integrations](https://docs.together.ai/docs/gpu-clusters-api.md): Manage clusters programmatically with the Together CLI, REST API, and SkyPilot.
- [Code sandbox](https://docs.together.ai/docs/together-code-sandbox.md): Run commands and code in isolated runtime environments built from Docker-image snapshots.
- [Code sandbox legacy SDK](https://docs.together.ai/docs/together-code-sandbox-legacy.md): Reference for the earlier code sandbox offering built on the CodeSandbox SDK.
- [Manage your account](https://docs.together.ai/docs/account-management.md): Sign up for Together AI, get your API key, and manage your account settings.
- [Authentication](https://docs.together.ai/docs/api-keys-authentication.md): Create, manage, and authenticate with project-scoped API keys.
- [IAM model](https://docs.together.ai/docs/identity-access-management.md): How users, credentials, and resources are organized across the Together platform.
- [Organizations](https://docs.together.ai/docs/organizations.md): Create and manage your Together organization, invite members, and configure billing.
- [Projects](https://docs.together.ai/docs/projects.md): Create isolated workspaces to organize resources, manage team access, and scope API keys.
- [Roles & permissions (RBAC)](https://docs.together.ai/docs/roles-permissions.md): Understand organization and project role-based access control (RBAC), including the admin, developer, and editor roles, and what each can do across Together.
- [Single sign-on (SSO)](https://docs.together.ai/docs/sso.md): Connect your Identity Provider for secure, automated team access to Together.
- [Privacy and security](https://docs.together.ai/docs/privacy-and-security.md): How Together handles your inputs, outputs, and account data, plus enterprise options for data residency and private networking.
- [Zero data retention](https://docs.together.ai/docs/zero-data-retention.md): Together's zero data retention policy and how to enable it.
- [Credits](https://docs.together.ai/docs/billing-credits.md): Understanding credits and billing basics on Together AI.
- [Payment methods & invoices](https://docs.together.ai/docs/billing-payment-methods.md): Managing payment cards, ACH transfers, viewing invoices, and updating billing details.
- [Spend controls](https://docs.together.ai/docs/billing-spend-controls.md): Manage Together AI spend with balance cut-offs, auto-recharge, cost attribution, and programmatic monitoring.
- [Cost analytics](https://docs.together.ai/docs/billing-cost-analytics.md): Track daily Together AI spend and billable units by product, line item, project, or API key.
- [Billing troubleshooting](https://docs.together.ai/docs/billing-troubleshooting.md): Resolving payment issues, understanding charges, and managing billing problems.
- [Support](https://docs.together.ai/docs/support.md): Search the support portal, file a ticket, or reach the Together AI team by email, Slack, or Discord.
- [Overview](https://docs.together.ai/docs/dedicated-endpoints/v1/overview.md): Reserved-hardware inference endpoints with predictable performance, no shared rate limits, and per-endpoint configuration.
- [Quickstart](https://docs.together.ai/docs/dedicated-endpoints/v1/quickstart.md): Pick a model, deploy a dedicated endpoint with one CLI command, and send your first request in under five minutes.
- [Available models](https://docs.together.ai/docs/dedicated-endpoints/v1/models.md): View the models you can deploy to dedicated endpoints.
- [Manage dedicated endpoints](https://docs.together.ai/docs/dedicated-endpoints/v1/manage.md): Create, start, stop, restart, list, update, and delete dedicated endpoints via the web UI or the Together API.
- [Endpoint settings](https://docs.together.ai/docs/dedicated-endpoints/v1/settings.md): Configure replica count, hardware, decoding optimizations, and prompt caching on a dedicated endpoint.
- [Scaling](https://docs.together.ai/docs/dedicated-endpoints/v1/scaling.md): How dedicated endpoints scale, how that affects cost, and when to choose vertical vs. horizontal scaling.
- [Upload a model](https://docs.together.ai/docs/dedicated-endpoints/v1/custom-models.md): Upload a custom or fine-tuned model from Hugging Face or S3 and serve it on a dedicated endpoint.
- [Upload a LoRA adapter](https://docs.together.ai/docs/dedicated-endpoints/v1/adapter.md): Upload a custom LoRA adapter from Hugging Face or S3 and serve it on a dedicated endpoint.
- [Serve multiple LoRA adapters on one endpoint](https://docs.together.ai/docs/dedicated-endpoints/v1/lora-adapter.md): Attach, list, and remove LoRA adapters on a running dedicated endpoint to serve several adapters from one deployment.
- [Overview](https://docs.together.ai/docs/guides.md): Quickstarts and step-by-step guides for building with Together AI.
- [Kimi K3 quickstart](https://docs.together.ai/docs/kimi-k3-quickstart.md): Call Kimi K3 on Together for long-horizon coding, vision-in-the-loop work, and deep reasoning.
- [GLM-5.3 quickstart](https://docs.together.ai/docs/glm-5.3-quickstart.md): Call GLM-5.3 and GLM-5.3 Flash on Together for long-horizon coding and agentic work.
- [DeepSeek V4 Pro quickstart](https://docs.together.ai/docs/deepseek-v4-quickstart.md): Call DeepSeek V4 Pro on Together for hybrid reasoning, long-context, and tool-using workloads.
- [GPT-OSS quickstart](https://docs.together.ai/docs/gpt-oss.md): Call OpenAI's GPT-OSS 120B open-weight reasoning model on Together.
- [FLUX quickstart](https://docs.together.ai/docs/quickstart-flux.md): Generate and edit images with the FLUX model family, including FLUX.2 and FLUX.1 Kontext.
- [Wan 2.7 quickstart](https://docs.together.ai/docs/wan2.7-quickstart.md): Generate videos from text, images, and reference materials with the Wan 2.7 model family.
- [Seedance 2.5 quickstart](https://docs.together.ai/docs/seedance2.5-quickstart.md): Generate multi-shot videos with synchronized audio from text, image, video, and audio inputs.
- [Build a phone voice agent with Together AI](https://docs.together.ai/docs/how-to-build-phone-voice-agent.md): Create a real-time phone voice agent from scratch with Twilio Media Streams, Together AI realtime STT, chat completions, realtime TTS, and local voice activity detection.
- [Build a Lovable clone with Kimi K3](https://docs.together.ai/docs/how-to-build-a-lovable-clone-with-kimi-k2.md): Build a full-stack Next.js app that generates React apps from a single prompt.
- [Build a resume-to-website app with structured outputs](https://docs.together.ai/docs/pdf-to-website-with-structured-outputs.md): Build an AI-powered site builder that turns PDF resumes into personal websites with structured outputs.
- [Build a real-time image generator with Flux](https://docs.together.ai/docs/how-to-build-a-real-time-image-generator.md): Create a real-time text-to-image app with Juggernaut Lightning Flux, Next.js, and Together AI.
- [Build an open source NotebookLM](https://docs.together.ai/docs/open-notebooklm-pdf-to-podcast.md): Build an open source NotebookLM that turns a PDF into a podcast.
- [Build an interactive AI tutor](https://docs.together.ai/docs/ai-tutor.md): Create LlamaTutor from scratch, an open source AI tutor with 90k users.
- [Build an audio transcription app with Whisper](https://docs.together.ai/docs/how-to-build-real-time-audio-transcription-app.md): Build a real-time AI audio transcription app with Whisper, Next.js, and Together AI.
- [Build a chat API on Render](https://docs.together.ai/docs/render-chat-api.md): Deploy an authenticated single-turn chat API backed by Together AI to a Render web service.
- [Build an AI search engine](https://docs.together.ai/docs/ai-search-engine.md): Build an open source AI search engine inspired by Perplexity with Next.js and Together AI.
- [Build a coding agent](https://docs.together.ai/docs/how-to-build-coding-agents.md): Build a simple code editing agent from scratch in 400 lines of code.
- [Agent workflows](https://docs.together.ai/docs/workflows.md): Orchestrate multiple language model calls to solve complex tasks.
- [Sequential workflow](https://docs.together.ai/docs/sequential-agent-workflow.md): Coordinate a chain of LLM calls to solve a complex task.
- [Parallel workflow](https://docs.together.ai/docs/parallel-workflows.md): Execute multiple LLM calls in parallel and aggregate afterwards.
- [Conditional workflow](https://docs.together.ai/docs/conditional-workflows.md): Adapt to different tasks by conditionally navigating to various LLMs and tools.
- [Iterative workflow](https://docs.together.ai/docs/iterative-workflow.md): Iteratively call LLMs to optimize task performance.
- [Together mixture of agents (MoA)](https://docs.together.ai/docs/mixture-of-agents.md): Combine responses from multiple open-source LLMs to produce higher-quality outputs with mixture of agents (MoA).
- [Agent integrations](https://docs.together.ai/docs/agent-integrations.md): Use OSS agent frameworks with Together AI.
- [CrewAI](https://docs.together.ai/docs/crewai.md): Use CrewAI with Together AI models to orchestrate AI agent systems.
- [LangGraph](https://docs.together.ai/docs/langgraph.md): Use LangGraph with Together AI to build agent workflows.
- [DSPy](https://docs.together.ai/docs/dspy.md): Use DSPy to program language models with Together AI.
- [PydanticAI](https://docs.together.ai/docs/pydanticai.md): Use PydanticAI with Together AI.
- [Agno](https://docs.together.ai/docs/agno.md): Use Agno with Together AI to build multimodal agents.
- [AutoGen(AG2)](https://docs.together.ai/docs/autogen.md): Use AutoGen (AG2) to build and orchestrate AI agents with Together AI models.
- [Composio](https://docs.together.ai/docs/composio.md): Use Composio with Together AI.
- [Power Claude Code with LiteLLM and Together AI](https://docs.together.ai/docs/using-together-with-litellm.md): Run Claude Code on Together coding models through a local LiteLLM proxy.
- [Configure Kimi Code with Together AI models](https://docs.together.ai/docs/how-to-use-kimi-code.md): Power Moonshot AI's coding agent with Together AI-hosted models.
- [Configure Cline with Together AI models](https://docs.together.ai/docs/how-to-use-cline.md): Power Cline, an AI coding agent, with Together AI models.
- [Configure OpenCode with Together AI models](https://docs.together.ai/docs/how-to-use-opencode.md): Power OpenCode, a terminal-based AI coding agent, with Together AI models.
- [Configure Qwen Code with Together AI models](https://docs.together.ai/docs/how-to-use-qwen-code.md): Power Qwen Code with Together AI models.
- [Configure OpenClaw with Together AI models](https://docs.together.ai/docs/how-to-use-openclaw.md): Power OpenClaw, an autonomous agent, with Together AI models.
- [Hugging Face inference quickstart](https://docs.together.ai/docs/quickstart-using-hugging-face-inference.md): Call Together models from Hugging Face Inference clients.
- [Vercel AI SDK quickstart](https://docs.together.ai/docs/using-together-with-vercels-ai-sdk.md): Call Together models from TypeScript, React, or Next.js with the Vercel AI SDK.
- [Mastra quickstart](https://docs.together.ai/docs/using-together-with-mastra.md): Install Mastra, point an agent at Together AI, and generate a response from a serverless model.
- [Next.js quickstart](https://docs.together.ai/docs/nextjs-chat-quickstart.md): Build an app that can ask a single question or chat with an LLM using Next.js and Together AI.
- [Introduction](https://docs.together.ai/learn/index.md): Explore the fundamental concepts of the Together AI platform: Tokens, context windows, when to use serverless vs. dedicated inference, and how the stack works in practice.
- [How LLMs work](https://docs.together.ai/learn/how-llms-work.md): How a large language model produces text from a prompt, one token at a time.
- [Tokens & tokenization](https://docs.together.ai/learn/tokens-and-tokenization.md): How tokenization works, what the model reads, and why tokens drive cost and context usage.
- [Context windows](https://docs.together.ai/learn/context-windows.md): The model's working memory. A hard limit on the way in, a soft constraint inside it.
- [Inference parameters & sampling](https://docs.together.ai/learn/inference-parameters-and-sampling.md): Temperature, top-k, top-p, and the other knobs that shape how a model picks each next token.
- [Context engineering](https://docs.together.ai/learn/prompt-engineering.md): How to structure context for chat, RAG, and coding agents.
- [Inference metrics: TTFT & TPS](https://docs.together.ai/learn/ttft-and-tps.md): Two numbers that describe how fast an LLM feels.
- [Function calling & tool use](https://docs.together.ai/learn/function-calling-and-tool-use.md): The model plans, your code runs the tool. How structured tool calls and agent loops actually work.
- [Structured outputs & JSON mode](https://docs.together.ai/learn/structured-outputs.md): Force the model to produce schema-valid output.
- [When to fine-tune vs. prompt](https://docs.together.ai/learn/finetune-vs-prompt.md): Prompting steers an existing model. Fine-tuning changes its weights. You should almost always try prompting first.
- [Choosing a deployment option](https://docs.together.ai/learn/choosing-a-deployment-option.md): Deploy your model with serverless, dedicated endpoints, or dedicated containers.
- [Quantization](https://docs.together.ai/learn/quantization.md): What quantization is, how lower precision speeds up inference, and the quality tradeoff.
- [Together Cookbooks & Example Apps](https://docs.together.ai/examples.md): Explore our vast library of open-source cookbooks & example apps
- [Create chat completion](https://docs.together.ai/reference/chat-completions.md): Generate a model response for a given chat conversation. Supports single queries and multi-turn conversations with system, user, and assistant messages.
- [Create image](https://docs.together.ai/reference/post-images-generations.md): Use an image model to generate an image for a given prompt.
- [Create audio generation request](https://docs.together.ai/reference/audio-speech.md): Generate audio from input text
- [Create realtime text-to-speech](https://docs.together.ai/reference/audio-speech-websocket.md): Establishes a WebSocket connection for real-time text-to-speech generation. This endpoint uses WebSocket protocol (wss://api.together.ai/v1/audio/speech/websocket) for bidirectional streaming communication.
- [List voices](https://docs.together.ai/reference/voices-list.md): Fetch available voices for each model
- [Create audio transcription request](https://docs.together.ai/reference/audio-transcriptions.md): Transcribes audio into text
- [Create audio translation request](https://docs.together.ai/reference/audio-translations.md): Translates audio into English
- [Real-time audio transcription via WebSocket](https://docs.together.ai/reference/audio-transcriptions-realtime.md): Establishes a WebSocket connection for real-time audio transcription. This endpoint uses WebSocket protocol (wss://api.together.ai/v1/realtime) for bidirectional streaming communication.
- [Create video](https://docs.together.ai/reference/create-videos.md): Create a video
- [Fetch video metadata](https://docs.together.ai/reference/get-videos-id.md): Fetch video metadata
- [Create a rerank request](https://docs.together.ai/reference/rerank.md): Rerank a list of documents by relevance to a query. Returns a relevance score and ordering index for each document.
- [Create completion](https://docs.together.ai/reference/completions.md): Generate text completions for a given prompt using a language, code, or image model.
- [List batch jobs](https://docs.together.ai/reference/batch-list.md): List all batch jobs for the authenticated user
- [Create a batch job](https://docs.together.ai/reference/batch-create.md): Create a new batch job with the given input file and endpoint
- [Get a batch job](https://docs.together.ai/reference/batch-get.md): Get details of a batch job by ID
- [Cancel a batch job](https://docs.together.ai/reference/batch-cancel.md): Cancel a batch job by ID
- [List supported models](https://docs.together.ai/reference/post-training/supported-models-list.md): Returns the models supported by the RL service and their limits for training/sampling operations.
- [List model resources](https://docs.together.ai/reference/post-training/model-resources-list.md): Lists the caller's model resources.
- [Create model resources](https://docs.together.ai/reference/post-training/model-resources-create.md): Provisions a standalone model resource that training sessions can attach to.
- [Estimate on-demand model resource cost](https://docs.together.ai/reference/post-training/model-resources-estimate-cost.md): Estimates a model resource's on-demand hourly price without creating it.
- [Get model resources](https://docs.together.ai/reference/post-training/model-resources-get.md): Gets a model resource by its ID and returns its details.
- [Stop model resources](https://docs.together.ai/reference/post-training/model-resources-stop.md): Stops the model resource and releases its allocated compute. If active training sessions are attached, the request fails unless `force=true`. A forced stop also stops all attached training sessions.
- [List training sessions](https://docs.together.ai/reference/post-training/training-sessions-list.md): Lists all training sessions.
- [Create training session](https://docs.together.ai/reference/post-training/training-sessions-create.md): Creates a training session and returns its details.
- [Get training session](https://docs.together.ai/reference/post-training/training-sessions-get.md): Gets a training session by its ID and returns its details.
- [Update a training session](https://docs.together.ai/reference/post-training/training-sessions-update.md): Updates the display name or metadata associated with a training session. Omitted fields remain unchanged, and empty strings clear existing values.
- [Stop training session](https://docs.together.ai/reference/post-training/training-sessions-stop.md): Stops a training session.
- [Forward-backward pass](https://docs.together.ai/reference/post-training/operations-forward-backward-create.md): Submits a forward-backward pass operation that will asynchronously compute gradients via backpropagation.
- [Get forward-backward operation](https://docs.together.ai/reference/post-training/operations-forward-backward-get.md): Retrieves the current status and result of a forward-backward operation.
- [Custom forward-backward pass](https://docs.together.ai/reference/post-training/operations-custom-forward-backward-create.md): Submits a forward-backward pass driven by externally computed gradients of the loss with respect to per-token log-probabilities.
- [Get custom forward-backward operation](https://docs.together.ai/reference/post-training/operations-custom-forward-backward-get.md): Retrieves the current status and result of a custom forward-backward operation.
- [Optimizer step](https://docs.together.ai/reference/post-training/operations-optim-step-create.md): Submits an optimizer step operation that will asynchronously apply accumulated gradients to update model parameters. Does not make the updated parameters available for sampling; call `weights-sync` afterwards when you want subsequent samples to use the updated policy.
- [Get optim-step operation](https://docs.together.ai/reference/post-training/operations-optim-step-get.md): Retrieves the current status and result of an optim-step operation.
- [Sample](https://docs.together.ai/reference/post-training/operations-sample-create.md): Submits a sample operation that will asynchronously generate text completions with logprobs.
- [Get sample operation](https://docs.together.ai/reference/post-training/operations-sample-get.md): Retrieves the current status and result of a sample operation.
- [Weights sync](https://docs.together.ai/reference/post-training/operations-weights-sync-create.md): Submits a weights-sync operation that makes the session's current trained parameters available for sampling. Call this after `optim-step` when you want subsequent samples to use the updated policy.
- [Get weights-sync operation](https://docs.together.ai/reference/post-training/operations-weights-sync-get.md): Retrieves the current status and result of a weights-sync operation.
- [Create inference checkpoint](https://docs.together.ai/reference/post-training/operations-inference-checkpoint-create.md): Submits an operation that will asynchronously save the current LoRA adapter as an inference checkpoint and upload it to object storage.
- [Get inference checkpoint operation](https://docs.together.ai/reference/post-training/operations-inference-checkpoint-get.md): Retrieves the current status and result of an inference checkpoint operation.
- [Save training checkpoint](https://docs.together.ai/reference/post-training/operations-training-checkpoint-create.md): Submits an operation that will asynchronously save the full training state (adapter + optimizer + step).
- [Get save training checkpoint operation](https://docs.together.ai/reference/post-training/operations-training-checkpoint-get.md): Retrieves the current status and result of a save training checkpoint operation.
- [List checkpoints](https://docs.together.ai/reference/post-training/checkpoints-list.md): Lists training and inference checkpoints owned by the caller, newest first. Filter by type, session, or base model, for example to recover a checkpoint ID for resume.
- [Get checkpoint](https://docs.together.ai/reference/post-training/checkpoints-get.md): Returns metadata for a checkpoint: type, base model, LoRA rank, step, and owning session.
- [Download checkpoint](https://docs.together.ai/reference/post-training/checkpoints-download.md): Returns presigned URLs for downloading a checkpoint's model files. Only inference checkpoints support downloading.
- [Prepare a fine-tuned model for FP4 inference](https://docs.together.ai/reference/post-training/fp4-inference-create.md): Prepares a fine-tuned model for FP4 inference: an adapter is merged into its base model and the result quantized to FP4; a full model is quantized as it is. The model is resolved when the request is made, so a request naming a model that does not exist in the project, was not produced by fine-tuning…
- [Estimate FP4 preparation cost](https://docs.together.ai/reference/post-training/fp4-inference-estimate.md): Reports the price and expected wall-clock time for preparing the model named by `model_object_id` or the deprecated `adapter_object_id` without creating a job. The request body is the same as POST /shaping/prepare-for-fp4-inference, so a caller can price exactly the job it is about to submit.
- [List FP4 preparation jobs](https://docs.together.ai/reference/post-training/shaping-jobs-list.md): List shaping jobs owned by the caller's project.
- [Get FP4 preparation job](https://docs.together.ai/reference/post-training/shaping-jobs-get.md): Retrieve a shaping job owned by the caller's project.
- [Cancel FP4 preparation job](https://docs.together.ai/reference/post-training/shaping-jobs-cancel.md): Request cancellation of a shaping job owned by the caller's project.
- [List FP4 preparation job events](https://docs.together.ai/reference/post-training/shaping-jobs-list-events.md): List events for a shaping job owned by the caller's project.
- [Create job](https://docs.together.ai/reference/post-fine-tunes.md): Create a fine-tuning job with the provided model and training data.
- [Estimate price](https://docs.together.ai/reference/post-fine-tunes-estimate-price.md): Estimate the price of a fine-tuning job.
- [Preview tokenized data](https://docs.together.ai/reference/post-fine-tunes-preview.md): Preview how sampled rows from a fine-tuning training file will be tokenized before packing.
- [List all jobs](https://docs.together.ai/reference/get-fine-tunes.md): List the metadata for all fine-tuning jobs. Returns a list of FinetuneResponseTruncated objects.
- [List job](https://docs.together.ai/reference/get-fine-tunes-id.md): List the metadata for a single fine-tuning job.
- [List job events](https://docs.together.ai/reference/get-fine-tunes-id-events.md): List the events for a single fine-tuning job.
- [List checkpoints](https://docs.together.ai/reference/get-fine-tunes-id-checkpoint.md): List the checkpoints for a single fine-tuning job.
- [Get metrics](https://docs.together.ai/reference/get-fine-tunes-id-metrics.md): Retrieves recorded training metrics for a fine-tuning job in chronological order. All query parameters are optional: omit them to retrieve all metrics.
- [Download tokenized dataset](https://docs.together.ai/reference/get-fine-tunes-id-download-tokenized-dataset.md): Get a presigned URL for the tokenized dataset archive generated for a fine-tune job.
- [Download model](https://docs.together.ai/reference/get-finetune-download.md): Receive a compressed fine-tuned model or checkpoint.
- [Cancel job](https://docs.together.ai/reference/post-fine-tunes-id-cancel.md): Cancel a currently running fine-tuning job. Returns a FinetuneResponseTruncated object.
- [Delete a fine-tune job](https://docs.together.ai/reference/delete-fine-tunes-id.md): Delete a fine-tuning job.
- [List supported models](https://docs.together.ai/reference/get-fine-tunes-models-supported.md): List models supported for fine-tuning.
- [Get model limits](https://docs.together.ai/reference/get-fine-tunes-models-limits.md): Get model limits for a specific fine-tuning model.
- [Create an evaluation job](https://docs.together.ai/reference/create-evaluation.md)
- [Get all evaluation jobs](https://docs.together.ai/reference/list-evaluations.md)
- [Get evaluation job details](https://docs.together.ai/reference/get-evaluation.md)
- [Get evaluation job status and results](https://docs.together.ai/reference/get-evaluation-status.md)
- [Get model list](https://docs.together.ai/reference/list-evaluation-models.md)
- [List all files](https://docs.together.ai/reference/get-files.md): List the metadata for all uploaded data files.
- [Retrieve file metadata](https://docs.together.ai/reference/get-files-id.md): Retrieve the metadata for a single uploaded data file.
- [Delete a file](https://docs.together.ai/reference/delete-files-id.md): Delete a previously uploaded data file.
- [Get file contents](https://docs.together.ai/reference/get-files-id-content.md): Get the contents of a single uploaded data file.
- [Upload a file](https://docs.together.ai/reference/upload-file.md): Upload a file with specified purpose, file name, and file type.
- [Create a GPU cluster](https://docs.together.ai/reference/clusters-create.md): Create an Instant Cluster on Together's high-performance GPU clusters. With features like on-demand scaling, long-lived resizable high-bandwidth shared DC-local storage, Kubernetes and Slurm cluster flavors, a REST API, and Terraform support, you can run workloads flexibly without complex infrastruc…
- [List all GPU clusters](https://docs.together.ai/reference/clusters-list.md): List all GPU clusters.
- [Get GPU cluster by cluster ID](https://docs.together.ai/reference/clusters-get.md): Retrieve information about a specific GPU cluster.
- [Update a GPU cluster](https://docs.together.ai/reference/clusters-update.md): Update the configuration of an existing GPU cluster.
- [Delete GPU cluster by cluster ID](https://docs.together.ai/reference/clusters-delete.md): Delete a GPU cluster by cluster ID.
- [Create a shared volume](https://docs.together.ai/reference/clusters_storages-create.md): Instant Clusters supports long-lived, resizable in-DC shared storage with user data persistence. You can dynamically create and attach volumes to your cluster at cluster creation time, and resize as your data grows. All shared storage is backed by multi-NIC bare metal paths, ensuring high-throughput…
- [List all shared volumes](https://docs.together.ai/reference/clusters_storages-list.md): List all shared volumes.
- [Get a shared volume by ID](https://docs.together.ai/reference/clusters_storages-get.md): Retrieve information about a specific shared volume.
- [Update a shared volume](https://docs.together.ai/reference/clusters_storages-update.md): Update the configuration of an existing shared volume.
- [Delete a shared volume by ID](https://docs.together.ai/reference/clusters_storages-delete.md): Delete a shared volume. Note that if this volume is attached to a cluster, deleting will fail.
- [Remediation get](https://docs.together.ai/reference/remediation-get.md): Retrieve the status of a specific remdiation on a specific instance in a specific cluster.
- [List remediations](https://docs.together.ai/reference/remediation-list.md): Lists remediations for an instance or cluster.
- [Remediation create](https://docs.together.ai/reference/remediation-create.md): Creates a new remediation for an instance.
- [Remediation approve](https://docs.together.ai/reference/remediation-approve.md): Approves a pending remediation.
- [Remediation cancel](https://docs.together.ai/reference/remediation-cancel.md): Cancels a pending remediation.
- [Remediation reject](https://docs.together.ai/reference/remediation-reject.md): Rejects a pending remediation.
- [List regions and corresponding supported driver versions](https://docs.together.ai/reference/clusters-list-regions.md)
- [Get the list of deployments](https://docs.together.ai/reference/deployments-list.md): Get a list of all deployments in your project
- [Create a new deployment](https://docs.together.ai/reference/deployments-create.md): Create a new deployment with specified configuration
- [Get a deployment by ID or name](https://docs.together.ai/reference/deployments-get.md): Retrieve details of a specific deployment by its ID or name
- [Update a deployment](https://docs.together.ai/reference/deployments-update.md): Update an existing deployment configuration
- [Delete a deployment](https://docs.together.ai/reference/deployments-delete.md): Delete an existing deployment
- [Get logs for a deployment](https://docs.together.ai/reference/deployments-logs.md): Retrieve logs from a deployment, optionally filtered by replica ID.
- [List revision history events of a deployment](https://docs.together.ai/reference/deployments-revisions-list.md): Returns the revision history of the deployment, in descending order. Defaults to the most recent events. Only the 200 most recent events are retained per deployment; paginating past that returns an empty list.
- [Get the full configuration of a specific revision](https://docs.together.ai/reference/deployments-revisions-get.md): Returns the deployment configuration defined by the specified revision. Only the 50 most recent revisions per deployment retain their configuration; older revisions return 404 even while they still appear in the revision history. The deployment's currently active revision is always available, regard…
- [Roll back a deployment to a previous revision](https://docs.together.ai/reference/deployments-rollback.md): Re-applies the spec of a previous revision. Pods running the target revision will be retained. Other pods will be drained and restarted with the target revision. Only the 50 most recent revisions per deployment can be rolled back to; older targets return 404.
- [Get the list of project secrets](https://docs.together.ai/reference/deployments-secrets-list.md): Retrieve all secrets in your project
- [Create a new secret](https://docs.together.ai/reference/deployments-secrets-create.md): Create a new secret to store sensitive configuration values
- [Get a secret by ID or name](https://docs.together.ai/reference/deployments-secrets-get.md): Retrieve details of a specific secret by its ID or name
- [Update a secret](https://docs.together.ai/reference/deployments-secrets-update.md): Update an existing secret's value or metadata
- [Delete a secret](https://docs.together.ai/reference/deployments-secrets-delete.md): Delete an existing secret
- [Download a file](https://docs.together.ai/reference/deployments-storage-get.md): Download a file by redirecting to a signed URL
- [Get a file download URL](https://docs.together.ai/reference/deployments-storage-get-url.md): Create a signed URL for downloading a deployment storage file
- [Get the list of project volumes](https://docs.together.ai/reference/deployments-storage-volumes-list.md): Retrieve all volumes in your project
- [Create a new volume](https://docs.together.ai/reference/deployments-storage-volumes-create.md): Create a new volume to preload files in deployments
- [Get a volume by ID or name](https://docs.together.ai/reference/deployments-storage-volumes-get.md): Retrieve details of a specific volume by its ID or name
- [Update a volume](https://docs.together.ai/reference/deployments-storage-volumes-update.md): Update an existing volume's configuration or contents
- [Delete a volume](https://docs.together.ai/reference/deployments-storage-volumes-delete.md): Delete an existing volume
- [Submit a queued job](https://docs.together.ai/reference/queue-submit.md): Submit a new job to the queue for asynchronous processing. Jobs are processed in strict priority order (higher priority first, FIFO within the same priority). Returns a request ID that can be used to poll status or cancel the job.
- [Get job status](https://docs.together.ai/reference/queue-status.md): Poll the current status of a previously submitted job. Provide the request_id and model as query parameters.
- [Cancel a queued job](https://docs.together.ai/reference/queue-cancel.md): Cancel a pending job. Only jobs in pending status can be canceled. Running jobs cannot be stopped. Returns the job status after the attempt. If the job is not pending, returns 409 with the current status unchanged.
- [Clear a model's pending jobs](https://docs.together.ai/reference/queue-clear.md): Cancel all pending jobs for the given model. Running jobs are left untouched. Returns the number of jobs that were canceled.
- [Get queue metrics](https://docs.together.ai/reference/queue-metrics.md): Get the current queue statistics for a model, including pending and running job counts.
- [Identify the caller](https://docs.together.ai/reference/whoami.md): Returns identity information for the authenticated caller. Accepts project API keys and user session JWTs. The response `type` indicates which fields are present. For project-scoped credentials, `project.slug` can be used with an endpoint slug as `<project_slug>/<endpoint_slug>` to form the `model`…
- [List projects](https://docs.together.ai/reference/projects-list.md): Retrieve a list of accessible projects.
- [Get billing usage](https://docs.together.ai/reference/billing-usage.md): Returns an organization's billing usage for a month as cost-annotated line items grouped into time windows. Finalized windows are returned through the end of yesterday at daily granularity, or the last completed hour at hourly granularity (UTC).
- [List endpoints](https://docs.together.ai/reference/dmi/endpoints-list.md): Lists the dedicated inference endpoints owned by the specified project.
- [Create an endpoint](https://docs.together.ai/reference/dmi/endpoints-create.md): Creates a stable, inference-addressable endpoint. Add one or more deployments and configure its traffic split before sending inference requests to the endpoint name.
- [Get an endpoint](https://docs.together.ai/reference/dmi/endpoints-get.md): Retrieves an endpoint and lightweight summaries of the deployments attached to it.
- [Update an endpoint](https://docs.together.ai/reference/dmi/endpoints-update.md): Updates mutable endpoint fields such as its endpoint string, visibility, or deployment traffic split. Use `updateMask` to select fields explicitly and `etag` in the request body for optimistic concurrency.
- [Delete an endpoint](https://docs.together.ai/reference/dmi/endpoints-delete.md): Permanently deletes an endpoint. Delete its deployments first; use `etag` to reject the request if the endpoint changed after it was read.
- [List endpoint events](https://docs.together.ai/reference/dmi/endpoints-list-events.md): Lists an endpoint's audit and lifecycle events newest first. The feed combines endpoint changes with provisioning, scaling, readiness, rollout, and other events from deployments under the endpoint.
- [Get endpoint analytics](https://docs.together.ai/reference/dmi/endpoints-analytics.md): Returns aggregated request, token, latency, throughput, error, and resource-utilization metrics for an endpoint over a time range. Optionally includes time-series buckets and a per-deployment breakdown.
- [List organization endpoints](https://docs.together.ai/reference/dmi/endpoints-list-organization.md): Lists endpoints shared with every project in the specified organization. Project-private and public endpoints are not included.
- [List deployments](https://docs.together.ai/reference/dmi/deployments-list.md): Lists the deployments attached to an endpoint, including their model, configuration, scaling settings, placement, and current status.
- [Create a deployment](https://docs.together.ai/reference/dmi/deployments-create.md): Creates a model deployment under an endpoint. The deployment provisions asynchronously; monitor its status before routing live traffic to it.
- [Get a deployment](https://docs.together.ai/reference/dmi/deployments-get.md): Retrieves a deployment's desired configuration, placement, runtime information, and current provisioning status.
- [Update a deployment](https://docs.together.ai/reference/dmi/deployments-update.md): Updates mutable deployment fields such as its model, configuration, autoscaling bounds, or LoRA support. Changes that affect serving may trigger asynchronous reprovisioning.
- [Delete a deployment](https://docs.together.ai/reference/dmi/deployments-delete.md): Permanently deletes a deployment from its endpoint. Remove the deployment from live traffic first; use `etag` to reject the request if it changed after it was read.
- [List deployment adapters](https://docs.together.ai/reference/dmi/adapters-list.md): Lists LoRA adapters attached to a deployment with per-cluster load state.
- [Add a deployment adapter](https://docs.together.ai/reference/dmi/adapters-add.md): Attaches a LoRA adapter to a deployment. If the deployment is at adapter capacity, force can evict the oldest adapter.
- [Get a deployment adapter](https://docs.together.ai/reference/dmi/adapters-get.md): Gets an adapter attachment by its `dad_` id and returns its per-cluster load state.
- [Update a deployment adapter](https://docs.together.ai/reference/dmi/adapters-update.md): Updates the pinned revision of an adapter attachment by its `dad_` id using its row-level etag for optimistic concurrency.
- [Remove a deployment adapter](https://docs.together.ai/reference/dmi/adapters-remove.md): Detaches an adapter attachment by its `dad_` id using its row-level etag for optimistic concurrency.
- [List placement profiles](https://docs.together.ai/reference/dmi/placement-profiles-list.md): Lists reusable, project-visible placement policies that control the regions where deployments may be scheduled.
- [Get a placement profile](https://docs.together.ai/reference/dmi/placement-profiles-get.md): Retrieves a reusable placement profile and its ordered region preferences.
- [List inference instance types](https://docs.together.ai/reference/dmi/instance-types-list.md): Lists hardware instance types currently available to inference deployments, including GPU resources, pricing, regions, and best-effort capacity headroom.
- [Get an inference instance type](https://docs.together.ai/reference/dmi/instance-types-get.md): Retrieves the GPU resources, pricing, regional availability, and best-effort capacity headroom for one inference instance type.
- [List A/B experiments](https://docs.together.ai/reference/dmi/ab-experiments-list.md): Lists the managed live-traffic experiments configured for an endpoint.
- [Create an A/B experiment](https://docs.together.ai/reference/dmi/ab-experiments-create.md): Creates a managed control/variant split across two to 20 deployments under the same endpoint. Exactly one member is the control, member percentages must add up to 100, and the split applies only to traffic that the endpoint would otherwise send to the control.
- [Get an A/B experiment](https://docs.together.ai/reference/dmi/ab-experiments-get.md): Retrieves an A/B experiment and its participating deployments, roles, and traffic percentages.
- [Update an A/B experiment](https://docs.together.ai/reference/dmi/ab-experiments-update.md): Updates an experiment's description or member traffic percentages. Use the experiment etag for optimistic concurrency.
- [Delete an A/B experiment](https://docs.together.ai/reference/dmi/ab-experiments-delete.md): Deletes an A/B experiment and removes its managed traffic split. The deployments themselves are not deleted.
- [List shadow experiments](https://docs.together.ai/reference/dmi/shadow-experiments-list.md): Lists experiments that mirror sampled endpoint traffic to target deployments without affecting client responses. Set `includeTargets=true` to include target details inline.
- [Create a shadow experiment](https://docs.together.ai/reference/dmi/shadow-experiments-create.md): Creates an experiment that mirrors a sampled portion of endpoint traffic to one or more target deployments without returning their responses to clients. Add a description with the update operation after creation.
- [Get a shadow experiment](https://docs.together.ai/reference/dmi/shadow-experiments-get.md): Retrieves a shadow experiment, including its sampling strategy and target deployments.
- [Update a shadow experiment](https://docs.together.ai/reference/dmi/shadow-experiments-update.md): Updates a shadow experiment's description or source sampling strategy. `updateMask` is required; source changes also require the current `etag` in the request body.
- [Delete a shadow experiment](https://docs.together.ai/reference/dmi/shadow-experiments-delete.md): Deletes a shadow experiment and its target records. The underlying deployments are not deleted.
- [List shadow experiment targets](https://docs.together.ai/reference/dmi/shadow-experiment-targets-list.md): Lists the deployments that receive mirrored requests from a shadow experiment.
- [Create a shadow experiment target](https://docs.together.ai/reference/dmi/shadow-experiment-targets-create.md): Adds a deployment under the same endpoint as a target for mirrored requests.
- [Get a shadow experiment target](https://docs.together.ai/reference/dmi/shadow-experiment-targets-get.md): Retrieves one target configured to receive mirrored requests from a shadow experiment.
- [Update a shadow experiment target](https://docs.together.ai/reference/dmi/shadow-experiment-targets-update.md): Updates a shadow target's name, deployment, or description. `updateMask` is required and must select at least one mutable field.
- [Delete a shadow experiment target](https://docs.together.ai/reference/dmi/shadow-experiment-targets-delete.md): Removes a target from a shadow experiment without deleting the underlying deployment.
- [List endpoint rollouts](https://docs.together.ai/reference/dmi/rollouts-list.md): Lists rollout histories for an endpoint. Use `filter=ROLLOUT_FILTER_ACTIVE` to return only the active rollout, if one exists.
- [Preview rollout defaults](https://docs.together.ai/reference/dmi/rollouts-preview-defaults.md): Returns the values a create request would pick for any field left unset, plus the capacity context needed to display them, without creating a rollout. Responses are display state only and re-validated authoritatively at create and start; do not copy response values back into a create request.
- [Create a rollout](https://docs.together.ai/reference/dmi/rollouts-create.md): Creates a rollout in the pending state without shifting traffic. Start the rollout in a separate request after reviewing its strategy and metric gates.
- [Get a rollout](https://docs.together.ai/reference/dmi/rollouts-get.md): Retrieves a rollout's strategy, lifecycle state, current traffic percentage, step history, and metric-gate results.
- [Delete a rollout](https://docs.together.ai/reference/dmi/rollouts-delete.md): Deletes a rollout record. An active rollout must be aborted or completed before it can be deleted.
- [Start a rollout](https://docs.together.ai/reference/dmi/rollouts-start.md): Starts a pending rollout and begins its configured traffic-shifting workflow.
- [Pause a rollout](https://docs.together.ai/reference/dmi/rollouts-pause.md): Requests a running or stabilizing rollout to pause and records an optional reason. The response returns the PAUSING snapshot; poll GetRollout until state is PAUSED to confirm the executor has parked.
- [Resume a rollout](https://docs.together.ai/reference/dmi/rollouts-resume.md): Resumes a pausing, paused, or system-paused rollout from its current step and traffic split.
- [Promote a rollout](https://docs.together.ai/reference/dmi/rollouts-promote.md): Completes a running or paused rollout immediately by sending all live traffic to the target deployment.
- [Cancel a rollout](https://docs.together.ai/reference/dmi/rollouts-cancel.md): Cancels a running, pausing, paused, system-paused, or stabilizing rollout by freezing the current traffic split into standing weights. Revert is removed and rejected; after canceling, start another canary rollout in either direction or rebalance the traffic split. The response is the accepted rollou…
- [List supported models](https://docs.together.ai/reference/dmi/supported-models-list.md): Lists Together-hosted base models that can be deployed for dedicated inference, together with their capabilities and certified deployment profiles.
- [Get a supported model](https://docs.together.ai/reference/dmi/supported-models-get.md): Retrieves a Together-hosted base model and the certified model, configuration, hardware, and performance profiles available for deployment.
- [List project models](https://docs.together.ai/reference/dmi/models-list.md): Lists custom model resources owned by the specified project. Use the organization endpoint to list models shared across projects or the supported-model catalog to discover Together-hosted base models.
- [Create a model](https://docs.together.ai/reference/dmi/models-create.md): Registers a custom model resource in the project. Registration creates the model's metadata; upload or import model files separately before deploying it.
- [Get a model](https://docs.together.ai/reference/dmi/models-get.md): Retrieves a custom model's metadata, visibility, weight information, and base-model relationship.
- [Update a model](https://docs.together.ai/reference/dmi/models-update.md): Updates mutable model metadata such as its inference name, description, base model, or visibility.
- [Delete a model](https://docs.together.ai/reference/dmi/models-delete.md): Permanently deletes a custom model resource. The model must not be in use by an active deployment.
- [List model files](https://docs.together.ai/reference/dmi/models-list-files.md): Lists files in the latest or specified revision of a model, including paths, sizes, and content hashes.
- [List model revisions](https://docs.together.ai/reference/dmi/models-list-revisions.md): Lists the immutable file revisions available for a custom model, newest first.
- [Get a model revision](https://docs.together.ai/reference/dmi/models-get-revision.md): Retrieves metadata for one immutable revision of a custom model's files.
- [List organization models](https://docs.together.ai/reference/dmi/models-list-organization.md): Lists custom models shared with every project in the specified organization. Project-private and public models are not included.
- [Create a remote model upload](https://docs.together.ai/reference/dmi/model-uploads-create.md): Starts an asynchronous job that imports model files from Hugging Face or a presigned URL into a registered model and creates a model revision when the import completes.
- [List remote model uploads](https://docs.together.ai/reference/dmi/model-uploads-list.md): Lists asynchronous jobs that import model files from Hugging Face or a presigned remote URL.
- [Get a remote model upload](https://docs.together.ai/reference/dmi/model-uploads-get.md): Retrieves the status, progress details, retry counts, and timestamps for a remote model import job.
- [List remote model upload events](https://docs.together.ai/reference/dmi/model-uploads-list-events.md): Lists progress and diagnostic events for a remote model import job.
- [List model configurations](https://docs.together.ai/reference/dmi/configs-list.md): Lists production-ready configuration revisions compatible with a reference model. Specify the model with `referenceModel` or the deprecated `referenceModelId`; if both are supplied, they must identify the same model. Results include public configurations and configurations owned by the specified pro…
- [Get a model configuration](https://docs.together.ai/reference/dmi/configs-get.md): Retrieves a model configuration revision by ID, including its runtime selectors and certifications.
- [Python Library](https://docs.together.ai/python-library.md)
- [TypeScript Library](https://docs.together.ai/typescript-library.md)
- [Python SDK v2 migration](https://docs.together.ai/docs/pythonv2-migration-guide.md): Migrate code written against the Together Python SDK v1 to the v2 SDK.
- [Error codes](https://docs.together.ai/docs/error-codes.md): An overview on error status codes, causes, and quick fix solutions.
- [Get started](https://docs.together.ai/reference/cli/getting-started.md): Install the Together CLI to deploy endpoints, fine-tune models, and manage GPU clusters from your terminal.
- [Telemetry](https://docs.together.ai/reference/cli/telemetry.md): Understand what the Together CLI tracks, how to opt out, and where the local config file lives.
- [Models](https://docs.together.ai/reference/cli/models.md): List Together AI models and upload your own from Hugging Face or S3.
- [Endpoints](https://docs.together.ai/reference/cli/endpoints.md): Create, update, and manage dedicated inference endpoints from your terminal.
- [Files](https://docs.together.ai/reference/cli/files.md): Upload and manage datasets for use in fine-tuning, evals, and batch inference.
- [Fine-tuning](https://docs.together.ai/reference/cli/finetune.md): Create, monitor, and manage fine-tuning jobs from your terminal.
- [Training](https://docs.together.ai/reference/cli/training.md): Inspect post-training resources, sessions, and checkpoints, and prepare trained LoRA adapters for FP4 inference from your terminal.
- [Evals](https://docs.together.ai/reference/cli/evals.md): Create and manage model-evaluation jobs from your terminal, including classify, score, and compare evals.
- [Batches](https://docs.together.ai/reference/cli/batches.md): Submit, monitor, download, and cancel batch inference jobs from your terminal.
- [Login](https://docs.together.ai/reference/cli/auth.md): Log in to the Together CLI with your browser.
- [Projects](https://docs.together.ai/reference/cli/projects.md): List the Together AI projects the CLI's credentials can access.
- [Whoami](https://docs.together.ai/reference/cli/whoami.md): Show the project and organization for the credentials the Together CLI uses.
- [Models](https://docs.together.ai/reference/cli/models-beta.md): Register, upload, and inspect models and adapters for dedicated model inference from your terminal.
- [Endpoints](https://docs.together.ai/reference/cli/endpoints-beta.md): Deploy and manage dedicated inference endpoints from your terminal.
- [Clusters](https://docs.together.ai/reference/cli/clusters.md): Reserve, configure, and manage GPU clusters from your terminal.
- [Jig CLI reference](https://docs.together.ai/reference/cli/jig.md): CLI commands, pyproject.toml configuration, environment variables, and Python SDK for dedicated containers.
- [Changelog](https://docs.together.ai/docs/changelog.md)
- [Deprecations](https://docs.together.ai/docs/deprecations.md): Together AI's model lifecycle policy, including upgrades, redirects, and deprecation schedules.

## OpenAPI Specs

- [deprecated-spec](/deprecated-spec.json)
- [openapi.base](/openapi-src/openapi.base.yaml)
- [openapi](/openapi.yaml)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.