Together AI llms.txt

The actual file

Source Markdown

The saved llms.txt text, displayed as source. The file belongs to its publisher; the live endpoint may change.

Showing the first 18000 characters. Load the complete file to read the remainder.

# Together AI docs

> Documentation for the Together AI platform for inference and training.

- [Overview](https://docs.together.ai/intro.md): Run, train, and serve open-source AI models on Together AI.
- [Quickstart](https://docs.together.ai/docs/quickstart.md): Make your first request to Together AI in a few minutes.
- [Coding agent setup](https://docs.together.ai/docs/agent-skills.md): Make your AI coding agent Together-AI-aware with a ready-made skill for code generation and an MCP server for live docs lookup.
- [Configure Claude Code, Codex, OpenCode & Pi Code with OSS models](https://docs.together.ai/docs/togetherlink.md): Use Together Link to run Claude Code, Codex, OpenCode, Pi Code, Claude Desktop and Cowork, and ChatGPT Desktop with models hosted by Together AI.
- [Recommended models](https://docs.together.ai/docs/inference/recommended-models.md): Our picks for common inference use cases.
- [Overview](https://docs.together.ai/docs/serverless/overview.md): Call 100+ open-source models with per-token pricing and no provisioning latency.
- [Available models](https://docs.together.ai/docs/serverless/models.md): Browse the catalog of available models for instant inference.
- [Rate limits](https://docs.together.ai/docs/serverless/rate-limits.md): Understand serverless performance, how to handle rate limits during high demand, and options for throughput guarantees.
- [Provisioned throughput](https://docs.together.ai/docs/inference/provisioned-throughput.md): Reserved inference capacity for production workloads.
- [Overview](https://docs.together.ai/docs/inference/overview.md): Run inference on 100+ open-source models.
- [OpenAI compatibility](https://docs.together.ai/docs/inference/openai-compatibility.md): Point your OpenAI Python or TypeScript client at Together AI to call open-source models without rewriting your app.
- [Third-party integrations](https://docs.together.ai/docs/inference/sdk-integrations.md): Use Together AI models through partner SDKs and integrations.
- [Send chat completions](https://docs.together.ai/docs/inference/chat/overview.md): Query chat models with single prompts, multi-turn conversations, and system prompts.
- [Parameters](https://docs.together.ai/docs/inference/chat/parameters.md): The full list of parameters you can pass to the chat completions endpoint.
- [Structured outputs](https://docs.together.ai/docs/inference/chat/structured-outputs.md): Use JSON mode to get structured outputs from supported chat models.
- [Reasoning](https://docs.together.ai/docs/inference/chat/reasoning.md): Use reasoning models that think step-by-step before answering.
- [Prompt caching](https://docs.together.ai/docs/inference/chat/prompt-caching.md): Together reuses previously processed prompt prefixes to lower input cost and time to first token.
- [Log probabilities](https://docs.together.ai/docs/inference/chat/logprobs.md): Return per-token log probabilities to measure model confidence and route low-confidence outputs to a stronger model.
- [Function calling patterns](https://docs.together.ai/docs/inference/function-calling/overview.md): Function calling lets LLMs respond with structured function names and arguments your application can execute.
- [Call functions](https://docs.together.ai/docs/inference/function-calling/single-call.md): One tool call per response, covering simple and multiple-tool patterns.
- [Call functions in parallel](https://docs.together.ai/docs/inference/function-calling/parallel.md): Multiple tool calls in one response, covering parallel calls to the same tool and to different tools.
- [Agentic function calling patterns](https://docs.together.ai/docs/inference/function-calling/agentic.md): Tool use across multiple steps or conversation turns, covering multi-step and multi-turn agent loops.
- [Function calling best practices](https://docs.together.ai/docs/inference/function-calling/best-practices.md): Design tools, write descriptions, and control tool selection so Together models call functions reliably.
- [Text-to-image generation](https://docs.together.ai/docs/inference/images/overview.md): Generate images from text prompts.
- [Image-to-image generation](https://docs.together.ai/docs/inference/images/reference-images.md): Edit or transform an existing image by passing image_url (Kontext) or reference_images (FLUX.2 and Google models).
- [Image generation parameters](https://docs.together.ai/docs/inference/images/parameters.md): Parameter reference for the images API: dimensions, quality control, base64 responses, safety checker, and troubleshooting.
- [Generate videos](https://docs.together.ai/docs/inference/videos/overview.md): Generate videos from text and image prompts.
- [Reference images and keyframes](https://docs.together.ai/docs/inference/videos/reference-and-keyframes.md): Guide visual style with reference images and control specific frames in your video.
- [Audio input for videos](https://docs.together.ai/docs/inference/videos/audio-input.md): Drive video generation with an audio file for lip sync, beat-matched motion, or narration.
- [Video generation parameters](https://docs.together.ai/docs/inference/videos/parameters.md): Reference for video generation parameters, including guidance scale and quality control.
- [Use image inputs](https://docs.together.ai/docs/inference/vision/overview.md): Run vision-language models on Together: pass images alongside text and get structured replies, transcripts, comparisons, or extracted data.
- [Vision input modes](https://docs.together.ai/docs/inference/vision/inputs.md): Send local images, video URLs, or multiple images to a vision model in a single request.
- [Structured extraction with vision](https://docs.together.ai/docs/inference/vision/structured-extraction.md): Combine image input with a JSON schema to extract typed data from screenshots, documents, and photos, or run OCR to pull text out of images.
- [Vision-language function calling](https://docs.together.ai/docs/inference/vision/function-calling.md): Combine image understanding with tool use on Together AI vision-language models.
- [Transcribe audio](https://docs.together.ai/docs/inference/transcription/overview.md): Transcribe and translate audio into text.
- [Streaming transcription](https://docs.together.ai/docs/inference/transcription/streaming.md): Use the real-time WebSocket API for low-latency, incremental speech-to-text.
- [Audio translation](https://docs.together.ai/docs/inference/transcription/translation.md): Translate speech in any language into English text.
- [Voice activity detection](https://docs.together.ai/docs/inference/transcription/voice-activity-detection.md): Configure voice activity detection to control how speech segments are detected in real-time transcription.
- [Advanced transcription options](https://docs.together.ai/docs/inference/transcription/features.md): Speaker diarization, word-level timestamps, response formats, async support, and best practices.
- [Generate speech](https://docs.together.ai/docs/inference/text-to-speech/overview.md): Generate speech audio from text with Together AI text-to-speech models.
- [Text-to-speech streaming](https://docs.together.ai/docs/inference/text-to-speech/streaming.md): Stream audio over HTTP for low time-to-first-byte and access raw PCM bytes.
- [WebSocket API](https://docs.together.ai/docs/inference/text-to-speech/websocket.md): Stream text in and audio out over a single WebSocket connection for the lowest interactive latency.
- [Rerank](https://docs.together.ai/docs/inference/embeddings/rerank.md): Reorder retrieved documents by relevance to a query for sharper search and RAG results.
- [Evaluations](https://docs.together.ai/docs/ai-evaluations.md): Use LLMs to classify, score, and compare model outputs on Together AI.
- [Run an evaluation](https://docs.together.ai/docs/run-an-evaluation.md): Prepare a dataset, launch an evaluation job, and download results with the Together CLI or API.
- [Parameters and result formats](https://docs.together.ai/docs/evaluations-reference.md): Parameters, result formats, and template syntax for the evaluations API.
- [Supported models](https://docs.together.ai/docs/evaluations-supported-models.md): Serverless models and external provider shortcuts supported by the evaluations API.
- [Overview](https://docs.together.ai/docs/inference/batch/overview.md): Run asynchronous batch workloads at up to 50% lower cost.
- [Run a batch job](https://docs.together.ai/docs/inference/batch/tutorial.md): Prepare a JSONL file, upload it, start a batch job, poll until it finishes, and retrieve results.
- [Manage batch jobs](https://docs.together.ai/docs/inference/batch/manage.md): Status, results, errors, and operational reference for the Together Batch API.
- [Overview](https://docs.together.ai/docs/dedicated-endpoints/overview.md): Deploy a model for inference on dedicated GPUs.
- [Quickstart](https://docs.together.ai/docs/dedicated-endpoints/quickstart.md): Deploy a model on dedicated hardware in a few minutes.
- [Concepts](https://docs.together.ai/docs/dedicated-endpoints/concepts.md): Understand the resource model and development workflow for dedicated model inference.
- [Pricing](https://docs.together.ai/docs/dedicated-endpoints/pricing.md): Billing and pricing details for dedicated model inference.
- [Supported models](https://docs.together.ai/docs/dedicated-endpoints/models.md): View the supported models you can deploy or fine-tune for dedicated model inference.
- [Upload a fine-tuned model](https://docs.together.ai/docs/dedicated-endpoints/custom-models.md): Serve a fine-tuned model uploaded from your machine, Hugging Face, or S3.
- [Upload a LoRA adapter](https://docs.together.ai/docs/dedicated-endpoints/adapter.md): Serve a custom LoRA adapter uploaded from Hugging Face or S3.
- [Choose a deployment profile](https://docs.together.ai/docs/dedicated-endpoints/configs.md): Pick the hardware deployment profile that your model runs on.
- [Manage endpoints and deployments](https://docs.together.ai/docs/dedicated-endpoints/manage.md): Create, update, and delete resources for dedicated model inference.
- [Configure autoscaling](https://docs.together.ai/docs/dedicated-endpoints/scaling.md): Autoscale a deployment between replica bounds, pick the right scaling metric, and stop idle deployments automatically.
- [Migrate from v1](https://docs.together.ai/docs/dedicated-endpoints/migrate-from-v1.md): Move a dedicated endpoint from the v1 API to the v2 dedicated model inference resource model.
- [Overview](https://docs.together.ai/docs/dedicated-endpoints/route-traffic.md): Route traffic to a deployment, and explore strategies that change how traffic moves over time.
- [Split traffic across deployments](https://docs.together.ai/docs/dedicated-endpoints/split-traffic.md): Run multiple deployments on one endpoint and split requests between them by weight.
- [Start a rollout](https://docs.together.ai/docs/dedicated-endpoints/rollouts.md): Shift traffic to a new deployment on the same endpoint.
- [Gate rollouts with metrics](https://docs.together.ai/docs/dedicated-endpoints/rollout-metric-gates.md): Pause a canary rollout automatically when the new deployment regresses.
- [Run an A/B test](https://docs.together.ai/docs/dedicated-endpoints/ab-tests.md): Compare a candidate deployment against a baseline on live traffic.
- [Run a shadow experiment](https://docs.together.ai/docs/dedicated-endpoints/shadow-experiments.md): Mirror a sampled fraction of endpoint traffic to a target deployment without affecting the client response.
- [Send requests](https://docs.together.ai/docs/dedicated-endpoints/requests.md): After deploying a model, send requests using the shared inference API.
- [Monitor endpoints and deployments](https://docs.together.ai/docs/dedicated-endpoints/monitoring.md): Monitor endpoint and deployment metrics with built-in dashboards and a Prometheus-compatible metrics endpoint.
- [Visualize endpoint metrics in Grafana](https://docs.together.ai/docs/dedicated-endpoints/grafana.md): Scrape the metrics endpoint with Prometheus and import the example Grafana dashboard for dedicated endpoints.
- [Overview](https://docs.together.ai/docs/dedicated-container-inference.md): Deploy your own containers on Together's managed GPU infrastructure with automatic scaling, job queues, and built-in observability.
- [Quickstart](https://docs.together.ai/docs/containers-quickstart.md): Deploy your first container in 20 minutes.
- [Image generation with Flux2](https://docs.together.ai/docs/dedicated-containers-image.md): Deploy a Flux2 image generation model on Together's managed GPU infrastructure using dedicated containers.
- [Video generation with Wan 2.1](https://docs.together.ai/docs/dedicated-containers-video.md): Deploy a multi-GPU video generation model on Together's managed GPU infrastructure using dedicated containers.
- [Serve an OpenAI-compatible endpoint](https://docs.together.ai/docs/dedicated-containers-openai.md): Deploy a custom model behind an OpenAI-compatible endpoint using dedicated containers.
- [Architecture](https://docs.together.ai/docs/together-deployments.md): Architecture, deployment lifecycle, and core concepts for dedicated container inference.
- [Jig CLI](https://docs.together.ai/docs/deployments-jig.md): Build, push, and deploy containers to Together's managed GPU infrastructure.
- [Sprocket SDK](https://docs.together.ai/docs/deployments-sprocket.md): A Python SDK for building inference workers that support both synchronous and asynchronous requests via Together's platform.
- [Queue API](https://docs.together.ai/docs/deployments-queue.md): Submit, monitor, and manage asynchronous jobs for your dedicated container inference deployments.
- [Sprocket SDK reference](https://docs.together.ai/reference/dci-reference-sprocket.md): API reference for Sprocket classes, functions, and configuration.
- [Overview](https://docs.together.ai/docs/post-training/overview.md): Train models from your own training loop on Together AI GPUs.
- [Supervised fine-tuning quickstart](https://docs.together.ai/docs/post-training/quickstart-sft.md): Run your first post-training session with supervised fine-tuning on instruction-following data, then download the trained weights.
- [Reinforcement learning with GRPO quickstart](https://docs.together.ai/docs/post-training/quickstart-rl.md): Train a model to solve grade-school math with GRPO, monitor the run, and retrieve the trained model.
- [Concepts](https://docs.together.ai/docs/post-training/concepts.md): See how a training run divides work between your code and a model resource on Together AI GPUs, how resources, sessions, and checkpoints relate, and how you deploy or download a trained model.
- [Pricing](https://docs.together.ai/docs/post-training/pricing.md): How post-training bills for GPU time, how to estimate the cost of a resource or a training run, and how refunds and the minimum credit balance work.
- [Supported models](https://docs.together.ai/docs/post-training/supported-models.md): Every base model available for post-training, with training modes, context lengths, GPU types, and adapter capacity.
- [Overview](https://docs.together.ai/docs/post-training/training-methods.md): Match a post-training method to your data, choose between adapter and full-weight training, and combine methods.
- [Supervised fine-tuning](https://docs.together.ai/docs/post-training/sft.md): Train a model on example conversations with supervised fine-tuning.
- [Direct preference optimization](https://docs.together.ai/docs/post-training/dpo.md): Train on preference pairs with a custom loss function that runs in your own process.
- [Reinforcement learning with GRPO](https://docs.together.ai/docs/post-training/grpo.md): Train a model against a reward function with group relative policy optimization.
- [Write a reward function](https://docs.together.ai/docs/post-training/reward-functions.md): How to write a reward function for GRPO, register it with your dataset, and test it before you start training.
- [Train in multi-turn environments](https://docs.together.ai/docs/post-training/multi-turn-environments.md): Train tool-use agents with GRPO over whole episodes, where the model acts, an environment responds, and a reward scores the outcome.
- [Write a custom loss function](https://docs.together.ai/docs/post-training/custom-loss-functions.md): Train with any objective you compute in PyTorch from the model's log-probabilities, for SFT, DPO, or RL.
- [Manage model resources](https://docs.together.ai/docs/post-training/model-resources.md): Configure, create, attach to, inspect, and stop the GPU resource that post-training sessions run on, and learn its lifecycle states, timers, error codes, and limits.
- [Manage checkpoints](https://docs.together.ai/docs/post-training/checkpoints.md): Save inference and training checkpoints, download trained weights, resume or fork a run, and find a session's checkpoints after it ends.
- [Train multiple adapters on one base model](https://docs.together.ai/docs/post-training/multi-lora.md): Share one provisioned model resource across several adapter sessions to run hyperparameter sweeps at the cost of a single base model.
- [Monitor training runs](https://docs.together.ai/docs/post-training/monitoring.md): Track sessions and model resources in the console, and read the run artifacts and Weights & Biases metrics that show whether a training run is healthy.
- [Cookbook reference](https://docs.together.ai/docs/post-training/cookbook.md): The post-training cookbook's runnable recipes, and every field on its GRPO, SFT, and DPO config classes.
- [Migrate from Tinker](https://docs.together.ai/docs/post-training/migrate-from-tinker.md): Run a Tinker training script on Together AI post-training by changing its import line.
- [Deploy a checkpoint](https://docs.together.ai/docs/post-training/deployment.md): Deploy a trained model or LoRA adapter from an inference checkpoint to dedicated model i