Fireworks AI llms.txt

The actual file

Source Markdown

The saved llms.txt text, displayed as source. The file belongs to its publisher; the live endpoint may change.

Showing the first 18000 characters. Load the complete file to read the remainder.

# Fireworks AI Docs

> Official Fireworks AI documentation for inference, deployments, training, Fireworks Nexus, APIs, SDKs, and CLI tools.

- [Build with Fireworks AI](https://docs.fireworks.ai/getting-started/introduction.md): Fast inference and training for open-source models on Fireworks AI, covering serverless APIs, dedicated deployments, fine-tuning, and multi-modal workflows.
- [Fireworks Nexus](https://docs.fireworks.ai/getting-started/nexus.md): Connect coding tools, agents, applications, and gateways to open models and Nexus Auto.
- [Serverless Quickstart](https://docs.fireworks.ai/getting-started/quickstart.md): Make your first serverless API call with GLM 5.3 Flash: create an API key and send a chat completion with curl or the Python SDK.
- [Deployments Quickstart](https://docs.fireworks.ai/getting-started/ondemand-quickstart.md): Deploy open-source models on dedicated GPUs on Fireworks in minutes, covering deployment creation, GPU sizing, and calling the endpoint from the API.
- [Which model should I use?](https://docs.fireworks.ai/guides/recommended-models.md): Pick the best Fireworks open-weight model for chat, coding, vision, embeddings, and agents, or migrate off Claude, GPT, and Gemini to comparable open models.
- [Concepts](https://docs.fireworks.ai/getting-started/concepts.md): Learn the core concepts for the Fireworks AI platform, including serverless inference, dedicated deployments, models, fine-tuning, and API keys.
- [Serverless Inference Overview](https://docs.fireworks.ai/serverless/overview.md): How Serverless inference works on Fireworks: serverless modes, per-token billing, request headers, prompt caching, model lifecycle, and when to pick Serverless.
- [Serverless Modes](https://docs.fireworks.ai/serverless/serverless-modes.md): Compare Standard, Priority, and Fast serverless modes on Fireworks Serverless to choose the right tradeoff between cost, latency, and throughput.
- [Serverless Pricing](https://docs.fireworks.ai/serverless/pricing.md): Per-token serverless pricing for text, vision, and embedding models on Fireworks, including Standard, Priority, and Fast serverless mode rates.
- [Reserved Throughput](https://docs.fireworks.ai/serverless/reserved-throughput.md): Reserve dedicated serverless throughput on Fireworks backed by an SLA, with guaranteed rate limits, additive capacity, and predictable per-token billing.
- [Serverless Rate Limits](https://docs.fireworks.ai/serverless/rate-limits.md): How adaptive rate limits work on Fireworks serverless inference: bounds, adjustment logic, and how RPS and TPS grow and shrink with your recent usage patterns.
- [US-only Serverless](https://docs.fireworks.ai/serverless/us-only-serverless.md): Run Fireworks Serverless inference exclusively on US infrastructure using US-scoped model routers like Kimi K3, for data-residency and compliance needs.
- [Deployments](https://docs.fireworks.ai/guides/ondemand-deployments.md): Create and manage on-demand Fireworks deployments on dedicated GPUs: choose accelerator shapes, autoscaling, replicas, and lifecycle controls for production.
- [Autoscaling](https://docs.fireworks.ai/deployments/autoscaling.md): Configure replica counts, scale-up/down windows, scale-to-zero, and load targets for on-demand Fireworks deployments using firectl flags.
- [Custom Models](https://docs.fireworks.ai/models/uploading-custom-models.md): Upload, verify, and deploy custom models on Fireworks from Hugging Face or local checkpoints, including supported architectures and quantization options.
- [Upload via REST API](https://docs.fireworks.ai/models/uploading-custom-models-api.md): Programmatically upload custom models to Fireworks using the REST API: create the model resource, upload weight shards, and finalize for dedicated deployment.
- [Quantization](https://docs.fireworks.ai/models/quantization.md): Reduce Fireworks model precision to FP16, FP8, BF16, or INT8 to cut cost by 30-50% and speed up inference, with firectl prepare-model examples.
- [Speculative Decoding](https://docs.fireworks.ai/deployments/speculative-decoding.md): Speed up token generation on Fireworks dedicated deployments with speculative decoding using draft models, n-gram speculation, and per-request configuration.
- [Regions](https://docs.fireworks.ai/deployments/regions.md): Multi-region defaults and single-region H100, H200, and B200 hardware and training availability on Fireworks, plus using and changing regions and quotas.
- [Routers](https://docs.fireworks.ai/deployments/routers.md): Distribute traffic across multiple deployments for A/B testing, traffic migration, and load distribution.
- [Reserved capacity](https://docs.fireworks.ai/deployments/reservations.md)
- [Performance benchmarking](https://docs.fireworks.ai/deployments/benchmarking.md): Measure and optimize your deployment's performance with load testing
- [Client-side performance optimization](https://docs.fireworks.ai/deployments/client-side-performance-optimization.md): Optimize your client code for maximum performance with dedicated deployments
- [Exporting Metrics](https://docs.fireworks.ai/deployments/exporting-metrics.md): Export metrics from your dedicated deployments to your observability stack
- [Deployment Tags](https://docs.fireworks.ai/deployments/deployment-tags.md): Attach key-value tags to dedicated Fireworks deployments to record environment, team, or workload, and manage them through firectl or the REST API.
- [Text Models](https://docs.fireworks.ai/guides/querying-text-models.md): Query, track, and manage inference for text models on Fireworks with chat completions, streaming, tool use, and structured output via the OpenAI-compatible API.
- [Spot Serverless](https://docs.fireworks.ai/guides/spot-serverless.md): Use lower-cost, best-effort serverless inference and fall back to Standard
- [Vision Models](https://docs.fireworks.ai/guides/querying-vision-language-models.md): Query Fireworks vision-language models with image inputs to analyze photos, screenshots, charts, and documents through the chat completions API and OpenAI SDKs.
- [Embeddings & Reranking](https://docs.fireworks.ai/guides/querying-embeddings-models.md): Generate text embeddings and rerank results on Fireworks using Voyage, Qwen, Nomic, and legacy models for semantic search, RAG, and retrieval pipelines.
- [Tool Calling](https://docs.fireworks.ai/guides/function-calling.md): Connect models to external tools and APIs
- [Reasoning](https://docs.fireworks.ai/guides/reasoning.md): How to enable reasoning on Fireworks chat and completion APIs, including reasoning_effort, extracting reasoning traces, and tuning depth for supported models.
- [Reliability and Error Handling](https://docs.fireworks.ai/guides/reliability.md): Recommended patterns for timeouts, retries, and error handling when building production applications on the Fireworks API.
- [Batch API](https://docs.fireworks.ai/guides/batch-inference.md): Use the Fireworks Batch API to run large-scale asynchronous inference jobs at a discount, with job-level system prompts, file inputs, and result retrieval.
- [Model Quality and Precision](https://docs.fireworks.ai/models/model-quality.md): How Fireworks tunes precision per model and validates every launch against the provider's official API before it serves production traffic.
- [Structured Outputs](https://docs.fireworks.ai/structured-responses/structured-response-formatting.md): Enforce Fireworks LLM output formats with JSON schemas or custom grammars, including supported schema features like pattern, length, size, and oneOf keywords.
- [Using predicted outputs](https://docs.fireworks.ai/guides/predicted-outputs.md): Use Predicted Outputs on Fireworks to boost token generation speed for code editing, document rewriting, and other tasks where most of the output is known.
- [Prompt caching](https://docs.fireworks.ai/guides/prompt-caching.md)
- [Inference for RL Rollouts](https://docs.fireworks.ai/guides/rollout-inference.md): Configure Fireworks inference deployments for RL rollout traffic, including session affinity, KV-cache behavior, weight-swap timing, and MoE Router Replay.
- [Video & Audio Inputs](https://docs.fireworks.ai/guides/video-audio-inputs.md): Query Fireworks multimodal models with video and audio inputs to transcribe, summarize, and reason over media directly through the chat completions API.
- [Completions API](https://docs.fireworks.ai/guides/completions-api.md): Use the completions API for raw text generation with custom prompt templates
- [Responses API](https://docs.fireworks.ai/guides/response-api.md): Use the Fireworks Responses API for stateful model calls with reasoning, tool calls, and persisted conversation state as an OpenAI-compatible alternative.
- [Inference Error Codes](https://docs.fireworks.ai/guides/inference-error-codes.md): Common error codes, their meanings, and resolutions for inference requests
- [Training Overview](https://docs.fireworks.ai/fine-tuning/finetuning-intro.md): Choose managed SFT, DPO, or ORPO, or run RL on the Training API, then pick serverless or dedicated infrastructure.
- [Agent Skills](https://docs.fireworks.ai/fine-tuning/agent/use-with-coding-agents.md): Install the Fireworks training Agent Skills in Claude Code, Cursor, Codex, or OpenCode so your agent can research, configure, and debug training runs.
- [Models](https://docs.fireworks.ai/fine-tuning/models.md): Base models supported for fine-tuning on Fireworks, with per-model training surfaces (managed, Training API, serverless, dedicated) and method availability.
- [Training cost estimator](https://docs.fireworks.ai/fine-tuning/cost-estimator.md): Estimate Managed Training, compare Fireworks Serverless with Dedicated, or compare Fireworks Dedicated with Tinker
- [Evaluating Trained Models](https://docs.fireworks.ai/fine-tuning/evaluating-fine-tuned-models.md): Evaluate a trained Fireworks model with in-session sampling during a serverless training run or a preemptible deployment for a promoted model.
- [Deploying Trained Models](https://docs.fireworks.ai/fine-tuning/deploying-loras.md): Deploy LoRA adapters trained or imported on Fireworks to on-demand deployments using live merge for one adapter or multi-LoRA for many adapters.
- [Managed Training Overview](https://docs.fireworks.ai/fine-tuning/managed-finetuning-intro.md): Fine-tune open-weight models on Fireworks-managed infrastructure with no custom training code, using supervised fine-tuning or preference optimization.
- [Supervised Fine-Tuning - Text](https://docs.fireworks.ai/fine-tuning/fine-tuning-models.md): Fine-tune Fireworks-hosted text LLMs with supervised learning on your labeled datasets, including LoRA and full-parameter options, formats, and deployment.
- [Thinking history in training](https://docs.fireworks.ai/fine-tuning/thinking-history.md): Choose how prior turn reasoning is rendered in Fireworks managed fine-tuning datasets so training behavior matches inference for thinking-capable models.
- [Preference Optimization with DPO or ORPO](https://docs.fireworks.ai/fine-tuning/dpo-fine-tuning.md): Train Fireworks-hosted models on preferred and non-preferred response pairs using managed DPO or ORPO preference optimization, with LoRA or full-parameter.
- [Overview](https://docs.fireworks.ai/fine-tuning/training-api/introduction.md): The Fireworks Training API lets you write custom Python training loops with full control over objectives while Fireworks manages distributed GPU infrastructure.
- [Serverless Training](https://docs.fireworks.ai/fine-tuning/training-api/serverless.md): Run LoRA training, preference optimization, and RL on the shared Fireworks Training API trainer pool with no provisioning and per-token pricing.
- [Dedicated Training](https://docs.fireworks.ai/fine-tuning/training-api/dedicated.md): Run Training API workloads with provisioned trainer and deployment resources on Fireworks, with explicit checkpoints, weight-sync, and lifecycle control.
- [Training Shapes](https://docs.fireworks.ai/fine-tuning/training-api/training-shapes.md): Training shapes are pre-configured GPU and model profiles that fix distributed training layout, context limits, and trainer images for Fireworks trainer jobs.
- [The Cookbook](https://docs.fireworks.ai/fine-tuning/training-api/cookbook/overview.md): Overview of the Training API cookbook: ready-to-run recipes for supervised fine-tuning, DPO, GRPO reinforcement learning, and distillation on Fireworks.
- [Cookbook: Muse Glimmer First Fine-Tune](https://docs.fireworks.ai/fine-tuning/training-api/cookbook/muse-glimmer.md): Run a free first LoRA fine-tune of Muse Glimmer 30B on Managed SFT or Serverless Training, then measure base versus tuned.
- [Cookbook: Supervised Fine-Tuning](https://docs.fireworks.ai/fine-tuning/training-api/cookbook/sft.md): Run supervised fine-tuning on Fireworks using the cookbook sft_loop recipe to handle dataset loading, tokenization, gradient accumulation, and checkpointing.
- [Cookbook: Preference Optimization](https://docs.fireworks.ai/fine-tuning/training-api/cookbook/dpo.md): Run Direct Preference Optimization (DPO) on Fireworks with the Training API cookbook recipe, using pairwise preference data and configurable reference models.
- [Cookbook: Reinforcement Learning](https://docs.fireworks.ai/fine-tuning/training-api/cookbook/rl.md): Run the experimental async GRPO cookbook recipe with a custom rollout function while Fireworks handles scheduling, training, checkpoints, and weight sync.
- [Cookbook: Agentic Reinforcement Learning](https://docs.fireworks.ai/fine-tuning/training-api/cookbook/agentic-rl.md): Run tool-using agents against environment-local TITO sidecars on the Fireworks Training API to return exact-token trajectories to async reinforcement learning.
- [Cookbook: Fireworks x HUD RL Training](https://docs.fireworks.ai/fine-tuning/training-api/cookbook/hud-rl-training.md): Train a LoRA adapter with reinforcement learning using HUD environments and Fireworks Serverless Training.
- [Cookbook: Distillation](https://docs.fireworks.ai/fine-tuning/training-api/cookbook/distillation.md): Run on-policy distillation on Fireworks using cookbook recipes that combine frozen teacher scoring, student sampling, and reward-shaped loss.
- [Cookbook: Reference](https://docs.fireworks.ai/fine-tuning/training-api/cookbook/reference.md): Reference for Training API cookbook configuration classes, TrainerConfig options, checkpoint helpers, and advanced recipe knobs used across SFT, DPO, and RL.
- [RL Rollouts with Your Own Trainer](https://docs.fireworks.ai/fine-tuning/rl-rollout-integration.md): Integrate an external RL trainer with Fireworks inference: hot-load new checkpoints from your bucket and run rollouts via the OpenAI-compatible API.
- [RL rollout cost comparison vs Tinker](https://docs.fireworks.ai/fine-tuning/multi-turn-cost-comparison.md): Estimate multi-turn agentic RL rollout cost on Fireworks and compare to Tinker per-token pricing with worked examples for GSM8K, SWE-bench, and browsing agents.
- [Fire Pass](https://docs.fireworks.ai/firepass.md): Set up Fire Pass to access open-weight model routers like kimi-fast-latest for agentic coding in Claude Code, OpenCode, Cline, Kilo Code, and LangChain.
- [Managing users](https://docs.fireworks.ai/accounts/users.md): Add, delete, and manage roles for users in your Fireworks account, including admin, member, viewer, and service account roles for API keys and dashboard access.
- [Custom SSO](https://docs.fireworks.ai/accounts/sso.md): Set up custom Single Sign-On (SSO) authentication and SCIM user provisioning for Fireworks accounts, including SAML setup, group mapping, and lifecycle sync.
- [Service Accounts](https://docs.fireworks.ai/accounts/service-accounts.md): Create Fireworks service accounts with firectl, issue API keys on their behalf, assign and update roles, and track their billing and audit log activity.
- [Account quotas](https://docs.fireworks.ai/guides/quotas_usage/account-quotas.md): Configure Fireworks account-wide request limits, spending tiers, monthly budget caps, and on-demand GPU quotas to control usage and costs.
- [Exporting Billing Metrics](https://docs.fireworks.ai/accounts/exporting-billing-metrics.md): Export Fireworks billing and usage metrics as a per-event CSV dump across serverless, on-demand deployments, and fine-tuning for downstream reporting.
- [Exporting Usage Costs](https://docs.fireworks.ai/accounts/exporting-usage-costs.md): Export rated Fireworks serverless usage costs to CSV using firectl billing export-usage-costs, grouped by model, model tier, user, or API key for reporting.
- [Usage & Cost Breakdown](https://docs.fireworks.ai/accounts/exporting-usage-and-costs.md): Break down Fireworks usage and rated costs by deployment, model, API key, or custom tags using firectl, billingUsage, and the usageCosts query API.
- [Enterprise features](https://docs.fireworks.ai/accounts/enterprise-features.md): Administrative capabilities available to Fireworks Enterprise accounts, including SSO, role-based access control, audit logs, model access policies, and quotas.
- [Model access policy](https://docs.fireworks.ai/accounts/model-access-policy.md): Restrict which models users on your Fireworks Enterprise account can access for inference, deployments, and fine-tuning using an admin-configured allowlist.
- [Data residency](https://docs.fireworks.ai/accounts/data-residency.md): Restrict inference for your Fireworks Enterprise account to a selected region such as US-only, with regional API endpoints, SDK configuration, and compliance.
- [Zero Data Retention policy](https://docs.fireworks.ai/accounts/zero-data-retention.md): Ensure no user actions on your account can persist or log customer content
- [Data Security](https://docs.fireworks.ai/guides/security_compliance/data_security.md): How Fireworks handles data for inference and training: zero data retention, encryption in transit and at rest, BYOB storage, and complianc