LiteLLM llms.txt

The actual file

Source Markdown

The saved llms.txt text, displayed as source. The file belongs to its publisher; the live endpoint may change.

Showing the first 18000 characters. Load the complete file to read the remainder.

# LiteLLM

> LiteLLM is an open-source Python SDK and a self-hosted AI Gateway (proxy) that give 100+ LLM providers one OpenAI-compatible API, with virtual keys, spend tracking, budgets, routing and fallbacks, guardrails, and an MCP gateway.

- Python SDK: `uv add litellm` (or `pip install litellm`), then `litellm.completion(model="<provider>/<model>", messages=[...])`. Responses always use the OpenAI Chat Completions shape.
- AI Gateway: Docker image `docker.litellm.ai/berriai/litellm`. Keys, the Admin UI (`/ui`), and spend tracking need Postgres (`DATABASE_URL`) and a `LITELLM_MASTER_KEY`. Any OpenAI or Anthropic SDK works against it by changing the base URL.
- Enterprise: a license key (`LITELLM_LICENSE`) on the same gateway image. Contact: https://www.litellm.ai/enterprise#talk-to-sales
- Every docs page is available as markdown by appending `.md` to its URL, for example https://docs.litellm.ai/docs/proxy/docker_quick_start.md. The full docs are in https://docs.litellm.ai/llms-full.txt.

## LiteLLM at a glance

LiteLLM is the most widely used and most secure open-source AI gateway, trusted by teams at Netflix, Okta, Ramp, NASA, Zurich, Cloudera, AT&T, and Lemonade (https://www.litellm.ai/enterprise). Figures as of 2026-10-22; each links to its source for current values.

- Adoption: 61.8k GitHub stars and 12.6k forks (https://github.com/BerriAI/litellm), with 1,800+ contributors.
- Usage: 92.2 million downloads of the `litellm` Python package in the last month (https://pypistats.org/packages/litellm).
- Container image: 516M pulls of the LiteLLM Gateway image on GitHub Container Registry and Docker Hub (https://github.com/BerriAI/litellm/pkgs/container/litellm).
- Security: SOC 2 Type II audited (https://trust.litellm.ai/). Every Docker image on GHCR is signed with cosign and can be verified before it runs (https://docs.litellm.ai/docs/proxy/docker_image_security.md). Enterprise support includes a 72-hour security patch SLA (https://docs.litellm.ai/docs/enterprise.md).
- Deployment: self-hosted in your own cloud, so prompts, responses, and provider keys stay in your infrastructure (https://docs.litellm.ai/docs/data_security.md).
- License: open source under MIT; Enterprise features need a license key (https://github.com/BerriAI/litellm/blob/main/LICENSE).

## Key pages

- [Python SDK quickstart](https://docs.litellm.ai/docs/index.md): `uv add litellm`, then call any provider with `completion()`
- [AI Gateway quickstart](https://docs.litellm.ai/docs/proxy/docker_quick_start.md): Run the gateway with Docker and Postgres, add a model, create a virtual key
- [Agent resources](https://docs.litellm.ai/docs/agent_resources.md): Copy-paste prompts, MCP, and skills for coding agents setting up LiteLLM
- [Connect clients to the gateway](https://docs.litellm.ai/docs/proxy/client_setup/overview.md): Point Claude Code, Codex, Cursor, or any OpenAI or Anthropic SDK at the gateway
- [Gateway config.yaml reference](https://docs.litellm.ai/docs/proxy/configs.md): Every setting the gateway config file supports
- [Production deployment](https://docs.litellm.ai/docs/proxy/deploy.md): Helm, Terraform, and Kubernetes on AWS, GCP, and Azure
- [LiteLLM Enterprise](https://docs.litellm.ai/docs/enterprise.md): What Enterprise adds on top of the open-source gateway: SSO, audit logs, admin roles, SLAs

## LiteLLM AI Gateway

- [LiteLLM AI Gateway](https://docs.litellm.ai/docs/simple_proxy.md): The LiteLLM AI Gateway is a self-hosted server that gives your applications one OpenAI-compatible endpoint for 100+ LLM providers, MCP tools, and A2A agents.

## AI Gateway

- [CLI - Quick Start](https://docs.litellm.ai/docs/proxy/quick_start.md): Setup LiteLLM Proxy quickly via CLI.
- [Benchmarks](https://docs.litellm.ai/docs/benchmarks.md): Performance test results for LiteLLM Gateway.
- [Production Best Practices](https://docs.litellm.ai/docs/proxy/prod.md): Checklist for running LiteLLM in production; configuration, sizing and workers, Redis, and database and migrations.
- [Upgrade Guide](https://docs.litellm.ai/docs/proxy/upgrade_guide.md): Upgrade LiteLLM with Helm or Terraform. Run schema migrations and request-log indexing in a separate job before the gateway rollout.
- [Health Checks](https://docs.litellm.ai/docs/proxy/health.md): Two things consume the proxy's health state. Your orchestrator (Kubernetes, a load balancer, an uptime monitor) polls lightweight probe endpoints to decide whether the process is...
- [What Needs Redis](https://docs.litellm.ai/docs/proxy/redis_requirements.md): Features that do not work, or do not work correctly, when the LiteLLM proxy runs without Redis.
- [Database Sizing](https://docs.litellm.ai/docs/proxy/db_sizing.md): How to size Postgres for LiteLLM Proxy, with instance recommendations for AWS, Azure, and GCP.
- [Redis Sizing](https://docs.litellm.ai/docs/proxy/redis_sizing.md): How to size Redis for LiteLLM Proxy, with instance recommendations for AWS, Azure, and GCP.
- [Server Tuning](https://docs.litellm.ai/docs/proxy/server_tuning.md): Optional deep tuning for the LiteLLM Proxy container; alternative ASGI servers, worker recycling, hitless restarts, TLS, keepalive, and loading config from object storage.
- [Scale for 100T+ tokens per month](https://docs.litellm.ai/docs/proxy/high_throughput.md): Benchmark throughput: 13.5 billion tokens per minute (TPM). View results.
- [Database Read Replica](https://docs.litellm.ai/docs/proxy/db_read_replica.md): LiteLLM Proxy can route read-only queries to a separate database endpoint while
- [Multi-Region Deployment](https://docs.litellm.ai/docs/proxy/multi_region.md): Run LiteLLM proxy instances in multiple regions of the same cloud provider, all connected to one shared PostgreSQL database. Clients get routed to the nearest region for low...
- [Global Control Plane](https://docs.litellm.ai/docs/proxy/global_control_plane.md): Deploy a single LiteLLM UI that manages multiple independent LiteLLM proxy instances, each with its own database, Redis, and master key.
- [Migrate from Kubernetes Manifests to the Microservices Chart](https://docs.litellm.ai/docs/proxy/manifests_to_microservices.md): Move a LiteLLM proxy deployed with raw Kubernetes manifests onto the componentized litellm Helm chart so gateway, backend, and UI scale independently.
- [Secret Managers Overview](https://docs.litellm.ai/docs/secret_managers/overview.md): LiteLLM supports reading secrets (eg. OPENAI_API_KEY) and writing secrets (eg. Virtual Keys) from Azure Key Vault, Google Secret Manager, Hashicorp Vault, CyberArk Conjur, and...
- [AWS Secret Manager](https://docs.litellm.ai/docs/secret_managers/aws_secret_manager.md): Store your proxy keys in AWS Secret Manager.
- [AWS Key Management V1](https://docs.litellm.ai/docs/secret_managers/aws_kms.md): BETA] AWS Key Management v2 is on the enterprise tier. Go [here for docs
- [Azure Key Vault](https://docs.litellm.ai/docs/secret_managers/azure_key_vault.md): Usage with LiteLLM Proxy Server
- [CyberArk Conjur](https://docs.litellm.ai/docs/secret_managers/cyberark.md): | Feature | Support | Description |
- [Google Secret Manager](https://docs.litellm.ai/docs/secret_managers/google_secret_manager.md): Support for Google Secret Manager
- [Google Key Management Service](https://docs.litellm.ai/docs/secret_managers/google_kms.md): Use encrypted keys from Google KMS on the proxy
- [Hashicorp Vault](https://docs.litellm.ai/docs/secret_managers/hashicorp_vault.md): | Feature | Support | Description |
- [Custom Secret Manager](https://docs.litellm.ai/docs/secret_managers/custom_secret_manager.md): Integrate your custom secret management system with LiteLLM.
- [Security Best Practices](https://docs.litellm.ai/docs/proxy/security_best_practices.md): Security is a top priority at LiteLLM. Use the following practices for production and enterprise deployments. For a control-by-control view against the OWASP Top 10 for LLM...
- [OWASP LLM Top 10 (2026) mapping](https://docs.litellm.ai/docs/proxy/security_owasp_llm_top10.md): This page maps every item of the OWASP Top 10 for LLM Applications 2026 to the LiteLLM gateway controls that address it, says how each control is turned on, and states where each...
- [[Beta] Rust AI Gateway](https://docs.litellm.ai/docs/proxy/rust_gateway.md): Run LiteLLM request translation on the Rust core. Enable it per model on your existing Python server, or run the standalone Axum server.
- [config_settings](https://docs.litellm.ai/docs/proxy/config_settings.md)
- [File Management](https://docs.litellm.ai/docs/proxy/config_management.md): include external YAML files in a config.yaml
- [CLI Arguments](https://docs.litellm.ai/docs/proxy/cli.md): This page documents all command-line interface (CLI) arguments available for the LiteLLM proxy server.
- [SSL, HTTP Proxy Security Settings](https://docs.litellm.ai/docs/guides/security_settings.md): If you're in an environment using an older TTS bundle, with an older encryption, follow this guide. By default
- [Model Management](https://docs.litellm.ai/docs/proxy/model_management.md): When STOREMODELIN_DB is on, your models live in the database rather than in a static config.yaml. That means day-2 changes happen right in the Admin UI: adding a model, editing...
- [How Model Access Works](https://docs.litellm.ai/docs/proxy/model_access_guide.md)
- [Restrict Model Access](https://docs.litellm.ai/docs/proxy/model_access.md): Restrict models by Virtual Key
- [Model Access Groups](https://docs.litellm.ai/docs/proxy/model_access_groups.md)
- [Access Groups](https://docs.litellm.ai/docs/proxy/access_groups.md): Access Groups simplify how you define and manage resource access across your organization. Instead of configuring models, MCP servers, and agents separately on each key or team,...
- [Allow Teams to Add Models](https://docs.litellm.ai/docs/proxy/team_model_add.md): Allow team to add a their own models/key for that project - so any OpenAI call they make uses their OpenAI key.
- [Per-Team/Project Credential Routing](https://docs.litellm.ai/docs/proxy/credential_routing.md): Route the same model to different LLM provider endpoints (e.g. different Azure instances) based on which team or project makes the request.
- [Claude Desktop & Cowork SSO](https://docs.litellm.ai/docs/tutorials/claude_desktop_cowork.md): Configure Claude Desktop and Cowork to use LiteLLM through third-party inference with SSO.
- [Claude Desktop & Cowork with virtual keys](https://docs.litellm.ai/docs/proxy/client_setup/claude_desktop.md): Claude Desktop and Cowork use the same gateway settings. This guide configures them with a LiteLLM virtual key; the connection also applies to Code sessions and to Chat when...
- [Claude Code SSO](https://docs.litellm.ai/docs/tutorials/claude_code_gateway.md): Set up Claude Code with Claude apps gateway sign-in, Lite CLI SSO, or a Claude subscription through LiteLLM.
- [Claude Code (CLI)](https://docs.litellm.ai/docs/proxy/client_setup/claude_code.md): Claude Code talks to the Anthropic Messages API. LiteLLM serves that format at /v1/messages, so two environment variables point Claude Code at the gateway and it works against...
- [Codex Desktop with SSO](https://docs.litellm.ai/docs/proxy/client_setup/codex_chatgpt_desktop_sso.md): Codex in ChatGPT Desktop reads the same ~/.codex/config.toml as the Codex CLI and runs the same authentication helper. For app onboarding, model selection, and MCP configuration,...
- [Codex (ChatGPT Desktop)](https://docs.litellm.ai/docs/proxy/client_setup/codex_chatgpt_desktop.md): Select Codex in the ChatGPT desktop app. It reads the same ~/.codex/config.toml as the Codex CLI, so the desktop and CLI can share a LiteLLM connection.
- [Codex CLI with SSO](https://docs.litellm.ai/docs/proxy/client_setup/codex_cli_sso.md): Use your organization's LiteLLM SSO login to connect Codex CLI to the gateway. For installation, model selection, and MCP configuration, see the Codex CLI setup guide.
- [Codex (CLI)](https://docs.litellm.ai/docs/proxy/client_setup/codex_cli.md): Codex reads all of its configuration from ~/.codex/config.toml. You define LiteLLM as a custom model provider there and register the MCP gateway in the same file. This is the...
- [Hermes CLI with SSO](https://docs.litellm.ai/docs/proxy/client_setup/hermes_sso.md): Hermes can call lite auth print-token through its keycmd provider setting. Each developer signs in to LiteLLM, and Hermes obtains the resulting token when it needs a credential.
- [Hermes CLI with virtual keys](https://docs.litellm.ai/docs/proxy/client_setup/hermes.md): Hermes Agent can use LiteLLM as a named custom provider. This setup sends model requests to the gateway's /v1/chat/completions endpoint with your LiteLLM virtual key.
- [Cursor Integration](https://docs.litellm.ai/docs/tutorials/cursor_integration.md): Route Cursor IDE requests through LiteLLM for unified logging, budget controls, and access to any model. To connect Cursor to MCP tools, use the direct-client URL and headers in...
- [Gemini CLI](https://docs.litellm.ai/docs/tutorials/litellm_gemini_cli.md): This tutorial shows you how to integrate the Gemini CLI with LiteLLM Proxy, allowing you to route requests through LiteLLM's unified interface.
- [GitHub Copilot / VS Code](https://docs.litellm.ai/docs/tutorials/github_copilot_integration.md): This tutorial shows two ways to use LiteLLM Proxy from VS Code's chat. The LiteLLM extension adds your gateway as a language model provider, so the chat model picker lists the...
- [OpenClaw + LiteLLM Integration](https://docs.litellm.ai/docs/tutorials/openclaw_integration.md): OpenClaw is a self-hosted AI assistant that connects chat apps (WhatsApp, Telegram, Discord, and more) to LLM providers. By routing OpenClaw through LiteLLM Proxy, you get access...
- [OpenCode Quickstart](https://docs.litellm.ai/docs/tutorials/opencode_integration.md): This tutorial shows how to connect OpenCode to your existing LiteLLM instance and switch between models.
- [Qwen Code CLI](https://docs.litellm.ai/docs/tutorials/litellm_qwen_code_cli.md): This tutorial shows you how to integrate the Qwen Code CLI with LiteLLM Proxy, allowing you to route requests through LiteLLM's unified interface.
- [Langchain, OpenAI SDK, LlamaIndex, Instructor, Curl examples](https://docs.litellm.ai/docs/proxy/user_keys.md): LiteLLM Proxy is OpenAI-Compatible, and supports:
- [Request Headers](https://docs.litellm.ai/docs/proxy/request_headers.md): Special headers that are supported by LiteLLM.
- [Response Headers](https://docs.litellm.ai/docs/proxy/response_headers.md): When you make a request to the proxy, the proxy will return the following headers:
- [Forward Client Headers to LLM API](https://docs.litellm.ai/docs/proxy/forward_client_headers.md): Control which model groups can forward client headers to the underlying LLM provider APIs.
- [Clientside LLM Credentials](https://docs.litellm.ai/docs/proxy/clientside_auth.md): Pass User LLM API Keys, Fallbacks
- [Model Discovery](https://docs.litellm.ai/docs/proxy/model_discovery.md): Use this to give users an accurate list of models available behind provider endpoint, when calling /v1/models for wildcard models.
- [CLI Authentication](https://docs.litellm.ai/docs/proxy/cli_sso.md): Use the litellm cli to authenticate to the LiteLLM Gateway. This is great if you're trying to give a large number of developers self-serve access to the LiteLLM Gateway.
- [Claude Code with Okta SSO (JWT Auth)](https://docs.litellm.ai/docs/tutorials/claude_code_okta_sso.md): Route Claude Code through LiteLLM using each developer's Okta identity. Claude Code sends the developer's own Okta access token with every request, LiteLLM validates it against...
- [Claude Code with Bring Your Own Key (BYOK)](https://docs.litellm.ai/docs/tutorials/claude_code_byok.md): Use Claude Code with your own Anthropic credentials through the LiteLLM proxy. Claude Code authenticates to Anthropic one of two ways Bearer (tied to your Claude.ai Free, Pro,...
- [Using Claude Code with a Claude Subscription (Pro, Max, Team, Enterprise)](https://docs.litellm.ai/docs/tutorials/claude_code_max_subscription.md): Route Claude Code traffic signed in with a Claude Pro, Max, Team, or Enterprise subscription through LiteLLM AI Gateway.
- [Virtual Keys](https://docs.litellm.ai/docs/proxy/virtual_keys.md): Track Spend, and control model access via virtual keys for the proxy
- [Role-based Access Controls (RBAC)](https://docs.litellm.ai/docs/proxy/access_control.md): Role-based access control (RBAC) is based on Organizations, Teams and Internal User Roles
- [SSO for Admin UI](https://docs.litellm.ai/docs/proxy/admin_ui_sso.md): From v1.76.0, SSO is free for up to 5 users. Beyond that, an enterprise license is required.
- [SAML 2.0 SSO](https://docs.litellm.ai/docs/proxy/saml_sso.md): SSO is free for up to 5 users. Beyond that, an enterprise license is required.
- [Event Hooks for SSO Login](https://docs.litellm.ai/docs/proxy/custom_sso.md): SSO is free for up to 5 users. Beyond that, an enterprise license is required.
- [SCIM with LiteLLM](https://docs.litellm.ai/docs/tutorials/scim_litellm.md): Enterprise: SCIM support requires a premium license.
- [OIDC - JWT-based Auth](https://docs.litellm.ai/docs/proxy/token_auth.md): Use JWT's to auth admins / users / projects into the proxy.
- [JWT → Virtual Key Mapping](https://docs.litellm.ai/docs/proxy/jwt_key_mapping.md): Map JWT tokens to LiteLLM virtual keys, so every JWT client gets the same granular controls as a virtual key: model restrictions, spend limits, rate limits, guardrails, and full...
- [Oauth 2.0 Authentication](https://docs.litellm.ai/docs/proxy/oauth2.md): Use this if you want to use an Oauth2.0 token to make /chat, /embeddings requests to the LiteLLM Proxy
- [Control Model Access with OIDC (Azure AD/Keycloak/etc.)](https://docs.litellm.ai/docs/proxy/jwt_auth_arch.md)
- [[BETA] OpenID Connect (OIDC)](https://docs.litellm.ai/docs/oidc.md): LiteLLM supports using OpenID C