Not found at this URL
https://cog.run/llms-full.txt ↗- Discovery
- Conventional sibling path checked
- HTTP response
- 404 · text/html; charset=utf-8
Artificial intelligence · llms.txt file breakdown
A directly retrieved Markdown index for Cog's documentation, with 203 links across 5 detected headings.
The actual file
The saved llms.txt text, displayed as source. The file belongs to its publisher; the live endpoint may change.
Showing the first 18000 characters. Load the complete file to read the remainder.
# Cog: Containers for machine learning
Cog is an open-source tool that lets you package machine learning models in a standard, production-ready container.
You can deploy your packaged model to your own infrastructure, or to [Replicate](https://replicate.com/).
## Highlights
- 📦 **Docker containers without the pain.** Writing your own `Dockerfile` can be a bewildering process. With Cog, you define your environment with a [simple configuration file](#how-it-works) and it generates a Docker image with all the best practices: Nvidia base images, efficient caching of dependencies, installing specific Python versions, sensible environment variable defaults, and so on.
- 🤬️ **No more CUDA hell.** Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.
- ✅ **Define the inputs and outputs for your model with standard Python.** Then, Cog generates an OpenAPI schema and validates the inputs and outputs.
- 🎁 **Automatic HTTP inference server**: Your model's types are used to dynamically generate a RESTful HTTP API using a high-performance Rust/Axum server.
- 🚀 **Ready for production.** Deploy your model anywhere that Docker images run. Your own infrastructure, or [Replicate](https://replicate.com).
## How it works
Define the Docker environment your model runs in with `cog.yaml`:
```yaml
build:
gpu: true
system_packages:
- "libgl1"
- "libglib2.0-0"
python_version: "3.13"
python_requirements: requirements.txt
run: "run.py:Runner"
```
Define how your model runs with `run.py`:
```python
from cog import BaseRunner, Input, Path
import torch
class Runner(BaseRunner):
def setup(self):
"""Load the model into memory to make running multiple inferences efficient"""
self.model = torch.load("./weights.pth")
# The arguments and types the model takes as input
def run(self,
image: Path = Input(description="Grayscale input image")
) -> Path:
"""Run the model"""
processed_image = preprocess(image)
output = self.model(processed_image)
return postprocess(output)
```
In the above we accept a path to the image as an input, and return a path to our transformed image after running it through our model.
Now, you can run the model:
```console
$ cog run -i image=@input.jpg
--> Building Docker image...
--> Running...
--> Output written to output.jpg
```
Or, build a Docker image for deployment:
```console
$ cog build -t my-classification-model
--> Building Docker image...
--> Built my-classification-model:latest
$ docker run -d -p 5000:5000 --gpus all my-classification-model
$ curl http://localhost:5000/predictions -X POST \
-H 'Content-Type: application/json' \
-d '{"input": {"image": "https://.../input.jpg"}}'
```
Or, combine build and run via the `serve` command:
```console
$ cog serve -p 8080
$ curl http://localhost:8080/predictions -X POST \
-H 'Content-Type: application/json' \
-d '{"input": {"image": "https://.../input.jpg"}}'
```
<!-- NOTE (bfirsh): Development environment instructions intentionally left out of readme for now, so as not to confuse the "ship a model to production" message.
In development, you can also run arbitrary commands inside the Docker environment:
```console
$ cog exec python train.py
...
```
Or, [spin up a Jupyter notebook](docs/notebooks.md):
```console
$ cog exec -p 8888 jupyter notebook --allow-root --ip=0.0.0.0
```
-->
## Why are we building this?
It's really hard for researchers to ship machine learning models to production.
Part of the solution is Docker, but it is so complex to get it to work: Dockerfiles, pre-/post-processing, Flask servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.
[Andreas](https://github.com/andreasjansson) and [Ben](https://github.com/bfirsh) created Cog. Andreas used to work at Spotify, where he built tools for building and deploying ML models with Docker. Ben worked at Docker, where he created [Docker Compose](https://github.com/docker/compose).
We realized that, in addition to Spotify, other companies were also using Docker to build and deploy machine learning models. [Uber](https://eng.uber.com/michelangelo-pyml/) and others have built similar systems. So, we're making an open source version so other people can do this too.
Hit us up if you're interested in using it or want to collaborate with us. [We're on Discord](https://discord.gg/replicate) or email us at [team@replicate.com](mailto:team@replicate.com).
## Prerequisites
- **macOS, Linux or Windows 11**. Cog works on macOS, Linux and Windows 11 with [WSL 2](docs/wsl2/wsl2.md)
- **Docker**. Cog uses Docker to create a container for your model. You'll need to [install Docker](https://docs.docker.com/get-docker/) before you can run Cog. If you install Docker Engine instead of Docker Desktop, you will need to [install Buildx](https://docs.docker.com/build/architecture/#buildx) as well.
## Install
Choose your platform for installation instructions.
<details>
<summary>macOS</summary>
The easiest way to install Cog on macOS is with Homebrew:
```console
brew install replicate/tap/cog
```
You can also use the install script:
```sh
# bash, zsh, and other shells
sh <(curl -fsSL https://cog.run/install.sh)
# fish shell
sh (curl -fsSL https://cog.run/install.sh | psub)
# download with wget and run in a separate command
wget -qO- https://cog.run/install.sh
sh ./install.sh
```
Or install manually:
```console
sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')"
sudo chmod +x /usr/local/bin/cog
sudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true
```
If you see a Gatekeeper warning saying the binary "cannot be opened because the developer cannot be verified", run:
```console
sudo xattr -d com.apple.quarantine /usr/local/bin/cog
```
</details>
<details>
<summary>Linux</summary>
You can install Cog using the install script:
```sh
# bash, zsh, and other shells
sh <(curl -fsSL https://cog.run/install.sh)
# fish shell
sh (curl -fsSL https://cog.run/install.sh | psub)
# download with wget and run in a separate command
wget -qO- https://cog.run/install.sh
sh ./install.sh
```
Or install manually:
```console
sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')"
sudo chmod +x /usr/local/bin/cog
```
</details>
<details markdown>
<summary>Windows</summary>
Cog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above.
</details>
<details>
<summary>Docker</summary>
To install Cog inside a Docker image:
```dockerfile
RUN sh -c "INSTALL_DIR=\"/usr/local/bin\" SUDO=\"\" $(curl -fsSL https://cog.run/install.sh)"
```
</details>
## Upgrade
If you're using macOS and you previously installed Cog with Homebrew, run the following:
```console
brew upgrade replicate/tap/cog
```
Otherwise, you can upgrade to the latest version by running the same commands you used to install it.
## Development
See [CONTRIBUTING.md](CONTRIBUTING.md) for how to set up a development environment and build from source.
## Next steps
- [Get started with an example model](docs/getting-started.md)
- [Get started with your own model](docs/getting-started-own-model.md)
- [Using Cog with notebooks](docs/notebooks.md)
- [Using Cog with Windows 11](docs/wsl2/wsl2.md)
- [Browse the example models in this repo](docs/examples.md)
- [Deploy models with Cog](docs/deploy.md)
- [`cog.yaml` reference](docs/yaml.md) to learn how to define your model's environment
- [Run interface reference](docs/python.md) to learn how the `Runner` interface works
- [Training interface reference](docs/training.md) to learn how to add a fine-tuning API to your model
- [HTTP API reference](docs/http.md) to learn how to use the HTTP API that models serve
## Need help?
[Join us in #cog on Discord.](https://discord.gg/replicate)
[](https://deepwiki.com/replicate/cog)
---
# Base images
Cog builds your model into a Docker image. To speed up builds and reduce cold boot times, Cog uses **prebuilt base images** by default. These images contain the common dependencies that most models need, so Cog doesn't have to install them from scratch every time you build or push.
## What is a Cog base image?
A Cog base image is a Docker image maintained by Replicate that includes:
- **Python runtime** — the Python version specified in your `cog.yaml`.
- **System libraries** — common dependencies for machine learning and media processing, including `ffmpeg`, `git`, `curl`, `build-essential`, `cmake`, OpenCV libraries, audio libraries (`sox`, `libsndfile1`), and graphics libraries (`libgl1`, `libsm6`, `libxext6`).
- **CUDA and PyTorch** (GPU images only) — the appropriate CUDA toolkit and PyTorch stack for your configuration.
Base images are tagged by their configuration, for example:
```
r8.im/cog-base:cuda12.4-python3.13-torch2.6.0
r8.im/cog-base:python3.13
```
When you run `cog build` or `cog push`, Cog selects the base image that matches your Python version, CUDA version, and PyTorch version. Because these images are pre-pulled on Replicate's infrastructure, models built on top of them start faster.
The entire list of available base images can be viewed at [https://r8.im/v2/cog-base/tags/list](https://r8.im/v2/cog-base/tags/list).
## Using the Cog base image
The `--use-cog-base-image` flag controls whether Cog uses a prebuilt base image. It is **enabled by default** on the following commands:
- [`cog build`](cli.md#cog-build)
- [`cog push`](cli.md#cog-push)
- [`cog run`](cli.md#cog-run)
- [`cog serve`](cli.md#cog-serve)
- [`cog exec`](cli.md#cog-exec)
Since it's on by default, you don't need to pass any flags:
```bash
cog push r8.im/your-username/my-model
```
This builds and pushes your model using a prebuilt Cog base image for faster cold boots.
## Disabling the Cog base image
If you encounter build issues or need a custom base layer, you can disable the Cog base image:
```bash
cog build --use-cog-base-image=false
```
When disabled, Cog generates a Dockerfile from scratch using either an NVIDIA CUDA base image or a slim Python base image, depending on the `--use-cuda-base-image` flag.
## Relationship to `--use-cuda-base-image`
The `--use-cuda-base-image` flag controls which underlying base image Cog uses **when the Cog base image is disabled**. It has no effect when `--use-cog-base-image` is enabled (the default), because the Cog base image already includes the appropriate CUDA and Python stack.
When you disable the Cog base image with `--use-cog-base-image=false`, Cog chooses the base image automatically:
- **GPU models** (`gpu: true` in `cog.yaml`): uses an NVIDIA CUDA base image.
- **CPU models**: uses a slim Python base image.
These flags — along with `--dockerfile` — are **mutually exclusive**: you can only set one of them explicitly on a given command. To customize the base image, disable the Cog base image and let Cog choose between CUDA and Python automatically.
## Troubleshooting
If `cog build` or `cog push` fails with the Cog base image enabled, try disabling it:
```bash
cog push --use-cog-base-image=false r8.im/your-username/my-model
```
This falls back to building from a standard CUDA or Python base image, which can help diagnose whether the issue is with the base image or your model's configuration.
---
# CLI reference
<!-- This file is auto-generated. Do not edit manually. -->
## `cog`
Containers for machine learning.
To get started, take a look at the documentation:
https://github.com/replicate/cog
**Examples**
```
To execute a command inside a Docker environment defined with Cog:
$ cog exec echo hello world
```
**Options**
```
--debug Show debugging output
-h, --help help for cog
--no-color Disable colored output
--version Show version of Cog
```
## `cog build`
Build a Docker image from the cog.yaml in the current directory.
The generated image contains your model code, dependencies, and the Cog
runtime. It can be run locally with 'cog run' or pushed to a registry
with 'cog push'.
```
cog build [flags]
```
**Examples**
```
# Build with default settings
cog build
# Build and tag the image
cog build -t my-model:latest
# Build without using the cache
cog build --no-cache
# Build with model weights in a separate layer
cog build --separate-weights -t my-model:v1
```
**Options**
```
-f, --file string The name of the config file. (default "cog.yaml")
-h, --help help for build
--no-cache Do not use cache when building the image
--openapi-schema string Load OpenAPI schema from a file
--progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default "auto")
--secret stringArray Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'
--separate-weights Separate model weights from code in image layers
-t, --tag string A name for the built image in the form 'repository:tag'
--use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)
--use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default "auto")
```
## `cog doctor`
Diagnose and fix common issues in your Cog project.
NOTE: cog doctor is experimental. Behavior and checks may change in future versions.
By default, cog doctor reports problems without modifying any files.
Pass --fix to automatically apply safe fixes.
```
cog doctor [flags]
```
**Options**
```
-f, --file string The name of the config file. (default "cog.yaml")
--fix Automatically apply fixes
-h, --help help for doctor
```
## `cog exec`
Execute a command inside a Docker environment defined by cog.yaml.
Cog builds a temporary image from your cog.yaml configuration and runs the
given command inside it. This is useful for debugging, running scripts, or
exploring the environment your model will run in.
```
cog exec <command> [arg...] [flags]
```
**Examples**
```
# Open a Python interpreter inside the model environment
cog exec python
# Run a script
cog exec python train.py
# Run with environment variables
cog exec -e HUGGING_FACE_HUB_TOKEN=abc123 python download.py
# Expose a port (e.g. for Jupyter)
cog exec -p 8888 jupyter notebook
```
**Options**
```
-e, --env stringArray Environment variables, in the form name=value
-f, --file string The name of the config file. (default "cog.yaml")
--gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.
-h, --help help for exec
--progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default "auto")
-p, --publish stringArray Publish a container's port to the host, e.g. -p 8000 or -p 0.0.0.0:8000
--use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)
--use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default "auto")
```
## `cog init`
Create a cog.yaml and run.py in the current directory.
These files provide a starting template for defining your model's environment
and run interface. Edit them to match your model's requirements.
```
cog init [flags]
```
**Examples**
```
# Set up a new Cog project in the current directory
cog init
```
**Options**
```
-h, --help help for init
```
## `cog login`
Log in to a container registry.
For Replicate's registry (r8.im), this command handles authentication
through Replicate's token-based flow.
For other registries, this command prompts for username and password,
then stores credentials using Docker's credential system.
```
cog login [flags]
```
**Options**
```
-h, --help help for login
--token-stdin Pass login token on stdin instead of opening a browser. You can find your Replicate login token at https://replicate.com/auth/token
```
## `cog playground`
Open a browser playground for talking to a running model.
Starts a local web server that serves a schema-driven UI (a Postman-like tool
for Cog models). Point it at any running Cog HTTP API -- for example one started
with 'cog serve' -- and the playground reflects that model's inputs and outputs
from its OpenAPI schema in real time.
Requests are reverse-proxied through this server, so the target API does not
need to set CORS headers. The server also hosts a webhook sink so async
predictions can be observed in the browser.
Async/webhook testing against a containerized model requires the webhook URL to
be reachable from inside the container. On Docker Desktop the default
'host.docker.internal' works once the server listens on a reachable interface
(e.g. --host 0.0.0.0).
```
cog playground [flags]
```
**Examples**
```
# Start a model API in one terminal
cog serve -p 8393
# Open the playground pointing at it
cog playground --target http://localhost:8393
```
**Options**
```
-h, --help help for playground
--host string Address to bind (use 0.0.0.0 to receive webhooks from containers) (default "127.0.0.1")
--no-open Measured from the saved file
Size measures the captured UTF-8 text, not compressed transfer size. Tokens are counted with a named tokenizer; other models can count differently. Neither measure establishes context compatibility or answer quality.
Context architecture
106 H2 sections organize 202 parsed Markdown links. 77 links point to .md or .markdown paths; 53 link occurrences use another hostname. Linked destinations have not been recursively validated.
| H2 section | Links | UTF-8 bytes | Tokens |
|---|---|---|---|
| HighlightsLine 7 | 2 | 1014 | 218 |
| How it worksLine 19 | 1 | 2180 | 552 |
| Why are we building this?Line 106 | 5 | 1088 | 245 |
| PrerequisitesLine 118 | 3 | 421 | 114 |
| InstallLine 123 | 1 | 1995 | 568 |
| UpgradeLine 210 | 0 | 241 | 52 |
| DevelopmentLine 220 | 1 | 108 | 28 |
| Next stepsLine 224 | 10 | 701 | 181 |
| Need help?Line 237 | 2 | 455 | 109 |
| What is a Cog base image?Line 251 | 1 | 1058 | 275 |
| Using the Cog base imageLine 270 | 5 | 504 | 151 |
| Disabling the Cog base imageLine 288 | 0 | 318 | 74 |
| Relationship to `--use-cuda-base-image`Line 298 | 0 | 767 | 174 |
| TroubleshootingLine 309 | 0 | 418 | 100 |
| `cog`Line 326 | 0 | 422 | 99 |
| `cog build`Line 349 | 0 | 1590 | 380 |
| H2 section | Links | UTF-8 bytes | Tokens |
|---|---|---|---|
| `cog doctor`Line 392 | 0 | 464 | 107 |
| `cog exec`Line 413 | 0 | 1551 | 391 |
| `cog init`Line 454 | 0 | 366 | 91 |
| `cog login`Line 478 | 0 | 512 | 115 |
| `cog playground`Line 499 | 0 | 1527 | 365 |
| `cog push`Line 542 | 0 | 2231 | 514 |
| `cog run`Line 592 | 0 | 2051 | 530 |
| `cog serve`Line 645 | 2 | 2683 | 647 |
| Build a Docker imageLine 715 | 0 | 221 | 53 |
| Push to a registryLine 726 | 0 | 3203 | 736 |
| Run the modelLine 803 | 0 | 921 | 244 |
| Make a predictionLine 842 | 1 | 3167 | 862 |
| Run a one-off predictionLine 962 | 0 | 429 | 108 |
| Health checksLine 976 | 1 | 425 | 100 |
| Stop the serverLine 986 | 0 | 255 | 66 |
| ConcurrencyLine 998 | 1 | 484 | 120 |
| Environment variablesLine 1013 | 1 | 388 | 85 |
| Next stepsLine 1022 | 3 | 350 | 70 |
| Build-time variablesLine 1035 | 0 | 3223 | 746 |
| CLI and local cache variablesLine 1112 | 0 | 1221 | 314 |
| Model reference and registry variablesLine 1162 | 0 | 1544 | 372 |
| Runtime server variablesLine 1212 | 0 | 4659 | 1123 |
| Push tuning variablesLine 1311 | 17 | 3483 | 842 |
| PrerequisitesLine 1403 | 3 | 404 | 110 |
| InitializationLine 1409 | 2 | 345 | 85 |
| Define the Docker environmentLine 1423 | 2 | 1596 | 332 |
| Define how to run your modelLine 1463 | 1 | 3031 | 753 |
| Using GPUsLine 1539 | 2 | 447 | 123 |
| Next stepsLine 1553 | 6 | 706 | 179 |
| PrerequisitesLine 1575 | 3 | 404 | 110 |
| Create a projectLine 1581 | 0 | 90 | 26 |
| Run commandsLine 1591 | 0 | 1187 | 216 |
| Run a modelLine 1626 | 0 | 2350 | 650 |
| Build an imageLine 1713 | 3 | 2755 | 690 |
| Next stepsLine 1789 | 6 | 3247 | 675 |
| Streaming predictions with server-sent eventsLine 1865 | 0 | 3885 | 942 |
| WebhooksLine 1996 | 0 | 1656 | 403 |
| Generating unique prediction IDsLine 2053 | 2 | 881 | 211 |
| File uploadsLine 2076 | 1 | 1531 | 391 |
| EndpointsLine 2144 | 8 | 7615 | 1937 |
| Install the jupyterlab Python packageLine 2421 | 1 | 215 | 62 |
| Run a notebookLine 2438 | 0 | 166 | 51 |
| Use notebook code in your runnerLine 2446 | 1 | 965 | 211 |
| Enable tracingLine 2478 | 0 | 867 | 197 |
| Getting started with tracingLine 2501 | 1 | 888 | 197 |
| Automatic spansLine 2537 | 0 | 1204 | 250 |
| Model-authored spansLine 2566 | 0 | 982 | 194 |
| Custom Python tracingLine 2589 | 0 | 2314 | 486 |
| Streaming predictionsLine 2640 | 0 | 651 | 123 |
| Caller-supplied trace tagsLine 2646 | 0 | 1213 | 294 |
| Calling Cog from another serviceLine 2685 | 0 | 689 | 174 |
| SamplingLine 2709 | 1 | 992 | 213 |
| Custom trace headersLine 2725 | 0 | 374 | 87 |
| Resource identityLine 2739 | 0 | 309 | 62 |
| Failure behaviorLine 2750 | 0 | 691 | 126 |
| What's nextLine 2760 | 0 | 268 | 51 |
| `pip.conf`Line 2771 | 0 | 428 | 98 |
| `cog.yaml`Line 2783 | 1 | 291 | 77 |
| BuildLine 2797 | 3 | 1425 | 351 |
| ContentsLine 2825 | 35 | 1409 | 423 |
| `BaseRunner`Line 2863 | 7 | 2921 | 665 |
| `async` runners and concurrencyLine 2925 | 0 | 1201 | 296 |
| `Input(**kwargs)`Line 2958 | 2 | 2015 | 484 |
| Deprecating inputsLine 2995 | 0 | 615 | 129 |
26 additional H2 sections are present in the source Markdown. This table previews the first 80 sections.
Link counts exclude parsed fenced and inline code and resolve relative destinations against the final endpoint. Section sizes cover the text below each H2 until the next H2.
Embedded material
Embedded example text and links to further documentation are counted separately. A code block does not establish that a sample runs or that an authentication flow succeeds.
Source line 133 · console
brew install replicate/tap/cogSource line 2785 · inline
pip installSource line 2805 · inline
pip installSource line 75 · console
$ curl http://localhost:5000/predictions -X POST \Source line 85 · console
$ curl http://localhost:8080/predictions -X POST \Source line 854 · console
curl http://localhost:5001/predictions -X POST \Source line 892 · console
curl http://localhost:5001/predictions -X POST \Source line 911 · bash
curl http://localhost:5001/predictions -X POST \Publisher prioritization
No H2 section named “Optional” was detected in this file.
The community proposal uses Optional by convention for secondary links that can be skipped for a shorter context. This label does not establish that everything outside it is critical, or that the material inside it is deprecated.
Machine-readable routes
No MCP or Model Context Protocol term was detected in the captured file. 0 links have MCP-related labels or paths. 0 fenced lines match mcpServers, mcp_servers or an mcp add command.
These are text and link observations. No MCP server connection, authentication or tool call was tested.
Index and bundle
Not found at checked URLs
A verified text response establishes a companion at the checked URL. It does not prove the bundle contains all product documentation, that generation is automated, or that it fits an assistant’s context window.
Change evidence
This capture establishes a baseline. Update frequency cannot be determined from one observation.
3 links have release, changelog, deprecation or migration terms in their labels or paths. These provide places to investigate product changes separately.
File changes can reveal documentation edits or navigation changes. They do not by themselves measure product release velocity, accuracy, agent adoption or AI visibility.
f7dd7b4bbf5bf15db8648e257998f6081f5dca34c0dac2fe67b3e057b0ae44aeCounts use a deterministic parser, not a spec-compliance grade. Unusual Markdown and indented code can fall outside its recognized patterns. See the llms.txt proposal and tokenizer implementation.