Skip to content
J·BJon BishopFractional growth marketer

Artificial intelligence · llms.txt file breakdown

Cog llms.txt

A directly retrieved Markdown index for Cog's documentation, with 203 links across 5 detected headings.

The actual file

Source Markdown

The saved llms.txt text, displayed as source. The file belongs to its publisher; the live endpoint may change.

Open saved text ↗

Showing the first 18000 characters. Load the complete file to read the remainder.

# Cog: Containers for machine learning

Cog is an open-source tool that lets you package machine learning models in a standard, production-ready container.

You can deploy your packaged model to your own infrastructure, or to [Replicate](https://replicate.com/).

## Highlights

- 📦 **Docker containers without the pain.** Writing your own `Dockerfile` can be a bewildering process. With Cog, you define your environment with a [simple configuration file](#how-it-works) and it generates a Docker image with all the best practices: Nvidia base images, efficient caching of dependencies, installing specific Python versions, sensible environment variable defaults, and so on.

- 🤬️ **No more CUDA hell.** Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.

- ✅ **Define the inputs and outputs for your model with standard Python.** Then, Cog generates an OpenAPI schema and validates the inputs and outputs.

- 🎁 **Automatic HTTP inference server**: Your model's types are used to dynamically generate a RESTful HTTP API using a high-performance Rust/Axum server.

- 🚀 **Ready for production.** Deploy your model anywhere that Docker images run. Your own infrastructure, or [Replicate](https://replicate.com).

## How it works

Define the Docker environment your model runs in with `cog.yaml`:

```yaml
build:
  gpu: true
  system_packages:
    - "libgl1"
    - "libglib2.0-0"
  python_version: "3.13"
  python_requirements: requirements.txt
run: "run.py:Runner"
```

Define how your model runs with `run.py`:

```python
from cog import BaseRunner, Input, Path
import torch

class Runner(BaseRunner):
    def setup(self):
        """Load the model into memory to make running multiple inferences efficient"""
        self.model = torch.load("./weights.pth")

    # The arguments and types the model takes as input
    def run(self,
          image: Path = Input(description="Grayscale input image")
    ) -> Path:
        """Run the model"""
        processed_image = preprocess(image)
        output = self.model(processed_image)
        return postprocess(output)
```

In the above we accept a path to the image as an input, and return a path to our transformed image after running it through our model.

Now, you can run the model:

```console
$ cog run -i image=@input.jpg
--> Building Docker image...
--> Running...
--> Output written to output.jpg
```

Or, build a Docker image for deployment:

```console
$ cog build -t my-classification-model
--> Building Docker image...
--> Built my-classification-model:latest

$ docker run -d -p 5000:5000 --gpus all my-classification-model

$ curl http://localhost:5000/predictions -X POST \
    -H 'Content-Type: application/json' \
    -d '{"input": {"image": "https://.../input.jpg"}}'
```

Or, combine build and run via the `serve` command:

```console
$ cog serve -p 8080

$ curl http://localhost:8080/predictions -X POST \
    -H 'Content-Type: application/json' \
    -d '{"input": {"image": "https://.../input.jpg"}}'
```

<!-- NOTE (bfirsh): Development environment instructions intentionally left out of readme for now, so as not to confuse the "ship a model to production" message.

In development, you can also run arbitrary commands inside the Docker environment:

```console
$ cog exec python train.py
...
```

Or, [spin up a Jupyter notebook](docs/notebooks.md):

```console
$ cog exec -p 8888 jupyter notebook --allow-root --ip=0.0.0.0
```
-->

## Why are we building this?

It's really hard for researchers to ship machine learning models to production.

Part of the solution is Docker, but it is so complex to get it to work: Dockerfiles, pre-/post-processing, Flask servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.

[Andreas](https://github.com/andreasjansson) and [Ben](https://github.com/bfirsh) created Cog. Andreas used to work at Spotify, where he built tools for building and deploying ML models with Docker. Ben worked at Docker, where he created [Docker Compose](https://github.com/docker/compose).

We realized that, in addition to Spotify, other companies were also using Docker to build and deploy machine learning models. [Uber](https://eng.uber.com/michelangelo-pyml/) and others have built similar systems. So, we're making an open source version so other people can do this too.

Hit us up if you're interested in using it or want to collaborate with us. [We're on Discord](https://discord.gg/replicate) or email us at [team@replicate.com](mailto:team@replicate.com).

## Prerequisites

- **macOS, Linux or Windows 11**. Cog works on macOS, Linux and Windows 11 with [WSL 2](docs/wsl2/wsl2.md)
- **Docker**. Cog uses Docker to create a container for your model. You'll need to [install Docker](https://docs.docker.com/get-docker/) before you can run Cog. If you install Docker Engine instead of Docker Desktop, you will need to [install Buildx](https://docs.docker.com/build/architecture/#buildx) as well.

## Install

Choose your platform for installation instructions.

<details>
  <summary>macOS</summary>

The easiest way to install Cog on macOS is with Homebrew:

```console
brew install replicate/tap/cog
```

You can also use the install script:

```sh
# bash, zsh, and other shells
sh <(curl -fsSL https://cog.run/install.sh)

# fish shell
sh (curl -fsSL https://cog.run/install.sh | psub)

# download with wget and run in a separate command
wget -qO- https://cog.run/install.sh
sh ./install.sh
```

Or install manually:

```console
sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')"
sudo chmod +x /usr/local/bin/cog
sudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true
```

If you see a Gatekeeper warning saying the binary "cannot be opened because the developer cannot be verified", run:

```console
sudo xattr -d com.apple.quarantine /usr/local/bin/cog
```

</details>

<details>
  <summary>Linux</summary>

You can install Cog using the install script:

```sh
# bash, zsh, and other shells
sh <(curl -fsSL https://cog.run/install.sh)

# fish shell
sh (curl -fsSL https://cog.run/install.sh | psub)

# download with wget and run in a separate command
wget -qO- https://cog.run/install.sh
sh ./install.sh
```

Or install manually:

```console
sudo curl -o /usr/local/bin/cog -L "https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')"
sudo chmod +x /usr/local/bin/cog
```

</details>

<details markdown>
  <summary>Windows</summary>

Cog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above.

</details>

<details>
  <summary>Docker</summary>

To install Cog inside a Docker image:

```dockerfile
RUN sh -c "INSTALL_DIR=\"/usr/local/bin\" SUDO=\"\" $(curl -fsSL https://cog.run/install.sh)"
```

</details>

## Upgrade

If you're using macOS and you previously installed Cog with Homebrew, run the following:

```console
brew upgrade replicate/tap/cog
```

Otherwise, you can upgrade to the latest version by running the same commands you used to install it.

## Development

See [CONTRIBUTING.md](CONTRIBUTING.md) for how to set up a development environment and build from source.

## Next steps

- [Get started with an example model](docs/getting-started.md)
- [Get started with your own model](docs/getting-started-own-model.md)
- [Using Cog with notebooks](docs/notebooks.md)
- [Using Cog with Windows 11](docs/wsl2/wsl2.md)
- [Browse the example models in this repo](docs/examples.md)
- [Deploy models with Cog](docs/deploy.md)
- [`cog.yaml` reference](docs/yaml.md) to learn how to define your model's environment
- [Run interface reference](docs/python.md) to learn how the `Runner` interface works
- [Training interface reference](docs/training.md) to learn how to add a fine-tuning API to your model
- [HTTP API reference](docs/http.md) to learn how to use the HTTP API that models serve

## Need help?

[Join us in #cog on Discord.](https://discord.gg/replicate)

[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/replicate/cog)



---

# Base images

Cog builds your model into a Docker image. To speed up builds and reduce cold boot times, Cog uses **prebuilt base images** by default. These images contain the common dependencies that most models need, so Cog doesn't have to install them from scratch every time you build or push.

## What is a Cog base image?

A Cog base image is a Docker image maintained by Replicate that includes:

- **Python runtime** — the Python version specified in your `cog.yaml`.
- **System libraries** — common dependencies for machine learning and media processing, including `ffmpeg`, `git`, `curl`, `build-essential`, `cmake`, OpenCV libraries, audio libraries (`sox`, `libsndfile1`), and graphics libraries (`libgl1`, `libsm6`, `libxext6`).
- **CUDA and PyTorch** (GPU images only) — the appropriate CUDA toolkit and PyTorch stack for your configuration.

Base images are tagged by their configuration, for example:

```
r8.im/cog-base:cuda12.4-python3.13-torch2.6.0
r8.im/cog-base:python3.13
```

When you run `cog build` or `cog push`, Cog selects the base image that matches your Python version, CUDA version, and PyTorch version. Because these images are pre-pulled on Replicate's infrastructure, models built on top of them start faster.

The entire list of available base images can be viewed at [https://r8.im/v2/cog-base/tags/list](https://r8.im/v2/cog-base/tags/list).

## Using the Cog base image

The `--use-cog-base-image` flag controls whether Cog uses a prebuilt base image. It is **enabled by default** on the following commands:

- [`cog build`](cli.md#cog-build)
- [`cog push`](cli.md#cog-push)
- [`cog run`](cli.md#cog-run)
- [`cog serve`](cli.md#cog-serve)
- [`cog exec`](cli.md#cog-exec)

Since it's on by default, you don't need to pass any flags:

```bash
cog push r8.im/your-username/my-model
```

This builds and pushes your model using a prebuilt Cog base image for faster cold boots.

## Disabling the Cog base image

If you encounter build issues or need a custom base layer, you can disable the Cog base image:

```bash
cog build --use-cog-base-image=false
```

When disabled, Cog generates a Dockerfile from scratch using either an NVIDIA CUDA base image or a slim Python base image, depending on the `--use-cuda-base-image` flag.

## Relationship to `--use-cuda-base-image`

The `--use-cuda-base-image` flag controls which underlying base image Cog uses **when the Cog base image is disabled**. It has no effect when `--use-cog-base-image` is enabled (the default), because the Cog base image already includes the appropriate CUDA and Python stack.

When you disable the Cog base image with `--use-cog-base-image=false`, Cog chooses the base image automatically:

- **GPU models** (`gpu: true` in `cog.yaml`): uses an NVIDIA CUDA base image.
- **CPU models**: uses a slim Python base image.

These flags — along with `--dockerfile` — are **mutually exclusive**: you can only set one of them explicitly on a given command. To customize the base image, disable the Cog base image and let Cog choose between CUDA and Python automatically.

## Troubleshooting

If `cog build` or `cog push` fails with the Cog base image enabled, try disabling it:

```bash
cog push --use-cog-base-image=false r8.im/your-username/my-model
```

This falls back to building from a standard CUDA or Python base image, which can help diagnose whether the issue is with the base image or your model's configuration.


---

# CLI reference

<!-- This file is auto-generated. Do not edit manually. -->

## `cog`

Containers for machine learning.

To get started, take a look at the documentation:
https://github.com/replicate/cog

**Examples**

```
   To execute a command inside a Docker environment defined with Cog:
      $ cog exec echo hello world
```

**Options**

```
      --debug      Show debugging output
  -h, --help       help for cog
      --no-color   Disable colored output
      --version    Show version of Cog
```

## `cog build`

Build a Docker image from the cog.yaml in the current directory.

The generated image contains your model code, dependencies, and the Cog
runtime. It can be run locally with 'cog run' or pushed to a registry
with 'cog push'.

```
cog build [flags]
```

**Examples**

```
  # Build with default settings
  cog build

  # Build and tag the image
  cog build -t my-model:latest

  # Build without using the cache
  cog build --no-cache

  # Build with model weights in a separate layer
  cog build --separate-weights -t my-model:v1
```

**Options**

```
  -f, --file string                  The name of the config file. (default "cog.yaml")
  -h, --help                         help for build
      --no-cache                     Do not use cache when building the image
      --openapi-schema string        Load OpenAPI schema from a file
      --progress string              Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default "auto")
      --secret stringArray           Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'
      --separate-weights             Separate model weights from code in image layers
  -t, --tag string                   A name for the built image in the form 'repository:tag'
      --use-cog-base-image           Use pre-built Cog base image for faster cold boots (default true)
      --use-cuda-base-image string   Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default "auto")
```

## `cog doctor`

Diagnose and fix common issues in your Cog project.

NOTE: cog doctor is experimental. Behavior and checks may change in future versions.

By default, cog doctor reports problems without modifying any files.
Pass --fix to automatically apply safe fixes.

```
cog doctor [flags]
```

**Options**

```
  -f, --file string   The name of the config file. (default "cog.yaml")
      --fix           Automatically apply fixes
  -h, --help          help for doctor
```

## `cog exec`

Execute a command inside a Docker environment defined by cog.yaml.

Cog builds a temporary image from your cog.yaml configuration and runs the
given command inside it. This is useful for debugging, running scripts, or
exploring the environment your model will run in.

```
cog exec <command> [arg...] [flags]
```

**Examples**

```
  # Open a Python interpreter inside the model environment
  cog exec python

  # Run a script
  cog exec python train.py

  # Run with environment variables
  cog exec -e HUGGING_FACE_HUB_TOKEN=abc123 python download.py

  # Expose a port (e.g. for Jupyter)
  cog exec -p 8888 jupyter notebook
```

**Options**

```
  -e, --env stringArray              Environment variables, in the form name=value
  -f, --file string                  The name of the config file. (default "cog.yaml")
      --gpus docker run --gpus       GPU devices to add to the container, in the same format as docker run --gpus.
  -h, --help                         help for exec
      --progress string              Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default "auto")
  -p, --publish stringArray          Publish a container's port to the host, e.g. -p 8000 or -p 0.0.0.0:8000
      --use-cog-base-image           Use pre-built Cog base image for faster cold boots (default true)
      --use-cuda-base-image string   Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default "auto")
```

## `cog init`

Create a cog.yaml and run.py in the current directory.

These files provide a starting template for defining your model's environment
and run interface. Edit them to match your model's requirements.

```
cog init [flags]
```

**Examples**

```
  # Set up a new Cog project in the current directory
  cog init
```

**Options**

```
  -h, --help   help for init
```

## `cog login`

Log in to a container registry.

For Replicate's registry (r8.im), this command handles authentication
through Replicate's token-based flow.

For other registries, this command prompts for username and password,
then stores credentials using Docker's credential system.

```
cog login [flags]
```

**Options**

```
  -h, --help          help for login
      --token-stdin   Pass login token on stdin instead of opening a browser. You can find your Replicate login token at https://replicate.com/auth/token
```

## `cog playground`

Open a browser playground for talking to a running model.

Starts a local web server that serves a schema-driven UI (a Postman-like tool
for Cog models). Point it at any running Cog HTTP API -- for example one started
with 'cog serve' -- and the playground reflects that model's inputs and outputs
from its OpenAPI schema in real time.

Requests are reverse-proxied through this server, so the target API does not
need to set CORS headers. The server also hosts a webhook sink so async
predictions can be observed in the browser.

Async/webhook testing against a containerized model requires the webhook URL to
be reachable from inside the container. On Docker Desktop the default
'host.docker.internal' works once the server listens on a reachable interface
(e.g. --host 0.0.0.0).

```
cog playground [flags]
```

**Examples**

```
  # Start a model API in one terminal
  cog serve -p 8393

  # Open the playground pointing at it
  cog playground --target http://localhost:8393
```

**Options**

```
  -h, --help                  help for playground
      --host string           Address to bind (use 0.0.0.0 to receive webhooks from containers) (default "127.0.0.1")
      --no-open         

Measured from the saved file

Links and embedded code

UTF-8 text size
150.8 KB154413 bytes
Token count
37224o200k_base encoding
Markdown links
202144 distinct URLs
Fenced code blocks
2554319 lines in the file

Size measures the captured UTF-8 text, not compressed transfer size. Tokens are counted with a named tokenizer; other models can count differently. Neither measure establishes context compatibility or answer quality.

Project H1
Cog: Containers for machine learning

Context architecture

How the content is divided

106 H2 sections organize 202 parsed Markdown links. 77 links point to .md or .markdown paths; 53 link occurrences use another hostname. Linked destinations have not been recursively validated.

H2 sectionLinksUTF-8 bytesTokens
HighlightsLine 721014218
How it worksLine 1912180552
Why are we building this?Line 10651088245
PrerequisitesLine 1183421114
InstallLine 12311995568
UpgradeLine 210024152
DevelopmentLine 220110828
Next stepsLine 22410701181
Need help?Line 2372455109
What is a Cog base image?Line 25111058275
Using the Cog base imageLine 2705504151
Disabling the Cog base imageLine 288031874
Relationship to `--use-cuda-base-image`Line 2980767174
TroubleshootingLine 3090418100
`cog`Line 326042299
`cog build`Line 34901590380
Show 64 more sections
H2 sectionLinksUTF-8 bytesTokens
`cog doctor`Line 3920464107
`cog exec`Line 41301551391
`cog init`Line 454036691
`cog login`Line 4780512115
`cog playground`Line 49901527365
`cog push`Line 54202231514
`cog run`Line 59202051530
`cog serve`Line 64522683647
Build a Docker imageLine 715022153
Push to a registryLine 72603203736
Run the modelLine 8030921244
Make a predictionLine 84213167862
Run a one-off predictionLine 9620429108
Health checksLine 9761425100
Stop the serverLine 986025566
ConcurrencyLine 9981484120
Environment variablesLine 1013138885
Next stepsLine 1022335070
Build-time variablesLine 103503223746
CLI and local cache variablesLine 111201221314
Model reference and registry variablesLine 116201544372
Runtime server variablesLine 1212046591123
Push tuning variablesLine 1311173483842
PrerequisitesLine 14033404110
InitializationLine 1409234585
Define the Docker environmentLine 142321596332
Define how to run your modelLine 146313031753
Using GPUsLine 15392447123
Next stepsLine 15536706179
PrerequisitesLine 15753404110
Create a projectLine 158109026
Run commandsLine 159101187216
Run a modelLine 162602350650
Build an imageLine 171332755690
Next stepsLine 178963247675
Streaming predictions with server-sent eventsLine 186503885942
WebhooksLine 199601656403
Generating unique prediction IDsLine 20532881211
File uploadsLine 207611531391
EndpointsLine 2144876151937
Install the jupyterlab Python packageLine 2421121562
Run a notebookLine 2438016651
Use notebook code in your runnerLine 24461965211
Enable tracingLine 24780867197
Getting started with tracingLine 25011888197
Automatic spansLine 253701204250
Model-authored spansLine 25660982194
Custom Python tracingLine 258902314486
Streaming predictionsLine 26400651123
Caller-supplied trace tagsLine 264601213294
Calling Cog from another serviceLine 26850689174
SamplingLine 27091992213
Custom trace headersLine 2725037487
Resource identityLine 2739030962
Failure behaviorLine 27500691126
What's nextLine 2760026851
`pip.conf`Line 2771042898
`cog.yaml`Line 2783129177
BuildLine 279731425351
ContentsLine 2825351409423
`BaseRunner`Line 286372921665
`async` runners and concurrencyLine 292501201296
`Input(**kwargs)`Line 295822015484
Deprecating inputsLine 29950615129

26 additional H2 sections are present in the source Markdown. This table previews the first 80 sections.

Link counts exclude parsed fenced and inline code and resolve relative destinations against the final endpoint. Section sizes cover the text below each H2 until the next H2.

Embedded material

Code, installation and API examples

Fenced blocks
255 closed
Declared languages
console (50), unlabeled (38), yaml (35), python (34), http (23), bash (19), py (18), json (10), sh (8), powershell (6), text (5), shell (4), dockerfile (1), none (1), dockerignore (1), js (1), conf (1)
Package-install commands
3 detected in fenced or inline code
Authentication examples
0 fenced lines match API-key, Authorization, Bearer, OAuth or access-token patterns
HTTP request examples
23 fenced lines match request-method patterns
Authentication links
0 links have authentication-related labels

Embedded example text and links to further documentation are counted separately. A code block does not establish that a sample runs or that an authentication flow succeeds.

Detected installation lines

Source line 133 · console

brew install replicate/tap/cog

Source line 2785 · inline

pip install

Source line 2805 · inline

pip install
Inspect detected API and authentication lines

Source line 75 · console

$ curl http://localhost:5000/predictions -X POST \

Source line 85 · console

$ curl http://localhost:8080/predictions -X POST \

Source line 854 · console

curl http://localhost:5001/predictions -X POST \

Source line 892 · console

curl http://localhost:5001/predictions -X POST \

Source line 911 · bash

curl http://localhost:5001/predictions -X POST \

Publisher prioritization

The Optional section

No H2 section named “Optional” was detected in this file.

The community proposal uses Optional by convention for secondary links that can be skipped for a shorter context. This label does not establish that everything outside it is critical, or that the material inside it is deprecated.

Machine-readable routes

MCP references and configuration text

No MCP or Model Context Protocol term was detected in the captured file. 0 links have MCP-related labels or paths. 0 fenced lines match mcpServers, mcp_servers or an mcp add command.

These are text and link observations. No MCP server connection, authentication or tool call was tested.

Index and bundle

llms-full.txt companion

Not found at checked URLs

A verified text response establishes a companion at the checked URL. It does not prove the bundle contains all product documentation, that generation is automated, or that it fits an assistant’s context window.

Change evidence

What can be said about freshness

This capture establishes a baseline. Update frequency cannot be determined from one observation.

Linked release and migration material

3 links have release, changelog, deprecation or migration terms in their labels or paths. These provide places to investigate product changes separately.

File changes can reveal documentation edits or navigation changes. They do not by themselves measure product release velocity, accuracy, agent adoption or AI visibility.

Capture identity and measurement method
Requested endpoint
https://cog.run/llms.txt
Final endpoint
https://cog.run/llms.txt
HTTP status
200
Content type
text/plain; charset=utf-8
Text retrieval
Firecrawl literal text capture
Tokenizer
tiktoken 0.14.0 · o200k_base
SHA-256 of saved UTF-8 text
f7dd7b4bbf5bf15db8648e257998f6081f5dca34c0dac2fe67b3e057b0ae44ae

Counts use a deterministic parser, not a spec-compliance grade. Unusual Markdown and indented code can fall outside its recognized patterns. See the llms.txt proposal and tokenizer implementation.

Related tools and resources

Contact Jon