Skip to content
J·BJon BishopFractional growth marketer

Developer tools · llms.txt file breakdown

Flyte llms.txt

A directly retrieved Markdown index for Flyte's documentation, with 754 links across 7 detected headings.

The actual file

Source Markdown

The saved llms.txt text, displayed as source. The file belongs to its publisher; the live endpoint may change.

Open saved text ↗

Showing the first 18000 characters. Load the complete file to read the remainder.

# Flyte Open Source Documentation
> Full documentation (single file): https://www.union.ai/docs/v2/flyte/llms-full.txt
> Site: https://www.union.ai/docs/v2/flyte

Each entry below is `- [Page title](URL)` followed by the H2/H3 headings found on that page. Each page's markdown is served at its own URL with `.md` appended, e.g. `/integrations/hydra.md`. A section landing page's markdown also lists every page directly beneath it, each with a one-line description, so it is the index for that section.

## User guide

- [Get started](https://www.union.ai/docs/v2/flyte/user-guide/get-started.md)
  - Pure Python, no DSL
  - Durability
  - Reproducibility
  - Recoverability
  - Built for scale
  - What this means in practice
  - Start here
  - [Quickstart](quickstart.md)
  - [Core concepts](core-concepts.md)
  - [Run modes](run-modes.md)
- [Get started > Quickstart](https://www.union.ai/docs/v2/flyte/user-guide/get-started/quickstart.md)
  - What you'll need
  - Install the SDK
  - Run something straight away
  - Configure
  - Write your first workflow
  - Run it
  - See the results
  - Next steps
- [Get started > Core concepts](https://www.union.ai/docs/v2/flyte/user-guide/get-started/core-concepts.md)
  - How a Flyte program fits together
- [Get started > Core concepts > TaskEnvironment](https://www.union.ai/docs/v2/flyte/user-guide/get-started/core-concepts/task-environment.md)
  - A minimal example
  - What TaskEnvironment controls
  - Configuring resources
  - Configuring container images
  - Multiple tasks, one environment
  - Multiple environments
  - Next steps
- [Get started > Core concepts > Tasks](https://www.union.ai/docs/v2/flyte/user-guide/get-started/core-concepts/tasks.md)
  - Defining a task
  - Type hints are required
  - Tasks calling tasks
  - The top-level task
  - Running tasks locally
  - Running tasks remotely
  - Next steps
- [Get started > Core concepts > Runs and actions](https://www.union.ai/docs/v2/flyte/user-guide/get-started/core-concepts/runs-and-actions.md)
  - What is a run?
  - What is an action?
  - Runs vs actions in practice
  - Viewing runs in the UI
  - Understanding the execution graph
  - Checking run status
  - Next steps
- [Get started > Core concepts > Where your data lives](https://www.union.ai/docs/v2/flyte/user-guide/get-started/core-concepts/where-data-lives.md)
  - The two stores
  - What goes in the database
  - What goes in the bucket
  - What "literal" and "raw data" mean
  - What "metadata" means
  - 1. "Metadata" as in the control plane database (Flyte's usage)
  - 2. "Metadata bucket" (a deployment/ops term you may see)
  - Per-run customization: `raw_data_path`
  - What happens if the bucket is purged
  - The short version
- [Get started > Core concepts > Apps](https://www.union.ai/docs/v2/flyte/user-guide/get-started/core-concepts/introducing-apps.md)
  - Tasks vs apps
  - AppEnvironment
  - A hello world app
  - Understanding the code
  - Serving the app
  - When to use apps vs tasks
  - Common patterns
  - Next steps
- [Get started > Core concepts > Projects and domains](https://www.union.ai/docs/v2/flyte/user-guide/get-started/core-concepts/projects-and-domains.md)
  - How projects and domains are used
  - Managing projects via CLI
  - Create a project
  - List projects
  - Update a project
  - Archive a project
  - Unarchive a project
  - Listing projects programmatically
  - Managing projects via the UI
  - Domains
  - Targeting a domain
- [Get started > Core concepts > Key capabilities](https://www.union.ai/docs/v2/flyte/user-guide/get-started/core-concepts/key-capabilities.md)
  - Environment and resources
  - Deployment
  - Data handling
  - Parallelism and composition
  - Security and automation
  - Durability and reliability
  - Apps and serving
  - Notebooks
  - Next steps
- [Get started > Core concepts > Basic project: RAG](https://www.union.ai/docs/v2/flyte/user-guide/get-started/core-concepts/basic-project.md)
  - Concepts covered
  - Part 1: The embedding pipeline
  - Setting up the environment
  - Fetching data
  - Creating embeddings
  - Orchestrating the pipeline
  - Running the pipeline
  - Part 2: The serving application
  - App environment configuration
  - The Streamlit application
  - Deploying the app
  - Key takeaways
- [Get started > Run modes](https://www.union.ai/docs/v2/flyte/user-guide/get-started/run-modes.md)
  - Local and remote
  - [Local](running-locally.md)
  - [Devbox](running-devbox.md)
  - [Deployed cluster](running-remote.md)
- [Get started > Run modes > Run locally in Python](https://www.union.ai/docs/v2/flyte/user-guide/get-started/run-modes/running-locally.md)
  - Getting started
  - Running tasks locally
  - Two ways to run a task
  - As a script: `python hello.py`
  - Via the CLI: `flyte run`
  - Terminal UI
  - Exploring past runs
  - What works locally
  - From in-process to on-cluster
  - Next steps
- [Get started > Run modes > Run on the Devbox](https://www.union.ai/docs/v2/flyte/user-guide/get-started/run-modes/running-devbox.md)
  - What you'll need
  - Install the SDK
  - Start the devbox
  - CPU
  - GPU
  - Check the devbox status
  - Configure
  - Run a workflow on the devbox
  - View results in the UI
  - Stop the devbox
  - Inline configuration
  - Programmatic
  - CLI
  - Delete the devbox
  - Using a CUDA-enabled GPU host
  - Next steps
- [Get started > Run modes > Run on a deployed cluster](https://www.union.ai/docs/v2/flyte/user-guide/get-started/run-modes/running-remote.md)
  - Prerequisites
  - Install the flyte package
  - Configuration file
  - Set up local Docker
  - Using the configuration
  - Explicit configuration
  - Programmatic
  - CLI
  - Programmatic
  - CLI
  - Check current configuration
  - Inline configuration
  - Programmatic
  - CLI
  - Next steps
- [Tasks](https://www.union.ai/docs/v2/flyte/user-guide/tasks.md)
  - [Configure tasks](task-configuration.md)
  - [Build tasks](task-programming.md)
  - [Run and deploy tasks](task-deployment.md)
  - Related
  - [Apps](../apps.md)
  - [Agents](../agents.md)
- [Tasks > Configure tasks](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration.md)
  - Task configuration levels
  - Example
  - Task configuration parameters
  - Scheduled and reactive automation
  - Distributed tasks with a clustered environment
- [Tasks > Configure tasks > Container images](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/container-images.md)
  - Specifying your own image directly
  - Specifying your own image with the `flyte.Image` object
  - Example: Defining a custom image with `Image.from_debian_base`
  - Example: Defining an image based on uv script metadata
  - Customizing an image with `with_*` methods
  - Installing dependencies from a `uv` or Poetry project
  - Installing dependencies from a pixi project
  - Copying local files into the image
  - More ways to define a base image
  - Starting from an existing image
  - Building from a Dockerfile
  - Referencing an image defined in configuration
  - Using different images for different tasks
  - Image building
  - Configuring the `builder`
  - Local image building
  - Remote `ImageBuilder`
  - Install private PyPI packages
- [Tasks > Configure tasks > Resources](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/resources.md)
  - Resources data class
  - Ephemeral storage
  - Examples
  - Usage in TaskEnvironment
  - Usage in a task-specific override
  - Accelerators
  - NVIDIA GPUs
  - Google TPUs
  - AMD GPUs
  - AWS Trainium and Inferentia (Neuron)
  - Intel Habana Gaudi
- [Tasks > Configure tasks > Secrets](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/secrets.md)
  - Creating a literal string secret
  - Creating a file secret
  - Scoping secrets
  - Listing secrets
  - Deleting secrets
  - Declaring secrets in a task environment
  - Using a literal string secret
  - Using a file secret
  - Using secrets in local runs
  - Environment variable secrets
  - File secrets
  - Overriding secrets at invocation time
- [Tasks > Configure tasks > Caching](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/caching.md)
  - Overview
  - Basic caching usage
  - `"auto"` - automatic versioning
  - `"override"`
  - `"disable"` - No caching
  - Advanced caching configuration
  - Ignoring specific inputs
  - Cache serialization
  - Salt for cache key variation
  - Caching with `flyte.map`
  - Excluding a bound argument from the per-element key
  - Content-based caching for DataFrames, files, and directories
  - DataFrames
  - Files
  - Directories
  - Cache policies
  - Function body policy (default)
  - Custom cache policies
  - Caching configuration at different levels
  - `TaskEnvironment` level
  - `@env.task` decorator level
  - `task.override` level
  - Runtime cache control
  - Project and domain cache isolation
  - Local development caching
- [Tasks > Configure tasks > Reusable containers](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/reusable-containers.md)
- [Tasks > Configure tasks > Pod templates](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/pod-templates.md)
  - How it works
  - Requirements
  - Basic usage
  - PodTemplate parameters
  - Volume mounts
  - GCS/S3 volume mounts
  - Sidecar containers
  - Image pull secrets
  - Cluster-specific configuration
  - Important notes
  - Best practices
  - Learn more
- [Tasks > Configure tasks > Multiple environments](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/multiple-environments.md)
  - Constraints on multiple environments
  - Task `depends_on` constraints
  - Dependency inclusion constraints
  - Example
- [Tasks > Configure tasks > Clustered task environment](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/clustered-task-environment.md)
  - When to use it
  - How it works
  - Prerequisites
  - Basic usage
  - Configuration parameters
  - `TorchRun`
  - `ClusterFailurePolicy`
  - The distributed context
  - Checkpointing
  - Using higher-level frameworks
  - Constraints
- [Tasks > Configure tasks > Retries and timeouts](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/retries-and-timeouts.md)
  - The action lifecycle
  - Retries
  - Retry count
  - Retries with exponential backoff
  - Skip retries for failures that can't be fixed
  - System retries
  - Timeouts
  - `max_runtime`: bound a single attempt's execution
  - `max_queued_time`: fail fast when capacity isn't available
  - `deadline`: bound the total wall-clock
  - Combining the bounds
  - Combining retries and timeouts
- [Tasks > Configure tasks > Interruptible tasks](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/interruptible-tasks-and-queues.md)
  - Setting at different levels
  - Behavior on preemption
  - Fall back to on-demand when spot capacity is unavailable
- [Tasks > Configure tasks > Task plugins](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/task-plugins.md)
  - Default execution: containers
  - Compute plugins
  - Available compute plugins
  - How compute plugins work
  - Using compute plugins
  - Using plugins on Union
  - Backend integrations
  - Next steps
- [Tasks > Configure tasks > Additional task settings](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/additional-task-settings.md)
  - Naming and metadata
  - `name`
  - `short_name`
  - `description`
  - `docs`
  - `report`
  - Source-code link (automatic)
  - `links`
  - Default inputs
  - Environment variables
  - Inline I/O threshold
- [Tasks > Configure tasks > Logging](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/logging.md)
  - Set the logging level with environment variables
  - Set the framework verbosity from the CLI
  - Set the logging level with `flyte.init()`
  - Write to the user logger
  - JSON logging and third-party libraries
  - Set it at registration with `FLYTE_RESET_ROOT_LOGGER`
  - Related
- [Tasks > Configure tasks > Overrides](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-configuration/overrides.md)
  - What you can override
  - What you cannot override
  - Overriding when a task is reusable
  - Overriding secrets
- [Tasks > Build tasks](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming.md)
  - What you'll learn
  - When to use these patterns
- [Tasks > Build tasks > Files and directories](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/files-and-directories.md)
  - Example usage
  - Synchronous and asynchronous APIs
  - File methods
  - Dir methods
  - Synchronous example
  - JSONL files
- [Tasks > Build tasks > Data classes and structures](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/dataclasses-and-structures.md)
  - Example: Combining dataclasses and Pydantic models
- [Tasks > Build tasks > DataFrames](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/dataframes.md)
  - Setting up the environment and sample data
  - Create a raw DataFrame
  - Create a flyte.io.DataFrame
  - Automatically convert between types
  - Downloading DataFrames
  - Run the example
  - Polars DataFrames
  - Setup
  - Eager DataFrames
  - Lazy DataFrames
  - Run the example
  - Lance datasets
  - See also
- [Tasks > Build tasks > Custom types](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/handling-custom-types.md)
  - Types of extensions
  - Creating a type transformer
  - Step 1: Define your custom type
  - Step 2: Create the type transformer
  - Step 3: Register the transformer
  - Distributing type plugins
  - Configure pyproject.toml
  - Automatic loading
  - Controlling plugin loading
  - Using custom types in tasks
  - DataFrame extensions
  - Best practices
  - See also
- [Tasks > Build tasks > Custom context](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/custom-context.md)
  - Overview
  - When to use it and when not to
  - Setting custom context
  - Run-level context
  - Overriding inside a task (local override that affects nested tasks)
  - Adding new keys for nested tasks
  - Accessing custom context
- [Tasks > Build tasks > Abort and cancel actions](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/abort-tasks.md)
  - Action lifetime
  - Canceling actions programmatically
  - External abort
  - Aborting via the CLI
  - Handling external aborts
- [Tasks > Build tasks > Run a bioinformatics tool](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/container-tasks.md)
  - What are container tasks?
  - How data flows in and out
  - Basic usage
  - Template syntax for inputs
  - Using container tasks in workflows
  - Advanced: Passing files and directories
  - Use case: Agentic sandbox execution
  - Use case: Legacy and specialized containers
  - Use case: Multi-language workflows
  - Configuration options
  - ContainerTask parameters
  - Supported input/output types
  - Best practices
  - Local execution
  - When to use container tasks
- [Tasks > Build tasks > Links](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/links.md)
  - Creating a link
  - Using execution metadata
  - Dynamic links with override
- [Tasks > Build tasks > Reports](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/reports.md)
  - A simple example
  - A more complex example
  - Streaming example
  - Rendering a custom type
  - Fetching a report after the run
- [Tasks > Build tasks > Notebooks](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/notebooks.md)
  - Iterating on and running a workflow
  - Accessing runs and downloading logs
- [Tasks > Build tasks > Remote tasks](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/remote-tasks.md)
  - Prerequisites
  - Basic usage
  - Understanding lazy loading
  - When tasks are fetched
  - Benefits of lazy loading
  - Error handling
  - Eager fetching with `fetch()`
  - Module-level vs dynamic loading
  - Complete example
  - Team A: Spark environment
  - Team B: ML environment
  - Team C: Orchestration
  - Invoke remote tasks in a script.
  - Why use remote tasks?
  - When to use remote tasks
  - How remote tasks work
  - Security model
  - Type system
  - Versioning options
  - Customizing remote tasks
  - Available overrides
  - Override examples
  - Chain overrides
  - Best practices
  - 1. Use meaningful task names
  - 2. Document task interfaces
  - 3. Prefer module-level loading
  - 4. Handle versioning thoughtfully
  - 5. Deploy remote tasks first
  - Limitations
  - Next steps
- [Tasks > Build tasks > Error handling](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/error-handling.md)
  - How Flyte represents failures
  - Catching and recovering from errors
  - Out-of-memory on the GPU
  - Falling back when GPU capacity isn't available
  - Telling the timeout bounds apart
  - Handling GPU hardware faults (Xid errors)
  - Handling the failure of a run you launched
  - Limiting inline I/O
  - Natively-supported exceptions
  - Related pages
- [Tasks > Build tasks > Traces](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/traces.md)
  - What are traced functions for?
  - What gets traced
  - Errors are not recorded
  - Supported function types
  - Task orchestration pattern
  - Relationship to caching and checkpointing
  - How they work together
  - Execution flow
  - Error handling and observability
  - Examples in practice
  - LLM pipeline with traces
- [Tasks > Build tasks > Intra-task checkpoints](https://www.union.ai/docs/v2/flyte/user-guide/tasks/task-programming/intra-task-checkpoints.md)
  - The checkpoint object
  - Basic usage
  - Async
  - Sync
  - Checkpointing ML training frameworks
  - PyTorch
  - PyTorch Lightning
  - Hugging Face Trainer
  - scikit-learn
  - Unsloth
  - How checkpoints are stored
- [Tasks > Build tasks > Grouping a

Measured from the saved file

Linked document

UTF-8 text size
189.4 KB193948 bytes
Token count
53833o200k_base encoding
Markdown links
762760 distinct URLs
Fenced code blocks
05406 lines in the file

Size measures the captured UTF-8 text, not compressed transfer size. Tokens are counted with a named tokenizer; other models can count differently. Neither measure establishes context compatibility or answer quality.

Project H1
Flyte Open Source Documentation

Opening blockquote

Full documentation (single file): https://www.union.ai/docs/v2/flyte/llms-full.txt Site: https://www.union.ai/docs/v2/flyte

Context architecture

How the content is divided

6 H2 sections organize 762 parsed Markdown links. 761 links point to .md or .markdown paths; 1 link occurrences use another hostname. Linked destinations have not been recursively validated.

H2 sectionLinksUTF-8 bytesTokens
User guideLine 72026603917519
TutorialsLine 183772150813885
IntegrationsLine 221751196335411
ReferenceLine 28834118308424182
CommunityLine 51341552651465
Platform deploymentLine 52791142511228

Link counts exclude parsed fenced and inline code and resolve relative destinations against the final endpoint. Section sizes cover the text below each H2 until the next H2.

Embedded material

Code, installation and API examples

Fenced blocks
0 closed
Declared languages
No fenced blocks detected
Package-install commands
0 detected in fenced or inline code
Authentication examples
0 fenced lines match API-key, Authorization, Bearer, OAuth or access-token patterns
HTTP request examples
0 fenced lines match request-method patterns
Authentication links
0 links have authentication-related labels

Embedded example text and links to further documentation are counted separately. A code block does not establish that a sample runs or that an authentication flow succeeds.

Publisher prioritization

The Optional section

No H2 section named “Optional” was detected in this file.

The community proposal uses Optional by convention for secondary links that can be skipped for a shorter context. This label does not establish that everything outside it is critical, or that the material inside it is deprecated.

Machine-readable routes

MCP references and configuration text

MCP-related text was detected in the captured file. 7 links have MCP-related labels or paths. 0 fenced lines match mcpServers, mcp_servers or an mcp add command.

These are text and link observations. No MCP server connection, authentication or tool call was tested.

Index and bundle

llms-full.txt companion

Verified text companion

Text companion verified

https://www.union.ai/docs/v2/flyte/llms-full.txt ↗
Discovery
Conventional sibling path checked
HTTP response
200 · text/plain; charset=utf-8
Text size
Whole-file size not established by the bounded response
Token count
Not measured for the whole file
Retrieval scope
Bounded prefix inspected; the whole body was not downloaded

A verified text response establishes a companion at the checked URL. It does not prove the bundle contains all product documentation, that generation is automated, or that it fits an assistant’s context window.

Change evidence

What can be said about freshness

This capture establishes a baseline. Update frequency cannot be determined from one observation.

Linked release and migration material

17 links have release, changelog, deprecation or migration terms in their labels or paths. These provide places to investigate product changes separately.

File changes can reveal documentation edits or navigation changes. They do not by themselves measure product release velocity, accuracy, agent adoption or AI visibility.

Capture identity and measurement method
Requested endpoint
https://docs.flyte.org/llms.txt
Final endpoint
https://www.union.ai/docs/v2/flyte/llms.txt
HTTP status
200
Content type
text/plain; charset=utf-8
Text retrieval
Firecrawl literal text capture
Tokenizer
tiktoken 0.14.0 · o200k_base
SHA-256 of saved UTF-8 text
44ed9b48c29a3652e888b6202f512841dbfd43cccc5e504aedf667566cd513f8

Counts use a deterministic parser, not a spec-compliance grade. Unusual Markdown and indented code can fall outside its recognized patterns. See the llms.txt proposal and tokenizer implementation.

Related tools and resources

Contact Jon