Showing the first 18000 characters. Load the complete file to read the remainder.
# Datafold
- [Datafold](https://docs.datafold.com/welcome.md): Datafold is the data engineering automation platform that combines specialized AI agents with a context layer and data quality tools — so data teams and their coding agents ship higher-quality data faster, migrate with confidence, and optimize platform costs.
- [MCP](https://docs.datafold.com/datafold-mcp.md): Connect your AI agent to Datafold and interact with your data through natural language
- [Migration Automation](https://docs.datafold.com/data-migration-automation/datafold-migration-automation.md): Modernize your data platform in weeks, not years. Datafold's Data Migration Agent delivers guaranteed-outcome migrations with fixed price, timeline, and data parity — over 6x faster and cheaper than traditional approaches.
- [Datafold Migration Agent](https://docs.datafold.com/data-migration-automation/datafold-migration-agent.md): The Data Migration Agent delivers guaranteed-outcome migrations with fixed price, timeline, and data parity — over 6x faster than traditional approaches.
- [What's a Data Diff?](https://docs.datafold.com/data-diff/what-is-data-diff.md): A data diff is the value-level comparison between two tables, used to identify critical changes to your data and guarantee data quality.
- [How Datafold Diffs Data](https://docs.datafold.com/data-diff/how-datafold-diffs-data.md): Data diffs allow you to perform value-level comparisons between any two datasets within the same database, across different databases, or even between files.
- [Creating a New Data Diff](https://docs.datafold.com/data-diff/in-database-diffing/creating-a-new-data-diff.md): Setting up a new data diff in Datafold is straightforward.
- [Results](https://docs.datafold.com/data-diff/in-database-diffing/results.md): Once your data diff is complete, Datafold provides a concise, high-level summary of the detected changes in the Overview tab
- [Best Practices](https://docs.datafold.com/data-diff/in-database-diffing/best-practices.md): We share best practices that will help you get the most accurate and efficient results from your data diffs.
- [Creating a New Data Diff](https://docs.datafold.com/data-diff/cross-database-diffing/creating-a-new-data-diff.md): Datafold's Data Diff can compare data across databases (e.g., PostgreSQL <> Snowflake, or between two SQL Server instances) to validate migrations, meet regulatory and compliance requirements, or ensure data is flowing successfully from source to target.
- [Results](https://docs.datafold.com/data-diff/cross-database-diffing/results.md): Once your data diff is complete, Datafold provides a concise, high-level summary of the detected changes in the Overview tab.
- [Best Practices](https://docs.datafold.com/data-diff/cross-database-diffing/best-practices.md): When dealing with large datasets, it's crucial to approach diffing with specific optimization strategies in mind. We share best practices that will help you get the most accurate and efficient results from your data diffs.
- [How Datafold in CI Works](https://docs.datafold.com/deployment-testing/how-it-works.md): Learn how Datafold integrates with your Continuous Integration (CI) process with Data Diffs and AI Code Reviews, catching issues before they make it into production.
- [AI Code Reviews](https://docs.datafold.com/deployment-testing/ai-code-reviews.md): Get automated, AI-powered code reviews on every pull request to catch SQL and data pipeline issues before they reach production.
- [Getting Started with CI/CD Testing](https://docs.datafold.com/deployment-testing/getting-started.md): Learn how to set up CI/CD testing with Datafold by integrating your data connections, code repositories, and CI pipeline for automated testing.
- [No-Code](https://docs.datafold.com/deployment-testing/getting-started/universal/no-code.md): Set up Datafold's No-Code CI integration to create and manage Data Diffs without writing code.
- [API](https://docs.datafold.com/deployment-testing/getting-started/universal/api.md): Learn how to set up and configure Datafold's API for CI/CD testing.
- [Configuration](https://docs.datafold.com/deployment-testing/configuration.md): Explore configuration options for CI/CD testing in Datafold.
- [Primary Key Inference](https://docs.datafold.com/deployment-testing/configuration/primary-key.md): Datafold requires a primary key to perform data diffs. Using dbt metadata, Datafold identifies the column to use as the primary key for accurate data diffs.
- [Column Remapping](https://docs.datafold.com/deployment-testing/configuration/column-remapping.md): Specify column renaming in your git commit message so Datafold can map renamed columns to their original counterparts in production for accurate comparison.
- [Running Data Diff for Specific PRs/MRs](https://docs.datafold.com/deployment-testing/configuration/datafold-ci/on-demand.md): By default, Datafold CI runs on every new pull/merge request and commits to existing ones.
- [Running Data Diff on Specific Branches](https://docs.datafold.com/deployment-testing/configuration/datafold-ci/specifc.md): By default, Datafold CI runs on every new pull/merge request and commits to existing ones.
- [SQL Filters](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/sql-filters.md): Use dbt YAML configuration to set model-specific filters for Datafold CI.
- [Time Travel](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/time-travel.md): Use `prod_time_travel` and `pr_time_travel` to diff tables from specific points in time.
- [Including/Excluding Columns](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/including-excluding-columns.md): Specify columns to include or exclude from the data diff using `include_columns` and `exclude_columns`.
- [Excluding Models](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/excluding-models.md): Use `never_diff` to exclude a model or subdirectory of models from data diffs.
- [Diff Timeline](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/diff-timeline.md): Specify a `time_column` to visualize match rates between tables for each column over time.
- [Slim Diff](https://docs.datafold.com/deployment-testing/best-practices/slim-diff.md): Choose which downstream tables to diff to optimize time, cost, and performance.
- [Handling Data Drift](https://docs.datafold.com/deployment-testing/best-practices/handling-data-drift.md): Ensuring Datafold in CI executes apples-to-apples comparison between staging and production environments.
- [Monitor Types](https://docs.datafold.com/data-monitoring/monitor-types.md): Monitoring your data for unexpected changes is one of the cornerstones of data observability.
- [Data Diff Monitors](https://docs.datafold.com/data-monitoring/monitors/data-diff-monitors.md): Data Diff monitors compare datasets across or within databases, identifying row and column discrepancies with customizable scheduling and notifications.
- [Metric Monitors](https://docs.datafold.com/data-monitoring/monitors/metric-monitors.md): Metric monitors detect anomalies in your data using ML-based algorithms or manual thresholds, supporting standard and custom metrics for tables or columns.
- [Data Test Monitors](https://docs.datafold.com/data-monitoring/monitors/data-test-monitors.md): Data Tests validate your data against off-the-shelf checks or custom business rules.
- [Schema Change Monitors](https://docs.datafold.com/data-monitoring/monitors/schema-change-monitors.md): Schema Change monitors notify you when a table’s schema changes, such as when columns are added, removed, or data types are modified.
- [Monitors as Code](https://docs.datafold.com/data-monitoring/monitors-as-code.md): Manage Datafold monitors via version-controlled YAML for greater scalability, governance, and flexibility in code-based workflows.
- [How It Works](https://docs.datafold.com/data-explorer/how-it-works.md): Datafold's Data Knowledge Graph maps your entire data ecosystem — lineage, business logic, usage, and ontology — providing essential context to your AI agents via MCP and helping you understand the impact of changes across systems.
- [Lineage](https://docs.datafold.com/data-explorer/lineage.md): Datafold offers a column-level and tabular lineage view.
- [Profile](https://docs.datafold.com/data-explorer/profile.md): View a data profile that summarizes key table and column-level statistics, and any upstream dependencies.
- [dbt Metadata Sync](https://docs.datafold.com/data-explorer/best-practices/dbt-metadata-sync.md): Datafold can automatically ingest dbt metadata from your production environment and display it in Data Explorer.
- [Set Up Your Data Connection](https://docs.datafold.com/integrations/databases.md): Set up your Data Connection with Datafold.
- [Snowflake](https://docs.datafold.com/integrations/databases/snowflake.md): Connect Datafold to Snowflake for data diffing, CI/CD testing, lineage, and migration validation. Includes setup instructions and required permissions.
- [Snowflake key-pair authentication](https://docs.datafold.com/integrations/databases/snowflake-key-pair-authentication.md): Move an existing Datafold Snowflake connection from password to key-pair authentication and make the Datafold user a Snowflake SERVICE user before Snowflake blocks password sign-ins.
- [BigQuery](https://docs.datafold.com/integrations/databases/bigquery.md)
- [Athena](https://docs.datafold.com/integrations/databases/athena.md)
- [Redshift](https://docs.datafold.com/integrations/databases/redshift.md)
- [Databricks](https://docs.datafold.com/integrations/databases/databricks.md): Connect Datafold to Databricks for data diffing, CI/CD testing, lineage, and migration validation. Includes setup instructions and required permissions.
- [PostgreSQL](https://docs.datafold.com/integrations/databases/postgresql.md)
- [Microsoft SQL Server](https://docs.datafold.com/integrations/databases/sql-server.md): Connect Datafold to Microsoft SQL Server for data diffing, reconciliation, and migration validation. Includes setup instructions and required permissions.
- [Azure Synapse Analytics](https://docs.datafold.com/integrations/databases/synapse.md): Connect Datafold to Azure Synapse Analytics for data diffing, reconciliation, and migration validation. Includes setup instructions and required permissions.
- [Oracle](https://docs.datafold.com/integrations/databases/oracle.md): Connect Datafold to Oracle Database for data diffing, reconciliation, and migration validation. Includes setup instructions and required permissions.
- [MySQL](https://docs.datafold.com/integrations/databases/mysql.md)
- [Dremio](https://docs.datafold.com/integrations/databases/dremio.md)
- [SAP HANA](https://docs.datafold.com/integrations/databases/sap-hana.md)
- [Starburst](https://docs.datafold.com/integrations/databases/starburst.md)
- [Teradata](https://docs.datafold.com/integrations/databases/teradata.md)
- [Netezza](https://docs.datafold.com/integrations/databases/netezza.md)
- [OAuth Support](https://docs.datafold.com/integrations/oauth.md): Set up OAuth App Connections in your supported data warehouses to securely execute data diffs on behalf of your users.
- [Slack Bot](https://docs.datafold.com/integrations/agents/slack-bot.md): Datafold Assistant — a conversational Slack bot that answers questions about your data using Datafold's MCP tools, scoped via a service account.
- [Bring Your Own LLM Provider](https://docs.datafold.com/integrations/llm-providers.md): Run the Datafold Migration Agent on your own LLM inference provider and credentials.
- [OpenAI-compatible](https://docs.datafold.com/integrations/llm-providers/openai-compatible.md)
- [OpenAI](https://docs.datafold.com/integrations/llm-providers/openai.md)
- [Anthropic](https://docs.datafold.com/integrations/llm-providers/anthropic.md)
- [Google Gemini](https://docs.datafold.com/integrations/llm-providers/google-gemini.md)
- [Azure AI Foundry](https://docs.datafold.com/integrations/llm-providers/azure-ai-foundry.md)
- [AWS Bedrock](https://docs.datafold.com/integrations/llm-providers/aws-bedrock.md)
- [Google Vertex AI](https://docs.datafold.com/integrations/llm-providers/google-vertex-ai.md)
- [Databricks](https://docs.datafold.com/integrations/llm-providers/databricks.md)
- [Snowflake Cortex](https://docs.datafold.com/integrations/llm-providers/snowflake-cortex.md)
- [Integrate with Orchestrators](https://docs.datafold.com/integrations/orchestrators.md): Integrate Datafold with dbt Core, dbt Cloud, Airflow, or custom orchestrators to streamline your data workflows with automated monitoring, testing, and seamless CI integration.
- [dbt Core](https://docs.datafold.com/integrations/orchestrators/dbt-core.md): Set up Datafold’s integration with dbt Core to automate Data Diffs in your CI pipeline.
- [dbt Cloud](https://docs.datafold.com/integrations/orchestrators/dbt-cloud.md): Integrate Datafold with dbt Cloud to automate Data Diffs in your CI pipeline, leveraging dbt jobs to detect changes and ensure data quality before merging.
- [Custom Integrations](https://docs.datafold.com/integrations/orchestrators/custom-integrations.md): Integrate Datafold with your custom orchestration using the Datafold SDK and REST API.
- [Looker](https://docs.datafold.com/integrations/bi-data-apps/looker.md): Integrate Datafold with Looker to track BI lineage and understand the downstream impact of data changes on your Looker dashboards and Explores.
- [Tableau](https://docs.datafold.com/integrations/bi-data-apps/tableau.md): Visualize downstream Tableau dependencies and understand how warehouse changes impact your BI layer.
- [Power BI](https://docs.datafold.com/integrations/bi-data-apps/power-bi.md): Include Power BI entities in Data Explorer and column-level lineage.
- [Mode](https://docs.datafold.com/integrations/bi-data-apps/mode.md)
- [Hightouch](https://docs.datafold.com/integrations/bi-data-apps/hightouch.md): Navigate to Settings > Integrations > Data Apps and add a Hightouch Integration.
- [Tracking Jobs](https://docs.datafold.com/integrations/bi-data-apps/tracking-jobs.md): Track the completion and success of your data app integration syncs.
- [Integrate with Code Repositories](https://docs.datafold.com/integrations/code-repositories.md): Connect your code repositories with Datafold.
- [GitHub](https://docs.datafold.com/integrations/code-repositories/github.md): Connect Datafold to GitHub to enable automated data diffs on pull requests, CI/CD testing integration, and code-level lineage tracking.
- [GitLab](https://docs.datafold.com/integrations/code-repositories/gitlab.md)
- [Bitbucket](https://docs.datafold.com/integrations/code-repositories/bitbucket.md)
- [Azure DevOps](https://docs.datafold.com/integrations/code-repositories/azure-devops.md)
- [Butterfly Deployment Options](https://docs.datafold.com/datafold-deployment/datafold-deployment-options.md): Butterfly is hosted and operated by Datafold, either in Datafold's cloud (multi-tenant or dedicated) or in your own AWS, GCP, or Azure account.
- [Classic Deployment Options](https://docs.datafold.com/datafold-deployment/classic-deployment-options.md): Deployment options for Datafold Classic, including multi-tenant SaaS and dedicated cloud (either customer- or Datafold-hosted).
- [Datafold VPC Deployment on AWS (Classic)](https://docs.datafold.com/datafold-deployment/dedicated-cloud/aws.md): Learn how to deploy Datafold Classic in a Virtual Private Cloud (VPC) on AWS.
- [Datafold VPC Deployment on GCP (Classic)](https://docs.datafold.com/datafold-deployment/dedicated-cloud/gcp.md): Learn how to deploy Datafold Classic in a Virtual Private Cloud (VPC) on GCP.
- [Datafold VPC Deployment on Azure (Classic)](https://docs.datafold.com/datafold-deployment/dedicated-cloud/azure.md): Learn how to deploy Datafold Classic in a Virtual Private Cloud (VPC) on Azure.
- [Compliance & Trust Center](https://docs.datafold.com/security/compilance-trust-center.md)
- [Securing Connections](https://docs.datafold.com/security/securing-connections.md): Datafold supports multiple options to secure connections between your resources (e.g., databases and BI tools) and Datafold.
- [User Roles and Permissions](https://docs.datafold.com/security/user-roles-and-permissions.md): Datafold uses role-based access control to manage user permissions and actions.
- [Service Accounts](https://docs.datafold.com/security/service-accounts.md): Machine identities for CI, integrations, and scripts. Service accounts own their own API keys, inherit permissions from groups, and are managed independently of human users.
- [MCP Tool Permissions](https://docs.datafold.com/security/mcp-tool-permissions.md): Which permissions each MCP tool requires. Use this reference when scoping a service account's group for MCP use.
- [Single Sign-On](https://docs.datafold.com/security/single-sign-on.md): Set up Single Sign-On with one of the following options.
- [Okta (OIDC)](https://docs.datafold.com/security/single-sign-on/okta.md): Configure Okta OIDC single sign-on (SSO) for Datafold. Step-by-step setup instructions for authenticating your team with Okta.
- [Google OAuth](https://docs.datafold.com/security/single-sign-on/google-oauth.md): Configure Google OAuth single sign-on (SSO) for Datafold. Step-by-step setup instructions for authenticating your team with Google.
- [SAML](https://docs.datafold.com/security/single-sign-on/saml.md): SAML (Security Assertion Markup Language) is a protocol that enables secure user authentication by integrating Identity Providers (IdPs) with Service Providers (SPs).
- [Group provisioning](https://docs.datafold.com/security/single-sign-on/saml/group-provisioning.md): Automatically sync group membership with your SAML Identity Provider (IdP).
- [Okta](https://docs.datafold.com/security/single-sign-on/saml/examples/okta.md)
- [Microsoft Entra ID](https://docs.datafold.com/security/sin
Measured from the saved file
Linked document
UTF-8 text size
33.4 KB34165 bytes
Token count
7710o200k_base encoding
Markdown links
200171 distinct URLs
Fenced code blocks
0210 lines in the file
Size measures the captured UTF-8 text, not compressed transfer size. Tokens are counted with a named tokenizer; other models can count differently. Neither measure establishes context compatibility or answer quality.
Project H1
Datafold
Context architecture
How the content is divided
2 H2 sections organize 200 parsed Markdown links. 196 links point to .md or .markdown paths; 3 link occurrences use another hostname. Linked destinations have not been recursively validated.
H2 section
Links
UTF-8 bytes
Tokens
OpenAPI SpecsLine 200
1
43
13
OptionalLine 204 · Optional
3
201
49
Link counts exclude parsed fenced and inline code and resolve relative destinations against the final endpoint. Section sizes cover the text below each H2 until the next H2.
Embedded material
Code, installation and API examples
Fenced blocks
0 closed
Declared languages
No fenced blocks detected
Package-install commands
0 detected in fenced or inline code
Authentication examples
0 fenced lines match API-key, Authorization, Bearer, OAuth or access-token patterns
HTTP request examples
0 fenced lines match request-method patterns
Authentication links
2 links have authentication-related labels
Embedded example text and links to further documentation are counted separately. A code block does not establish that a sample runs or that an authentication flow succeeds.
Publisher prioritization
The Optional section
1 H2 section named “Optional” contain 3 parsed links. 197 links sit outside those sections. That puts 1.5% of the file’s parsed links under Optional.
The community proposal uses Optional by convention for secondary links that can be skipped for a shorter context. This label does not establish that everything outside it is critical, or that the material inside it is deprecated.
Machine-readable routes
MCP references and configuration text
MCP-related text was detected in the captured file. 3 links have MCP-related labels or paths. 0 fenced lines match mcpServers, mcp_servers or an mcp add command.
A verified text response establishes a companion at the checked URL. It does not prove the bundle contains all product documentation, that generation is automated, or that it fits an assistant’s context window.
Change evidence
What can be said about freshness
This capture establishes a baseline. Update frequency cannot be determined from one observation.
Linked release and migration material
2 links have release, changelog, deprecation or migration terms in their labels or paths. These provide places to investigate product changes separately.
File changes can reveal documentation edits or navigation changes. They do not by themselves measure product release velocity, accuracy, agent adoption or AI visibility.
Counts use a deterministic parser, not a spec-compliance grade. Unusual Markdown and indented code can fall outside its recognized patterns. See the llms.txt proposal and tokenizer implementation.