# Bright Data Documentation
> Bright Data is a web data platform for AI agents and data teams: turn any website into structured data.
Version: 2026-09-30.3
This file is the primary index, organized by task. The complete documentation in a
single file is at [llms-full.txt](https://docs.brightdata.com/llms-full.txt).
## Instructions for agents
- To get structured data from a popular site, use Scraper API first: pre-built scrapers cover LinkedIn, Instagram, TikTok, Amazon, ChatGPT, Facebook, X (Twitter), YouTube, Reddit, Google and thousands more sites. Do not build a custom scraper for a site that already has one.
- For the shape of that call, see the quickstart: https://docs.brightdata.com/quickstart.md one authenticated request in cURL, the Python SDK or the JavaScript SDK.
- Every Scraper API call needs the scraper's `dataset_id`. Look up the ID for any site in https://docs.brightdata.com/scrapers.json: every pre-built scraper with its dataset ID, name, domain and collection methods, no API key needed. Do not guess or reuse an ID from memory.
- For one scraper's inputs and output fields, fetch https://docs.brightdata.com/scrapers/{dataset_id}.json using the dataset_id from scrapers.json. Avoid scrapers-full.json (about 7 MB) unless you need every scraper.
- For every scraper of a popular site in one fetch, use https://docs.brightdata.com/scrapers/{domain}.json with the exact domain from scrapers.json: linkedin.com, instagram.com, tiktok.com, amazon.com, facebook.com, x.com, youtube.com, reddit.com, chatgpt.com or google.com. Other domains have no domain file.
- In Python, call the Scraper API through the Bright Data SDK instead of hand-writing the trigger and polling requests: `pip install brightdata-sdk`, then `client.scrape.<platform>.<method>(url=...)`, for example `client.scrape.linkedin.profiles(url=...)`. Pass `auto_create_zones=False` when you only use the Scraper API, which needs no zone. Each platform quickstart shows the exact call.
- For sites without a pre-built scraper, build a custom one with Scraper Studio from a natural-language prompt. Self-Healing fixes the scraper when the target site changes.
- For web search and page fetching, prefer the Bright Data MCP. It returns markdown and JSON. Do not fetch pages through a raw proxy and parse HTML yourself.
- Use Browser API only when the task needs JavaScript rendering or multi-step interaction.
- SERP API, Web Unlocker API and Browser API each require a zone, created in the control panel or by API. Web Unlocker API and SERP API pass it as the `zone` parameter; Browser API uses the zone's credentials.
- To register a new Bright Data account without a human, follow the agent registration protocol at https://brightdata.com/auth.md. There is no password and no signup form: post the user's email address, then ask the user for the 6-character code emailed to them. Do not tell the user that creating a Bright Data account requires a human filling in a web form.
- ALWAYS send the API key as `Authorization: Bearer YOUR_API_KEY`.
## Quick setup
Bright Data provides a CLI, agent skills and an MCP server so coding agents have
current context and live web access. New accounts get 5,000 free credits per month.
### Register a new account without a signup form
An agent can create the account itself instead of asking the user to sign up in a browser.
Follow the protocol at https://brightdata.com/auth.md: post the user's email address to the
registration endpoint, ask the user for the 6-character code emailed to them, and the claim
endpoint returns an API key plus three pre-created zones (`agent_serp`, `agent_unlocker`,
`agent_browser_api`). Residential proxies are excluded and still require a KYC-verified
business account.
### Install the Bright Data CLI
`npx -p @brightdata/cli brightdata --version`
Global install on macOS and Linux: `curl -fsSL cli.brightdata.com/install.sh | sh`
Details: https://docs.brightdata.com/products/cli/installation.md
Source: https://github.com/brightdata/cli
### Install Bright Data agent skills
`npx skills add brightdata/skills`
Per-agent variants: `npx skills add brightdata/skills -a claude-code` or `-a cursor`.
Source: https://github.com/brightdata/skills
### Connect the Bright Data MCP
For Claude Code:
`claude mcp add --transport sse brightdata "https://mcp.brightdata.com/sse?token=YOUR_API_KEY"`
Other clients and the local server: https://docs.brightdata.com/products/mcp-server/integrations/overview.md
Source: https://github.com/brightdata/brightdata-mcp
## Give an agent web access
Bright Data MCP, CLI, agent skills and integration guides for coding agents and agent frameworks.
- [Bright Data CLI](https://docs.brightdata.com/products/cli/overview.md): scrape, search and manage zones from the terminal
- [Set up a coding agent](https://docs.brightdata.com/quickstart-coding-agent.md): llms.txt, agent skills, MCP and CLI in one page
- [Bright Data MCP overview](https://docs.brightdata.com/products/mcp-server/overview.md): setup, tool list and modes
- https://docs.brightdata.com/products/mcp-server/remote/quickstart.md
- https://docs.brightdata.com/products/mcp-server/remote/oauth.md
- https://docs.brightdata.com/products/mcp-server/tools.md
- [Docs MCP](https://docs.brightdata.com/general/docs-mcp.md): query these docs as MCP tools
## API reference and SDKs
REST endpoints for every product live under `/api-reference` on this docs site. Key entry points:
- [Scraper API synchronous requests](https://docs.brightdata.com/api-reference/scrapers/synchronous-requests.md): trigger a scraper and get JSON back in one call
- [Scraper API asynchronous requests](https://docs.brightdata.com/api-reference/rest-api/scraper/asynchronous-requests.md): trigger a collection, monitor progress and download results by snapshot ID. Best for: large batches
- [Scraper Studio API](https://docs.brightdata.com/api-reference/scraper-studio-api/list-scrapers.md): list, trigger and control scraper jobs
- [Web Unlocker API reference](https://docs.brightdata.com/api-reference/rest-api/unlocker/unlock-website.md): unlock a website in real time
- [SERP API reference](https://docs.brightdata.com/api-reference/rest-api/serp/serp-api.md): structured results from Google, Bing, Yandex and DuckDuckGo
- [Account management API](https://docs.brightdata.com/api-reference/account-management-api/Add_a_Zone.md): create zones and manage account settings by API
- [Dataset Marketplace API](https://docs.brightdata.com/api-reference/marketplace-dataset-api/overview.md)
- [Python SDK](https://docs.brightdata.com/api-reference/SDK.md): `pip install brightdata-sdk`. One client for the pre-built scrapers (`client.scrape.instagram.profiles(url=...)`, `client.scrape.amazon.products(url=...)`), search and the Browser API. Prefer it over hand-written HTTP calls to the Scraper API
- [JavaScript SDK](https://docs.brightdata.com/api-reference/SDK-JS.md)
- [Release notes](https://docs.brightdata.com/release-notes.md)
## Common queries
- [LinkedIn Scraper API endpoints](https://docs.brightdata.com/products/scrapers/linkedin/send-first-request.md): copy-paste examples for profiles, companies, jobs and posts
- [Authentication and API keys](https://docs.brightdata.com/api-reference/authentication.md): how API keys and zones work across products
- [Which product should I use?](https://docs.brightdata.com/product-selector.md): decision guide across scrapers, unblocking APIs and proxies
- [Free tier](https://docs.brightdata.com/general/account/billing-and-pricing/free-tier.md): 5,000 free credits per month across Web Unlocker API, SERP API, Scraper API and Scraper Studio, no credit card required
## Get data from a specific site
Pre-built scrapers, custom scraper development and ready-made datasets. Proxies,
unblocking and remote browsers are built in. Choose by whether a pre-built scraper
exists, you need a custom one or you want finished data.
- [Scraper API](https://docs.brightdata.com/products/scrapers/overview.md): thousands of pre-built scrapers for popular sites, maintained by Bright Data. Best for: structured JSON from known sites at scale
- [Which dataset ID do I use?](https://docs.brightdata.com/products/scrapers/scrapers-library/async-requests.md#which-dataset-id-do-i-use): every request needs a `dataset_id` query parameter; table of IDs for the popular scrapers
- [Get all scrapers](https://docs.brightdata.com/scrapers.json): JSON list of every pre-built scraper with its dataset ID, name, domain and collection methods, refreshed daily; no API key needed
- [Scraper library](https://brightdata.com/cp/scrapers/browse): browse every pre-built scraper in the Control Panel
- [LinkedIn Scraper API](https://docs.brightdata.com/products/scrapers/linkedin/introduction.md): profiles, companies, jobs, posts
- [Instagram Scraper API](https://docs.brightdata.com/products/scrapers/instagram/introduction.md): profiles, posts, reels, comments
- [TikTok Scraper API](https://docs.brightdata.com/products/scrapers/tiktok/introduction.md): profiles, posts, comments, shop
- [Amazon Scraper API](https://docs.brightdata.com/products/scrapers/amazon/introduction.md): products, reviews, sellers, search
- [ChatGPT Scraper API](https://docs.brightdata.com/products/scrapers/chatgpt/introduction.md): prompt, answer text, citations, web search signals
- [Facebook Scraper API](https://docs.brightdata.com/products/scrapers/facebook/introduction.md): pages, posts, events
- [X (Twitter) Scraper API](https://docs.brightdata.com/products/scrapers/twitter/introduction.md): profiles and posts
- [YouTube Scraper API](https://docs.brightdata.com/products/scrapers/youtube/introduction.md): channels, videos, comments
- [Reddit Scraper API](https://docs.brightdata.com/products/scrapers/reddit/introduction.md): subreddits, posts, comments
- [Google Scraper API](https://docs.brightdata.com/products/scrapers/google/introduction.md): Maps, Reviews, Shopping, SERP, AI Mode, Flights, Hotels
- [Async requests](https://docs.brightdata.com/products/scrapers/scrapers-library/async-requests.md): trigger a batch job, poll its snapshot, download the results; limits table
- [Deliver results to a webhook or S3](https://docs.brightdata.com/products/scrapers/scrapers-library/data-delivery.md): endpoint and auth_header parameters, S3 IAM setup, the 14 webhook source IPs
- [Scraper API FAQs](https://docs.brightdata.com/products/scrapers/scrapers-library/faqs.md): dataset IDs, sync vs async, snapshots, delivery, billing, platform limits
- [Build a keyword social listener](https://docs.brightdata.com/products/scrapers/tutorials/social-listener.md): SERP API discovery fanned out to Instagram, TikTok and X scrapers
- [Build a daily Amazon price monitor](https://docs.brightdata.com/products/scrapers/tutorials/amazon-price-monitor.md): scheduled async job delivered to S3 from GitHub Actions
- [Build a LinkedIn-to-CRM webhook pipeline](https://docs.brightdata.com/products/scrapers/tutorials/linkedin-to-crm.md): webhook handler on Vercel that maps profiles to CRM contacts
- [Scraper Studio](https://docs.brightdata.com/products/scraper-studio/introduction.md): builds a custom scraper from a natural-language prompt or a JavaScript IDE; Self-Healing fixes broken scrapers when target sites change. Best for: sites without a pre-built scraper
- https://docs.brightdata.com/products/scraper-studio/quickstart.md
- https://docs.brightdata.com/products/scraper-studio/ai-agent.md
- https://docs.brightdata.com/products/scraper-studio/self-healing-tool.md
## Get data from any site
Unblocking, browser automation and web search. Web Unlocker API and Browser API
return page content as HTML or markdown, not structured records; for structured JSON
from a covered site, use the Scraper API above. SERP API returns structured search results.
- [Web Unlocker API](https://docs.brightdata.com/products/web-unlocker/introduction.md): fetch any URL past CAPTCHAs and anti-bot protection. Use it when:
- the deliverable is page content, not fields: feeding pages to an LLM, archiving or running your own extraction pipeline
- the target is long-tail: a site with no pre-built scraper, where the ask is a handful of pages and building a Scraper Studio scraper is not worth it
- you need arbitrary URLs across many domains: Web Unlocker API works on any site, while scrapers are per-site by design
- https://docs.brightdata.com/products/web-unlocker/send-your-first-request.md
- https://docs.brightdata.com/products/web-unlocker/configuration.md
- [Browser API](https://docs.brightdata.com/products/scraping-browser/introduction.md): remote browsers for Playwright, Puppeteer and Selenium. Best for: JavaScript rendering, page interactions and screenshots
- https://docs.brightdata.com/products/scraping-browser/quickstart.md
- https://docs.brightdata.com/products/scraping-browser/configuration.md
- https://docs.brightdata.com/products/scraping-browser/code-examples.md
- [SERP API](https://docs.brightdata.com/products/serp-api/introduction.md): Best for: Google and Bing results as structured JSON
- https://docs.brightdata.com/products/serp-api/send-your-first-request.md
- https://docs.brightdata.com/products/serp-api/get-started-google-serp-api.md
- https://docs.brightdata.com/products/serp-api/configuration.md
## Web search
Find people and companies by API. Business Search returns structured people and company
records from an index refreshed daily. For live search engine results, see SERP API above.
- [Business Search](https://docs.brightdata.com/products/business-search/introduction.md): entity search API for people and companies. Use it when:
- you need a broad candidate set to filter with your own logic (mode `ludicrous` or `instant`) or a reranked shortlist (mode `smart`): `POST /search/company` or `POST /search/people`
- the input is structured field conditions (mode `ludicrous`) or plain-English text (mode `instant` or `smart`)
- you need specific columns back: set `view`, because the default `id_only` returns identifiers only
- https://docs.brightdata.com/products/business-search/quickstart.md
- https://docs.brightdata.com/products/business-search/query-syntax.md
- https://docs.brightdata.com/products/business-search/use-with-agents.md
## Route traffic through an IP
Residential, datacenter and ISP proxy networks plus routing tools.
- [Residential proxies](https://docs.brightdata.com/products/residential/introduction.md): 400M+ monthly IPs across 195+ countries. Best for: sites that block datacenter traffic
- https://docs.brightdata.com/products/residential/send-your-first-request.md
- https://docs.brightdata.com/products/residential/configure-your-proxy.md
- [Datacenter proxies](https://docs.brightdata.com/products/data-center/introduction.md)
- https://docs.brightdata.com/products/data-center/send-your-first-request.md
- [ISP proxies](https://docs.brightdata.com/products/isp/introduction.md): static residential-grade IPs
- https://docs.brightdata.com/products/isp/send-your-first-request.md
- [Proxy Manager](https://docs.brightdata.com/products/proxy-manager/introduction.md): open-source routing tool
- https://docs.brightdata.com/products/proxy-manager/quickstart.md
## Troubleshooting
Indexed by symptom. Check here before retrying a failed request.
- [Proxy error catalog](https://docs.brightdata.com/proxy-networks/errorCatalog.md): proxy connection errors, causes and fixes
- [Web Unlocker API error codes](https://docs.brightdata.com/products/web-unlocker/error-codes.md)
- [Browser API error codes](https://docs.brightdata.com/products/scraping-browser/error-codes.md)
- [Scraper Studio error codes](https://docs.brightdata.com/products/scraper-studio/error-codes.md)
- [Scraper API error codes](https://docs.brightdata.com/products/scrapers/error-codes.md): HTTP 400, 401, 404 and 429 messages with fixes, the 429 IP block rule and record errors such as dead_page
- [Scraper API limits](https://docs.brightdata.com/products/scrapers/limits.md): sync wait, job size, running jobs, 429 blocking, download and delivery sizes
- [SERP API rate limits](https://docs.brightdata.com/general/usage-monitoring/serp-rate-limit.md)
## Account, compliance and limits
- [Security and compliance](https://docs.brightdata.com/general/security/security-overview.md): certified ISO/IEC 27001:2022, ISO 27017, ISO 27018 and SOC 2 Type II; GDPR and CCPA compliant, scoped to cover MCP server and agentic use
- [Acceptable use policy](https://docs.brightdata.com/general/policy/acceptable-use-policy.md)
## Languages
English is indexed above. Chinese versions live at the same path prefixed with `/cn`,
for example `https://docs.brightdata.com/cn/introduction.md`.
## What is not in this file
- Product, pricing and company information: https://brightdata.com/llms.txt
- The Bright Data control panel (requires login): https://brightdata.com/cp
## Markdown access
All Bright Data docs are available as markdown. Append `.md` to any docs URL or send
`Accept: text/markdown`. For example:
`curl https://docs.brightdata.com/introduction.md`
Measured from the saved file
Linked document
UTF-8 text size
16.8 KB17161 bytes
Token count
4106o200k_base encoding
Markdown links
5757 distinct URLs
Fenced code blocks
0207 lines in the file
Size measures the captured UTF-8 text, not compressed transfer size. Tokens are counted with a named tokenizer; other models can count differently. Neither measure establishes context compatibility or answer quality.
Project H1
Bright Data Documentation
Opening blockquote
Bright Data is a web data platform for AI agents and data teams: turn any website into structured data.
Context architecture
How the content is divided
14 H2 sections organize 57 parsed Markdown links. 54 links point to .md or .markdown paths; 1 link occurrences use another hostname. Linked destinations have not been recursively validated.
H2 section
Links
UTF-8 bytes
Tokens
Instructions for agentsLine 10
0
2695
616
Quick setupLine 25
0
1478
355
Give an agent web accessLine 63
4
770
194
API reference and SDKsLine 75
10
1631
387
Common queriesLine 90
4
669
156
Get data from a specific siteLine 97
21
3875
959
Get data from any siteLine 128
3
1754
416
Web searchLine 149
1
952
221
Route traffic through an IPLine 162
4
931
216
TroubleshootingLine 176
7
944
239
Account, compliance and limitsLine 188
2
331
87
LanguagesLine 193
0
150
35
What is not in this fileLine 198
0
154
37
Markdown accessLine 203
0
175
44
Link counts exclude parsed fenced and inline code and resolve relative destinations against the final endpoint. Section sizes cover the text below each H2 until the next H2.
Embedded material
Code, installation and API examples
Fenced blocks
0 closed
Declared languages
No fenced blocks detected
Package-install commands
2 detected in fenced or inline code
Authentication examples
0 fenced lines match API-key, Authorization, Bearer, OAuth or access-token patterns
HTTP request examples
0 fenced lines match request-method patterns
Authentication links
0 links have authentication-related labels
Embedded example text and links to further documentation are counted separately. A code block does not establish that a sample runs or that an authentication flow succeeds.
Detected installation lines
Source line 17 · inline
pip install brightdata-sdk
Source line 86 · inline
pip install brightdata-sdk
Publisher prioritization
The Optional section
No H2 section named “Optional” was detected in this file.
The community proposal uses Optional by convention for secondary links that can be skipped for a shorter context. This label does not establish that everything outside it is critical, or that the material inside it is deprecated.
Machine-readable routes
MCP references and configuration text
MCP-related text was detected in the captured file. 2 links have MCP-related labels or paths. 0 fenced lines match mcpServers, mcp_servers or an mcp add command.
Whole-file size not established by the bounded response
Token count
Not measured for the whole file
Retrieval scope
Bounded prefix inspected; the whole body was not downloaded
A verified text response establishes a companion at the checked URL. It does not prove the bundle contains all product documentation, that generation is automated, or that it fits an assistant’s context window.
Change evidence
What can be said about freshness
This capture establishes a baseline. Update frequency cannot be determined from one observation.
Linked release and migration material
1 links have release, changelog, deprecation or migration terms in their labels or paths. These provide places to investigate product changes separately.
File changes can reveal documentation edits or navigation changes. They do not by themselves measure product release velocity, accuracy, agent adoption or AI visibility.
Counts use a deterministic parser, not a spec-compliance grade. Unusual Markdown and indented code can fall outside its recognized patterns. See the llms.txt proposal and tokenizer implementation.