Choose an llms.txt entry point for the job

An llms.txt file may be a short map of useful pages, a categorized index, an instruction sheet for an agent, or a much larger content bundle. Start with the task and retrieval limits you actually have.

Steps to use

1. If an agent needs to find a guide, prefer a compact index with stable links and useful section names. Check how relative links should be resolved.
2. If a task needs a specific procedure, retrieve the linked, current documentation page. Do not assume the index itself contains the full instructions.
3. If the file contains a very large corpus, select the relevant page or section instead of loading everything by default. Check whether the vendor also offers smaller indexes.
4. Record the requested URL and final URL separately, then recheck redirected destinations and linked pages before depending on them.

Where judgment matters

A larger file is not necessarily more complete, accurate or useful. These records describe what Firecrawl retrieved on a date; they do not show which files models consume or whether a particular workflow succeeds.