Guide
What is llms.txt?
The llms.txt file is an emerging convention for giving AI agents a concise Markdown map of a website's most useful public resources.
Robots.txt vs llms.txt
Robots.txt tells crawlers which URLs they may access. A sitemap lists URLs for search engines. LLMs.txt is different: it summarizes the site and points AI agents to curated resources that are easier to consume than a full navigation-heavy website.
What to include
- A clear H1 with the site or product name.
- A short blockquote summary of what the site does.
- Core links with one-line descriptions.
- Optional resources that are helpful but not central.
- Usage notes when summaries, citations, or API documentation need context.
Important limitation
LLMs.txt is not an access-control file and does not guarantee ranking or citation in AI answers. It is best treated as a helpful agent-readable index, not a replacement for good content, structured pages, sitemaps, and crawler policy.