One link per line: URL | title | short note. Title and note are optional.
# Your site name ## Pages
Upload it to the root of your site so it loads at https://yourdomain.com/llms.txt.
What an llms.txt file looks like
llms.txt is plain Markdown with a fixed structure: a title, a one-paragraph summary, optional background, then sections of links with short notes. A model, or the tool feeding it, can read it in one pass and decide which pages to fetch:
# Example Analytics
> Example Analytics is a privacy-friendly web analytics tool for small
> businesses. This file lists our product docs and guides.
Pricing is per website, not per user. The free plan covers up to
10,000 monthly pageviews.
## Docs
- [Quickstart](https://example.com/docs/quickstart/): Add the tracking script and verify your first pageview
- [API reference](https://example.com/docs/api/): REST endpoints for exporting data
## Optional
- [Changelog](https://example.com/changelog/)The format rules
| Part | Rule |
|---|---|
| H1 | Required, first line, and the only H1: the site or project name. |
| Blockquote | Recommended. A short summary of what the site is. |
| Details | Optional paragraphs or lists with key facts. No headings here. |
| H2 sections | Zero or more, each holding a Markdown list of links: - [Title](URL): notes. |
| “Optional” section | An H2 named Optional marks links that can be skipped when the model's context is short. |
| Location | The root of the domain: /llms.txt. |
The proposal also suggests offering a Markdown version of each page at the same URL with .md appended, so tools can read content without HTML, navigation and scripts. Documentation platforms increasingly do this automatically.
How to write a useful llms.txt
- Curate, don't export. The sitemap import is a starting point. Cut it down to the pages that answer real questions about what you offer.
- Write notes for each link. “Pricing” tells a model little; “Pricing: plans from $9/month, billed per website” lets it answer without fetching the page.
- State the facts models get wrong. Old product names, discontinued plans, regions you don't serve. The details section is the place.
- Use absolute URLs. The file is often copied or loaded on its own, where relative links break.
- Keep it current. A stale llms.txt pointing to removed pages is worse than none. Regenerate it when your navigation changes.
llms.txt, robots.txt and sitemap.xml compared
| robots.txt | sitemap.xml | llms.txt | |
|---|---|---|---|
| Purpose | Allow or block crawling | List every indexable URL | Curated guide to the most useful pages |
| Audience | All crawlers | Search engines | Language models and AI tools |
| Format | Directives | XML | Markdown |
| Enforced or used by Google | Yes | Yes | No |
If your aim is to keep AI companies from training on your content, llms.txt won't do it. Block their crawlers in robots.txt instead, and check the rules with the Robots.txt Tester.
Does llms.txt actually matter?
Honestly: not for search rankings, so far. Google has said it doesn't use the file, and its Search Advocate John Mueller has compared it to the keywords meta tag. None of the large AI search products has documented fetching it while crawling.
Where it does help is with tools that act on a user's request. When a developer points a coding assistant or an AI agent at your documentation, a clean llms.txt gets it to the right pages without wading through navigation and marketing copy. Many developer documentation sites, including Anthropic's, Stripe's and Cloudflare's, publish one for exactly that reason. For a product with documentation or an API, it takes an hour and costs nothing. For a small blog, your time is better spent on the content itself and on clear structured data, which the Schema Markup Generator helps with.
Frequently asked questions
What is llms.txt?
llms.txt is a Markdown file at the root of a website, such as example.com/llms.txt, that gives large language models a short description of the site and a curated list of links to its most useful pages. Jeremy Howard of Answer.AI proposed the format in September 2024.
Does llms.txt improve SEO or rankings in AI search?
There is no evidence that it does. Google has said it doesn't use llms.txt, and none of the major AI search products has documented reading it when crawling. Its practical use today is giving coding assistants and AI agents a clean map of your documentation when someone points them at it.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers which URLs they may fetch. llms.txt doesn't allow or block anything; it only suggests which pages are worth reading. To stop AI crawlers, use robots.txt rules for their user agents.
What is the difference between llms.txt and llms-full.txt?
llms.txt is an index: a summary plus links. llms-full.txt is a widely used companion convention, popularised by documentation platforms, that puts the full text of the documentation into one file so a model can load everything at once. It isn't part of the original proposal.
Which pages should I include?
The pages someone would need to understand and use what you offer: product or service overviews, documentation, pricing, key guides and policies. Ten to fifty well-described links beat an export of the whole sitemap. Put useful but skippable pages under an Optional section.
Where do I put the llms.txt file?
At the root of the domain, so it loads at https://yourdomain.com/llms.txt, served as plain text. In most frameworks that means the public or static folder; in WordPress, upload it to the site root or use an SEO plugin that generates one.
Related Tools
- Robots.txt Tester & Generator — Test URLs against a live or pasted robots.
- Meta Tag Checker — Check a page's title, meta description, robots directives, canonical, viewport and hreflang tags, with a Google result preview and a fix for each issue.
- Schema Markup Generator — Generate valid JSON-LD structured data for 31 schema types — Article, Product, FAQ, Local Business, Organization, Review and more.
- SEO Title Checker — Check a title tag for length, keyword position and truncation, preview it in desktop and mobile results, and get a 0–100 headline score.
- SEO Slug Generator — Turn any title or phrase into a clean, SEO-friendly URL slug.
- Page Indexability Checker — Instantly analyze whether a webpage can be indexed by search engines by checking HTTP status codes, robots.