AEO glossary

llms.txt

llms.txt is a proposed markdown file, suggested by Jeremy Howard, that gives AI agents a short, curated guide to a site's most useful content.

llms.txt is a proposal, not a standard. Its author, Jeremy Howard, writes: "We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content." The proposal was published in September 2024 and last modified in August 2026.

The format is simple. The only required part is "An H1 with the name of the project or site." A blockquote summary, more detail, and H2 sections listing links with short notes can follow. A section called "Optional" holds links an agent can skip when it needs a shorter context. The file can sit at the site root or under a path such as /docs/.

It does not control access. The proposal says "robots.txt and llms.txt have different purposes": robots.txt says which automated access is acceptable, while llms.txt is context that is "used on demand" when an agent is helping a user.

Support varies. OpenAI's and Perplexity's own docs sites publish an llms.txt, but none of the crawler pages from OpenAI, Anthropic or Perplexity says their bots read one. Google says you "don't need to create new machine readable files, AI text files, or markup" to appear in its AI features. Get crawler access and rendering right first.

Example

Markdown
A minimal llms.txt in the proposed format (example.com is a placeholder)
# Example Garden Supply

> Online garden supplies shop. Delivery is free on orders over $50.

## Help

- [Delivery and returns](https://www.example.com/help/delivery.md): times, costs and returns policy

## Optional

- [Company history](https://www.example.com/about.md)

Learn it properly

The course covers this in Make your site reachable by AI. Lesson 1 is free.

Related terms

Sources

  1. The /llms.txt file (opens in a new tab) · llmstxt.org (Jeremy Howard) · Updated 10 Aug 2026
  2. AI features and your website (opens in a new tab) · Google Search Central · Updated 10 Dec 2025
  3. Overview of OpenAI Crawlers (opens in a new tab) · OpenAI · Checked 28 Oct 2026
  4. Does Anthropic crawl data from the web, and how can site owners block the crawler? (opens in a new tab) · Claude Help Center · Updated 7 Apr 2026
  5. Perplexity Crawlers (opens in a new tab) · Perplexity · Checked 28 Oct 2026

All 30 glossary terms