llms.txt

llms.txt is a standard for exposing website content to AI developer tools. Fern implements this standard, automatically generating and maintaining an llms.txt Markdown file so AI tools can discover and index your documentation. For single pages, agents can also fetch Markdown directly.

llms.txt is a root-level file Fern serves to non-human consumers, alongside robots.txt. robots.txt decides which crawlers reach your site and what AI training signals you broadcast; llms.txt shapes what AI agents receive once they do.

What llms.txt contains

llms.txt contains a lightweight summary of your documentation site with each page distilled into a one-sentence description and URL. For sites with API endpoints, it also links to your OpenAPI specification as a standalone, machine-readable file so AI tools can parse your full API schema directly. For sites with WebSocket channels, it also links to your AsyncAPI specification. Fetching an endpoint page listed in llms.txt returns its parameters, body, response, and authentication as a readable property list.

Fern serves llms.txt at the site root and for every navigation node that contains pages: sections, products, versions, tabs, API References, and changelogs (/llms.txt, /docs/llms.txt, /docs/ai-features/llms.txt, etc.). Individual pages don’t have their own llms.txt. A request for /<page>/llms.txt returns a 301 redirect to the page’s Markdown at /<page>.md, with the query string preserved (for example, ?lang=python), so existing links and crawlers still receive the page content.

Example: Eleven Labs llms.txt.

Localized sites

For sites with localization enabled, each non-default locale gets its own llms.txt index at /{locale}/llms.txt (or /{basepath}/{locale}/llms.txt on basepath-mounted sites). Every link in that index is locale-prefixed, so guides/intro.md becomes /ja/guides/intro.md.

Section-level indexes work the same way: /{locale}/{section}/llms.txt returns the translated section index with locale-prefixed links. Pages that fall back to default-language content in a locale are marked noindex and left out of that locale’s index.

Custom llms.txt files (configured via agents in docs.yml) are only served for the default locale. Non-default locales always receive the auto-generated index.

Page descriptions

llms.txt includes page descriptions pulled from frontmatter. Fern uses the description field if present, otherwise falls back to subtitle.

---
title: Fern Docs
subtitle: Build beautiful documentation websites with Fern.
---

Each page appears in its section’s llms.txt as a link followed by its description:

- [Fern Docs](https://example.com/docs): Build beautiful documentation websites with Fern.

Learn more