AI featuresllms-txt

llms.txt

View as Markdown

llms.txt is a standard for exposing website content to AI developer tools. Fern implements this standard, automatically generating and maintaining an llms.txt Markdown file so AI tools can discover and index your documentation. For single pages, agents can also fetch Markdown directly.

llms.txt is a root-level file Fern serves to non-human consumers, alongside robots.txt. robots.txt decides which crawlers reach your site and what AI training signals you broadcast; llms.txt shapes what AI agents receive once they do.

Example of using llms.txt

What llms.txt contains

llms.txt contains a lightweight summary of your documentation site with each page distilled into a one-sentence description and URL. For sites with API endpoints, it also links to your OpenAPI specification as a standalone, machine-readable file so AI tools can parse your full API schema directly. For sites with WebSocket channels, it also links to your AsyncAPI specification.

llms.txt is available at any level of your documentation hierarchy (/llms.txt, /docs/llms.txt, /docs/ai-features/llms.txt, etc.).

Example: Eleven Labs llms.txt.

Fern doesn’t generate llms-full.txt. Full-site concatenation exceeded most model context windows, added heavy serving overhead, and saw little use compared to llms.txt combined with per-page Markdown access. Use llms.txt to discover pages and fetch them individually.

Page descriptions

llms.txt includes page descriptions pulled from frontmatter. Fern uses the description field if present, otherwise falls back to subtitle.

Frontmatter
1---
2title: Fern Docs
3subtitle: Build beautiful documentation websites with Fern.
4---

The output format depends on whether you’re requesting an individual page or a section:

.../page/llms.txt
# Fern Docs
> Build beautiful documentation websites with Fern.

Learn more

Customize LLM output

Exclude pages, filter content with tags, or serve your own custom files.

Agent directives

Configure the default directive prepended to every page served to AI agents.

Analytics and integration

Track LLM traffic and surface llms.txt endpoints to readers.