llms.txt and llms-full.txt

llms.txt is a standard for exposing website content to AI developer tools. Fern implements this standard, automatically generating and maintaining llms.txt and llms-full.txt Markdown files so AI tools can discover and index your documentation.

Fern also serves clean Markdown instead of HTML on any page URL when the request includes an Accept: text/markdown header, ensuring agents only receive the content they need. Together, these features reduce token consumption by 90%+.

Generated files

Fern generates two files for LLMs:

• llms.txt contains a lightweight summary of your documentation site with each page distilled into a one-sentence description and URL. For sites with API endpoints, it also links to your OpenAPI specification as a standalone, machine-readable file so AI tools can parse your full API schema directly. For sites with WebSocket channels, it also links to your AsyncAPI specification.

• llms-full.txt contains complete documentation content including the full text of all pages. For API documentation, this includes your complete API Reference with resolved OpenAPI specifications and SDK code examples for enabled languages.

Both files are available at any level of your documentation hierarchy (/llms.txt, /llms-full.txt, /docs/llms.txt, /docs/ai-features/llms-full.txt, etc.).

Examples: Eleven Labs llms.txt, Cash App llms-full.txt.

Custom files

To serve your own llms.txt or llms-full.txt instead of the auto-generated versions, point to your files under the agents key in docs.yml:

1
agents:
2
  llms-txt: ./path/to/llms.txt
3
  llms-full-txt: ./path/to/llms-full.txt

Paths are relative to the docs.yml file. The CLI validates that each file exists and uploads it as part of your docs deployment. Your custom files are served at the root-level /llms.txt and /llms-full.txt endpoints. Nested paths (e.g., /api-reference/llms.txt) continue to use the auto-generated output.

You can provide one or both files. Any file you don’t specify falls back to the auto-generated version.

1
---
2
title: Fern Docs
3
subtitle: Build beautiful documentation websites with Fern.
4
---

# Fern Docs
> Build beautiful documentation websites with Fern.

/llms.txt?lang=python
/llms-full.txt?excludeSpec=true
/llms-full.txt?lang=python&excludeSpec=true

Control visibility

You can exclude whole pages from LLM endpoints (llms.txt and llms-full.txt) by adding noindex: true to the page’s frontmatter. Pages marked noindex aren’t indexed by search engines but remain visible in your sidebar navigation and can still be accessed directly by URL. To also hide a page from navigation, see Hiding content.

1
---
2
title: Internal notes
3
noindex: true
4
---

Within pages, use <llms-only> and <llms-ignore> tags to control what content is exposed to AI versus human readers on your documentation site. These tags affect all Markdown output intended for AI consumption: LLM endpoints, Copy page, and View as Markdown.

To preview and debug what AI sees for any page, you can append .md or .mdx to the page URL to view its Markdown source.

1
<llms-only>
2
  This technical context is only visible to AI, not on your documentation site.
3
</llms-only>

1
<llms-ignore>
2
{/* TODO: Update this section after v2 launch */}
3
</llms-ignore>
4
5
## Authentication
6
7
To authenticate your requests, include your API key in the header.
8
9
<llms-ignore>
10
  <Callout intent="info">
11
    Need an API key? [Sign up for free](https://example.com/signup) to get started.
12
  </Callout>
13
</llms-ignore>

Per-page directives

Every page served to AI agents is automatically prepended with a default directive that tells agents how to navigate your documentation programmatically:

For clean Markdown of any page, append .md to the page URL. For a complete documentation index, see https://docs.example.com/llms.txt. For full documentation content, see https://docs.example.com/llms-full.txt.

The URLs in the directive are generated from your site’s domain and basepath. The directive is injected after the frontmatter metadata section but before the page body, so agents see it first even if they truncate the rest of the page. It applies to individual page Markdown (.md/.mdx URLs) and to each page section within llms-full.txt, and human-facing documentation is unaffected.

To override the default, set a custom directive using the agents key in docs.yml:

1
agents:
2
  page-directive: "For a complete page index, fetch https://docs.example.com/llms.txt"

To disable the directive entirely, set page-directive to an empty string:

1
agents:
2
  page-directive: ""

Analytics and monitoring

The Fern Dashboard provides comprehensive analytics for llms.txt usage including:

• Traffic by LLM provider (Claude, ChatGPT, Cursor, etc.)
• Page-level breakdowns of bot vs. human visitors for Markdown and llms.txt files

This visibility helps you understand LLM traffic patterns and optimize your documentation for AI consumption.

Integrate llms.txt into your docs

Add buttons or navigation links to surface your llms.txt endpoints.

1
<Button href="/api-reference/llms-full.txt?lang=python&excludeSpec=true" target="_blank">
2
  Open Python API Reference for LLMs
3
</Button>

1
navbar-links:
2
  - type: dropdown
3
    text: LLMs
4
    icon: fa-solid fa-robot
5
    links:
6
      - text: Full docs
7
        href: /llms-full.txt
8
      - text: Python SDK
9
        href: /api-reference/llms-full.txt?lang=python&excludeSpec=true
10
      - text: TypeScript SDK
11
        href: /api-reference/llms-full.txt?lang=typescript&excludeSpec=true
12
      - text: Go SDK
13
        href: /api-reference/llms-full.txt?lang=go&excludeSpec=true