Configuration

Additional content sources

View as Markdown

Extend Ask Fern’s knowledge beyond your core documentation by adding additional content sources like internal FAQs, support tickets, blog posts, and knowledge base articles.

There are two APIs for adding content. Each one helps you index a different type of data source:

  • Documents API - Upload markdown documents with precise control over what gets indexed
  • Websites API - Provide URLs to websites that Ask Fern will automatically crawl and index

Documents API

Make a request to the Documents API to index markdown documents with precise control over what gets indexed. This is ideal for content that isn’t publicly accessible, like internal documentation, support ticket summaries, or proprietary knowledge base articles. Upload the full markdown content directly along with an optional title and URL.

Example
1{
2    "document": "Ferns are plants native to the tropical and subtropical regions of the world. They are characterized by their fronds, which are large, leaf-like structures that are often found in the understory of forests.",
3    "title": "What are ferns?",
4    "url": "https://en.wikipedia.org/wiki/Fern"
5}

The URL is used solely for citations—Ask Fern doesn’t crawl it. You provide the full content in the document field.

Websites API

Make a request to the Websites API to crawl and index publicly accessible web content like marketing sites and blog posts. Provide a base URL, and Ask Fern will automatically crawl and index the website.

You can use optional filters to control what gets crawled:

  • domain_filter - Restrict crawling to a specific subdomain. For example, help.example.com will only crawl pages on that subdomain, not www.example.com or docs.example.com
  • path_filter - Restrict crawling to a specific section of the site. For example, /getting-started will only crawl URLs containing /getting-started in the path, like docs.example.com/getting-started or docs.example.com/getting-started/sdks, but not docs.example.com/api-reference
Example
1{
2    "base_url": "https://docs.example.com",
3    "path_filter": "/getting-started"
4}

The API returns a job_id to track the crawling progress. When referenced, Ask Fern cites the original URL where the content was found.