Fern DefinitionEndpoints

Server-sent events and streaming APIs

Use the response-stream key to model streaming endpoints

View as Markdown
Pro and Enterprise feature

This feature is available only for the Pro and Enterprise plans. To get started, reach out to support@buildwithfern.com.

Specifying response-stream on an endpoints allows you to represent endpoint responses that are streaming.

JSON streaming

If your API returns a series of JSON chunks as seen below

1{ "text": "Hi, I am a" }
2{ "text": "chatbot. Do you have any"}
3{ "text": "questions for me"}

then simply specify the response under response-stream for your endpoint.

chat.yml
1service:
2  base-path: /chat
3  endpoints:
4    stream:
5      method: POST
6      path: ""
7      response-stream: Chat
8
9types:
10  Chat:
11    properties:
12      text: string

Server-sent events

If your API returns server-sent-events, with the data and event keys as seen below

1data: { "text": "Hi, I am a" }
2data: { "text": "chatbot. Do you have any"}
3data: { "text": "questions for me"}

then make sure to include format: sse.

chat.yml
1service:
2  base-path: /chat
3  endpoints:
4    stream:
5      method: POST
6      path: ""
7      response-stream:
8        type: Chat
9        format: sse
10
11types:
12  Chat:
13    properties:
14      text: string

Stream parameter

It has become common practice for endpoints to have a stream parameter that controls whether the response is streamed or not. Fern supports this pattern in a first class way.

Simply specify the stream-condition as well as the ordinary response and the streaming response:

chat.yml
1service:
2  base-path: /chat
3  endpoints:
4    stream:
5      method: POST
6      path: ""
7      stream-condition: $request.stream
8      request:
9        name: StreamChatRequest
10        body: 
11          properties: 
12            stream: boolean
13      response: Chat
14      response-stream:
15        type: ChatChunk
16        format: sse
17
18types:
19  Chat:
20    properties:
21      text: string
22      tokens: integer
23  ChatChunk:
24    properties:
25      text: string