Server Sent Events and Streaming APIs

Use the response-stream key to model streaming endpoints

Specifying response-stream on an endpoints allows you to represent endpoint responses that are streaming.

JSON streaming

If your API returns a series of JSON chunks as seen below

1{ "text": "Hi, I am a" }
2{ "text": "chatbot. Do you have any"}
3{ "text": "questions for me"}

then simply specify the response under response-stream for your endpoint.

chat.yml
1service:
2  base-path: /chat
3  endpoints:
4    stream:
5      method: POST
6      path: ""
7      response-stream: Chat
8
9types:
10  Chat:
11    properties:
12      text: string

Server sent events

If your API returns server-sent-events, with the data and event keys as seen below

1data: { "text": "Hi, I am a" }
2data: { "text": "chatbot. Do you have any"}
3data: { "text": "questions for me"}

then make sure to include format: sse.

chat.yml
1service:
2  base-path: /chat
3  endpoints:
4    stream:
5      method: POST
6      path: ""
7      response-stream:
8        type: Chat
9        format: sse
10
11types:
12  Chat:
13    properties:
14      text: string

Stream parameter

It has become common practice for endpoints to have a stream parameter that controls whether the response is streamed or not. Fern supports this pattern in a first class way.

Simply specify the stream-condition as well as the ordinary response and the streaming response:

chat.yml
1service:
2  base-path: /chat
3  endpoints:
4    stream:
5      method: POST
6      path: ""
7      stream-condition: $request.stream
8      request:
9        name: StreamChatRequest
10        body: 
11          properties: 
12            stream: boolean
13      response: Chat
14      response-stream:
15        type: ChatChunk
16        format: sse
17
18types:
19  Chat:
20    properties:
21      text: string
22      tokens: integer
23  ChatChunk:
24    properties:
25      text: string