All posts

From handwritten to generated—Cohere's SDK journey

From handwritten to generated—Cohere's SDK journey

Cohere is one of the leading companies in today’s AI rush. Cohere is known for its Retrieval Augmented Generation (RAG) toolkit that allows LLMs to accurately answer questions and perform tasks. Developers use Cohere’s API to accomplish this—and, often, rely on its SDK to make those API calls.

Today, Cohere builds its SDKs on Fern. Prior to Fern, Cohere dealt with a classic SDK challenge. Like most dev-products, their users utilized a variety of languages. This created a design problem: each language had different type systems, primitives, and concurrency models. Accordingly, if the SDKs across languages were line-for-line alike, they would implicitly be compromising advantages boasted by each language.

For instance, TypeScript has a native type system. It supports convenient features like discriminated unions, helpful for handling deserialized objects in a type-safe manner. Other languages like Go have no such first-class language support. So Cohere needed to generate SDKs that offered symmetrical support for Cohere’s API with asymmetrical design patterns. Anything else would be short-sighted.

Fern enabled Cohere to deliver idiomatic SDKs without having to staff an SDK team. If we were to say it in CS 101 terms, Cohere reduced an O(n) problem to O(1) via Fern. Today, Cohere doesn’t have to spend time triaging hordes of small SDK errors. Instead, they focus on crafting a centralized, tightly-defined OpenAPI spec that encompasses requests, responses, errors, and examples. Cohere entrusts Fern to take care of the rest—SDK generation alongside publishing to targets like GitHub and package managers (e.g. npm, PyPI).

Let’s discuss Cohere’s implementation in detail.

Cohere used to rely on handwritten SDKs

Prior to discovering a robust automation tool like Fern, Cohere published a Go and Typescript SDK. However, both SDKs required significant maintenance, stealing (expensive) engineering team. In short, handwriting SDKs was simultaneously time-consuming and low-quality—human error creates inconsistent practices and poor code. Additionally, because SDKs were staffed by different engineers with different issue stacks, they were updated at different frequencies. This created a drift in feature coverage between SDKs.

This put Cohere in a tricky spot. SDKs had profuse errors and were collectively inconsistent. GitHub issues sometimes took months to resolve. Eventually, Cohere decided they needed to explore auto-generated SDKs. After evaluating severals tools, they settled on Fern.

Fern appealed to Cohere for a few reasons. First, it was open-source. This addressed some reasonable concerns about underlying transparency. It also dissolved fears of being commercially locked-in to a closed-source solution. Second, Fern worked with the widely accepted OpenAPI spec. And third—and most importantly—Fern could seamlessly generate well-designed, tightly-written SDKs for both Go and Typescript.

Within months, the results were obvious. GitHub issues were resolved in days. SDKs stayed auto-updated and consistent across repositories. This meant less errors, less developer hours, and better SDK experiences for users.

Today, Cohere continues to build its Typescript and Go SDKs via Fern. Let’s dive into how.

How Cohere uses Fern

Cohere manages an internal monorepo where they build their OpenAPI spec. OpenAPI, previously known as Swagger, is an open-source API specification. Cohere then syncs their OpenAPI repo with a separate repo centered around Fern’s specification file. From there, the SDKs are built, including the TypeScript SDK and Go SDK.

Cohere’s TypeScript SDK

The TypeScript SDK was Cohere’s first SDK re-vamp powered by Fern. Typescript was an apt choice given its wide-spread popularity for both frontend and backend applications.

Cohere’s TypeScript SDK can be installed via npm, yarn, or whatever package manager is preferred. It can be easily imported and configured with an API key thereafter.

import { CohereClient } from "cohere-ai";

const cohere = new CohereClient({
    token: "YOUR_API_KEY",
});

Cohere’s SDK offers async functionality. This is particularly helpful because AI processing often takes non-trivial time.

(async () => {
    const response = await cohere.chat({
        message: "Hello cohere!",
    });

    console.log("Response: ", response);
})();

Cohere’s Javascript SDK also has functionality to support streaming, directly plugging into streaming API endpoints.

(async () => {
    const stream = await cohere.chatStream({
        message: "Hello cohere!",
    });

    for await (const chat of stream) {
        if (chat.eventType === "text-generation") {
            process.stdout.write(chat.text);
        }
    }
})();

All of these functions are strongly-typed, making them easy to explore in modern text editors such as VSCode.

Cohere’s Go SDK

After TypeScript, Cohere launched Go. Go has more limitations than Typescript when it comes to type-safety. For instance, TypeScript supports discriminated unions—helpful for handling deserialized objects in a type-safe way. However, Go has no equivalent out-of-the-box support. Regardless, Fern makes it easy to export a Go SDK from the same origin as the TypeScript SDK’s, all without compromising TypeScript’s type-safety.

Reading time

3 to 5 minutes

Industry

AI

Fern Stack

Docs + API

Date

Jan 20, 2024

Author

Danny Fern

Time to value

1 month