Sync your gRPC specification | Fern Documentation

Keeping your gRPC specifications in sync with your codebase is crucial for maintaining accurate SDKs and documentation. Fern provides several automation options to streamline this process.

GitHub Actions

Use Fern’s GitHub Action to automatically update SDKs and documentation when your Protocol Buffer files change.

If you’re using local-generation: true in your generators.yml, you must install buf.

.github/workflows/fern.yml

1 name: Fern
2 
3 on:
4   push:
5     branches:
6       - main
7     paths:
8       - 'proto/**/*.proto'
9       - 'fern/**/*.yml'
10   pull_request:
11     branches:
12       - main
13     paths:
14       - 'proto/**/*.proto'
15       - 'fern/**/*.yml'
16 
17 jobs:
18   fern-check:
19     runs-on: ubuntu-latest
20     steps:
21       - name: Checkout repo
22         uses: actions/checkout@v4
23         
24       - name: Check gRPC spec
25         uses: fern-api/action@v0
26         with:
27           command: check
28         env:
29           FERN_TOKEN: ${{ secrets.FERN_TOKEN }}
30 
31   fern-generate:
32     runs-on: ubuntu-latest
33     if: github.event_name == 'push' && github.ref == 'refs/heads/main'
34     steps:
35       - name: Checkout repo
36         uses: actions/checkout@v4
37 
38       # Required if using local-generation: true
39       - name: Setup buf
40         uses: bufbuild/buf-setup-action@v1
41         with:
42           github_token: ${{ secrets.GITHUB_TOKEN }}
43 
44       - name: Generate SDKs and docs
45         uses: fern-api/action@v0
46         with:
47           command: generate
48         env:
49           FERN_TOKEN: ${{ secrets.FERN_TOKEN }}

Protocol buffer validation

Validate, lint, and check for breaking changes in your Protocol Buffer files before generating SDKs.

.github/workflows/proto-validation.yml

1 name: Protocol Buffer Validation
2 
3 on:
4   push:
5     paths:
6       - 'proto/**/*.proto'
7       - 'buf.yaml'
8   pull_request:
9     paths:
10       - 'proto/**/*.proto'
11 
12 jobs:
13   validate-proto:
14     runs-on: ubuntu-latest
15     steps:
16       - name: Checkout repo
17         uses: actions/checkout@v4
18 
19       - name: Setup Protocol Buffer Compiler
20         uses: arduino/setup-protoc@v2
21         with:
22           version: '23.4'
23 
24       - name: Setup buf
25         uses: bufbuild/buf-setup-action@v1
26         with:
27           github_token: ${{ secrets.GITHUB_TOKEN }}
28 
29       - name: Lint Protocol Buffers
30         run: buf lint
31 
32       - name: Validate Protocol Buffer files
33         run: |
34           find proto -name "*.proto" -exec protoc --proto_path=proto --descriptor_set_out=/dev/null {} \;
35 
36       - name: Check for breaking changes
37         run: buf breaking --against '.git#branch=main'
38 
39       - name: Generate and validate with Fern
40         uses: fern-api/action@v0
41         with:
42           command: check
43         env:
44           FERN_TOKEN: ${{ secrets.FERN_TOKEN }}

Optional: Push to Buf Schema Registry

If you’re using Buf Schema Registry, add a step to publish your schemas:

.github/workflows/buf-sync.yml

1 name: Buf Sync
2 
3 on:
4   push:
5     paths:
6       - 'proto/**/*.proto'
7       - 'buf.yaml'
8 
9 jobs:
10   buf-sync:
11     runs-on: ubuntu-latest
12     steps:
13       - name: Checkout repo
14         uses: actions/checkout@v4
15 
16       - name: Setup buf
17         uses: bufbuild/buf-setup-action@v1
18         with:
19           github_token: ${{ secrets.GITHUB_TOKEN }}
20 
21       - name: Lint Protocol Buffers
22         run: buf lint
23 
24       - name: Check for breaking changes
25         run: buf breaking --against '.git#branch=main'
26 
27       - name: Generate and push to Buf Registry
28         run: |
29           buf generate
30           buf push
31         env:
32           BUF_TOKEN: ${{ secrets.BUF_TOKEN }}
33 
34       - name: Generate SDKs with Fern
35         uses: fern-api/action@v0
36         with:
37           command: generate
38         env:
39           FERN_TOKEN: ${{ secrets.FERN_TOKEN }}

Optional: Customize buf configuration

Optionally create a buf.yaml file to customize buf’s linting rules, breaking change detection, and dependencies:

buf.yaml

1 version: v1
2 deps:
3   - buf.build/googleapis/googleapis
4   - buf.build/envoyproxy/protoc-gen-validate
5 lint:
6   use:
7     - DEFAULT
8   except:
9     - UNARY_RPC
10 breaking:
11   use:
12     - FILE

Auto-sync from source

Configure Fern to automatically pull Protocol Buffer files from various sources:

From git repository

generators.yml

1 api:
2   specs:
3     - spec:
4         git:
5           repository: https://github.com/your-org/proto-definitions
6           path: services/user_service.proto
7           branch: main
8   generators:
9     - name: fernapi/fern-typescript-sdk
10       version: 0.8.8

From local directory

generators.yml

1 api:
2   specs:
3     - spec: proto/user_service.proto
4       auto-sync: true
5   generators:
6     - name: fernapi/fern-typescript-sdk
7       version: 0.8.8

CI/CD integration

Integrate Fern into your existing CI/CD pipelines to automatically generate SDKs and documentation.

CircleCI

If you’re using local-generation: true in your generators.yml, you must install buf.

.circleci/config.yml

1 version: 2.1
2 
3 orbs:
4   fern: fernapi/fern@1.0
5 
6 workflows:
7   version: 2
8   build-and-generate:
9     jobs:
10       - build
11       - test:
12           requires:
13             - build
14       - validate-proto:
15           requires:
16             - build
17       - fern/generate:
18           requires:
19             - test
20             - validate-proto
21           filters:
22             branches:
23               only: main
24           context:
25             - fern-context
26 
27 jobs:
28   validate-proto:
29     docker:
30       - image: namely/protoc-all:1.51_1
31     steps:
32       - checkout
33       # Required if using local-generation: true
34       - run:
35           name: Install buf
36           command: |
37             curl -sSL "https://github.com/bufbuild/buf/releases/latest/download/buf-Linux-x86_64" -o /usr/local/bin/buf
38             chmod +x /usr/local/bin/buf
39       - run:
40           name: Validate Protocol Buffers
41           command: |
42             find proto -name "*.proto" -exec protoc --proto_path=proto --descriptor_set_out=/dev/null {} \;

GitLab CI

If you’re using local-generation: true in your generators.yml, you must install buf.

.gitlab-ci.yml

1 stages:
2   - build
3   - test
4   - validate
5   - generate
6 
7 variables:
8   FERN_TOKEN: $FERN_TOKEN
9 
10 build:
11   stage: build
12   script:
13     - echo "Building gRPC service..."
14 
15 validate-proto:
16   stage: validate
17   image: namely/protoc-all:1.51_1
18   script:
19     - find proto -name "*.proto" -exec protoc --proto_path=proto --descriptor_set_out=/dev/null {} \;
20   only:
21     changes:
22       - proto/**/*.proto
23 
24 generate-sdks:
25   stage: generate
26   image: fernapi/fern:latest
27   # Required if using local-generation: true
28   before_script:
29     - curl -sSL "https://github.com/bufbuild/buf/releases/latest/download/buf-Linux-x86_64" -o /usr/local/bin/buf
30     - chmod +x /usr/local/bin/buf
31   script:
32     - fern generate
33   only:
34     - main

Scheduled updates

Set up scheduled updates to ensure your SDKs stay current:

.github/workflows/scheduled-update.yml

1 name: Scheduled gRPC Update
2 
3 on:
4   schedule:
5     - cron: '0 2 * * 1'  # Every Monday at 2 AM UTC
6   workflow_dispatch:
7 
8 jobs:
9   update-proto:
10     runs-on: ubuntu-latest
11     steps:
12       - name: Checkout repo
13         uses: actions/checkout@v4
14         
15       - name: Sync Protocol Buffer files
16         run: |
17           # Sync from upstream proto repository
18           git subtree pull --prefix=proto https://github.com/your-org/proto-definitions main --squash
19           
20       - name: Generate with latest spec
21         uses: fern-api/action@v0
22         with:
23           command: generate
24         env:
25           FERN_TOKEN: ${{ secrets.FERN_TOKEN }}
26           
27       - name: Create PR if changes
28         uses: peter-evans/create-pull-request@v5
29         with:
30           token: ${{ secrets.GITHUB_TOKEN }}
31           commit-message: "chore: update Protocol Buffer definitions"
32           title: "Update Protocol Buffer definitions"
33           body: "Automated update of Protocol Buffer definitions from upstream repository"

Code generation from gRPC server

For servers that can generate their own Protocol Buffer definitions:

.github/workflows/auto-generate.yml

1 name: Auto-generate from gRPC server
2 
3 on:
4   push:
5     paths:
6       - 'src/**/*.py'  # Trigger on server code changes
7       - 'src/**/*.go'
8       - 'src/**/*.java'
9 
10 jobs:
11   generate-proto:
12     runs-on: ubuntu-latest
13     steps:
14       - name: Checkout repo
15         uses: actions/checkout@v4
16         
17       - name: Setup environment
18         uses: actions/setup-python@v4
19         with:
20           python-version: '3.9'
21           
22       - name: Install dependencies
23         run: |
24           pip install grpcio-tools
25           
26       - name: Generate Protocol Buffer files
27         run: |
28           python -m grpc_tools.protoc \
29             --proto_path=src/protos \
30             --python_out=. \
31             --grpc_python_out=. \
32             --descriptor_set_out=proto/service.protoset \
33             src/protos/*.proto
34           
35       - name: Convert to Protocol Buffer text format
36         run: |
37           protoc --decode_raw < proto/service.protoset > proto/user_service.proto
38           
39       - name: Generate SDKs
40         uses: fern-api/action@v0
41         with:
42           command: generate
43         env:
44           FERN_TOKEN: ${{ secrets.FERN_TOKEN }}

Monitoring changes

Track changes to your Protocol Buffer specifications:

generators.yml

1 api:
2   specs:
3     - spec: proto/user_service.proto
4   change-detection:
5     enabled: true
6     breaking-changes: error
7     notifications:
8       slack: ${{ secrets.SLACK_WEBHOOK }}
9       email: team@yourcompany.com
10   generators:
11     - name: fernapi/fern-typescript-sdk
12       version: 0.8.8

Multi-service sync

Sync different Protocol Buffer services for different components:

generators.yml

1 environments:
2   user-service:
3     specs:
4       - spec: proto/user_service.proto
5         overlays:
6           - user-service-overlay.yml
7     generators:
8       - name: fernapi/fern-typescript-sdk
9         version: 0.8.8
10         output:
11           location: npm
12           package-name: "@yourcompany/user-service-sdk"
13   order-service:
14     specs:
15       - spec: proto/order_service.proto
16         overlays:
17           - order-service-overlay.yml
18     generators:
19       - name: fernapi/fern-typescript-sdk
20         version: 0.8.8
21         output:
22           location: npm
23           package-name: "@yourcompany/order-service-sdk"
24   payment-service:
25     specs:
26       - spec: proto/payment_service.proto
27     generators:
28       - name: fernapi/fern-typescript-sdk
29         version: 0.8.8
30         output:
31           location: npm
32           package-name: "@yourcompany/payment-service-sdk"

gRPC reflection sync

Automatically sync Protocol Buffer definitions from running gRPC services that have server reflection enabled:

scripts/sync_from_reflection.py

1 import grpc
2 from grpc_reflection.v1alpha import reflection_pb2
3 from grpc_reflection.v1alpha import reflection_pb2_grpc
4 import subprocess
5 
6 def sync_from_grpc_reflection(server_address, output_dir):
7     """Sync Protocol Buffer definitions from gRPC reflection"""
8     
9     channel = grpc.insecure_channel(server_address)
10     reflection_stub = reflection_pb2_grpc.ServerReflectionStub(channel)
11     
12     # List services
13     request = reflection_pb2.ServerReflectionRequest(
14         list_services=""
15     )
16     
17     response = reflection_stub.ServerReflectionInfo(iter([request]))
18     
19     for resp in response:
20         if resp.HasField('list_services_response'):
21             for service in resp.list_services_response.service:
22                 print(f"Found service: {service.name}")
23                 
24                 # Get file descriptor for service
25                 file_request = reflection_pb2.ServerReflectionRequest(
26                     file_containing_symbol=service.name
27                 )
28                 
29                 file_response = reflection_stub.ServerReflectionInfo(iter([file_request]))
30                 
31                 for file_resp in file_response:
32                     if file_resp.HasField('file_descriptor_response'):
33                         # Save descriptor to file
34                         descriptor_path = f"{output_dir}/{service.name}.protoset"
35                         with open(descriptor_path, 'wb') as f:
36                             f.write(file_resp.file_descriptor_response.file_descriptor_proto[0])
37                         
38                         # Convert to text format
39                         proto_path = f"{output_dir}/{service.name}.proto"
40                         subprocess.run([
41                             'protoc',
42                             '--decode_raw',
43                             '--proto_path', output_dir,
44                             descriptor_path
45                         ], stdout=open(proto_path, 'w'))
46 
47 if __name__ == "__main__":
48     sync_from_grpc_reflection("localhost:50051", "proto/")

This ensures that any changes to your gRPC services are automatically reflected in your SDKs and documentation, maintaining consistency across your entire API ecosystem.