Forward Impact Team

Codegen Internals

Why Generated Code is Installation-Specific

Generated gRPC code is never bundled in npm packages. Each installation runs fit-codegen to produce code tailored to its own set of services.

This design exists because external installations can define custom proto files for custom gRPC services. Guide dynamically picks up these services through the service management system (fit-rc) and the tool configuration in config/config.json. A custom proto file produces a custom service client that Guide can use at runtime — without any changes to the product code itself.

If generated code were bundled, it would only reflect the standard services and miss any custom services the installation defines.

Proto Distribution Model

Proto files are co-located with the packages that own them. Each package includes a proto/ subdirectory published in its npm tarball.

Package Proto files Role
@forwardimpact/guide common.proto, resource.proto Shared Guide types
@forwardimpact/svcagent agent.proto Agent orchestration
@forwardimpact/svcgraph graph.proto Graph queries
@forwardimpact/svcllm llm.proto LLM inference
@forwardimpact/svcmemory memory.proto Conversation memory
@forwardimpact/svctool tool.proto Tool execution
@forwardimpact/svctrace trace.proto Distributed tracing
@forwardimpact/svcvector vector.proto Vector search
User project proto/*.proto Custom services

fit-codegen auto-discovers proto/ subdirectories from all installed @forwardimpact/* packages in node_modules/, plus the project's own proto/ directory. All discovered directories are used as include paths so cross-file imports (e.g. import "common.proto") resolve correctly.

Pipeline

Proto discovery → Type compilation → Service generation → Definition generation → Symlink

1. Proto discovery

CodegenBase.collectProtoFiles() scans all discovered proto directories, deduplicates by filename, and ensures common.proto is ordered first (other protos import it).

2. Type compilation

CodegenTypes compiles all proto files into a single generated/types/types.js using the pbjs compiler from protobufjs-cli. Proto source files are also copied to generated/proto/ for runtime loading.

3. Service and client generation

CodegenServices generates gRPC service base classes and client stubs for each proto file that defines a service. Output structure: 

generated/services/
  {service}/
    service.js    — Base class for the server implementation
    client.js     — Client stub for calling the service
  exports.js      — Aggregates all services and clients

4. Definition generation

CodegenDefinitions generates pre-compiled gRPC service definitions used by librpc at runtime for serialization and deserialization: 

generated/definitions/
  {service}.js    — Pre-compiled service definition
  exports.js      — Aggregates all definitions

Finder.createPackageSymlinks() creates symlinks from librpc/generated/ and libtype/generated/ to the central generated/ directory, so relative imports in those packages resolve correctly.

Module Index

Module Purpose
base.js Shared utilities: proto parsing, template loading, rendering
types.js Proto-to-JavaScript type compilation via pbjs
services.js Service base class and client stub generation
definitions.js Pre-compiled gRPC definition generation
bin/fit-codegen.js CLI entry point: discovery, orchestration, bundling
templates/ Mustache templates for generated artifacts

How librpc Consumes Generated Code

librpc imports generated code via relative paths:

  • librpc/index.js imports ./generated/services/exports.js (service bases and clients)
  • librpc/base.js imports ./generated/definitions/exports.js (service definitions for gRPC registration)

These paths resolve through the symlink created by fit-codegen during the symlink step.

Adding Custom Services

External users follow this workflow to add a custom gRPC service:

  1. Define the proto — create proto/custom.proto in the project root with a service definition
  2. Run codegen — npx fit-codegen --all discovers the new proto automatically
  3. Implement the server — create a service using the generated base class
  4. Register the tool — add a tool endpoint mapping in config/config.json and a tool descriptor in config/tools.yml
  5. Use the client — createClient("custom") from librpc returns a configured client

Guide's agent will automatically see the new tool through the dynamic tool configuration.

Internal vs External Workflow

Step Internal (monorepo) External (npm)
Install bun install (workspaces) npm install @forwardimpact/guide
Proto source services/*/proto/, products/guide/proto/ node_modules/@forwardimpact/*/proto/ + project proto/
Run codegen just codegen or bunx fit-codegen --all npx fit-codegen --all
Output generated/ at monorepo root generated/ at project root