Embed Text Using a Shared Service

You build a product feature that needs semantic vectors. Examples are search, deduplication, clustering, and retrieval-augmented generation. You do not want each product to run its own inference backend. The embedding gRPC service exposes one method. Pass an array of strings. The method returns an array of dense vectors, in order. The inference model runs in a sidecar process. The service starts that process on boot. Products see only the typed gRPC surface.

In this guide you connect to the embedding service. You call its one RPC with a batch of inputs. You then check that the response shape matches what your feature expects.

Prerequisites

• Node.js 18+
• Generated client code available (run npx fit-codegen generate --all if not)
• Services running (npx fit-rc start)

Install the transport and type packages:

npm install @forwardimpact/librpc @forwardimpact/libtype

Architecture overview

The embedding service is a thin gRPC adapter over a Text Embeddings Inference (TEI) backend that runs as a sidecar process. On boot, the service spawns text-embeddings-router with a configured model (default BAAI/bge-small-en-v1.5) that listens on a local port. The service translates each gRPC request into a single HTTP call to TEI's OpenAI-compatible /v1/embeddings endpoint. It then translates the response back into the proto shape.

Product A ──┐                                    ┌── BAAI/bge-small-en-v1.5
            ├── gRPC ── embedding ── HTTP/JSON ──┤
Product B ──┘                                    └── (or other TEI model)

The service exposes one RPC:

RPC	Purpose	Request type
CreateEmbeddings	Embed one or more strings in a single call	embedding.EmbeddingsRequest

The response is embedding.EmbeddingsResponse with one EmbeddingVector per input string, in the same order.

Connect to the embedding service

Create an embedding client with the generated EmbeddingClient class:

import { createClient, createTracer } from "@forwardimpact/librpc";
import { createLogger } from "@forwardimpact/libtelemetry";

const logger = createLogger("my-product");
const tracer = await createTracer("my-product");

const embeddingClient = await createClient("embedding", logger, tracer);

createClient("embedding") resolves the host and port from config/config.json, creates an EmbeddingClient instance, and establishes the gRPC channel with automatic retry.

Embed one or more strings

Call CreateEmbeddings with an array of input strings:

import { embedding } from "@forwardimpact/libtype";

const request = embedding.EmbeddingsRequest.fromObject({
  input: [
    "the quick brown fox jumps over the lazy dog",
    "a fast auburn vulpine vaults a sluggish canine",
  ],
});

const result = await embeddingClient.CreateEmbeddings(request);
console.log(result.data.length); // 2
console.log(result.data[0].values.length); // depends on model

result.data is an array of EmbeddingVector objects in the same order as the input. Each EmbeddingVector has a values field that holds the dense embedding as a Float32-compatible array of numbers.

The vector dimensionality depends on the configured model. The default model (BAAI/bge-small-en-v1.5) emits 384-dimensional vectors. Check the model card for the configured model if you need a fixed shape.

Compare two strings

Most products use embedding vectors to compute cosine similarity against other embeddings:

function cosineSimilarity(a, b) {
  let dot = 0;
  let normA = 0;
  let normB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    normA += a[i] * a[i];
    normB += b[i] * b[i];
  }
  return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

const [a, b] = result.data.map((v) => v.values);
console.log(cosineSimilarity(a, b)); // ~0.7 for paraphrases, ~0.1 for unrelated

For storage and similarity search at scale, hand the vectors to the vector service. At that scale, do not compute cosine similarity in-process.

Handle backend failures

If the TEI sidecar process is unhealthy or unreachable, CreateEmbeddings throws an error whose message includes the HTTP status code from the backend:

try {
  await embeddingClient.CreateEmbeddings(request);
} catch (err) {
  console.error(err.message);
  // "TEI request failed: 503"
}

When the service starts but the sidecar fails, the service process exits. Restart with npx fit-rc restart embedding to relaunch both.

Verify

You reach the outcome of this guide when:

• createClient("embedding") connects without error.
• CreateEmbeddings returns one EmbeddingVector per input string, in order.
• Each values array has the expected dimensionality for the configured model.
• Errors from the TEI sidecar surface as exceptions on the client. They do not become silent empty responses.

If the connection fails, confirm the service runs with npx fit-rc status. Then check that config/config.json lists the correct host and port for the embedding service.

What's next