Deploy functions

• Prompts: LLM prompts with model configuration and templating (see Deploy prompts)
• Tools: General-purpose code that LLMs can invoke to perform operations or access external data
• Scorers: Functions for evaluating LLM output quality (returning a number from 0 to 1)
• Workflows: Chains of two or more prompts for multi-step workflows
• Parameters: Configuration schemas for evaluations with runtime values (see Write parameters)

​

Composability

• A vector search tool that embeds a query, searches for relevant documents, and returns them
• A system prompt with instructions for how to retrieve content and synthesize answers using the tool

​

Deploy tools

import * as braintrust from "braintrust";
import { z } from "zod";

const project = braintrust.projects.create({ name: "calculator" });

project.tools.create({
  handler: ({ op, a, b }) => {
    switch (op) {
      case "add":
        return a + b;
      case "subtract":
        return a - b;
      case "multiply":
        return a * b;
      case "divide":
        return a / b;
    }
  },
  name: "Calculator",
  slug: "calculator",
  description: "A simple calculator that can add, subtract, multiply, and divide.",
  parameters: z.object({
    op: z.enum(["add", "subtract", "multiply", "divide"]),
    a: z.number(),
    b: z.number(),
  }),
  returns: z.number(),
  ifExists: "replace",
});

npx braintrust push calculator.ts

from typing import Literal

import braintrust
from pydantic import BaseModel, RootModel

project = braintrust.projects.create(name="calculator")

class CalculatorInput(BaseModel):
    op: Literal["add", "subtract", "multiply", "divide"]
    a: float
    b: float

class CalculatorOutput(RootModel[float]):
    pass

def calculator(op, a, b):
    if op == "add":
        return a + b
    elif op == "subtract":
        return a - b
    elif op == "multiply":
        return a * b
    elif op == "divide":
        return a / b
    else:
        raise ValueError(f"Unsupported operation: {op}")

project.tools.create(
    handler=calculator,
    name="Calculator",
    slug="calculator",
    description="A simple calculator that can add, subtract, multiply, and divide.",
    parameters=CalculatorInput,
    returns=CalculatorOutput,
)

braintrust push calculator.py

​

View and manage tools

​

Add tools to prompts

​

Call tools directly

import { invoke } from "braintrust";

const result = await invoke({
  projectName: "calculator",
  slug: "calculator",
  input: {
    op: "add",
    a: 5,
    b: 3,
  },
});

console.log(result); // 8

​

Deploy scorers

​

Deploy workflows

1. Navigate to Playgrounds
2. Click + Workflow
3. Add prompt nodes by selecting + in the comparison pane
4. Use template variables to pass data between prompts:
   • {{dataset.input}} - Access dataset inputs
   • {{input}} - Access previous prompt’s output
   • {{input.field}} - Access structured output fields
5. Save the agent

​

Invoke functions

• Slug: The unique identifier within a project for any function type (e.g., slug: "calculator")
• Global function name: Built-in Braintrust scorers only - globally unique functions like Factuality from autoevals

import { invoke } from "braintrust";

const result = await invoke({
  projectName: "my-project",
  slug: "my-function",
  input: { query: "hello" },
});

import { invoke } from "braintrust";

const score = await invoke({
  globalFunction: "Factuality",
  functionType: "scorer",
  input: {
    output: "The capital of France is Paris",
    expected: "Paris is the capital of France",
  },
});

​

Version functions

import { invoke } from "braintrust";

const result = await invoke({
  projectName: "calculator",
  slug: "calculator",
  version: "a1b2c3d4", // Pin to specific version
  input: { op: "add", a: 5, b: 3 },
});

​

Sync with SDK

• Push to Braintrust: Push changes from your codebase to the UI using braintrust push. You can push one or more files or directories. If you specify a directory, all files under that directory are pushed.
  Report incorrect code

  Copy

  Ask AI

  npx braintrust push calculator.ts
  
  Report incorrect code

  Copy

  Ask AI

  braintrust push calculator.py
  
  Python tools and prompts can be pushed to Braintrust using braintrust push, but pulling from Braintrust is not yet available for Python.
• Pull from Braintrust: Pull changes from the UI to your codebase using braintrust pull:
  Report incorrect code

  Copy

  Ask AI

  npx braintrust pull
  
  Use pull to:
  • Download functions from public projects so others can use them
  • Pin your production environment to a specific prompt version without routing requests through Braintrust
  • Review changes to functions in pull requests
  Currently, braintrust pull only works for TypeScript prompts and tools.

​

Handle dependencies

• TypeScript: Braintrust uses esbuild to bundle your code. Bundling creates a single JavaScript file containing all necessary code, reducing the risk of version mismatches and dependency errors when deploying functions. Since esbuild statically analyzes your code, it cannot handle:
  • Dynamic imports
  • Runtime code modifications
  • Native libraries like SQLite
• Python: Braintrust uses uv to cross-bundle dependencies to the target platform (Linux). This works for most binary dependencies, except for libraries that require on-demand compilation. If you encounter bundling issues, file an issue on GitHub.

​

Use the REST API

curl https://api.braintrust.dev/v1/function \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BRAINTRUST_API_KEY" \
  -d '{
    "project_name": "calculator",
    "slug": "calculator",
    "input": {
      "op": "add",
      "a": 5,
      "b": 3
    }
  }'

​

Function features

• Well-defined parameters and return types: Type-safe interfaces using Zod (TypeScript) or Pydantic (Python)
• Streaming and non-streaming invocation: Handle real-time and batch operations
• Automatic tracing and logging: All function calls are traced in Braintrust
• OpenAI argument format: Prompts can be loaded directly in OpenAI-compatible format
• Version control: Functions are automatically versioned with each deployment

​

Organize functions

import * as braintrust from "braintrust";

// Get a handle to the project (creates if it doesn't exist)
const project = braintrust.projects.create({ name: "my-project" });

// Use the project to create functions
project.tools.create({...});
project.prompts.create({...});
project.scorers.create({...});

​

Next steps

• Write scorers to create custom evaluation functions
• Deploy prompts that use your tools
• Monitor deployments to track function performance
• Manage environments to version functions across environments