LLM Configuration

Configure an LLM provider to enable AI features in workflows, agents, and chat.

Supported providers

Pick OpenAI, Anthropic, or Google as the provider type. Available models are fetched live from the provider after you save the API key — Bifrost does not ship a hard-coded model list, so you always see what your account actually has access to.

The provider type controls which SDK Bifrost uses to talk to the endpoint. Any service that speaks the OpenAI or Anthropic wire format works under the matching provider type, even if it isn’t OpenAI or Anthropic itself — see OpenAI-compatible endpoints below.

Configure an LLM provider

1. Navigate to Settings > AI Configuration.

2. Select OpenAI, Anthropic, or Google.

3. Leave API Endpoint at its default for the official provider, or set a custom URL for a compatible endpoint.

4. Paste your API key.

5. Click Test Connection. Bifrost lists the models your key can access and caches them.

6. Pick a default model and save.

The page configures the provider, Chat model tiers, task-specific model overrides, embeddings, private memory, required instructions, and per-model pricing.

Configure Chat model tiers

The Chat Models section maps user-facing choices to provider model IDs:

Tier	Purpose
Fast	Low-latency, lower-cost requests
Balanced	Default general-purpose requests
Pro	Deeper or more demanding requests

Rename the labels to match your organization. Balanced is always available; leave the Fast or Pro model blank to hide that choice from Chat. Provider model IDs stay private to administrators.

Memory and required instructions

The Memory switch enables private, user-controlled memory for Bifrost-connected assistants. It depends on the embedding configuration. Users can still opt out and remove saved memories from User Settings → Preferences.

Global Instructions apply to every task performed through the default Bifrost MCP endpoint. Organization instructions are managed from each organization’s Manage → Instructions tab and apply after the global layer. See Manage AI memory and instructions.

OpenAI-compatible endpoints

The API Endpoint field overrides the base URL passed to the provider SDK. Anything that implements the OpenAI or Anthropic API surface will work — OpenRouter, Azure OpenAI, Together, Groq, Ollama, vLLM, LM Studio, etc.

Example: OpenRouter

OpenRouter exposes an OpenAI-compatible API and gives you a single key for hundreds of models across many providers.

1. Set Provider to OpenAI.

2. Set API Endpoint to https://openrouter.ai/api/v1.

3. Paste your OpenRouter key (sk-or-...) into API Key.

4. Click Test Connection. Bifrost calls OpenRouter’s /models endpoint and populates the model dropdown with everything available to your key — anthropic/claude-..., openai/gpt-..., meta-llama/..., and so on.

5. Pick a default model (the OpenRouter model ID, e.g. anthropic/claude-sonnet-4-5) and save.

Tip

OpenRouter model IDs include the upstream provider as a prefix (openai/gpt-4o, anthropic/claude-sonnet-4-5). Use the full prefixed ID when overriding model= per call.

Other compatible endpoints

Service	Provider type	API Endpoint
Azure OpenAI	OpenAI	https://<resource>.openai.azure.com/openai/deployments/<deployment>
Ollama (local)	OpenAI	http://localhost:11434/v1
LM Studio (local)	OpenAI	http://localhost:1234/v1
Groq	OpenAI	https://api.groq.com/openai/v1
Together	OpenAI	https://api.together.xyz/v1

The endpoint override also works under the Anthropic provider type for any service that implements Anthropic’s Messages API.

Embedding configuration

For knowledge base / RAG features, configure embeddings:

1. In Settings > AI Configuration, scroll to Embeddings.

2. Choose to use the same provider as the LLM or configure separately.

3. For OpenAI, embeddings use text-embedding-3-small (1536 dimensions).

Anthropic and Google do not provide the OpenAI-compatible embedding surface Bifrost uses here, so those providers require a dedicated OpenAI-compatible embedding key. When the LLM provider is OpenAI-compatible, the embedding card inherits its key and endpoint automatically — click Override to point embeddings at a different endpoint.

Caution

Changing the embedding model changes the vector dimensions. If the knowledge store already holds rows embedded with the old model, Bifrost prompts you to re-embed them — search won’t return those rows until they’re re-indexed. Use Reindex knowledge store to re-embed on demand; progress shows in the notification center.

Usage in workflows

Once configured, use AI in your workflows:

from bifrost import ai

@workflow
async def summarize_ticket(description: str):
    response = await ai.complete(f"Summarize this ticket: {description}")
    return {"summary": response.content}

Model overrides

Override the default model per call. Use whatever model ID your configured provider exposes:

response = await ai.complete(
    "Complex analysis task",
    model="anthropic/claude-sonnet-4-5",  # OpenRouter-style ID
)

Usage tracking

Bifrost tracks all AI usage:

• Input/output tokens per call
• Cost calculation based on configured pricing
• Aggregation by workflow, conversation, and organization

View usage reports in Settings > Usage Reports.

Pricing configuration

Configure per-model pricing in Settings > AI Pricing:

Field	Description
Provider	OpenAI or Anthropic
Model	Model ID as returned by the provider
Input Price	Cost per 1M input tokens
Output Price	Cost per 1M output tokens

Note

If pricing isn’t configured for a model, usage is still tracked but costs show as $0. For OpenRouter, set pricing using the prefixed model ID (e.g. anthropic/claude-sonnet-4-5) so it matches what the SDK reports.

Next steps

• Using AI in Workflows — Completions and streaming
• Knowledge Bases — RAG with hybrid semantic and full-text search