AI Router

The Chalk AI Router is a managed, OpenAI-compatible LLM gateway hosted alongside your Chalk deployment. Point any OpenAI-compatible client at the router and it forwards your request to the right provider — OpenAI, Anthropic, or Google Gemini — behind a single API. Because the router sits in front of every provider, it is also where you centralize the things you do not want scattered across application code: API keys and budgets, rate limits, automatic fallback between models, and provider credentials.

The same endpoint also serves the Anthropic Messages API, so Anthropic SDKs and Claude Code route through the router without a translation layer.

The router is hosted with the Chalk API and requires no installation or self-hosting.

​

Endpoint

The router exposes the standard OpenAI-compatible paths under this base URL:

The same base URL also serves the Anthropic Messages API. Anthropic SDKs append their own /v1 segment to the base URL they are given, so those paths read with the extra segment:

Use whichever format your client already speaks — the router resolves the requested model against your configured providers either way, and a key’s restrictions are enforced on both. See Use with Claude Code for a client configured this way.

If you are on a dedicated or self-hosted Chalk deployment, replace api.chalk.ai with your own API server host. You can find it in the apiServer field of chalk config --format json. Requests are routed to the calling environment’s router, selected by the X-Chalk-Env-Id header described below.

​

Authentication

Every request carries an issued router API key as a bearer token, plus the environment to route to:

Router API keys are issued and revoked from the dashboard — see API keys below. A key is shown only once when it is issued, so copy it immediately.

Router API keys are sensitive: they spend against your configured providers. Treat them like any other secret, scope them with a model allow-list and daily budget, and revoke any key you suspect is compromised.

​

Using the router

The router speaks the OpenAI API, so any OpenAI-compatible client works with no code changes beyond the base URL, key, and environment header.

from openai import OpenAI

client = OpenAI(
    base_url="https://<your-chalk-api-host>/v1/router",
    api_key="<YOUR_ROUTER_API_KEY>",
    default_headers={"X-Chalk-Env-Id": "<env-id>"},
)

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)

The same request with curl:

curl https://<your-chalk-api-host>/v1/router/chat/completions \
  -H "Authorization: Bearer $CHALK_ROUTER_API_KEY" \
  -H "X-Chalk-Env-Id: <env-id>" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

The model you request is resolved against the providers you have configured. A key’s provider restriction, model allow-list, team, and daily token budget are all enforced on every request.

An Anthropic client is configured the same way — the base URL, the key, and the environment header — and calls the Messages API:

from anthropic import Anthropic

client = Anthropic(
    base_url="https://<your-chalk-api-host>/v1/router",
    api_key="<YOUR_ROUTER_API_KEY>",
    default_headers={"X-Chalk-Env-Id": "<env-id>"},
)

resp = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.content[0].text)

​

Use with Claude Code

Claude Code speaks the Anthropic Messages API, so the router works as its LLM gateway: model requests route through Chalk, where the provider credential, model allow-list, daily budget, and rate limits already live, and each developer holds a revocable router key instead of a provider key.

Set the base URL and the key in your shell:

export ANTHROPIC_BASE_URL=https://<your-chalk-api-host>/v1/router
export ANTHROPIC_AUTH_TOKEN=<YOUR_ROUTER_API_KEY>
export ANTHROPIC_CUSTOM_HEADERS="X-Chalk-Env-Id: <env-id>"

Use ANTHROPIC_AUTH_TOKEN rather than ANTHROPIC_API_KEY: the router reads the key from the Authorization header, and ANTHROPIC_API_KEY sends it in x-api-key. The router accepts either, but ANTHROPIC_API_KEY also needs a one-time approval prompt in Claude Code before it takes effect.

Confirm the router answers before starting Claude Code, so a failure points at the router rather than at your configuration:

curl -X POST "$ANTHROPIC_BASE_URL/v1/messages" \
  -H "Authorization: Bearer $ANTHROPIC_AUTH_TOKEN" \
  -H "X-Chalk-Env-Id: <env-id>" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'

A body starting with {"id":"msg_ means the URL, key, and environment header all work. An unknown-model error also confirms them, since the router authenticated the request before rejecting the model.

To make the configuration apply everywhere Claude Code runs, including background agents, move the values into the env block of ~/.claude/settings.json:

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://<your-chalk-api-host>/v1/router",
    "ANTHROPIC_AUTH_TOKEN": "<YOUR_ROUTER_API_KEY>",
    "ANTHROPIC_CUSTOM_HEADERS": "X-Chalk-Env-Id: <env-id>"
  }
}

Run /status in Claude Code to confirm: the Status tab shows an Anthropic base URL line with your Chalk host and an Auth token line naming the variable you set.

Two things to know before rolling this out:

• The router key replaces a claude.ai login. While ANTHROPIC_AUTH_TOKEN is set, a saved claude.ai subscription is unused and its limits do not apply; usage bills against the provider credential the router forwards. Features that need a claude.ai identity, such as Remote Control and voice dictation, are unavailable while it is set.
• Serve the router’s model names. If a router model route exposes a name that is not one of Claude Code’s built-in models, set CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1 so the /model picker is populated from the router’s /v1/models response.

​

Calling the router from Chalk

Because the router is OpenAI-compatible, you can also call it from inside feature computation with chalk.functions. F.openai_complete issues a chat completion while a query runs. To route it through the AI Router rather than calling OpenAI directly, set api_server to your Chalk API host — or set the OPENAI_BASE_URL environment variable on the execution host and omit the argument.

import chalk.functions as F
from chalk.features import _
from chalkdf import DataFrame

df = DataFrame({"questions": ["Recommend some movies like High and Low by Akira Kurosawa"]})
(
    df.with_columns(
        F.openai_complete(
            prompt=_.x,
            model="o4-mini",
            service_tier="flex",
            api_server="https://api.chalk.ai",  # route through the AI Router
        ).alias("result")
    )
    .run()
    .to_arrow()
)

F.openai_complete takes the following arguments:

It returns a struct with the completion text plus prompt_tokens, completion_tokens, total_tokens, model, finish_reason, and the upstream ratelimit_remaining_tokens / ratelimit_remaining_requests headers.

​

Throttling with rate limits

LLM calls are blocking and metered, so throttle them with the policy modifiers chained onto the expression. with_rate_limit caps how often the call may run across every expression that shares its key:

import chalk.functions as F
from chalk.features import _
from chalkdf import DataFrame

df = DataFrame({"x": ["Recommend some movies like Buzzard by Joel Potrykus"]})
(
    df.with_columns(
        F.openai_complete(prompt=_.x, model="o4-mini", service_tier="flex")
        .with_rate_limit(rate=3, key="openai", per="minute")
        .alias("result")
    )
    .run()
    .to_arrow()
)

The same modifier works on a feature defined in a @features class — chain .with_rate_limit(...) before selecting the .completion field.

with_rate_limit takes:

These policy modifiers compose — chain with_concurrency, with_rate_limit, and with_retry on the same expression to bound in-flight calls, cap the call rate, and retry transient failures with backoff:

(
    F.openai_complete(prompt=_.x, model="o4-mini", service_tier="flex")
    .with_concurrency(max_concurrent=4, key="my_api")
    .with_rate_limit(rate=100, key="my_api")
    .with_retry(max_retries=3, key="my_api")
)

Reusing one key ties the policies to the same logical resource, so every expression that calls "my_api" shares a single budget — here, at most 4 concurrent calls and 100 calls per second (per defaults to "second"), with each failed call retried up to 3 times.

​

API keys

Issue and revoke router keys from the AI Router → API keys tab of the Chalk dashboard. Each key can be scoped at issue time so that a leaked or over-eager client cannot do more than you intend:

Each key tracks its total token usage, and revoking a key takes effect immediately.

​

Teams

Teams group API keys so you can manage and limit access by team rather than key by key. Create a team, then assign keys to it at issue time. Rate limits can be scoped to a team so the whole team shares a single budget. Deleting a team leaves its keys working but removes their team association.

​

Rate limits

Rate limit policies cap throughput and protect against runaway spend. Each policy has a limit type, an optional target, and a scope:

• Limit type — tokens per minute, requests per minute, or concurrent requests.
• Target (optional) — narrow the policy to a specific provider or model. Leave it unset to apply across all traffic.
• Scope — apply the limit per individual key (token) or shared across a team.

Policies can be enabled or disabled without deleting them.

​

Fallback policies

A fallback policy keeps requests succeeding when a primary model is unavailable. For a primary model you define an ordered list of fallbacks; if a request against the primary fails, the router retries the fallbacks in order. For example:

gpt-4o → [gpt-4o-mini, claude-3-5-sonnet]

If gpt-4o is unavailable, the router tries gpt-4o-mini, then claude-3-5-sonnet, before giving up. Edit fallback rules in the AI Router → Fallback tab.

​

Providers

Configure provider credentials in the AI Router → Settings tab. The router supports:

For each provider you set an API key and, optionally, a base URL override to point at a custom or OpenAI-compatible endpoint (for example, Gemini’s OpenAI-compatible API). Credentials are applied at runtime — no redeploy is required — and each provider shows a connected or not-configured status.

​

Playground

The AI Router → Playground tab is an in-dashboard tester for the router. It authenticates with your Chalk session — no separate API key needed — and lets you exercise the router across three tabs:

• Chat — send streaming chat completions with a configurable system prompt, temperature, and max tokens.
• Embeddings — generate embeddings and inspect their dimensions and token counts.
• Images — generate images with configurable size, quality, and count.

The model picker is populated from the router’s /models endpoint, so it reflects the providers you have configured.

​

See also

• Authentication — service credentials and RBAC for the Chalk API.
• MCP Gateway — govern the external MCP servers your agents reach.
• LLM Toolchain — building AI features and agents on Chalk.