Functions

---

​

Overview

A Chalk Function is a Python callable that you deploy onto a Scaling Group and register in the Chalk Catalog. Decorate a function with @chalkcompute.function and it becomes reachable from anywhere Chalk runs: local scripts and notebooks, other functions, and (because of the Catalog registration) the feature engine, including Chalk SQL, resolvers, the query server, and Chalk DataFrames (chalkdf) via F.catalog_call(). Arguments and results are serialized automatically; you write ordinary Python and Chalk handles scheduling, scaling, and delivery.

On top of what a bare Scaling Group provides, functions add async / streaming invocation, nested calls, automatic retries, concurrency caps, rate limits, and request-queue-driven autoscaling. Reach for a function when you want any of those features, or when you want the callable to be visible to the rest of Chalk.

Functions support three return styles:

• Single value: return a result and the caller receives it.
• Async generator: yield results incrementally and the caller receives them as they’re produced. See Streaming results for an end-to-end example.
• Nested calls: a function can call other functions, composing pipelines without manual orchestration. See Nested function calls for a pipeline example.

To define and deploy a function, see Defining a function.

For jobs with multiple steps and separate task environments, see Workflows.

---

​

Calling a function

Call a deployed function from Python with the chalkcompute SDK, using .remote() or .defer(), or from inside Chalk through the Chalk Catalog. Functions have no HTTP or REST route, and the Python SDK is the only published client for calling them.

​

Calling from SQL

Call a deployed function on each row with functions.<name>(...). For example, if you deploy an extract_entities function that finds people and organizations in text, call it on each transaction’s memo:

SELECT
    transaction_id,
    functions.extract_entities(memo) AS entities
FROM transactions

functions.extract_entities(memo) resolves the same deployed function as catalog_call('function.extract_entities', memo). You can pass the same named call options, including metadata, row_metadata, timeout, and on_error:

SELECT
    transaction_id,
    functions.extract_entities(memo, timeout => 30, on_error => 'struct') AS extraction
FROM transactions

Use a quoted identifier for function names containing punctuation, such as functions."extract-entities"(memo). SQL normalizes function names to lowercase, including quoted names. To call a catalog entry with uppercase letters in its name, use catalog_call('function.ExtractEntities', memo). Existing catalog_call(...) and catalog_call_async(...) calls remain supported.

​

Calling from other languages

To call code that runs on Chalk from TypeScript or another language, use one of these routes:

• A resolver: put the logic in a resolver and query its features with the TypeScript client or another Chalk client.
• A scaling group: serve the logic as an HTTP app on a scaling group with authenticated=True, and send a Chalk access token as a bearer token on each request. A service can get a token from POST /v1/oauth/token.

---

​

How it works

A synchronous .remote() call goes through the environment’s gateway to a worker and returns the result. A .defer() call, or a .remote() call to a generator function, goes through three moving parts:

┌──────────┐         ┌──────────────────┐         ┌───────────┐
│  Caller  │──call──▸│  Function Queue  │──run───▸│  Worker   │
│          │◂results─│                  │◂results─│ (Python)  │
└──────────┘         └──────────────────┘         └───────────┘

• The caller (local script, notebook, another function) submits an invocation and receives results over a single bidirectional stream.
• The Function Queue durably accepts the work, assigns each call an ID, enforces configured policies (concurrency, rate limit, retry), and relays results back to the caller.
• The worker is the Python process that actually runs your function code.

Callers and workers are loosely coupled: the caller never opens a connection directly to a worker. Workers can scale up, scale down, or get rescheduled mid-flight and in-flight calls are not lost. This is what enables streaming async generators, nested calls, and policy-enforced dispatch.

​

Policy enforcement

The queue enforces the policies declared on each function (rate limit, concurrency, and retries) before dispatching work to the worker.

Policy	What it caps
rate_limit	Outbound requests per second / minute / hour, optionally shared across functions via a key.
concurrency	In-flight invocations at once, optionally shared across functions via a key. Defaults to 1 per function.
retries	Retries of a failed call, after a fixed delay. Any exception the function raises is retried until the count runs out. Applies to .defer(...) calls and to .remote(...) calls to generator functions.

Concurrency is acquired before dispatching, so a burst of queued calls backpressures on the semaphore rather than spawning unbounded tasks. Rate-limit tokens are consumed per attempt: each retry takes a fresh token.

​

Retries and streamed output

When a generator function has retries set, the queue holds each attempt’s output until that attempt succeeds or is the last one. The caller does not receive values from an attempt that is then retried, so a retry does not repeat values the caller already has.