Evaluation Tasks

Write the task as an ordinary @chalkcompute.function. Its parameters bind to your dataset’s columns by name, so triage in the Quick start declares message: str and requires the dataset to have a message column.

import chalkcompute

@chalkcompute.function
def triage(message: str) -> str:
    ...

@chalkcompute.function starts deploying in the background as soon as the module runs, so consecutive definitions build concurrently. Evaluation.create waits on those handles before reading their immutable version IDs, so you do not need to call deploy() yourself for decorated functions.

---

​

Attaching deployed functions by reference

Attach functions that already exist rather than redefining them. In this example, dataset is the dataset uploaded in the Quick start:

import chalkcompute

evaluation = chalkcompute.Evaluation.create(
    "support-triage",
    dataset=dataset,
    task=chalkcompute.RemoteFunction.from_name("triage"),
    scorers=[chalkcompute.RemoteFunction.from_version_id("fn_intent_match_v2")],
)

RemoteFunction.from_name(name) resolves the currently selected version at lookup time, and creating the evaluation then pins that version. A decorated function registers under its Python name, so triage here is the def triage from the Quick start. RemoteFunction.from_version_id(id) attaches an exact, immutable version; from_id is a compatibility alias for it.

Deploy an imperative RemoteFunction before you use it. Evaluation.create never deploys on your behalf: an undeployed handle raises EvaluationError before any request goes out, and a bare callable raises TypeError.