Volumes

---

​

Overview

Volumes provide persistent file storage that can be shared across sandboxes. Under the hood, volumes are backed by a Rust-based FUSE driver that mounts directly into the container’s filesystem. This is file-level storage, not block storage — workloads interact with volumes through normal filesystem operations, and the driver handles tiered caching and replication to object storage transparently.

Volumes can be managed through the Python SDK, the CLI, or the web dashboard.

---

​

Creating and using volumes

​

CLI

# Create a volume
chalk volume create chalk://my-data

# Upload a file
chalk volume cp ./local-model.bin chalk://my-data/models/latest.bin

# Upload a directory
chalk volume cp ./models/ chalk://my-data/models --recursive

# List contents
chalk volume ls chalk://my-data

# Download a file
chalk volume cp chalk://my-data/models/latest.bin ./downloaded-model.bin

​

Chalk Volume URIs

A Chalk Volume URI starts with chalk://VOLUME, where VOLUME is the volume name. Everything after the next / is a path inside that volume.

chalk://my-data/models/latest.bin
        └─────┘
        volume name

Chalk Volume URI	Meaning
chalk://my-data	Volume named my-data
chalk://my-data/models/latest.bin	File inside my-data
chalk://my-data:experiment	Fork experiment of my-data
chalk://my-data@3	Version 3 of my-data

​

Python SDK

from chalkcompute import Volume

vol = Volume("my-data")

# Upload files
vol.put_file("models/latest.bin", model_bytes)

# Batch upload for efficiency
with vol.batch_upload() as batch:
    batch.put("data/train.csv", train_csv)
    batch.put("data/eval.csv", eval_csv)

# Read files
data = vol.read_file("models/latest.bin")

# List files
for f in vol.listdir("data/"):
    print(f"{f.path}  {f.size} bytes")

​

Mounting into sandboxes

Create a sandbox with a volumes parameter to mount that volume before starting it. The mounted volume is available as an ordinary directory inside the sandbox:

from chalkcompute import Image, Sandbox, Volume

vol = Volume("my-data")
vol.put_file("inputs/message.txt", b"hello from the volume\n")

sandbox = Sandbox(
    image=Image.debian_slim("3.12"),
    name="volume-reader",
    cpu="1",
    memory="2Gi",
    volumes=[("my-data", "/data")],
).run()

result = sandbox.exec(
    "python3",
    "-c",
    "print(open('/data/inputs/message.txt').read())",
)
print(result.stdout_text)

sandbox.terminate()

​

Syncing a mounted volume

To persist modified files, run sync from the mount to commit modifications:

sync /data

This creates a new immutable version of the mounted volume. This can be run after writing files from a shell session or long-running process.

The mount also contains a control file at .chalk-volume/commit. Run it to commit the files instead of sync. The .chalk-volume directory is not saved in the volume.

/data/.chalk-volume/commit

To reload the mount with the latest committed version, run the reload control file:

/data/.chalk-volume/reload

​

Web browser

Volumes are also browsable from the Chalk dashboard. You can navigate the file tree, preview file contents, and upload or download files directly from the browser.

---

​

Architecture

​

FUSE driver

The volume mount is implemented as a Rust-based FUSE (Filesystem in Userspace) driver that runs inside the sandbox. The driver exposes the volume as a standard POSIX directory, so workloads don’t need special libraries or APIs to read and write files — any tool that works with the filesystem works with a volume.

The FUSE driver handles:

• Tiered caching. Frequently accessed files are cached on the sandbox’s local disk. Cold data is fetched from object storage on demand.
• Transparent replication. All persisted data is durably stored in object storage (S3 or GCS, depending on your environment). Local cache is ephemeral and rebuilt automatically.

​

Copy-on-write semantics

Volumes use batch copy-on-write. When a sandbox writes to a mounted volume, the writes are buffered locally on the sandbox’s filesystem. These local writes are not visible to other sandboxes until a commit is made to persist writes via a sync call and sandboxes reload their current state.

# Writes inside the sandbox are local until committed
sandbox.exec("cp", "output.parquet", "/volumes/my-data/output.parquet")

This design avoids partial-write visibility — other consumers of the volume see a consistent snapshot, not a stream of in-progress file mutations. If the sandbox terminates before sync is called, uncommitted writes are discarded.

---

​

Versioning and fork semantics

Every commit creates an immutable snapshot of the volume’s state. Past versions are retained and can be retrieved by version ID:

# List available versions
versions = vol.versions()

# Open a previous version (read-only)
old = vol.at_version(versions[-2].id)
data = old.read_file("models/latest.bin")

Because prior versions are immutable and cheaply addressable, volumes support fork semantics. You can spawn multiple sandboxes from the same volume version, let them diverge independently, and sync their results into separate version lineages — without copying the underlying data.

This is particularly useful for coding agents that need to explore multiple solution paths in parallel:

from chalkcompute import Image, Sandbox

base_version = vol.latest_version()

# Fork two sandboxes from the same starting state
sandbox_a = Sandbox(
    image=Image.debian_slim(),
    volumes=[vol.at_version(base_version.id).fork("sandbox_a")],
).run()
sandbox_b = Sandbox(
    image=Image.debian_slim(),
    volumes=[vol.at_version(base_version.id).fork("sandbox_b")],
).run()

# Each sandbox writes independently — no interference
sandbox_a.exec("python", "approach_a.py")
sandbox_b.exec("python", "approach_b.py")
sandbox_a.terminate()
sandbox_b.terminate()

Each sandbox’s writes are isolated until explicitly synced, and the original version remains available regardless of what the forks produce.

---

​

Setting up a volume bucket

To use volumes on your own EKS or GKE cluster, link an object storage bucket and bind it to the cluster. Chalk volumes are backed by that bucket.

The volume service manages file metadata, versions, and signed upload and download URLs. File contents live in your bucket. Grant bucket access to the cloud identity of the service handling volume requests:

Volume backend	Identity to use
Dataplane API server	The account associated with the dataplane-api-server Kubernetes service account in namespace chalk-dataplane-api-server
Legacy cluster manager	The account associated with the cluster-manager pod’s Kubernetes service account, configured through background persistence

Chalk routes versioned-volume requests to the dataplane API server when its endpoint is available. Older deployments can serve supported operations through the cluster manager. The two services can have different cloud identities. A running dataplane API pod still needs a volume bucket binding and a reachable endpoint before it can serve volume operations.

​

Bucket permissions

Apply these permissions to the volume backend’s identity for each environment.

GCP:

• Grant roles/storage.objectAdmin on the bucket to the volume service’s Google service account.
• Ensure that account has iam.serviceAccounts.signBlob on its signing service account, typically itself. Existing IAM grants may already provide this permission. A custom role containing iam.serviceAccounts.signBlob, or roles/iam.serviceAccountTokenCreator scoped to the signing service account, can provide signing access. See Google’s signed URL permissions.

AWS: The volume service obtains AWS credentials through its pod’s IRSA role. The role needs s3:GetObject, s3:PutObject, s3:DeleteObject, s3:ListBucket, and s3:GetBucketCors on the bucket.

​

Find the volume service identity

Select the environment you are configuring. Repeat the lookup for other environments because their service accounts can differ.

For the dataplane API server:

1. Open Kubernetes → Service accounts.
2. Set Filter by namespace to chalk-dataplane-api-server.
3. Open dataplane-api-server. Copy GCP Service Account for Google Cloud or AWS Role ARN for AWS.

For the legacy background-persistence deployment:

1. Open Kubernetes → All pods. In Pod visibility, enable Show non environment-scoped pods and clear the namespace filter if needed.
2. Open the cluster-manager pod, such as go-metrics-bus-writer.
3. Expand Service Account and copy GCP Service Account or AWS Role ARN.

The background-persistence setting Service Account Name identifies a Kubernetes service account. For GCP IAM grants, use the Google service account email shown in GCP Service Account. You can also find a pod’s Kubernetes account by name and namespace under Kubernetes → Service accounts.

​

CORS policy

The Chalk dashboard uploads and downloads volume files directly from the bucket using signed URLs, so the bucket needs a CORS rule that allows:

• Origin: the URL you open the dashboard at, for example https://chalk.ai
• Methods: GET and PUT
• Headers: Content-Type

Without it, dashboard uploads and downloads are proxied via the metadata server instead of going directly to the bucket, and the cluster’s volume health check reports a CORS error. To add the rule, follow Set up and view CORS configurations for GCS or Using cross-origin resource sharing for S3.

​

1. Create a cloud storage reference

Create the bucket first if you haven’t already. In the team navigation (click on the Chalk logo in the top left of the page), go to Cloud storage and click Link Storage. Fill in:

Field	Description
Storage Type	The bucket’s provider: Google Cloud Storage, AWS S3, or Azure Blob Storage.
URI	The bucket URI, optionally with a prefix (for example gs://my-bucket/volumes, s3://my-bucket/volumes, or https://myaccount.blob.core.windows.net/my-container).
Cloud Account	The saved cloud connection associated with the bucket. The volume service also needs the runtime permissions listed above.

​

2. Bind the storage reference to the cluster

In the team navigation click on Clusters and select the cluster to open the Settings tab. Under Volume Bucket Binding, select the cloud storage reference created in step 1 and click Save Binding. This sets the bucket the cluster’s volume service uses. To change or remove the binding later, select a different reference and save, or click Delete Binding.

​

Verify volume access

For a dataplane API deployment, open Settings → Connections in the environment and inspect the Dataplane API server checks for bucket-access or CORS errors. Then list volumes through the normal Chalk API, for example with chalk volume ls after selecting the environment with chalk login.

To verify writes and signed URLs, upload a small file to a test volume and download it again. Finding a service account in Kubernetes confirms its identity; the health checks and volume operations verify that the configured service can reach and use the bucket.