Sandbox Snapshots

A sandbox snapshot saves the state of a running sandbox at a point in time. You can restore it into new sandboxes without repeating initialization, such as installing dependencies, loading a model, or warming an application. Restore the same snapshot multiple times to start separate sandboxes from the same state, for example to run experiments or investigate a problem from a common starting point.

You can use the same capture mechanism in two ways:

Snapshot and fork: sandbox.snapshot() saves a reusable snapshot and leaves the source sandbox running. snapshot.restore() creates a new sandbox from it.
Suspend and restore: sandbox.suspend() saves the sandbox’s state and releases its compute resources, including its CPU, memory, and any GPUs. sandbox.restore() starts it again with the same sandbox ID.

Snapshots and suspension require a running sandbox on the host compute class and configured object storage. To check a sandbox’s status and compute class, use chalk sandbox get --name my-sandbox.

To check whether you can take a snapshot, create one from a disposable running sandbox. If the request reports that the snapshot store is not configured, ask your cluster administrator to check the volume bucket. A successful capture confirms that the sandbox can be captured; restoration also requires a compatible host.

What a snapshot captures

Chalk uses gVisor checkpointing to capture the sandbox’s process memory and runtime state together with its writable root filesystem and the default in-memory /tmp. For a supported NVIDIA GPU sandbox, Chalk also uses cuda-checkpoint to capture CUDA state and GPU memory.

The sandbox’s main process and processes it starts can continue from their captured state when restored. To preserve an initialized application or model in memory, start it as the sandbox’s entrypoint:

import chalkcompute

sandbox = chalkcompute.Sandbox(
    image="my-app:tag",  # An image containing /app/server.py
    entrypoint=["python", "/app/server.py"],
    compute_class=chalkcompute.ComputeClass.HOST,
).run()

Wait for the application to finish initializing before taking the snapshot. The return from run() indicates sandbox readiness, so use your application’s own readiness signal to determine when its state is ready to capture.

The following boundaries apply:

Commands started through exec: Processes started with Sandbox.exec, exec_start, exec_stream, or a CLI exec session, and their descendants, are killed on restore. This includes processes detached or backgrounded from an exec session. Files they wrote to the sandbox’s root filesystem before the snapshot are captured. Start a long-running process from the entrypoint if it must survive restore.
Bind-mounted data: Chalk volumes and host-backed empty-directory mounts are external to the checkpoint. The restored sandbox uses the recorded mounts, but the snapshot does not freeze or copy their contents. This also applies when one is mounted at /tmp. Commit volume writes and manage volume versions separately; see Volumes.
External connections and services: A snapshot does not capture the state of external services. Open network connections may be disconnected on restore, so applications should reconnect and handle retries. External side effects are not rolled back.
Copied state: Restored processes begin with the saved memory, including cached credentials, random generator state, and application identifiers. Refresh credentials and generate new identifiers or random seeds where each restored sandbox needs its own values.

When you create a reusable snapshot, Chalk pauses the workload during capture and then leaves the source sandbox running. Each snapshot.restore() creates a new sandbox with its own ID and root filesystem. The source and restored sandboxes can still interact through external services or shared volumes. Restoration uses the snapshot’s recorded resources and configuration and requires a compatible host, including compatible CPU, runtime, and GPU hardware and software. GPU snapshots support NVIDIA CUDA; AMD GPU snapshots are not supported. A restore can fail if a compatible host is unavailable or the source image reference no longer resolves to the same image. Pin images by digest when the image reference might change.

Suspend and restore a sandbox

Suspend a running sandbox when you want to stop using its compute resources and continue from its saved state later. After suspension succeeds, the sandbox no longer holds its CPU, memory, or GPUs. Restoring it allocates compatible compute resources and resumes the sandbox under the same ID. The capture and restore boundaries above also apply to suspension.

For example, clone a repository, leave a note in its working tree, and suspend the workspace until you are ready to continue:

import chalkcompute

sandbox = chalkcompute.Sandbox(
    compute_class=chalkcompute.ComputeClass.HOST,
).run()
sandbox.exec("git", "clone", "--depth", "1", "https://github.com/psf/requests.git", "/workspace/requests")
sandbox.exec("sh", "-c", "echo 'Investigate the failing login test' > /workspace/requests/NOTES.md")
print(sandbox.id)  # Save the ID so you can reconnect later.
sandbox.suspend()  # Release the sandbox's compute resources.

# Later, restore the same sandbox and workspace.
sandbox.restore()
print(sandbox.refresh().status)

sandbox.restore() initiates restoration. Once sandbox.refresh().status is running, inspect the working tree and continue without cloning it again:

print(sandbox.exec("git", "-C", "/workspace/requests", "status", "--short").stdout_text)  # ?? NOTES.md
sandbox.terminate()

If you are reconnecting from another process, use chalkcompute.Sandbox.from_id("sandbox-id") with the saved ID. With the CLI, restore waits for the sandbox to start:

chalk sandbox suspend --id <sandbox-id>
chalk sandbox restore --id <sandbox-id>

Create and restore a snapshot

Create a sandbox, write a file, and capture a snapshot. Restore it twice to create two independent sandboxes, then clean up the sandboxes and snapshot:

import chalkcompute

source = chalkcompute.Sandbox(
    image=chalkcompute.Image.debian_slim(),
    compute_class=chalkcompute.ComputeClass.HOST,
).run()
source.exec("sh", "-c", "printf 'ready' > /tmp/state.txt")

snapshot = source.snapshot()
print(snapshot.id)

first = snapshot.restore()
second = snapshot.restore()
print(first.exec("cat", "/tmp/state.txt").stdout_text)  # ready
print(second.exec("cat", "/tmp/state.txt").stdout_text)  # ready

first.terminate()
second.terminate()
source.terminate()
snapshot.delete()

snapshot() waits for a completed snapshot by default. Pass wait=False to skip the SDK’s completion polling, then call snapshot.wait() before restoring. This option does not guarantee that the initial capture request returns immediately. snapshot.refresh() updates its status, which can be pending, in_progress, completed, or failed. wait() raises SandboxSnapshotError on failure and TimeoutError if its timeout expires. snapshot.restore() waits for the new sandbox by default; it also accepts wait=False.

To use a saved snapshot later, load it by ID. A SandboxClient can also list snapshots for a source sandbox:

import chalkcompute

with chalkcompute.SandboxClient.from_env() as client:
    snapshot = client.get_snapshot("snapshot-id")
    print(snapshot.status)
    print(client.list_snapshots("sandbox-id"))
    restored = snapshot.restore()
    restored.terminate()
    snapshot.delete()

chalkcompute.SandboxSnapshot.from_id("snapshot-id") also loads a snapshot.

Snapshot registry and storage

Chalk records each snapshot in an environment-scoped registry, with its ID, source sandbox, status, and creation time. Open Compute > Snapshots in the dashboard to browse the registry, or use the SDK and CLI commands on this page. Listing and restoring snapshots require the appropriate permissions in the selected environment.

The checkpoint data and a manifest are stored in object storage. For managed host pools, Chalk uses the cluster’s configured volume bucket under a snapshots prefix. On your own cluster, see Setting up a volume bucket. Snapshots can contain application memory, filesystem data, and credentials that were present at capture time.

A reusable snapshot remains available after its source sandbox terminates. Delete snapshots you no longer need with snapshot.delete(), the CLI, or the dashboard. Deletion removes the registry entry and prevents future restores; it does not terminate the source or sandboxes already restored from it. Chalk attempts to delete the stored checkpoint data, but cleanup is best effort and objects can remain if cleanup fails. A snapshot used by a suspended sandbox or scaling group cannot be deleted until it is no longer in use.

Use the CLI

Find a sandbox’s ID and check that it is running on the host compute class:

chalk sandbox list --all
chalk sandbox get --name my-sandbox

The get output includes the sandbox ID, status, and compute class. list --all also shows suspended and terminated sandboxes. In Python, use sandbox.id to get the ID of a started sandbox, or chalkcompute.Sandbox.from_name("my-sandbox") to attach by name.

Create a snapshot by name:

chalk sandbox snapshot create --name my-sandbox

You can use --id <sandbox-id> instead of --name. The output includes a snapshot ID, which is separate from the sandbox ID. Use it to inspect the snapshot and fork it once its status is completed:

chalk sandbox snapshot get --snapshot-id <snapshot-id>
chalk sandbox fork --snapshot-id <snapshot-id>

chalk sandbox fork waits for the new sandbox to start. List and delete snapshots with:

chalk sandbox snapshot list --sandbox-id <sandbox-id>
chalk sandbox snapshot delete --snapshot-id <snapshot-id>

The list command returns one page at a time. Use the returned cursor with --cursor to retrieve the next page. You can filter by --status, for example --status completed.

Troubleshooting

A process is missing after restore: Processes started through exec, including backgrounded processes, are killed on restore. Start work that must continue from the sandbox’s entrypoint instead.
Volume data is missing or differs after restore: Mounted volume contents are outside the checkpoint. Commit writes before capturing and manage volume versions separately; see Syncing a mounted volume.
A connection fails after restore: Open network connections can be disconnected. Reconnect to external services and retry the operation as appropriate.
A restored sandbox stays pending or fails: Check its status and status message with chalk sandbox get --id <sandbox-id>. Restoration requires a compatible host with available capacity and the original image. If the problem persists, contact Chalk Support with the sandbox and snapshot IDs and the status message.
A request is denied: Check that you selected the intended environment and that your credentials have permission to manage sandboxes and snapshots. See Permissions.

If suspension fails or a suspended sandbox cannot be restored, contact Chalk Support with the error and sandbox ID. A failed suspension can leave the source paused, so do not assume it is still running.

​What a snapshot captures

​Suspend and restore a sandbox

​Create and restore a snapshot

​Snapshot registry and storage

​Use the CLI