# Sandbox Snapshots
source: https://docs.chalk.ai/docs/sandboxes/snapshots

## Capture a sandbox's filesystem and runtime state, restore independent sandboxes, or suspend and resume one sandbox.

A sandbox snapshot saves the state of a running sandbox at a point in time. You can
restore it into new sandboxes without repeating initialization, such as installing
dependencies, loading a model, or warming an application. Restore the same snapshot
multiple times to start separate sandboxes from the same state, for example to run
experiments or investigate a problem from a common starting point.

You can use the same capture mechanism in two ways:

- Snapshot and fork: sandbox.snapshot() saves a reusable snapshot and leaves
the source sandbox running. snapshot.restore() creates a new sandbox from it.
- Suspend and restore: sandbox.suspend() saves the sandbox's state and
releases its compute resources, including its CPU, memory, and any GPUs.
sandbox.restore() starts it again with the same sandbox ID.

Snapshots and suspension require a running sandbox on the host compute class and
configured object storage. To check a sandbox's status and compute class, use
chalk sandbox get --name my-sandbox.

To check whether you can take a snapshot, create one from a disposable
running sandbox. If the request reports that the snapshot store is not configured,
ask your cluster administrator to check the
volume bucket. A successful capture confirms that
the sandbox can be captured; restoration also requires a compatible host.

### What a snapshot captures

Chalk uses gVisor checkpointing to capture the sandbox's process memory and runtime
state together with its writable root filesystem and the default in-memory /tmp. For a
supported NVIDIA GPU sandbox, Chalk also uses
cuda-checkpoint to capture CUDA state
and GPU memory.

The sandbox's main process and processes it starts can continue from their captured
state when restored. To preserve an initialized application or model in memory, start
it as the sandbox's entrypoint:

```
import chalkcompute

sandbox = chalkcompute.Sandbox(
    image="my-app:tag",  # An image containing /app/server.py
    entrypoint=["python", "/app/server.py"],
    compute_class=chalkcompute.ComputeClass.HOST,
).run()
```

Wait for the application to finish initializing before taking the snapshot. The
return from run() indicates sandbox readiness, so use your application's own
readiness signal to determine when its state is ready to capture.

The following boundaries apply:

- Commands started through exec: Processes started with Sandbox.exec,
exec_start, exec_stream, or a CLI exec session, and their descendants, are
killed on restore. This includes processes detached or backgrounded from an exec
session. Files they wrote to the sandbox's root filesystem before the snapshot are
captured. Start a long-running process from the entrypoint if it must survive restore.
- Bind-mounted data: Chalk volumes and host-backed empty-directory mounts are
external to the checkpoint. The restored sandbox uses the recorded mounts, but the
snapshot does not freeze or copy their contents. This also applies when one is
mounted at /tmp. Commit volume writes and manage volume versions separately; see
Volumes.
- External connections and services: A snapshot does not capture the state of
external services. Open network connections may be disconnected on restore, so
applications should reconnect and handle retries. External side effects are not
rolled back.
- Copied state: Restored processes begin with the saved memory, including cached
credentials, random generator state, and application identifiers. Refresh credentials
and generate new identifiers or random seeds where each restored sandbox needs its
own values.

When you create a reusable snapshot, Chalk pauses the workload during capture and then
leaves the source sandbox running. Each snapshot.restore() creates a new sandbox with
its own ID and root filesystem. The source and restored sandboxes can still interact
through external services or shared volumes.
Restoration uses the snapshot's recorded resources and configuration and requires a
compatible host, including compatible CPU, runtime, and GPU hardware and software.
GPU snapshots support NVIDIA CUDA; AMD GPU snapshots are not supported. A restore can
fail if a compatible host is unavailable or the source image reference no longer resolves
to the same image. Pin images by digest when the image reference might change.

### Suspend and restore a sandbox

Suspend a running sandbox when you want to stop using its compute resources and
continue from its saved state later. After suspension succeeds, the sandbox no longer
holds its CPU, memory, or GPUs. Restoring it allocates compatible compute resources
and resumes the sandbox under the same ID. The capture and restore boundaries above
also apply to suspension.

For example, clone a repository, leave a note in its working tree, and suspend
the workspace until you are ready to continue:

```
import chalkcompute

sandbox = chalkcompute.Sandbox(
    compute_class=chalkcompute.ComputeClass.HOST,
).run()
sandbox.exec("git", "clone", "--depth", "1", "https://github.com/psf/requests.git", "/workspace/requests")
sandbox.exec("sh", "-c", "echo 'Investigate the failing login test' > /workspace/requests/NOTES.md")
print(sandbox.id)  # Save the ID so you can reconnect later.
sandbox.suspend()  # Release the sandbox's compute resources.

# Later, restore the same sandbox and workspace.
sandbox.restore()
print(sandbox.refresh().status)
```

sandbox.restore() initiates restoration. Once sandbox.refresh().status is
running, inspect the working tree and continue without cloning it again:

```
print(sandbox.exec("git", "-C", "/workspace/requests", "status", "--short").stdout_text)  # ?? NOTES.md
sandbox.terminate()
```

If you are reconnecting from another process, use
chalkcompute.Sandbox.from_id("sandbox-id") with the saved ID. With the CLI,
restore waits for the sandbox to start:

```
chalk sandbox suspend --id <sandbox-id>
chalk sandbox restore --id <sandbox-id>
```

### Create and restore a snapshot

Create a sandbox, write a file, and capture a snapshot. Restore it twice to create two
independent sandboxes, then clean up the sandboxes and snapshot:

```
import chalkcompute

source = chalkcompute.Sandbox(
    image=chalkcompute.Image.debian_slim(),
    compute_class=chalkcompute.ComputeClass.HOST,
).run()
source.exec("sh", "-c", "printf 'ready' > /tmp/state.txt")

snapshot = source.snapshot()
print(snapshot.id)

first = snapshot.restore()
second = snapshot.restore()
print(first.exec("cat", "/tmp/state.txt").stdout_text)  # ready
print(second.exec("cat", "/tmp/state.txt").stdout_text)  # ready

first.terminate()
second.terminate()
source.terminate()
snapshot.delete()
```

snapshot() waits for a completed snapshot by default. Pass wait=False to skip
the SDK's completion polling, then call snapshot.wait() before restoring. This
option does not guarantee that the initial capture request returns immediately.
snapshot.refresh() updates its status, which can be pending, in_progress,
completed, or failed. wait() raises SandboxSnapshotError on failure and
TimeoutError if its timeout expires. snapshot.restore() waits for the new sandbox
by default; it also accepts wait=False.

To use a saved snapshot later, load it by ID. A SandboxClient can also list snapshots
for a source sandbox:

```
import chalkcompute

with chalkcompute.SandboxClient.from_env() as client:
    snapshot = client.get_snapshot("snapshot-id")
    print(snapshot.status)
    print(client.list_snapshots("sandbox-id"))
    restored = snapshot.restore()
    restored.terminate()
    snapshot.delete()
```

chalkcompute.SandboxSnapshot.from_id("snapshot-id") also loads a snapshot.

### Snapshot registry and storage

Chalk records each snapshot in an environment-scoped registry, with its ID, source
sandbox, status, and creation time. Open Compute > Snapshots in the dashboard to
browse the registry, or use the SDK and CLI commands on this page. Listing and
restoring snapshots require the appropriate permissions in the selected environment.

The checkpoint data and a manifest are stored in object storage. For managed host
pools, Chalk uses the cluster's configured volume bucket under a snapshots prefix.
On your own cluster, see Setting up a volume bucket.
Snapshots can contain application memory, filesystem data, and credentials that were
present at capture time.

A reusable snapshot remains available after its source sandbox terminates. Delete
snapshots you no longer need with snapshot.delete(), the CLI, or the dashboard.
Deletion removes the registry entry and prevents future restores; it does not terminate
the source or sandboxes already restored from it. Chalk attempts to delete the stored
checkpoint data, but cleanup is best effort and objects can remain if cleanup fails.
A snapshot used by a suspended sandbox or scaling group cannot be deleted until it is
no longer in use.

### Use the CLI

Find a sandbox's ID and check that it is running on the host compute class:

```
chalk sandbox list --all
chalk sandbox get --name my-sandbox
```

The get output includes the sandbox ID, status, and compute class. list --all
also shows suspended and terminated sandboxes. In Python, use sandbox.id to get
the ID of a started sandbox, or chalkcompute.Sandbox.from_name("my-sandbox")
to attach by name.

Create a snapshot by name:

```
chalk sandbox snapshot create --name my-sandbox
```

You can use --id <sandbox-id> instead of --name. The output includes a
snapshot ID, which is separate from the sandbox ID. Use it to inspect the
snapshot and fork it once its status is completed:

```
chalk sandbox snapshot get --snapshot-id <snapshot-id>
chalk sandbox fork --snapshot-id <snapshot-id>
```

chalk sandbox fork waits for the new sandbox to start. List and delete snapshots
with:

```
chalk sandbox snapshot list --sandbox-id <sandbox-id>
chalk sandbox snapshot delete --snapshot-id <snapshot-id>
```

The list command returns one page at a time. Use the returned cursor with --cursor
to retrieve the next page. You can filter by --status, for example --status completed.

### Troubleshooting

- A process is missing after restore: Processes started through exec, including
backgrounded processes, are killed on restore. Start work that must continue from
the sandbox's entrypoint instead.
- Volume data is missing or differs after restore: Mounted volume contents are
outside the checkpoint. Commit writes before capturing and manage volume versions
separately; see Syncing a mounted volume.
- A connection fails after restore: Open network connections can be disconnected.
Reconnect to external services and retry the operation as appropriate.
- A restored sandbox stays pending or fails: Check its status and status message
with chalk sandbox get --id <sandbox-id>. Restoration requires a compatible host
with available capacity and the original image. If the problem persists, contact
Chalk Support with the sandbox and snapshot IDs and the status message.
- A request is denied: Check that you selected the intended environment and that
your credentials have permission to manage sandboxes and snapshots. See
Permissions.

If suspension fails or a suspended sandbox cannot be restored, contact Chalk Support
with the error and sandbox ID. A failed suspension can leave the source paused, so
do not assume it is still running.