Tunnels

​

Overview

A Chalk container has no inbound port. It reaches the Chalk API, not the other way around, which is what keeps it isolated — and what normally puts ssh out of reach.

chalk session proxy closes that gap. It connects a TCP port inside a container to its own standard input and output, so any local client that can be pointed at a command instead of a socket can talk to a server running in the container. ssh is exactly such a client: that is what ProxyCommand is for.

The container still opens nothing. The SSH connection is encrypted end to end between your machine and the container’s sshd, and it travels inside a stream that Chalk has already authenticated with your Chalk credentials — so reaching a container requires both a Chalk token and an SSH key.

​

Requirements

Three things have to be true before a container is reachable this way.

The container must be host-backed. Sessions are only available on containers created with --compute-class host. A Kubernetes-backed container rejects every session call with container sessions require a host-backed container.

The image must already contain an sshd. Containers have no outbound internet, so apt-get install openssh-server inside a running container will fail. The SSH server has to be baked into the image.

The image must contain socat, nc, or bash. The proxy runs one of these inside the container to make the loopback connection. Most base images already carry at least one; socat is the most reliable.

​

Building an image with sshd

Bake the server and your public key into the image. This example starts from Debian, but the shape is the same anywhere: install openssh-server and socat, generate a host key, drop in an authorized_keys, and bind to loopback.

FROM debian:trixie-slim

RUN apt-get update \
 && apt-get install -y --no-install-recommends openssh-server socat ca-certificates \
 && rm -rf /var/lib/apt/lists/*

# A host key has to exist in the image, or sshd has nothing to present.
RUN ssh-keygen -q -t ed25519 -N '' -f /etc/ssh/ssh_host_ed25519_key

# Only the tunnel can reach sshd, so it never needs to listen on a public address.
RUN mkdir -p /run/sshd /root/.ssh && chmod 700 /root/.ssh \
 && printf '%s\n' \
      'ListenAddress 127.0.0.1' \
      'PermitRootLogin prohibit-password' \
      'PasswordAuthentication no' \
      'AuthorizedKeysFile /root/.ssh/authorized_keys' \
      > /etc/ssh/sshd_config.d/chalk.conf

COPY id_ed25519.pub /root/.ssh/authorized_keys
RUN chmod 600 /root/.ssh/authorized_keys

CMD ["/usr/sbin/sshd", "-D", "-e"]

Build and push it to a registry your Chalk environment can pull from, or build the same Dockerfile through Chalk with Image.from_dockerfile — see Images. Then start a container from it. The image’s own entrypoint is what keeps sshd running, so there is no --entrypoint here:

chalk container run \
  --image my-registry/dev-box:latest \
  --name my-box \
  --compute-class host \
  --lifetime 8h

​

Configuring ssh

chalk sandbox configure-ssh writes the necessary block into ~/.ssh/config. Check what it will write first:

chalk sandbox configure-ssh --dry-run

# >>> chalk sandbox configure-ssh >>>
# Managed by 'chalk sandbox configure-ssh'. Edits between these markers are
# overwritten on the next run; delete the markers too to stop managing it.
#
# 'ssh chalk-sbx-<sandbox name or id>' tunnels to a Chalk sandbox over the Chalk API.
# The sandbox must be host-backed ('--compute-class host') and running an sshd.
Host chalk-sbx-*
  User root
  ProxyCommand sh -c 'exec "/usr/local/bin/chalk" session proxy --dir "/path/to/project" "${0#chalk-sbx-}"' %n
  # Sandboxes are ephemeral and regenerate their host keys with them, so
  # recording those keys would only produce false mismatch warnings. The
  # connection is authenticated by Chalk before ssh ever sees it.
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  LogLevel ERROR
# <<< chalk sandbox configure-ssh <<<

Then write it:

chalk sandbox configure-ssh

The block is delimited by markers and rewritten in place on each run, so it is safe to re-run after moving the chalk binary or switching projects. It is placed at the top of the file, because ssh keeps the first value it finds for each option and a catch-all Host * block later in the file would otherwise win.

Any host you name with the chalk-sbx- prefix now resolves to a container, addressed by either its name or its ID:

ssh chalk-sbx-my-box
ssh chalk-sbx-54b4aa4c-6adf-46d0-a3ed-a92a24f19052

The prefix is what keeps the pattern from matching real hosts. %n hands ssh’s original host name to sh, which strips the prefix back off before the name reaches the proxy — ssh_config has no string editing of its own.

Use --prefix, --user, and --port when the defaults do not fit, and --ssh-config to write somewhere other than ~/.ssh/config:

chalk sandbox configure-ssh --prefix box- --user dev --port 2222

​

Using the connection

Once ssh works, everything built on ssh works. Copy files:

scp ./train.py chalk-sbx-my-box:/workspace/
rsync -av ./src/ chalk-sbx-my-box:/workspace/src/

Forward a port, so a server inside the container answers on your machine — useful for a notebook, a profiler UI, or a debugger:

ssh -N -L 8888:127.0.0.1:8888 chalk-sbx-my-box

Attach an editor. VS Code’s Remote-SSH and the JetBrains remote clients read the same ~/.ssh/config, so chalk-sbx-my-box appears as a target with no further setup.

Interactive latency is the network round trip to your Chalk region and nothing more — the tunnel itself adds about a millisecond. A container in the same region as your laptop feels like any other remote host; one across the country feels like one across the country.

​

Tunnelling other services

ssh is the common case, not the only one. chalk session proxy will carry any TCP connection, so a database, a debugger, or an inference server inside a container can be reached the same way with --port.

Point it at a Postgres running inside the container:

chalk session proxy my-box --port 5432

On its own that only wires the service to the command’s own stdin and stdout. To get a local port you can connect to, put a listener in front of it. With socat:

socat TCP-LISTEN:15432,reuseaddr,fork \
  EXEC:'chalk session proxy my-box --port 5432'

psql -h 127.0.0.1 -p 15432 -U postgres

Each incoming connection starts its own proxy, and its own connector process inside the container, which is cleaned up when the connection ends.

If the container already has an sshd, prefer an ssh port forward — it is one process, it multiplexes every forwarded port over a single tunnel, and it needs nothing on your machine beyond ssh:

ssh -N -L 15432:127.0.0.1:5432 -L 8888:127.0.0.1:8888 chalk-sbx-my-box

​

How it works

ssh does not have to speak TCP. Given a ProxyCommand, it runs that command and speaks the SSH protocol over the command’s standard input and output, leaving the transport to somebody else. chalk session proxy is that somebody:

1. It resolves the container and opens an authenticated, bidirectional stream to the Chalk API — an ordinary outbound HTTPS connection from your machine.
2. Chalk starts one connector process inside the container (socat, nc, or bash), attached to that stream, which opens a TCP connection to 127.0.0.1:22.
3. Bytes move in both directions: ssh’s stdout to the container’s sshd, and back.

Everything ssh sees is a byte stream that happens to be tunnelled, so nothing above it — authentication, channels, forwarding, SFTP — knows the difference.

Two details matter in practice. Nothing but the proxied bytes is ever written to stdout, because stdout is the connection. And the connector inside the container is killed when the local side goes away, so a connection leaves no process behind — even though a Chalk session’s process would normally outlive the stream that started it.

​

Troubleshooting

container sessions require a host-backed container — the container was created without --compute-class host. Recreate it; the setting cannot be changed in place.

chalk session proxy: no socat, nc, or bash in the container — the image has no connector. Add socat to the image.

Connection closed by UNKNOWN port 65535 — the proxy reached the container but sshd did not answer on the port. Check that it is running and bound where you expect:

chalk session exec my-box sh -- -c 'ss -ltn || netstat -ltn'

User root not allowed because account is locked — sshd refuses accounts with no password set, even for key authentication. Unlock the account in the image (passwd -u root), or log in as a user that is not locked.

Authentication fails with the right key — confirm which key ssh offered with ssh -v, and that authorized_keys in the image matches it and is mode 600.

​

Related

• Sandbox — the isolation model containers run under.
• Images — building the image that carries your sshd.
• Volumes — persistent storage, for work that should outlive a container.