Infrastructure
Build a Chalk engine image and push it to a registry you own, without creating a deployment — and mirror it into a separate cloud partition such as AWS GovCloud.
chalk deploy build builds the engine image for your project and pushes it to container
registries that you name. Unlike chalk apply, it does not deploy
anything.
Concretely, it does not:
:latest tag on your environment’s engine repository.The result is an image in a registry you control, and nothing else. Your active deployment keeps serving exactly what it was serving before.
Use chalk deploy build when the image is an artifact you want to handle yourself rather
than a change you want live:
If you just want to deploy, use chalk apply.
chalk deploy build --destination 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1Repeat --destination to push the same image to several registries in one build:
chalk deploy build \
--destination 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:2026-08-25 \
--destination 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:release-candidateThe build pushes to every destination in a single pass, so a second destination costs almost nothing.
On success the command prints the build’s identifier and the references it pushed to:
Build ID: chalk-build-argo-8f3c21d4
→ 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:2026-08-25
→ 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:release-candidate
The command runs from your project directory, exactly like chalk apply — it archives the
same source tree, honouring .chalkignore.
Every destination must be a fully-qualified reference with an explicit tag:
<registry-host>/<repository>:<tag>
| Rule | Accepted | Rejected |
|---|---|---|
| Registry host | 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 | myorg/engine:v1 (Docker Hub shorthand) |
| Explicit tag | .../engine:v1 | .../engine |
| Tag, not digest | .../engine:v1 | .../engine@sha256:… |
| Supported registry | Amazon ECR, Google Artifact Registry, GCR | Azure Container Registry, other hosts |
An untagged repository is rejected rather than defaulted to :latest, because silently
writing latest in a registry you own can overwrite an image your other systems already
pull.
Chalk pushes with the build’s own cloud identity. It never accepts, stores, or transmits registry credentials.
That means you grant Chalk’s build identity push access to the destination before running the command. What that identity is depends on your cloud:
This model implies two limits on AWS:
Cross-account works. A destination in a different AWS account is fine, as long as that repository’s policy grants your environment’s management role.
Cross-region does not. An ECR authorization token is only valid in the region that issued it, so the destination must be in the same region as your environment’s own registry. A destination elsewhere is rejected before the build starts:
image destination "123456789012.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine:v1"
is in region us-gov-west-1 but this environment's registry is in us-east-1;
an ECR authorization token is only valid in the region that issued it
Destinations must also be on the same cloud as your environment — a GCP environment cannot push to ECR, and vice versa — because the build pod holds exactly one cloud identity.
Because there is no deployment record, this build does not appear in
chalk deployment list, the deployments page, or chalk deployment build-steps. Those
surfaces are all keyed on a deployment.
Today, the way to observe completion is to watch for the image to appear at its destination:
crane digest 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1or equivalently aws ecr describe-images. The GovCloud example below builds this wait into
the mirroring workflow.
Automating on top of this? Poll the destination registry rather than any Chalk deployment API. The build ID that `chalk deploy build` prints identifies the build workflow, not a deployment, so deployment-scoped endpoints will not find it.
The image is the same engine image chalk apply produces: Chalk’s platform software, your
project source, and the resolved Python virtualenv for your requirements.
Runtime configuration is *not* baked in. When Chalk deploys an image it supplies environment variables pointing at your online store, offline store, and graph artifacts. An image you run outside Chalk's control plane needs equivalent configuration supplied by whatever runs it.
AWS GovCloud is a separate AWS partition (aws-us-gov). This is a stronger boundary
than a region:
sts:AssumeRole does not cross partitions, so you cannot assume a GovCloud role from a
commercial account.So when your Chalk environment runs in the commercial partition, Chalk cannot push into
GovCloud directly, and --destination will reject a GovCloud reference on the region rule
above. The split is:
Step 2 is your code, running in your account, holding your GovCloud credentials. Chalk is not involved and never sees them. What follows is a working example you can adapt.
Because role assumption cannot cross partitions, the copy needs a credential issued in GovCloud. In practice that is an IAM user in your GovCloud account with permission to push to the destination repository, stored as a Kubernetes secret:
kubectl create secret generic govcloud-ecr \
--from-literal=access_key_id="$GOVCLOUD_ACCESS_KEY_ID" \
--from-literal=secret_access_key="$GOVCLOUD_SECRET_ACCESS_KEY"The commercial side needs no secret — give the workflow’s service account an IRSA role with read access to the source repository.
This WorkflowTemplate logs into both registries, copies the image, and verifies that the
digest survived the copy. All three containers share one pod so they can share a docker
config through an emptyDir.
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: mirror-chalk-engine-to-govcloud
spec:
entrypoint: mirror
# IRSA role with ECR read on the commercial source repository.
serviceAccountName: chalk-image-mirror
arguments:
parameters:
- name: source-image
value: 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1
- name: destination-image
value: 210987654321.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine:v1
- name: source-region
value: us-east-1
- name: destination-region
value: us-gov-west-1
templates:
- name: mirror
volumes:
- name: docker-config
emptyDir: {}
containerSet:
volumeMounts:
- name: docker-config
mountPath: /docker
containers:
- name: login
image: amazon/aws-cli:2
env:
- name: GOVCLOUD_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: govcloud-ecr
key: access_key_id
- name: GOVCLOUD_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: govcloud-ecr
key: secret_access_key
command: [sh, -c]
args:
- |
set -eu
SRC='{{workflow.parameters.source-image}}'
DST='{{workflow.parameters.destination-image}}'
SRC_HOST="${SRC%%/*}"
DST_HOST="${DST%%/*}"
# Commercial partition: the pod's own IRSA role.
SRC_PW=$(aws ecr get-login-password \
--region '{{workflow.parameters.source-region}}')
# GovCloud is a different partition, so this must use the static
# credential. `env -u` drops the IRSA variables so the SDK cannot
# prefer web identity over the keys we are handing it.
DST_PW=$(env -u AWS_ROLE_ARN -u AWS_WEB_IDENTITY_TOKEN_FILE \
AWS_ACCESS_KEY_ID="$GOVCLOUD_ACCESS_KEY_ID" \
AWS_SECRET_ACCESS_KEY="$GOVCLOUD_SECRET_ACCESS_KEY" \
aws ecr get-login-password \
--region '{{workflow.parameters.destination-region}}')
SRC_AUTH=$(printf 'AWS:%s' "$SRC_PW" | base64 -w0)
DST_AUTH=$(printf 'AWS:%s' "$DST_PW" | base64 -w0)
umask 077
cat > /docker/config.json <<EOF
{"auths":{
"$SRC_HOST":{"auth":"$SRC_AUTH"},
"$DST_HOST":{"auth":"$DST_AUTH"}
}}
EOF
echo "wrote docker config for $SRC_HOST and $DST_HOST"
- name: copy
image: gcr.io/go-containerregistry/crane:debug
dependencies: [login]
env:
- name: DOCKER_CONFIG
value: /docker
command: [crane]
args:
- copy
- '{{workflow.parameters.source-image}}'
- '{{workflow.parameters.destination-image}}'
- name: verify
image: gcr.io/go-containerregistry/crane:debug
dependencies: [copy]
env:
- name: DOCKER_CONFIG
value: /docker
command: [sh, -c]
args:
- |
set -eu
SRC_DIGEST=$(crane digest '{{workflow.parameters.source-image}}')
DST_DIGEST=$(crane digest '{{workflow.parameters.destination-image}}')
echo "source: $SRC_DIGEST"
echo "destination: $DST_DIGEST"
if [ "$SRC_DIGEST" != "$DST_DIGEST" ]; then
echo "digest mismatch; the copy did not preserve the image" >&2
exit 1
fi
echo "digests match"Run it with:
argo submit --from workflowtemplate/mirror-chalk-engine-to-govcloud \
-p source-image=123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 \
-p destination-image=210987654321.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine:v1crane copycrane copy transfers the manifest and its layers as-is, so the image keeps its digest
across partitions. That is what makes the verify step meaningful: a matching digest is
evidence that the bits authorized in GovCloud are byte-identical to the ones Chalk built,
which is usually what an authorization boundary review wants to see. A docker pull and
docker push through an intermediate host can rewrite the manifest and change the digest.
skopeo copy --all is an equivalent choice if it fits your tooling better.
# 1. Chalk builds and pushes to your commercial repository.
chalk deploy build \
--destination 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1
# 2. Wait for the image to land (there is no deployment to poll).
until crane digest 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 \
>/dev/null 2>&1; do sleep 30; done
# 3. Mirror it into GovCloud.
argo submit --from workflowtemplate/mirror-chalk-engine-to-govcloud \
-p source-image=123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 \
-p destination-image=210987654321.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine:v1Tag each build with something immutable — a git SHA or a release version — rather than reusing one tag. The digest check in step 3 only proves the copy was faithful; an immutable tag is what lets you say which build was authorized.
chalk apply Works — the deploying counterpart, and what the
image build stage does in detail.