# Building Images Without Deploying source: https://docs.chalk.ai/docs/chalk-deploy-build ## Build a Chalk engine image and push it to a registry you own without creating a deployment. Mirror it into a separate cloud partition such as AWS GovCloud. chalk deployment build builds the engine image for your project and pushes it to container registries that you name. It does not deploy anything. chalk deploy build is an alias for the same command. It requires Chalk CLI v1.54.1 or later. The command does not: - create a deployment record, - promote the image to your environment, - change what your environment is currently serving, - move the floating :latest tag on your environment's engine repository. The result is an image in a registry you control. Your active deployment keeps serving the same version it served before the build. ### When to use it Use chalk deployment build when you want the image as an artifact to handle yourself: - Delivery into a separate cloud partition or an air-gapped network. Chalk builds in the partition your environment lives in. To get the image somewhere Chalk's build identity cannot reach, such as AWS GovCloud, Chalk hands you the image and you move it the rest of the way. See Mirroring into AWS GovCloud below. - Promotion through your own pipeline. Build once, scan or sign the image, run it through your own approval gates, then deploy the exact bits you approved. - Vulnerability scanning before rollout. Get the image into a registry your scanner watches without exposing it to traffic first. If you just want to deploy, use chalk apply. ### Usage ``` chalk deployment build --destination 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 ``` Repeat --destination to push the same image to several registries in one build: ``` chalk deployment build \ --destination 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:2026-08-25 \ --destination 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:release-candidate ``` The build pushes to every destination in a single pass. On success the command prints the build's identifier and the references it pushed to: ``` Build ID: chalk-build-argo-8f3c21d4 ↗ 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:2026-08-25 ↗ 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:release-candidate ``` Run the command from your project directory, as you would chalk apply. It archives the same source tree and honors .chalkignore. ### Destination rules Every destination must be a fully-qualified reference with an explicit tag: ``` /: ``` | Rule | Accepted | Rejected | | ------------------ | -------------------------------------------------------------- | ---------------------------------------- | | Registry host | `123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1` | `myorg/engine:v1` (Docker Hub shorthand) | | Explicit tag | `.../engine:v1` | `.../engine` | | Tag, not digest | `.../engine:v1` | `.../engine@sha256:…` | | Supported registry | Amazon ECR, Google Artifact Registry, GCR | Azure Container Registry, other hosts | Chalk rejects an untagged repository instead of tagging it :latest, because writing latest in a registry you own can overwrite an image your other systems already pull. ACR destinations are rejected. The builder authenticates only to the environment's own ACR, so it cannot push to a second registry. The command fails before the build starts. ### Authentication Chalk pushes with the build's own cloud identity. It never accepts, stores, or transmits registry credentials. That means you grant Chalk's build identity push access to the destination before running the command. What that identity is depends on your cloud: - Google Cloud: the build's service account. One access token authenticates every Artifact Registry or GCR host that account can push to, so any such destination works once IAM allows it. - AWS: the environment's ECR authorization token, issued for the environment's management role. Grant access by adding that role to the destination repository's policy. This model implies two limits on AWS: Cross-account works. A destination in a different AWS account is fine, as long as that repository's policy grants your environment's management role. Cross-region does not. An ECR authorization token is only valid in the region that issued it, so the destination must be in the same region as your environment's own registry. A destination elsewhere is rejected before the build starts: ``` image destination "123456789012.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine:v1" is in region us-gov-west-1 but this environment's registry is in us-east-1; an ECR authorization token is only valid in the region that issued it ``` Destinations must also be on the same cloud as your environment, because the build pod holds one cloud identity. A GCP environment cannot push to ECR, and an AWS environment cannot push to Artifact Registry or GCR. ### Knowing when the build finished Because there is no deployment record, this build does not appear in chalk deployment list, the deployments page, or chalk deployment build-steps. Those surfaces are all keyed on a deployment. To find out when the build has finished, watch for the image to appear at its destination: ``` crane digest 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 ``` or equivalently aws ecr describe-images. The GovCloud example below builds this wait into the mirroring workflow. To automate this, poll the destination registry. The build ID that chalk deployment build prints identifies the build workflow, not a deployment, so deployment endpoints do not find it. ### What the image contains The image is the same engine image chalk apply produces: Chalk's platform software and the resolved Python virtualenv for your requirements. Your project source is not baked in. The engine downloads it at boot, from the source bundle the deploy uploaded to your environment's bucket. One image therefore serves every deployment whose requirements are unchanged. This is also what makes chalk apply --from-image possible. Runtime configuration is not baked in. When Chalk deploys an image it supplies environment variables pointing at your online store, offline store, and graph artifacts. An image you run outside Chalk's control plane needs equivalent configuration supplied by whatever runs it. ### Mirroring into AWS GovCloud AWS GovCloud is a separate AWS partition (aws-us-gov). This is a stronger boundary than a region: - Credentials issued in the commercial partition have no identity in GovCloud at all. - sts:AssumeRole does not cross partitions, so you cannot assume a GovCloud role from a commercial account. So when your Chalk environment runs in the commercial partition, Chalk cannot push into GovCloud directly, and --destination will reject a GovCloud reference on the region rule above. The split is: - Chalk builds and pushes to a commercial-partition ECR repository that you own. - You copy that image into GovCloud, using credentials issued in the GovCloud partition. Step 2 is your code, running in your account, holding your GovCloud credentials. Chalk is not involved and never sees them. The example below shows one way to do it. ### GovCloud credentials Because role assumption cannot cross partitions, the copy needs a credential issued in GovCloud. Create an IAM user in your GovCloud account with permission to push to the destination repository, and store its keys as a Kubernetes secret: ``` kubectl create secret generic govcloud-ecr \ --from-literal=access_key_id="$GOVCLOUD_ACCESS_KEY_ID" \ --from-literal=secret_access_key="$GOVCLOUD_SECRET_ACCESS_KEY" ``` Partition boundaries rule out short-lived, federated credentials. Scope the IAM user to ecr:PutImage and the related upload actions on the single destination repository, rotate it on a schedule, and keep it out of the commercial account's blast radius. The commercial side needs no secret. Give the workflow's service account an IRSA role with read access to the source repository. ### The mirroring workflow This WorkflowTemplate logs into both registries, copies the image, and verifies that the digest survived the copy. All three containers share one pod so they can share a docker config through an emptyDir. ``` apiVersion: argoproj.io/v1alpha1 kind: WorkflowTemplate metadata: name: mirror-chalk-engine-to-govcloud spec: entrypoint: mirror # IRSA role with ECR read on the commercial source repository. serviceAccountName: chalk-image-mirror arguments: parameters: - name: source-image value: 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 - name: destination-image value: 210987654321.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine:v1 - name: source-region value: us-east-1 - name: destination-region value: us-gov-west-1 templates: - name: mirror volumes: - name: docker-config emptyDir: {} containerSet: volumeMounts: - name: docker-config mountPath: /docker containers: - name: login image: amazon/aws-cli:2 env: - name: GOVCLOUD_ACCESS_KEY_ID valueFrom: secretKeyRef: name: govcloud-ecr key: access_key_id - name: GOVCLOUD_SECRET_ACCESS_KEY valueFrom: secretKeyRef: name: govcloud-ecr key: secret_access_key command: [sh, -c] args: - | set -eu SRC='{{workflow.parameters.source-image}}' DST='{{workflow.parameters.destination-image}}' SRC_HOST="${SRC%%/*}" DST_HOST="${DST%%/*}" # Commercial partition: the pod's own IRSA role. SRC_PW=$(aws ecr get-login-password \ --region '{{workflow.parameters.source-region}}') # GovCloud is a different partition, so this must use the static # credential. `env -u` drops the IRSA variables so the SDK cannot # prefer web identity over the keys we are handing it. DST_PW=$(env -u AWS_ROLE_ARN -u AWS_WEB_IDENTITY_TOKEN_FILE \ AWS_ACCESS_KEY_ID="$GOVCLOUD_ACCESS_KEY_ID" \ AWS_SECRET_ACCESS_KEY="$GOVCLOUD_SECRET_ACCESS_KEY" \ aws ecr get-login-password \ --region '{{workflow.parameters.destination-region}}') SRC_AUTH=$(printf 'AWS:%s' "$SRC_PW" | base64 -w0) DST_AUTH=$(printf 'AWS:%s' "$DST_PW" | base64 -w0) umask 077 cat > /docker/config.json <&2 exit 1 fi echo "digests match" ``` Run it with: ``` argo submit --from workflowtemplate/mirror-chalk-engine-to-govcloud \ -p source-image=123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 \ -p destination-image=210987654321.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine:v1 ``` ### Copying with crane copy crane copy transfers the manifest and its layers as-is, so the image keeps its digest across partitions. The verify step compares the two digests, and a match shows that the image in GovCloud is byte-identical to the one Chalk built. A docker pull and docker push through an intermediate host can rewrite the manifest and change the digest. skopeo copy --all is an equivalent choice if it fits your tooling better. ### End to end ``` # 1. Chalk builds and pushes to your commercial repository. chalk deployment build \ --destination 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 # 2. Wait for the image to land (there is no deployment to poll). until crane digest 123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 \ >/dev/null 2>&1; do sleep 30; done # 3. Mirror it into GovCloud. argo submit --from workflowtemplate/mirror-chalk-engine-to-govcloud \ -p source-image=123456789012.dkr.ecr.us-east-1.amazonaws.com/chalk/engine:v1 \ -p destination-image=210987654321.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine:v1 ``` Tag each build with an immutable value, such as a git SHA or a release version, instead of reusing one tag. The digest check in step 3 shows that the copy matches its source. An immutable tag records which build was authorized. ### Deploying a pre-built image Once an image exists in a registry your environment's cluster can pull from, deploy it with chalk apply --from-image. This flag and the --with-* flags below require Chalk CLI v1.54.0 or later. They do not appear in chalk apply --help. ``` chalk apply --from-image \ 210987654321.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine@sha256: ``` This creates a normal deployment, with a deployment record, a graph, and a rollout, but it builds nothing. The reference you give becomes the deployment's engine image verbatim, and the Python source uploaded by that same chalk apply is what the engine downloads and serves. Pin the image by its digest instead of a tag. The digest is the identifier the mirroring step verified, so pinning it makes the deployment run the image that was authorized. Nothing checks that the image matches the source you are deploying. The image carries the virtualenv for the requirements it was built from; if your source imports something that virtualenv does not have, the failure appears when the engine boots, not when you run chalk apply. Deploy from source whose requirements.txt produced the image. Because no image is built, Chalk rejects any option that describes how to build one: a base image override or a build profile cannot be combined with --from-image. --from-image also cannot be combined with --branch. A deployment created from a pre-built image cannot be rebuilt or redeployed. To change its image, run chalk apply --from-image again with the image you want. ### Carrying artifacts forward An image build also produces artifacts besides the image: the source bundle it was built from, the virtualenv tarball, and the graph export an indexing job wrote. Three optional flags serve those instead of producing them again: | Flag | What it replaces | | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `--with-source-bundle` | The source bundle the engine downloads at boot, in place of the archive `chalk apply` just uploaded | | `--with-export` | The deployment's `export.pb`, in place of the one an indexing job would have written | | `--with-venv` | Seeds your environment's virtualenv cache, so a later indexing or branch build reuses it instead of resolving the requirements again | Each takes an object that is already in your environment's source bundle bucket, given as a bare object key or a full s3://, gs:// or az:// URI naming that same bucket. Chalk copies within that one bucket; it does not read from anywhere else, and an object in another bucket is rejected. Copy the artifacts across the partition boundary yourself, the same way you copied the image. --with-venv must keep the filename the cache uses, shallow_copy_.tar.zst, because that name is the cache key. --with-export is optional in a different sense: without it the engine builds its graph from the source bundle at boot, which is slower but correct. ``` chalk apply \ --from-image 210987654321.dkr.ecr.us-gov-west-1.amazonaws.com/chalk/engine@sha256: \ --with-source-bundle s3://my-chalk-bundles/authorized/build-v1.tar.gz \ --with-export s3://my-chalk-bundles/authorized/build-v1-export.pb \ --with-venv s3://my-chalk-bundles/authorized/shallow_copy_.tar.zst ``` ### Related - How chalk apply Works: the deploying counterpart, and what the image build stage does in detail. - Artifact Repository: private Python indexes the build resolves packages from.