# Chalk Machine Types
source: https://docs.chalk.ai/docs/chalk-machine-types

## Configure your Chalk Kubernetes cluster

### Overview

Chalk Machine Types provide a simple and robust interface for configuring cloud resource consumption for workloads
running in your Chalk environment. They are a replacement for the legacy configuration system that required more direct
configuration of the underlying Kubernetes API and provide improvements in performance, simplicity, and durability of your
workloads, as well as enhanced usage reporting.

### What is a Chalk Machine Type

Chalk Machine Types are a simple way of categorizing cloud instances of different sizes, based on the amount of CPU and memory they
have available. For instance, small corresponds to a cloud instance with 4 vCPU and 16GiB of memory. Each Chalk Machine Type corresponds
to a single concrete instance type in the appropriate cloud provider backend, along with an ordered list of fallback instance types that
will automatically be used if the primary instance type is not available due to a cloud provider stockout or other availability issue (like
regional availability restrictions). As an example, small in an AWS region may map to an m7a.xlarge instance, but if the region or availability
zone are out of that instance type, it can fall back to m7i.xlarge, and then m8a.xlarge, etc. This list of fallbacks is configurable per-cluster
and per-environment.

When you pick a Chalk Machine Type, your workload is guaranteed to run isolated from other Chalk workloads on its own cloud instance, with appropriate
requests and limits automatically configured to ensure that it will schedule and that it can make full use of the resources on the machine.

### Usage Labeling for Chalk Machine Types

Workloads configured with a Chalk Machine Type will run on Karpenter nodepools created and configured by Chalk in AWS and Azure, and will run on
equivalent ComputeClasses in GCP. These nodepools and ComputeClasses will provide appropriate labeling for the Chalk credits consumed by these
workloads, depending on the type of workload: online, offline, compute, infrastructure, etc. These nodepools and ComputeClasses are not
directly configurable and are automatically created and configured by Chalk.

### Disk Spilling for Offline Workloads

For offline workloads, using a Chalk Machine Type will automatically enable disk spilling via local SSDs. Offline workloads will be automatically configured
to take advantage of this feature, which significantly reduces memory usage for large workloads, reducing their cost and improving their efficiency. No
manual configuration will be required to set up disk spilling when using a Chalk Machine Type.

### GPU Support for Online Workloads

Some inference and training workloads require access to a GPU on the selected instance type. GPU instance support for Chalk Machine Types is not yet enabled
but will be available soon.

### Chalk Machine Types and Resource Groups

Chalk Machine Types integrate nicely with the high-level categorization and isolation provided by Resource Groups. Using a Chalk Machine Type already
provides inter-service isolation, and moving workloads with different logical isolation requirements (often useful for separating different business use-cases)
to different resource groups allows both for isolation between use-cases and also allows you to select different machine sizes according to the compute needs
of each different use-case.

For offline workloads like the Job Queue Consumer, it can be convenient to make several different Resource Groups, each configured to run the consumer
against a different Chalk Machine Type, and potentially each with different scaling limits. Then, workloads like offline queries can be directed at
the appropriate Resource Group depending on the resources they need, and resource consumption can be controlled via the different scaling configurations.

### Customizing Chalk Machine Types

Chalk Machine Types are mapped to a primary instance type along with an ordered list of fallback instance types, depending on regional availability.
The default list of instance types can be customized in the organization-level settings page for viewing and
configuring attached clusters. You can change the order in which different instance types will be chosen as fallbacks, and you can change which instance
types are available at all for a given Chalk Machine Type. This can be useful, for example, if you want to ensure that your environment always tries
to use the fastest instance types available for maximum performance, or less expensive instance types for cost savings.

Customizing Chalk Machine Type definitions requires organization-level (team-level) permissions.

### List of Available Chalk Machine Types

Machine types larger than xlarge are only available for offline workloads. "High-CPU" machine types are only available for online
workloads. "High-memory" and "high-cpu" indicate a larger and smaller ratio of memory to vCPU respectively on the instances compared to the "standard"
machine type sizes, approximately:

- High-CPU: 2GiB per vCPU
- Standard: 4GiB per vCPU
- High-memory: 8GiB per vCPU

This is the full list of machine type sizes:

Standard

| AWS Machine type | vCPU | Memory |
| ---------------- | ---- | ------ |
| `small`          | 4    | 16Gi   |
| `medium`         | 8    | 32Gi   |
| `large`          | 16   | 64Gi   |
| `xlarge`         | 32   | 128Gi  |
| `2xlarge`        | 48   | 192Gi  |
| `3xlarge`        | 64   | 256Gi  |
| `4xlarge`        | 96   | 384Gi  |
| `5xlarge`        | 128  | 512Gi  |

High-CPU

| AWS Machine type | vCPU | Memory |
| ---------------- | ---- | ------ |
| `small-highcpu`  | 4    | 8Gi    |
| `medium-highcpu` | 8    | 16Gi   |
| `large-highcpu`  | 16   | 32Gi   |
| `xlarge-highcpu` | 32   | 64Gi   |

High-memory

| AWS Machine type  | vCPU | Memory |
| ----------------- | ---- | ------ |
| `small-highmem`   | 4    | 32Gi   |
| `medium-highmem`  | 8    | 64Gi   |
| `large-highmem`   | 16   | 128Gi  |
| `xlarge-highmem`  | 32   | 256Gi  |
| `2xlarge-highmem` | 48   | 384Gi  |
| `3xlarge-highmem` | 64   | 512Gi  |
| `4xlarge-highmem` | 96   | 768Gi  |
| `5xlarge-highmem` | 128  | 1024Gi |

Standard

| Azure Machine type | vCPU | Memory |
| ------------------ | ---- | ------ |
| `small`            | 4    | 16Gi   |
| `medium`           | 8    | 32Gi   |
| `large`            | 16   | 64Gi   |
| `xlarge`           | 32   | 128Gi  |
| `2xlarge`          | 48   | 192Gi  |
| `3xlarge`          | 64   | 256Gi  |
| `4xlarge`          | 96   | 384Gi  |
| `5xlarge`          | 128  | 512Gi  |

High-CPU

| Azure Machine type | vCPU | Memory |
| ------------------ | ---- | ------ |
| `small-highcpu`    | 4    | 8Gi    |
| `medium-highcpu`   | 8    | 16Gi   |
| `large-highcpu`    | 16   | 32Gi   |
| `xlarge-highcpu`   | 32   | 64Gi   |

High-memory

| Azure Machine type | vCPU | Memory |
| ------------------ | ---- | ------ |
| `small-highmem`    | 4    | 32Gi   |
| `medium-highmem`   | 8    | 64Gi   |
| `large-highmem`    | 16   | 128Gi  |
| `xlarge-highmem`   | 32   | 256Gi  |
| `2xlarge-highmem`  | 48   | 384Gi  |
| `3xlarge-highmem`  | 64   | 512Gi  |
| `4xlarge-highmem`  | 96   | 768Gi  |
| `5xlarge-highmem`  | 128  | 1024Gi |

Standard

| GCP Machine type | vCPU | Memory |
| ---------------- | ---- | ------ |
| `small`          | 4    | 15Gi   |
| `medium`         | 8    | 31Gi   |
| `large`          | 16   | 62Gi   |
| `xlarge`         | 32   | 124Gi  |
| `2xlarge`        | 48   | 186Gi  |
| `3xlarge`        | 64   | 248Gi  |
| `4xlarge`        | 96   | 372Gi  |
| `5xlarge`        | 192  | 744Gi  |

High-CPU

| GCP Machine type | vCPU | Memory |
| ---------------- | ---- | ------ |
| `small-highcpu`  | 4    | 7Gi    |
| `medium-highcpu` | 8    | 15Gi   |
| `large-highcpu`  | 16   | 30Gi   |
| `xlarge-highcpu` | 32   | 60Gi   |

High-memory

| GCP Machine type  | vCPU | Memory |
| ----------------- | ---- | ------ |
| `small-highmem`   | 4    | 31Gi   |
| `medium-highmem`  | 8    | 63Gi   |
| `large-highmem`   | 16   | 126Gi  |
| `xlarge-highmem`  | 32   | 252Gi  |
| `2xlarge-highmem` | 48   | 378Gi  |
| `3xlarge-highmem` | 64   | 504Gi  |
| `4xlarge-highmem` | 96   | 756Gi  |
| `5xlarge-highmem` | 192  | 1512Gi |