# Chalk Serverless source: https://docs.chalk.ai/docs/serverless ## Run Chalk on Chalk's infrastructure while you provide the offline store and data sources. In a Chalk Serverless deployment, Chalk runs both the metadata plane and the data plane on clusters that Chalk operates, and each environment includes an online store. You provide the offline store and the data sources your resolvers read from, and you can connect your own online store. Serverless clusters are shared: environments from several Chalk customers run on the same clusters, and each environment runs in its own namespace. To compare Chalk Serverless with the other deployment models, see Deployment models. To install the CLI and start working in a new Chalk Serverless environment, see Starting in a Chalk Serverless environment. ### What Chalk runs and what you provide Chalk runs: - The metadata plane: deployments, access control, monitoring, alerting, and feature discovery - The data plane compute: online query serving, offline queries, resolvers, streaming, sandboxes, and scaling groups - Storage for datasets, deployment source, and model registry artifacts - Storage for your environment's logs, traces, and metrics - Upgrades to the clusters and the services that run on them - An included online store for the environment You provide: - The offline store - The data sources your resolvers read from - Optionally, your own online store Serverless deployments use an offline store that you host. Other hosting options are available. Contact your Chalk representative for more information. For the online and offline store types Chalk supports, see Online and Offline Stores. ### Included online store If you have not connected your own online store, a serverless environment uses a small Redis online store that Chalk runs in the environment. This keeps features that read from or write to the online store working until you connect your own. For production workloads, connect your own online store. Chalk does not bill for the included online store, and it does not count against your quotas. When you connect your own online store, it replaces the included one. Chalk does not copy data from the included online store to your store. ### Connecting your stores and data sources Serverless workloads connect to your offline store, your data sources, and your own online store if you connect one, from Chalk's clusters. Serverless deployments do not support private network connections, such as VPC peering, PrivateLink, or SSH tunnels, so each store and data source must accept connections from Chalk's clusters. To discuss private connectivity, contact your Chalk representative. Serverless clusters connect from fixed IP addresses, so you can allowlist them on your stores and data sources. To get the addresses, contact your Chalk representative. To connect your own online store, enter its connection URI in Integrations > Online Stores in the dashboard. Chalk stores the URI as a secret. For URI formats, see Online Store Setup. Store the credentials for your offline store and data sources as secrets in your Chalk environment. Serverless environments do not read secrets from your cloud provider's secret manager. ### Quotas Quotas limit the compute a serverless environment can use. A quota does not reserve capacity on the cluster. Each quota counts the workloads running in the environment at the same time. When a workload stops, its usage no longer counts against the quota. To see your environment's quotas, open Settings > Quotas in the dashboard. The page lists each quota with its current usage, its limit, and the amount still available: - CPU, in cores (vCPUs) - Memory, in GiB - GPUs (all families) - Concurrent sandboxes - Concurrent scaling groups - Functions - Model deployments - Concurrent notebook runtimes The CPU and memory a workload requests count against the CPU and Memory quotas, including workloads that also use GPUs. If the page lists Other GPU types with a limit of 0, workloads can use only the GPU types listed on the page, and each GPU request must name a GPU type. If starting a workload would exceed a quota, Chalk rejects the request and the workload does not start. The error names the quota, the amount requested, and the amount still available. A workload within its quotas can still find the cluster without enough free CPU, memory, or GPUs. What happens then depends on the workload: | Workload | When capacity is not available | | --------------------------------------------------- | ----------------------------------------------------------------------------------------------- | | Scaling groups | Wait until capacity is available. | | Sandboxes on the `host` compute class (the default) | Fail immediately. If Chalk is bringing more capacity online, the sandbox waits up to 5 minutes. | | Sandboxes on the `k8s` compute class | Wait up to about 5 minutes, then fail. | A quota without a limit shows No limit. To change a quota that has a limit, request an increase from Chalk: click Request increase next to the quota. Chalk reviews the request and changes the quota. ### Billing Chalk bills serverless compute by the second. For current rates, see Pricing. - Billing period: a container is billed from the second it reports Ready until the second it exits. Image pulls, driver initialization, and model downloads before Ready are not billed. - GPUs: billed per GPU-second, at the rate for the GPU type. - CPU and memory: billed per vCPU-second and per GiB-second. Containers that use GPUs are also billed for their CPU and memory. - Volumes: billed per GiB per month. - Network egress: billed per GiB. There is no minimum charge, minimum duration, or per-request fee. Each replica of a scaling group is billed while it exists, whether or not it is serving traffic. Chalk's charges do not include your offline store, your data sources, or your own online store. Their costs come from whoever hosts them. For committed or reserved pricing, contact your Chalk representative. To learn how Chalk bills for each kind of workload, see Billing. ### Limitations Some Chalk features are unavailable or work differently in serverless environments. ### Networking | Feature | In serverless environments | | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | Private connections to stores and data sources (VPC peering, PrivateLink, SSH tunnels) | Not available. Your stores and data sources must accept connections from Chalk's clusters, which connect from fixed IP addresses that you can allowlist. | | Private gateway | Not available. Services in the environment are reachable only through the public gateway. | ### Secrets | Feature | In serverless environments | | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | References to AWS Secrets Manager, GCP Secret Manager, or Azure Key Vault | Not available. Store each value as a [Chalk secret](/docs/env-vars#secrets). This applies to environment secrets, data source credentials, and notebook secrets. | ### Compute and resources | Feature | In serverless environments | | -------------------------------------- | ---------------------------------------------------------------------------------------- | | [Host pools](/docs/compute/host-pools) | Not available. Workloads run on host pools that Chalk manages for the cluster. | | Resource group machine types | Online and offline workloads must use a [Chalk machine type](/docs/chalk-machine-types). | | GPU types | Limited to the GPU types available on serverless clusters. | ### Dashboard pages The pages and tabs below are not shown in serverless environments. | Area | Not shown | | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Kubernetes | The Kubernetes and Nodepools pages, and the detail pages for Kubernetes objects. | | Workload detail pages | Pod tables and Kubernetes event tabs on deployments, branch servers, functions, sandboxes, scaling groups, model deployments, scheduled queries, and offline queries. | | Infrastructure | Cloud overview, Artifact repositories, Metrics Database, Background Persistence, Workflow Orchestrator Engine, and Resource group jobs. | | Settings | Compute and Shared resources. | | Built-in dashboards | Kubernetes pods overview, Kubernetes nodes overview, and Bloom filters overview. | | Ingestion health | The ingestion health dashboard. | ### Debugging | Feature | In serverless environments | | -------------------------------------------------- | -------------------------- | | Terminal and TCP proxy access to running workloads | Not available. | ### Observability | Feature | In serverless environments | | ----------------------- | ----------------------------------------------------------------------------------------------- | | Log and trace retention | Set by Chalk. You cannot change the retention period. | | Summary metrics | Some summary metrics are not available, such as run counts for scheduled queries and resolvers. |