Aug 13: Context layer webinar
Chalk home page
  1. Enterprise Deployment

Chalk integrates with your data sources, transform this data with feature pipelines, store this data in online and offline storage, and provide monitoring on feature computation and distributions.

​
Enterprise Service Architecture

Chalk offers a hosted model (“Chalk Cloud”), or a customer-hosted model (“Customer Cloud”).

There are a few main components of a Chalk deployment:

  • Management - serves non-customer data (like alert configuration and RBAC configuration).
  • Builder - builds containers that run your feature pipelines.
  • Compute - machines that run your feature pipelines. In both AWS and GCP, the compute runs on Kubernetes with Knative.
  • Customer Data - the online and offline stores for your features.
  • Secrets - environment variables and configuration for your data sources.

These components are organized as follows:

Architecture diagram
  1. 1Creating secrets:The API server can be configured to have write-only access to the secret store.
  2. 2Reading secrets:Secret access can be restricted entirely to the data plane.
  3. 3Online store:Chalk supports several online feature stores, which are used for caching feature values. On AWS, Chalk supports DynamoDB and Elasticache.

​
Data Isolation

In the enterprise deployment, your API clients talk directly to the compute engine deployed into your cloud. Feature values, historical and online, do not escape your cloud project.

​
Storage

Chalk uses different storage technologies to support online and offline use-cases.

The online store is optimized for serving the latest version of any given feature for any given entity with the minimum possible latency. Behind the scenes, Chalk uses a key-value store for this purpose. Chalk can be configured to use Redis or Cloud Memory Store for smaller resident data sets with high latency requirements, or DynamoDB when horizontal scalability is required.

The offline store is optimized for storing all historical feature values, serving point-in-time correct queries, and tracking provenance of features. Chalk supports a variety of storage backends depending on data scale and latency requirements. Typically, Chalk uses TimescaleDB, Redshift, or BigQuery.