Overview

​

Integrate any API or Data Source

Instead of complex ETL jobs and streaming pipelines, make direct calls to your data sources. Chalk makes it fast and easy to call external APIs, query production databases and data warehouses, or fetch parquet files from S3. Chalk orchestrates pipeline stages automatically to achieve maximum possible parallelism, and makes it easy to pre-compute + cache features when necessary.

​

What to read next

Chalk enables integration with any data source, in addition to some more customized logic for common data sources. For some of our out-of-the-box solutions for popular storage providers, find more below:

• AWS - Integrate with Amazon Web Services to read from data sources such as Redshift, S3, and Kinesis.
• GCP - Integrate with Google CLoud Platform to read from data sources such as BigQuery, Spanner, PubSub, and Cloud Storage.

• SQL Integrations - Chalk has special support for SQLAlchemy and can push down filters into SQL queries for more efficient data fetching.
• Snowflake - Chalk offers custom support for Snowflake as a SQL source.
• Databricks - Chalk provides out-of-the-box support for Databricks as a SQL source.
• MySQL - Read from your MySQL data sources.
• PostgreSQL - Integrate your PostgreSQL data sources.
• Custom Data Sources - Chalk offers extra optimizations for some data sources, but you don’t need Chalk to officially support a data source to use it!