BigQuery

Chalk has an integration with BigQuery that makes it easy to read queries and tables into your feature store.

​

Authorization

To use BigQuery in your resolvers, you first need to add the Chalk GCP integration to the environments where you would like to use BigQuery.

When querying your BigQuery data source, Chalk will push down filters on top of your queries to optimize the amount of data read from your tables. For larger queries, rather than interpolating values directly in the SQL string for the query, which has length limits in BigQuery, Chalk will use a table to temporarily hold the values against which to query. As such, we require bigquery.tables.create and bigquery.jobs.create permissions to create and use these temporary tables.

​

Integrations Setup

After configuring your BigQuery integration with the GCP integration, define your data sources in Python:

from chalk.sql import BigQuerySource

risk = BigQuerySource(name="RISK")
marketing = BigQuerySource(name="MARKETING")

You can then reference them in SQL file resolvers using the name parameter. For example, to query from the RISK source:

-- type: online
-- resolves: User
-- source: RISK
SELECT id, credit_score FROM users

And to query from the MARKETING source:

-- type: online
-- resolves: User
-- source: MARKETING
SELECT id, email, campaign_status FROM users