Aug 13: Context layer webinar
Chalk home page
  1. Notebooks
  2. SQL result caching

When a SQL cell queries a configured datasource, such as Snowflake, BigQuery, or PostgreSQL, Chalk can reuse the result of the cell’s last run instead of sending the query to the datasource again. This applies when the cell’s SQL and datasource have not changed since that run, and the run finished successfully within the cache’s max staleness.

​
Cached results

A SQL cell that used a cached result shows Cached · before the time of its last run, in the top right of the cell header.

A run that uses a cached result gets its own copy of the result. Paging, downloads, and column aggregations on it work the same as on a result from the datasource.

​
Running without cached results

To send a query to the datasource even when a cached result is available:

  • One cell: click the arrow next to the cell’s run button and select Run without cached results.
  • Every SQL cell: click the arrow next to Run all in the notebook toolbar and select Run all without cached results.

The new result replaces the cached result, so the next run of the cell can reuse it.

From the CLI, pass --no-cache to chalk notebook run to run without cached results.

​
Max staleness

Max staleness is the oldest a cached result can be and still be reused. A value of 0 turns caching off. Chalk uses the first of these settings that is set:

  1. Notebook: the SQL cache setting in the notebook’s Notebook settings panel. The default, Environment default, uses the environment setting. Off (always run fresh) turns caching off for the notebook.
  2. Environment: the SQL cache max staleness setting under Settings > Notebooks. Changing it requires the team.admin permission. Disabled (always run fresh) turns caching off for every notebook in the environment, and a notebook setting cannot turn it back on.
  3. Default: 60 minutes.

From the CLI, chalk notebook run --max-staleness sets a lower max staleness for one run, such as --max-staleness 5m. It cannot make the run reuse an older result than the settings allow.

​
Which queries use cached results

A SQL cell can use a cached result only when all of the following are true:

  • It queries a configured datasource. Chalk SQL and DataFrames cells always run.
  • Its statement is a read: it starts with SELECT or WITH.
  • It does not call a function whose result depends on when the query runs, such as now(), current_timestamp, or random().
  • The cached result is 512 MiB or smaller.

Changing the cell’s SQL or its datasource means the next run sends the query to the datasource. Renaming the cell or its result variable does not.

​
Retention

Chalk keeps notebook run history, including cached results, for 30 days. A result older than 30 days is never reused, whatever the max staleness setting.