This reference page documents every command and flag available in Chalk's command-line interface.
The Chalk CLI allows you to create, update, and manage your feature pipelines directly from your terminal.
With the Chalk CLI, you can:
Installing Chalk is easy! On Linux and Mac, run the command below to install the latest version of Chalk.
$ curl -s -L https://api.chalk.ai/install.sh | sh
On Windows, download the latest version of the executable from one of the following links:
https://storage.googleapis.com/cli-go.chalk-prod.chalk.ai/latest/chalk-windows-amd64
https://storage.googleapis.com/cli-go.chalk-prod.chalk.ai/latest/chalk-windows-386
Subsequent updates for Mac, Linux, and Windows can be installed using chalk update.
After installing, you may need to restart your terminal, or source your shell's rc file
(e.g. ~/.bashrc, ~/.zshrc, etc.)
$ curl -s -L https://api.chalk.ai/install.sh | shInstalling Chalk...Downloading binary... Done!Version: v1.12.2Hash: de2bf781c52f39825840c89863bde17c32b9d1d1Build Time: 2023-07-18T18:14:26+00:00Chalk was installed successfully to /Users/kate/.chalk/bin/chalk-v1.0.3Run 'chalk --help' to get started.You may need to open a new terminal window for these changes to take effect.
The Chalk CLI supports a number of flags for every command.
Get help for any command in CLI with chalk help [command].
Every command also accepts the -h or --help flag to see help for that command.
Override the API host. This flag is only relevant if you're running on a Chalk Isolated deployment. You can also set the API host with the environment variable CHALK_API_HOST. Even then, it is not required unless this is your first time connecting your terminal to Chalk.
Override the environment name, if desired. To see the default environment, run chalk environment.
Override the GRPC API host. This flag is only relevant if you're running on a Chalk Isolated deployment. You can also set the API host with the environment variable CHALK_GRPC_API_HOST. Even then, it is not required unless this is your first time connecting your terminal to Chalk.
Hides loading spinners. This flag can be useful in CI and other non-TTY environments, where spinners may render poorly.
By default, Chalk samples traces for feature requests. If --trace is set, Chalk will generate trace logs for feature requests.
Connect the CLI to your Chalk account by logging in to persist your client ID and secret locally.
The Chalk CLI runs commands using a global configuration or project-specific configurations. To configure the CLI globally, run:
$ chalk login
You'll be redirected to the dashboard to confirm that you want to give the CLI access to your account.
All configurations are stored in ~/.config/chalk.yml but
you can use the
XDG_CONFIG_HOME
environment variable to override this location.
If set, Chalk will open the CLI authorization window in your browser without asking. Useful for automated scripts.
$ chalk login✓ Created sessionOpen the authorization page in your browser? YesComplete authorization at https://chalk.ai/cli/ls-cldtwriw700ds62e07sr55✓ Opened login window in browser⡿ Waiting...
Use the environment command to view and edit the current environment.
In Chalk, each team has many projects, and each project has many environments. Within an environment, you may define different data sources and different resolvers.
The name of the environment to activate. When <env> is not specified, this command prints information about the current environment.
$ chalk environment prod✓ Fetched available environments✓ Set environment to 'prod'
Use the config command to view information about
the authorization configuration that the CLI is using to make requests.
When you connect the CLI to your Chalk account by logging in with
chalk login, Chalk stores your client ID and secret
in ~/.config.yml (unless overridden by the
XDG_CONFIG_HOME
environment variable).
The config will change according to the active environment (determined by
chalk environment and active project
(determined by chalk project.
$ chalk configPath: /Users/emarx/dev/example-projectName: Test TokenClient Id: client-XXXXXClient Secret: secret-XXXXXEnvironment: devAPI Server: https://api.chalk.aiValid Until: 2023-01-23T18:05:05.007000
Use the init command to set up a new Chalk project.
This command creates two files for you: chalk.yaml and .chalkignore.
The first file, chalk.yaml, contains configuration information about
your project. The second file, .chalkignore, tells the CLI
which files to ignore when deploying to your Chalk environment.
You can edit this file and use it just like a .gitignore file.
$ chalk initCreated project config file chalk.yamlCreated .chalkignore file
Use the project command to view and edit the current project.
In Chalk, each team has many projects, and
each project has many environments.
A project is defined by a folder with a
chalk.yml or
chalk.yaml file.
The contents within that folder define the features and resolvers of the project.
That code is deployed once per environment.
The project configuration file allows you to view and modify:
Your current working directory (or any parent directory)
must contain a
chalk.yml or
chalk.yaml file.
$ chalk projectName: CreditEnvironment: defaultRequirements: requirements.txtRuntime: python311Environment: stagingRequirements: requirements-staging.txtRuntime: python311
Use the infra command to manage and update various infrastucture elements
of your chalk environment.
The infrastructure under management will change according to the active environment (determined by
chalk environment and active project
(determined by chalk project.
Use the apply command to deploy your feature pipelines.
Chalk projects have a configuration file in the root of the project named
chalk.yml or chalk.yaml.
Typically, chalk apply is run from the project root,
but it can also be run from any child directory of the project.
The deploy is composed of three steps:
chalkpy to check for errors in any Chalk resolvers (for example, resolvers that take incompatible inputs and outputs).--force is specified, you'll see a diff of the features, and asked to confirm before deployment..gitignore or .chalkignore.By default, the chalk apply command deploys your features and resolvers to a production serving
environment. These deployments roll out gradually over the course of ~1 minute to eliminate downtime.
However, you may wish to iterate more quickly on your feature pipelines. For development,
you can use the --branch flag to deploy to a named and ephemeral environment.
Branch deployments are optimized for iterating on your feature and resolver definitions and deploy in
~5 seconds.
Once you've deployed to a branch, you can query new features and resolvers in the branch:
> chalk apply --branch feat1
> chalk query --in user.name --out user --branch feat1
Branch deployments also allow you to create or modify features and resolvers from a Jupyter notebook, in real time.
You can also use the --reset flag to deploy a clean version of the working directory to a branch, resetting any changes that may have been made from a notebook.
> chalk apply --branch feat1 --reset
We can make the iteration loop even tighter by adding the --watch flag to the
chalk apply --branch command. This will watch for changes to your feature and resolver definitions
and deploy them to the branch automatically.
> chalk apply --branch --watch
If true, wait for the deployment to go live before exiting. Otherwise, exit and print a deployment URL that tracks the deployment progress.
If true, the deployment will not execute. The diff will be printed, and validation errors will be reported.
By default, Chalk will deploy to your active environment. However, running with --no-promote, pipelines will be available on a Preview Deployment.
Resets the given branch to the state of the working directory by removing any features or resolvers that were created/updated in a notebook. Must also supply ''--branch''.
$ chalk apply✓ Found resolvers✓ Successfully validated features and resolvers!✓ Checked against live resolversAdded ResolversName ← →────────────────────────────────────────────────────────────+ features.underwriting.get_user_details_44 1 1Added FeaturesName Cache? Owner────────────────────────────────────────────────────────────+ example_user.id+ example_user.name+ example_user.fraud_score 30m fraud@company.comWould you like to deploy? [y/n]
Use the query command to quickly test your features pipelines.
Chalk supports several API clients for production use. But in development, it is convenient
to quickly pull feature values without using an API client or writing code. The chalk query command allows you to pull features and test deployments.
You can even request entire feature classes by asking for the feature namespace. For example,
$ chalk query --in user.id=4 --out user
will return all of the features on user. It will not, however, return all the features of all the has-one relationships of user.
If you do want to return has-one features, you can specify them in the query:
$ chalk query --in user.id=4 \
--out user \
--out user.card
In the above example, the Chalk CLI will return all the scalar features of user and of user.card.
Known feature value. Prefix the feature with . to disable namespace inference in --out.
Examples: --in user.id=1232, --in .user.id=1232
Feature or feature namespace to compute. When a feature namespace is not specified, the namespace will be inferred from --in if possible.
Examples: --out user.name, --out user, --out user.organization.name, --out name
Specify a max-staleness for a feature.
Example: --staleness user.some_feature=5m
For applications where the underlying data is changing frequently (as in streaming applications), it can be helpful to poll the same query. When --repeat is specified, the Chalk CLI will run your query every <duration> period.
Optional. Overrides 'now' in resolver execution within the query. Pass as an ISO8601 instant, e.g.: '2023-01-01T09:30:00Z'
Run a benchmark on the query. Specify parameters: --benchmark --qps=8000 --duration=10s
File to which to save the output of a benchmark. For use only when --benchmark is set
Duration to warm up the benchmark benchmark in seconds. For use only when --benchmark is set
Use online query with multiple inputs per feature and has-many outputs. With --bulk, you can pass --in multiple times for the same feature to, for example, pass a list of ids.
$ chalk query --in user.id=4 \--out user \--out user.card \--out user.organization.name \--staleness user.fraud_score=20m
Use the lint command to check for errors in your feature pipelines.
The lint command is composed of two steps:
chalkpy to check for errors in any Chalk resolvers (for example, resolvers that take incompatible inputs and outputs).$ chalk lint✓ Found resolvers✓ Successfully validated features and resolvers!
Use the delete command to remove features for specific primary keys.
This command is irreversible, and will drop all data for the given features and primary keys in the online and offline stores.
You can also drop data by tag, which
can be helpful to meet GDPR requirements.
For example, if you tag PII features with pii, you can run
$ chalk delete --tags pii --keys=user2342
The namespace of the feature to be deleted. If not provided, features are expected to be namespace (e.g. ''user.name'').
$ chalk delete --keys=1,2,3 --features user.name,email,ageAre you sure you want to delete these features? [y/n]Successfully deleted features
Use the drop command to drop all
instances of a feature.
This command is irreversible, and will drop all data for the given features in the online and offline stores. After dropping a feature, you can re-create the feature with a different type.
The namespace of the feature to be deleted. If not provided, features are expected to be namespace (e.g. ''user.name'').
$ chalk drop --features user.name,email,ageThe following features will be dropped:- user.name- user.email- user.ageAre you sure you want to drop all instances of the features? [y/n]Successfully dropped features
Use the trigger command to run resolvers from the CLI.
In addition to scheduling resolver executions,
Chalk allows you to trigger resolver executions from the CLI.
The trigger endpoint, also supported in Chalk's API clients,
allows you to build custom integrations with other
data orchestration tools like Airflow.
If you trigger a resolver that takes arguments, Chalk will sample the latest value of all temporally-consistent feature values. Then, it will execute the resolver for each of those arguments.
$ chalk trigger --resolver my.module.fnID: j-2qtwuxpskm2pbgStatus: ReceivedURL: https://chalk.ai/runs/j-2qtwuxpskm2pbg
Use the incremental status command to get the current progress state for an
incremental resolver.
Specifically, this returns the timestamps used by the resolver to only process recent input data.
$ chalk incremental status --resolver my.module.fnResolver: my.module.fnEnvironment: my_environment_idMax Ingested Timestamp: 2023-01-01T09:30:00+00:00Last Execution Timestamp: N/A
Use the incremental drop command to clear the current progress state for an
incremental resolver.
Specifically, this erases the timestamps the resolver uses to only ingest recent data. The next time the resolver runs, it will ingest all historical data that is available.
$ chalk incremental drop --resolver my.module.fnSuccessfully cleared incremental progress state for resolver: my.module.fn
Use the incremental set command to set the current progress state for an
incremental resolver.
Specifically, this configures the timestamps used by the resolver to only process recent input data.
max_ingested_ts represents the latest timestamp found in the input data on the resolver's previous run.last_execution_ts represents the most recent time at which this resolver was run.Both of these values must be given as ISO-8601 timestamps with a time zone specified.
$ chalk incremental set --resolver my.module.fn --max_ingested_ts "2023-01-01T09:30:00Z"Successfully updated incremental progress state for resolver: my.module.fn
Use the aggregate backfill command to backfill a materialized window aggregation.
The names of the feature to backfill. Chalk will backfill potentially many aggregations for a single feature
An ISO8601 instant string to set the lower bound on the feature time above which to backfill.
An ISO8601 instant string to set the upper bound on the feature time below which to backfill.
If set, the command will print the plan of the backfill without executing it. The output will show the expected number of tiles to be backfilled and anticipated storage needs.
If true, the backfill will execute the underlying SQL source to determine the exact number of rows that need to migrate.
$ chalk aggregate backfill --feature user.transaction_sum
$ chalk aggregate listSeries Namespace Group Agg Bucket Retention Aggregation Dependent Features────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────1 transaction user_id merchant_id amount 1d 30d sum user.txn_sum_by_merchant merchant.txn_sum_by_user1 transaction user_id merchant_id amount 1d 30d count user.txn_count_by_merchant2 transaction user_id amount 1d 30d sum user.txn_sum
Use the topic push command to push an message to one of your configured Kafka topics.
This command can be useful for testing streaming applications.
The value field accepts special template syntax to generate random values. The following template functions are available:
rand(): Generate a random float between 0 and 1.rand(max): Generate a random float between 0 and max.rand(min, max): Generate a random float between min and max, inclusive.randint(): Generate a random integer between 0 and 1,000,000, inclusive.randint(max): Generate a random integer between 0 and max, inclusive.randint(min, max): Generate a random integer between min and max, inclusive.randstr(length): Generate a random string of length length.now(): Generate the current datetime in ISO 8601 format.For example, we might push a message of the form:
--value '{"ip": "121.32.randint(255).randint(255)"}'
which would generate messages like {"ip": "121.32.43.232"}.
Repeat sending the message. When --repeat is specified, the Chalk CLI will run your query every <duration> period.
$ chalk topic push --value '{"key": "value"}'✓ Asked Chalk to push message to topic
Use the stubgen command to generate
Python stubs
using your feature types.
Watch the current files for changes and run stubgen when there are changes in the project.
The root directory of your project. By default, Chalk will find the root directory by looking for a chalk.yaml file.
$ chalk stubgen --watch⡿ Waiting for changes...
This command generates features that can be
used with the ChalkClient class in the
chalkpy pip package.
Whether to include the dtype explicitly for each feature, e.g. feature(dtype=pa.int64())
Path of output file with the filename included. Creates the file if it does not exist.
$ chalk codegen python --out=output.pyWrote features to file 'output.py'
This command generates Go structs that mirror the defined features, and create references to the available features. For details on using the resulting generated code, see the Go client.
Path of output file with the filename included. Creates the file if it does not exist.
Package name to use. If unspecified, we will guess the appropriate package based on neighboring files of the output file specified.
$ chalk codegen go --out=/codegen/features.go✓ Wrote features to file '/codegen/features.go'
Java codegen generates classes that mirror the features
classes defined in Python and creates Feature objects
for each available feature. If that also fails, please use the
--package flag to specify the package name explicitly.
For details on using the resulting generated code, see the
Java client.
Java package name that the generated code should use. If not specified, we will infer from existing files in the output directory. If that fails, we will concatenate folder names in the path after src/main/java.
$ chalk codegen java --out=/java_project/src/main/java/codegen/✓ Wrote features to folder '/java_project/src/main/java/codegen/'
Typescript codegen generates TypeScript types that mirror the defined features.
It will also generate a type FeaturesType that can be used to
parameterize the Chalk TypeScript client in order to provide automatic
autocomplete and type checking for queries.
Path of output file with the filename included. Creates the file if it does not exist.
$ chalk codegen typescript --out=/ts_project/src/codegen/features.ts✓ Wrote features to file '/ts_project/src/codegen/features.ts'
This command will generate Pydantic models that can
be used to describe streaming messages and
structs from proto files given with --in.
Output filepath to dump the pydantic models. The path should include the filename, and the file will be created if it does not exist.
$ chalk codegen pydantic --in proto/ --out=output.py✓ Wrote Pydantic models to file 'output.py'
Get an access token.
$ chalk tokeneyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOmYRzLmxpc3QiLCJ0b2tlbnM3cml0ZSIsInRlYW0uYWRkbmUiLCJjcm9uLmNyZWF0ZSIsImRlcGxveS5jcmVhdGUiLCJtaWdyYXRlLmV4ZWN1dGUiLCJzZWNyZXRzLndyaXRlIiwidGVhbS5hZGQiLCJ0ZWFtLmFkbWluIiwidGVhbS5kZWxldGUiLCJ0b2tlbnMud3JpdGUiXX0sImltcGVyc29uYXRlZCI6ZmFsc2V9.ZcEUIsroLoE3sgXovnZUBYZeEZC93VVsFXSP92qw0cdAMDxKavZnZv3L8k2eRVHK-KbNOEnwntk15RmopDzYzIKt8ne3u9mqgJg4YHseooskZkrHIy9F8zj94dFr3oJ9_2JDULsfKUxQmYHV9-u2mxu-AKNqtA
User the user command to print the currently logged-in user.
Useful for checking that your CLI can communicate with Chalk.
$ chalk userUser: cl0wpcey0770609l56lazvc52Environment: 900nw89h0wz0613l57la8v958Team: 131cpe52r000009l0662k3m11
Print the build, platform, and version information for the Chalk CLI tool.
$ chalk versionchalk versionVersion: v0.9.5Platform:Hash: a9297a32e5d2e6507f27d2ea98b831fbcb775e21Build Time: 2023-01-23T21:19:53+00:00$ chalk version --tag-onlyv0.9.5
Print out Chalk's public changelog in an interactive viewer
$ chalk changelog# ChangelogImprovements to Chalk are published here!See our [public roadmap](https://github.com/orgs/chalk-ai/projects/1) for upcoming changes.---## January 26, 2023### SQL File ResolversSQL-integrated resolvers can be completely written in SQL files: no Python required!...
Use the chalk branch source command to download the
source code for a branch.
Output folder where code should be output. If not specified, will use the deployment ID as the containing folder.
$ chalk branch source✓ Fetched branchesWhich branch would you like to download?: elliot-testWhich deployment id would you like to download?: clkhcspz1000201or1uw21rrz✓ Fetched download link✓ Downloaded source✓ Extracted source to clkhcspz1000201or1uw21rrz
Use the chalk branch list command to view information about
branches that have been deployed in this environment.
Branches can be created with chalk apply --branch.
$ chalk branch list✓ Fetched branchesName Deploys Last─────────────────────────────────────elliot-test 3 2h agotesting 33 3w agogabs 54 3w agotest-mc 6 1mo agotest_credit_score 1 1mo ago
Use the healthcheck command to view information on
the health of Chalk's api server and its services.
The healthcheck will change according to the active environment (determined by
chalk environment and active project
(determined by chalk project.
$ chalk healthcheckName Status───────────────────────────────────────────────────────Metadata DB HEALTH_CHECK_STATUS_OKgRPC Client Query HEALTH_CHECK_STATUS_OKLogging Client HEALTH_CHECK_STATUS_OKMetrics DB HEALTH_CHECK_STATUS_OKPush Registry HEALTH_CHECK_STATUS_OKSource Bucket HEALTH_CHECK_STATUS_OK
Use the chalk secret list command to view the secrets that are available in this environment.
Secrets can be deleted with chalk secret delete.
$ chalk secret list --decrypt✓ Fetched secretsName Value Integration Updated────────────────────────────────────────────────────PGDATABASE abc pgdemo 1mo agoPGHOST 44.444.444.444 pgdemo 1mo agoPGPASSWORD abvdg$Qabw3-m!zP pgdemo 1mo agoPGPORT 5432 pgdemo 2mo agoPGUSER developer pgdemo 1mo agoAPI_KEY PRZ2fyw3ynf.cvm! 1mo ago
Use the chalk secret set command to upsert secrets.
This command can be used to set one or more secrets at once.
If no arguments are provided, chalk secret set will enter interactive mode,
which will prompt you for the secret name and value. You can also provide only
the secret name as an argument, and the CLI will prompt you for the value.
chalk secret set
Using stdin is helpful for setting secrets whose value is the contents of a file:
cat key.pem | chalk secrets set TLS_CERT
You can also use stdin to set the value of a secret to the output of a command or script:
base64 -i chalk.p12 | chalk secrets set PKCS12_CERT
You can also provide key-value pairs as arguments to chalk secret set.
chalk secrets set HOSTNAME=73.62.143.151
chalk secrets set API_KEY=0x9Xz4#3 PORT=1234
Note that using key-value pairs will cause the secret value to be stored in
your shell history. To avoid this, use interactive mode, stdin, or the web dashboard.
Or, follow the instructions below to exclude chalk secrets set commands from
being stored in your shell history.
Bash and ZSH will ignore chalk secrets set commands in
your shell history if you set the HISTIGNORE or HISTORY_IGNORE environment
variables, respectively.
For Bash, add the following to your ~/.bashrc file:
export HISTIGNORE='*chalk secrets set*'
For ZSH, add the following to your ~/.zshrc file:
HISTORY_IGNORE="(chalk secrets set*)"
$ chalk secrets set HOSTNAME=73.62.143.151✓ Secrets saved
Use the chalk secret delete command to delete a secret.
You can either specify the secret name with the --name flag, select a secret from a list,
or specify the secret names as arguments.
$ chalk secret delete --name my-secret✓ Deleted secret
Use the flare command to send your code to a partner at Chalk for assistance.
$ chalk flareCode uploaded successfully
Submit feedback to the Chalk team.
$ chalk feedback_ _ _| | | | |___| |__ __ _| | | __ _____/ __| '_ \ / _` | | |/ / | || (__| | | | (_| | | < | |\___|_| |_|\__,_|_|_|\_\ |_____|We love hearing your feedback and suggestions!Feel free to add issues to our public roadmap: https://github.com/chalk-ai/roadmap/
Use the 'export' command to retrieve and export Chalk usage data based on specified criteria.
The 'export' command allows you to fetch usage data from Chalk and save it as a CSV file for external analysis. You can group the data by 'cluster' or 'instance' and specify the reporting period as either 'daily' or 'monthly'. Additionally, you can set custom date ranges to filter the exported data based on start and end dates.
Optionally save the usage chart to a file. If not specified, this command will print the chart to the terminal.
$ chalk usage export --period monthlyDate Credits Label─────────────────────────────────────────────────────────────────2024-05-17 00:00:00 UTC 8925.98 sandbox-eks-cluster2024-06-17 00:00:00 UTC 8283.77 sandbox-eks-cluster2024-07-17 00:00:00 UTC 3769.53 sandbox-eks-cluster
$ chalk usage ratesCloud Machine vCPUs Memory (gb) Credits/hr──────────────────────────────────────────────────────────GCP n2d-standard-4 4 16 0.85GCP n2d-standard-8 8 32 1.7GCP n2d-standard-16 16 64 3.4GCP n2d-standard-32 32 128 6.8
Help provides help for any command in the application. Simply type chalk help [path to command] for full details.
Open the Chalk dashboard in your web browser.
$ chalk dashboardOpening https://chalk.ai/projects
The chalk doctor command will remove all cached JWTs, leading to new credentials exchange for the next new request. This can be helpful if your ~/.config/chalk.yml file changed or has been corrupted.
Update the Chalk CLI tool. You can always switch back to your old build, stored in ~/.chalk/bin/.
The version of the CLI to install. By default latest is installed. Otherwise, the format should match something like v1.12.4
$ chalk updateInstalling Chalk...Downloading binary... Done!Version: v1.12.4Platform:Hash: 01e05fcb93cfe81fcfb6a871e27f46299d536740Build Time: 2023-02-08T07:28:55+00:00Chalk was installed successfully to /Users/emarx/.chalk/bin/chalk-v1.12.4Run 'chalk --help' to get started
Use the files command to see which of your files will be uploaded to Chalk.
Chalk respects .chalkignore and .gitignore files and only uploads files that don't match patterns in those files.
$ chalk files/home/user/project_dir/.chalkignore/home/user/project_dir/.gitignore/home/user/project_dir/resolvers.py/home/user/project_dir/features.py