Chalk home page
Docs
API
CLI
  1. Chalk CLI

Chalk CLI Reference

This reference page documents every command and flag available in Chalk's command-line interface.

The Chalk CLI allows you to create, update, and manage your feature pipelines directly from your terminal.

With the Chalk CLI, you can:

  • Create, update, and validate feature pipelines, including migrating existing feature data
  • Query for online and offline data from your feature pipelines, right in your terminal
  • Retrieve detailed information about your projects, environments, and data

Installing Chalk is easy! Run the command below to install the latest version of Chalk.

$ curl -s -L https://api.chalk.ai/install.sh | sh

Subsequent updates can be installed using chalk update. After installing, you may need to restart your terminal, or source your shell's rc file (e.g. ~/.bashrc, ~/.zshrc, etc.)

$ curl -s -L https://api.chalk.ai/install.sh | sh
Installing Chalk...
Downloading binary... Done!
Version: v1.0.3
Hash: de2bf781c52f39825840c89863bde17c32b9d1d1
Build Time: 2023-07-18T18:14:26+00:00
Chalk was installed successfully to /Users/kate/.chalk/bin/chalk-v1.0.3
Run 'chalk --help' to get started.
You may need to open a new terminal window for these changes to take effect.

The Chalk CLI supports a number of flags for every command.

Get help for any command in CLI with chalk help [command].

Every command also accepts the -h or --help flag to see help for that command.

Flags

Override the API host. This flag is only relevant if you're running on a Chalk Isolated deployment. You can also set the API host with the environment variable Even then, it is not required unless this is your first time connecting your terminal to Chalk.

Send the request to a given branch.

Override the client id.

Override the client secret.

Override the environment name, if desired. To see the default environment, run chalk environment.

--json
bool

Output all commands in JSON instead of the default human-readable format.

By default, Chalk checks for updates to the CLI every 24 hours. If --skip-version-check is set, version checks will not be performed.

Enable verbose logging.

Connect the CLI to your Chalk account by logging in to persist your client ID and secret locally.

The Chalk CLI runs commands using a global configuration or project-specific configurations. To configure the CLI globally, run:

$ chalk login

You'll be redirected to the dashboard to confirm that you want to give the CLI access to your account.

All configurations are stored in ~/.config/chalk.yml but you can use the XDG_CONFIG_HOME environment variable to override this location.

Flags

If set, Chalk will open the CLI authorization window in your browser without asking. Useful for automated scripts.

$ chalk login
✓ Created session
Open the authorization page in your browser? Yes
Complete authorization at https://chalk.ai/cli/ls-cldtwriw700ds62e07sr55
✓ Opened login window in browser
⡿ Waiting...

Use the config command to view information about the authorization configuration that the CLI is using to make requests.

When you connect the CLI to your Chalk account by logging in with chalk login, Chalk stores your client ID and secret in ~/.config.yml (unless overridden by the XDG_CONFIG_HOME environment variable).

The config will change according to the active environment (determined by chalk environment and active project (determined by chalk project.

$ chalk config
Path: /Users/emarx/dev/example-project
Name: Test Token
Client Id: client-XXXXX
Client Secret: secret-XXXXX
Environment: dev
API Server: https://api.chalk.ai
Valid Until: 2023-01-23T18:05:05.007000

Use the init command to set up a new Chalk project. This command creates two files for you: chalk.yaml and .chalkignore. The first file, chalk.yaml, contains configuration information about your project. The second file, .chalkignore, tells the CLI which files to ignore when deploying to your Chalk environment. You can edit this file and use it just like a .gitignore file.

$ chalk init
Created project config file chalk.yaml
Created .chalkignore file

Use the project command to view and edit the current project.

In Chalk, each team has many projects, and each project has many environments. A project is defined by a folder with a chalk.yml or chalk.yaml file. The contents within that folder define the features and resolvers of the project. That code is deployed once per environment.

The project configuration file allows you to view and modify:

Your current working directory (or any parent directory) must contain a chalk.yml or chalk.yaml file.

Flags

Opens the configuration file in your default editor.

$ chalk project
Name: Credit
Environment: default
Requirements: requirements.txt
Runtime: python310
Environment: staging
Requirements: requirements-staging.txt
Runtime: python310

Use the environment command to view and edit the current environment.

In Chalk, each team has many projects, and each project has many environments. Within an environment, you may define different data sources and different resolvers.

Arguments
<env>
string

The name of the environment to activate. When <env> is not specified, this command prints information about the current.

$ chalk environment prod
✓ Fetched available environments
✓ Set environment to 'prod'

Use the apply command to deploy your feature pipelines.

Chalk projects have a configuration file in the root of the project named chalk.yml or chalk.yaml. Typically, chalk apply is run from the project root, but it can also be run from any child directory of the project.

The deploy is composed of three steps:

  • Local validation: The Chalk CLI uses chalkpy to check for errors in any Chalk resolvers (for example, resolvers that take incompatible inputs and outputs).
  • Remote validation: Once local validation passes, Chalk sends the features and resolvers to the server to validate against the deployment.
  • Confirmation: Unless --force is specified, you'll see a diff of the features, and asked to confirm before deployment.
  • Code upload: The Chalk CLI bundles the code in the project root and sends it to Chalk's servers. Chalk does not upload files matched in your .gitignore or .chalkignore.

Branch deploys

By default, the chalk apply command deploys your features and resolvers to a production serving environment. These deployments roll out gradually over the course of ~1 minute to eliminate downtime.

However, you may wish to iterate more quickly on your feature pipelines. For development, you can use the --branch flag to deploy to a named and ephemeral environment. Branch deployments are optimized for iterating on your feature and resolver definitions and deploy in ~5 seconds.

Once you've deployed to a branch, you can query new features and resolvers in the branch:

> chalk apply --branch feat1
> chalk query --in user.name --out user --branch feat1

Branch deployments also allow you to create or modify features and resolvers from a Jupyter notebook, in real time. You can also use the --reset flag to deploy a clean version of the working directory to a branch, resetting any changes that may have been made from a notebook.

> chalk apply --branch feat1 --reset 

File watches

We can make the iteration loop even tighter by adding the --watch flag to the chalk apply --branch command. This will watch for changes to your feature and resolver definitions and deploy them to the branch automatically.

> chalk apply --branch --watch
Flags

If true, skip confirmation before creating a new deployment.

If true, wait for the deployment to go live before exiting. Otherwise, exit and print a deployment URL that tracks the deployment progress.

If true, the deployment will not execute. The diff will be printed, and validation errors will be reported.

By default, Chalk will deploy to your active environment. However, running with --no-promote, pipelines will be available on a Preview Deployment.

Watch the current files for changes. Must also supply --branch to run with watch

Resets the given branch to the state of the working directory by removing any features or resolvers that were created/updated in a notebook. Must also supply --branch.

$ chalk apply
✓ Found resolvers
✓ Successfully validated features and resolvers!
✓ Checked against live resolvers
Added Resolvers
Name ← →
────────────────────────────────────────────────────────────
+ features.underwriting.get_user_details_44 1 1
Added Features
Name Cache? Owner
────────────────────────────────────────────────────────────
+ example_user.id
+ example_user.name
+ example_user.fraud_score 30m fraud@company.com
Would you like to deploy? [y/n]

Use the query command to quickly test your features pipelines.

Chalk supports several API clients for production use. But in development, it is convenient to quickly pull feature values without using an API client or writing code. The chalk query command allows you to pull features and test deployments.

You can even request entire feature classes by asking for the feature namespace. For example,

$ chalk query --in user.id=4 --out user

will return all of the features on user. It will not, however, return all the features of all the has-one relationships of user.

If you do want to return has-one features, you can specify them in the query:

$ chalk query --in user.id=4 \
			 --out user \
			 --out user.card

In the above example, the Chalk CLI will return all the scalar features of user and of user.card.

Flags

Deploy the code to a branch before running the query.

Known feature value. Example: --in user.id=1232

Feature or feature namespace to compute. For example, --out user.name, --out user, --out user.organization.name

Specify a max-staleness for a feature. Example: --staleness user.some_feature=5m

For applications where the underlying data is changing frequently (as in streaming applications), it can be helpful to poll the same query. When --repeat is specified, the Chalk CLI will run your query every <duration> period.

The deployment to query. This flag is useful when you wish to query code that was deployed with --no-promote.

Associate a query name with this request.

Optional. Overrides 'now' in resolver execution within the query. Pass as an ISO8601 instant, e.g.: '2023-01-01T09:30:00Z'

--tag <tag>
string[]

Tags to use when requesting these features.

Run a benchmark on the query. Specify parameters: --benchmark --qps=8000 --duration=10s

Queries per second. For use only when --benchmark is set

Duration to benchmark in seconds. For use only when --benchmark is set

File to which to save the output of a benchmark. For use only when --benchmark is set

Duration to warm up the benchmark benchmark in seconds. For use only when --benchmark is set

Persistently store all intermediate computed values for debugging. This dramatically decreases performance.

Explain this query

$ chalk query --in user.id=4 \
--out user \
--out user.card \
--out user.organization.name \
--staleness user.fraud_score=20m

Run a backfill or migration on your resolvers

Flags

Choose which resolver to migrate.

--plan
bool

Only return statistics about what the migration would do without modifying any data.

Number of parallel samples to process per worker.

Maximum number of features to sample from output.

If most-recent (default), sample only the most recent values of the arguments. If all, sample all historical values.

Pass a fully-qualified Python function name that refers to a valid 'sampling' function to fully control the sampled arguments.

Mark data from previous runs of this resolver as ineligible for serving.

Re-ingest all data, ignoring previous high-water marks. Relevant only for incremental resolvers.

Pass an ISO8601 instant string. Resolvers using incremental ingest will use this datetime as the timestamp lower-bound for ingesting rows.

Pass an ISO8601 instant string. Samples will only be considered if they are timestamped with a newer timestamp (inclusive) than this boundary

Pass an ISO8601 instant string. Samples will only be considered if they are timestamped with an older timestamp (inclusive) than this boundary

Use the lint command to check for errors in your feature pipelines.

The lint command is composed of two steps:

  • Local validation: The Chalk CLI uses chalkpy to check for errors in any Chalk resolvers (for example, resolvers that take incompatible inputs and outputs).
  • Remote validation: Once local validation passes, Chalk sends the features and resolvers to the server to validate against the deployment.
Flags

Hides loading spinners. This flag can be useful in CI and other non-TTY environments, where spinners may render poorly.

Only perform local validation. Do not call to Chalk for remote validation.

$ chalk lint
✓ Found resolvers
✓ Successfully validated features and resolvers!

Use the trigger command to run resolvers from the CLI.

In addition to scheduling resolver executions, Chalk allows you to trigger resolver executions from the CLI. The trigger endpoint, also supported in Chalk's API clients, allows you to build custom integrations with other data orchestration tools like Airflow.

If you trigger a resolver that takes arguments, Chalk will sample the latest value of all temporally-consistent feature values. Then, it will execute the resolver for each of those arguments.

Flags

The resolver to trigger.

The Chalk deployment id to trigger.

Persist data to the online store, if features are eligible.

Persist data to the offline store.

$ chalk trigger --resolver my.module.fn
ID: j-2qtwuxpskm2pbg
Status: Received
URL: https://chalk.ai/runs/j-2qtwuxpskm2pbg

Use the delete command to remove features for specific primary keys.

This command is irreversible, and will drop all data for the given features and primary keys in the online and offline stores.

You can also drop data by tag, which can be helpful to meet GDPR requirements. For example, if you tag PII features with pii, you can run

$ chalk delete --tags pii --keys=user2342
Flags

The namespace of the feature to be deleted. If not provided, features are expected to be namespace (e.g. user.name).

The features to be deleted.

The tags to be deleted.

Primary keys to be deleted.

If true, skip confirmation before deleting features.

$ chalk delete --keys=1,2,3 --features user.name,email,age
Are you sure you want to delete these features? [y/n]
Successfully dropped features

Use the drop command to drop all instances of a feature.

This command is irreversible, and will drop all data for the given features in the online and offline stores. After dropping a feature, you can re-create the feature with a different type.

Flags

The features to be dropped. May be specified as namespace.feature or without the namespace as feature if the namespace is specified with the --namespace flag.

The namespace of the feature to be dropped. If not provided, features are expected to be namespace (e.g. user.name).

If true, skip confirmation before dropping features.

If true, do not drop feature values from the online store

If true, do not drop feature values from the offline store

$ chalk drop --features user.name,email,age
The following features will be dropped:
- user.name
- user.email
- user.age
Are you sure you want to drop all instances of the features? [y/n]
Successfully dropped features

Use the incremental status command to get the current progress state for an incremental resolver.

Specifically, this returns the timestamps used by the resolver to only process recent input data.

  • Max Ingested Timestamp: the latest timestamp found in the input data on the resolver's previous run.
  • Last Execution Timestamp: the most recent time at which this resolver was run. By default, incremental resolvers rely only on the max ingested timestamp.
Flags

Name of an incremental resolver.

$ chalk incremental status --resolver my.module.fn
Resolver: my.module.fn
Environment: my_environment_id
Max Ingested Timestamp: 2023-01-01T09:30:00+00:00
Last Execution Timestamp: N/A

Use the incremental drop command to clear the current progress state for an incremental resolver.

Specifically, this erases the timestamps the resolver uses to only ingest recent data. The next time the resolver runs, it will ingest all historical data that is available.

Flags

Name of an incremental resolver.

$ chalk incremental drop --resolver my.module.fn
Successfully cleared incremental progress state for resolver: my.module.fn

Use the incremental set command to set the current progress state for an incremental resolver.

Specifically, this configures the timestamps used by the resolver to only process recent input data.

  • max_ingested_ts represents the latest timestamp found in the input data on the resolver's previous run.
  • last_execution_ts represents the most recent time at which this resolver was run.

Both of these values must be given as ISO-8601 timestamps with a time zone specified.

Flags

Name of an incremental resolver.

Latest timestamp found in ingested data (ISO 8601 Timestamp).

Timestamp of latest run of this resolver (ISO 8601 Timestamp).

$ chalk incremental set --resolver my.module.fn --max_ingested_ts "2023-01-01T09:30:00Z"
Successfully updated incremental progress state for resolver: my.module.fn

Use the topic push command to push an message to one of your configured Kafka topics.

This command can be useful for testing streaming applications.

The value field accepts special template syntax to generate random values. The following template functions are available:

  • rand(): Generate a random float between 0 and 1.
  • rand(max): Generate a random float between 0 and max.
  • rand(min, max): Generate a random float between min and max, inclusive.
  • randint(): Generate a random integer between 0 and 1,000,000, inclusive.
  • randint(max): Generate a random integer between 0 and max, inclusive.
  • randint(min, max): Generate a random integer between min and max, inclusive.
  • randstr(length): Generate a random string of length length.

For example, we might push a message of the form:

--value '{"ip": "121.32.randint(255).randint(255)"}'

which would generate messages like {"ip": "121.32.43.232"}.

Flags

Name of the topic to push to.

Value of the message to push to the topic.

Key of the message to push to the topic.

Name of the integration to use.

Format of the message. JSON or binary accepted.

Repeat sending the message. When --repeat is specified, the Chalk CLI will run your query every <duration> period.

$ chalk topic push --value '{"key": "value"}'
✓ Asked Chalk to push message to topic

This command generates features that can be used with the ChalkClient class in the chalkpy pip package.

Flags

Path of output file with the filename included. We will create the file if it does not exist.

$ chalk codegen python --out=output.py
Wrote features to file 'output.py'

This command generates Go structs that mirror the defined features, and create references to the available features. For details on using the resulting generated code, see the Go client.

Flags

Path of output file with the filename included. We will create the file if it does not exist.

Package name to use. If unspecified, we will guess the appropriate package based on neighboring files of the output file specified.

$ chalk codegen go --out=/codegen/features.go
✓ Wrote features to file '/codegen/features.go'

Typescript codegen generates TypeScript types that mirror the defined features. It will also generate a type FeaturesType that can be used to parameterize the Chalk TypeScript client in order to provide automatic autocomplete and type checking for queries.

Flags

Path of output file with the filename included. We will create the file if it does not exist.

$ chalk codegen typescript --out=/ts_project/src/codegen/features.ts
✓ Wrote features to file '/ts_project/src/codegen/features.ts'

This command will generate Pydantic models that can be used to describe streaming messages and structs from proto files given with --in.

Flags

Output filepath to dump the pydantic models. The path should include the filename, and the file will be created if it does not exist.

--in <in>
string

The path of the input proto file or directory of proto files.

$ chalk codegen pydantic --in proto/ --out=output.py
✓ Wrote Pydantic models to file 'output.py'

Java codegen generates classes that mirror the features classes defined in Python and creates Feature objects for each available feature. If that also fails, please use the --package flag to specify the package name explicitly. For details on using the resulting generated code, see the Java client.

Flags

Output directory to dump generated Java class files.

The Java package name that the generated code should use. If not specified, we will infer from existing files in the output directory. If that fails, we will concatenate folder names in the path after src/main/java.

$ chalk codegen java --out=/java_project/src/main/java/codegen/
✓ Wrote features to folder '/java_project/src/main/java/codegen/'

Use the stubgen command to generate Python stubs using your feature types.

Flags

Watch the current files for changes and run stubgen when there are changes in the project.

The root directory of your project. By default, Chalk will find the root directory by looking for a chalk.yaml file.

$ chalk stubgen --watch
⡿ Waiting for changes...

Show the resolvers you have defined and the features they resolve

User the user command to print the currently logged-in user.

Useful for checking that your CLI can communicate with Chalk.

$ chalk user
User: cl0wpcey0770609l56lazvc52
Environment: 900nw89h0wz0613l57la8v958
Team: 131cpe52r000009l0662k3m11

Print the build, platform, and version information for the Chalk CLI tool.

Flags

Output only the tag, and no other information

$ chalk version
chalk version
Version: v0.9.5
Platform:
Hash: a9297a32e5d2e6507f27d2ea98b831fbcb775e21
Build Time: 2023-01-23T21:19:53+00:00
$ chalk version --tag-only
v0.9.5

Print out Chalk's public changelog in an interactive viewer

$ chalk changelog
# Changelog
Improvements to Chalk are published here!
See our [public roadmap](https://github.com/orgs/chalk-ai/projects/1) for upcoming changes.
---
## January 26, 2023
### SQL File Resolvers
SQL-integrated resolvers can be completely written in SQL files: no Python required!
...

Use the chalk branch list command to view information about branches that have been deployed in this environment.

Branches can be created with chalk apply --branch.

$ chalk branch list
✓ Fetched branches
Name Deploys Last
─────────────────────────────────────
elliot-test 3 2h ago
testing 33 3w ago
gabs 54 3w ago
test-mc 6 1mo ago
test_credit_score 1 1mo ago

Use the chalk branch source command to download the source code for a branch.

Flags

Output folder where code should be output. If not specified, will use the deployment ID as the containing folder.

Use the latest branch deployment instead of prompting for a deployment ID.

The deployment to query. If not specified, you will be asked to pick one.

$ chalk branch source
✓ Fetched branches
Which branch would you like to download?: elliot-test
Which deployment id would you like to download?: clkhcspz1000201or1uw21rrz
✓ Fetched download link
✓ Downloaded source
✓ Extracted source to clkhcspz1000201or1uw21rrz

Use the chalk secret list command to view the secrets that are available in this environment.

Secrets can be deleted with chalk secret delete.

Flags

Decrypt the secrets.

$ chalk secret list --decrypt
✓ Fetched secrets
Name Value Integration Updated
────────────────────────────────────────────────────
PGDATABASE abc pgdemo 1mo ago
PGHOST 44.444.444.444 pgdemo 1mo ago
PGPASSWORD abvdg$Qabw3-m!zP pgdemo 1mo ago
PGPORT 5432 pgdemo 2mo ago
PGUSER developer pgdemo 1mo ago
API_KEY PRZ2fyw3ynf.cvm! 1mo ago

Use the chalk secret set command to upsert secrets. This command can be used to set one or more secrets at once.

Interactive Mode (recommended)

If no arguments are provided, chalk secret set will enter interactive mode, which will prompt you for the secret name and value. You can also provide only the secret name as an argument, and the CLI will prompt you for the value.

chalk secret set

stdin (recommended)

Using stdin is helpful for setting secrets whose value is the contents of a file:

cat key.pem | chalk secrets set TLS_CERT

You can also use stdin to set the value of a secret to the output of a command or script:

base64 -i chalk.p12 | chalk secrets set PKCS12_CERT

Key-Value Pairs

You can also provide key-value pairs as arguments to chalk secret set.

chalk secrets set HOSTNAME=73.62.143.151
chalk secrets set API_KEY=0x9Xz4#3 PORT=1234

Note that using key-value pairs will cause the secret value to be stored in your shell history. To avoid this, use interactive mode, stdin, or the web dashboard. Or, follow the instructions below to exclude chalk secrets set commands from being stored in your shell history.

Exclude from Shell History

Bash and ZSH will ignore chalk secrets set commands in your shell history if you set the HISTIGNORE or HISTORY_IGNORE environment variables, respectively.

For Bash, add the following to your ~/.bashrc file:

export HISTIGNORE='*chalk secrets set*'

For ZSH, add the following to your ~/.zshrc file:

HISTORY_IGNORE="(chalk secrets set*)"
$ chalk secrets set HOSTNAME=73.62.143.151
✓ Secrets saved

Use the chalk secret delete command to delete a secret.

You can either specify the secret name with the --name flag, select a secret from a list, or specify the secret names as arguments.

Flags

Names of the secrets to delete.

$ chalk secret delete --name my-secret
✓ Deleted secret

Use the flare command to send your code to a partner at Chalk for assistance.

$ chalk flare
Code uploaded successfully

Submit feedback to the Chalk team.

$ chalk feedback
_ _ _
| | | | |
___| |__ __ _| | | __ _____
/ __| '_ \ / _` | | |/ / | |
| (__| | | | (_| | | < | |
\___|_| |_|\__,_|_|_|\_\ |_____|
We love hearing your feedback and suggestions!
Feel free to add issues to our public roadmap: https://github.com/chalk-ai/roadmap/

Help provides help for any command in the application. Simply type chalk help [path to command] for full details.

Open the Chalk dashboard in your web browser.

Flags

If true, print the URL instead of opening the browser.

$ chalk dashboard
Opening https://chalk.ai/projects

The chalk doctor command will remove all cached JWTs, leading to new credentials exchange for the next new request. This can be helpful if your ~/.config/chalk.yml file changed or has been corrupted.

Update the Chalk CLI tool. You can always switch back to your old build, stored in ~/.chalk/bin/.

Flags

The version of the CLI to install. By, default latest is installed. Otherwise, the format should match something like v0.9.4

$ chalk update
Installing Chalk...
Downloading binary... Done!
Version: v0.9.4
Platform:
Hash: 01e05fcb93cfe81fcfb6a871e27f46299d536740
Build Time: 2023-02-08T07:28:55+00:00
Chalk was installed successfully to /Users/emarx/.chalk/bin/chalk-v0.8.0
Run 'chalk --help' to get started

Use the files command to see which of your files will be uploaded to Chalk.

Chalk respects .chalkignore and .gitignore files and only uploads files that don't match patterns in those files.

$ chalk files
/home/user/project_dir/.chalkignore
/home/user/project_dir/.gitignore
/home/user/project_dir/resolvers.py
/home/user/project_dir/features.py