Feature Engine
Connect your data sources, then define, deploy, and query your first features.
This page starts where the Setup Guide ends: you have a project that chalk init created, and
the CLI is logged in. It walks through connecting your data sources, defining feature classes and resolvers, and
deploying and querying them.
Your resolvers load data from the data sources that you add in the Chalk dashboard.
chalk dashboard, or go to
https://chalk.ai/projects. The dashboard shows every environment that your login has
access to.src/datasources.py file that refers to the data sources you added:
root_directory/
├── src/
│ └── datasources.py
├── chalk.yaml
├── README.md
└── pyproject.toml
For example, for a PostgreSQL data source named PG, datasources.py contains:
from chalk.sql import PostgreSQLSource
pg = PostgreSQLSource(name='PG')
For the settings of each kind of data source, see Integrations.
A feature class is a Python class of features, and a resolver tells Chalk how to compute feature values. Each feature that you define is the output of a resolver.
We recommend starting with a small feature class and adding to it, testing as you go. A project with several feature classes, resolvers, and tests looks something like this:
root_directory/
├── src/
│ ├── resolvers/
│ │ ├── .../
│ │ ├── __init__.py
│ │ └── pipelines.py
│ ├── __init__.py
│ ├── datasources.py
│ └── feature_sets.py
├── tests/
│ └── ...
├── .chalkignore
├── chalk.yaml
├── README.md
└── pyproject.toml
For the kinds of features and resolvers that you can write, see Features and Resolvers. For help structuring your feature classes and resolvers, ask in your support channel.
Deploy your features and resolvers with chalk apply. During development, deploy to a
branch with chalk apply --branch <branch-name>, so that several people can
work in one environment at the same time. Deploying without --branch requires the deploy.create permission,
which the built-in Admin and Owner roles include. For the permissions each task needs, see
Permissions.
After you deploy, query your features from the command line with chalk query, or from code with a
Chalk client for Python, Go,
Java, or TypeScript.
Repeat these steps (write, deploy, and query) until the feature values that you get back are the values you expect.
Once your features return the values you expect, decide how the rest of your machine learning platform uses them. You can trigger resolvers and run queries from other orchestrated pipelines, such as Airflow. You can also run resolvers on a cron schedule, and set staleness values on features so that the data you query meets your freshness requirements.
For a tutorial that builds a fraud detection pipeline with Chalk, see Tutorial: Fraud Detection Pipeline.