Streaming Resource Groups

Streaming resolvers can be assigned to resource groups, enabling you to isolate streaming workloads onto dedicated infrastructure. This is useful for separating high-throughput streams from lighter workloads, or for running specific resolvers on nodes with different resource profiles.

By default, streaming resolvers without a resource_group run on the Default resource group. When a resource_group is specified, the resolver only runs on the streaming server deployed for that resource group.

​

Setting up a resource group

​

1. Configure infrastructure

In the Chalk dashboard:

1. Navigate to Infrastructure in the left sidebar
2. Click Resource Configuration
3. Under Resource Groups, click Add Resource Group and give it a name
4. Add a Streaming Server to the new resource group and configure CPU, memory, and scaling
5. Add a gRPC Query Server so the streaming server can route computed features to the engine
6. Click Save and Apply Service to deploy

We recommend starting with the same resource settings as your Default group for simplicity.

​

2. Assign resolvers to the resource group

In your deployment source code, pass the resource_group parameter to make_stream_resolver:

from chalk.features import features, make_stream_resolver
from chalk.streams import KinesisSource

source = KinesisSource(name="transactions")

@features(max_staleness="1d", etl_offline_to_online=True)
class Transaction:
    id: str
    amount: float

# This resolver only runs on the "high-throughput" streaming server
transaction_resolver = make_stream_resolver(
    name="transaction_resolver",
    source=source,
    resource_group="high-throughput",
    output_features={
        Transaction.id: _.transaction_id,
        Transaction.amount: _.transaction_amount,
    },
)

Resolvers without a resource_group run on the Default streaming server.

​

3. Deploy

Run chalk apply to deploy your updated resolver definitions. The streaming server in the new resource group will automatically pick up only the resolvers assigned to it.

​

Example: isolating a high-throughput stream

Suppose you have five streaming resolvers, but one handles significantly more traffic than the others. You can isolate it onto dedicated infrastructure:

# Runs on the "high-throughput" resource group only
high_volume_resolver = make_stream_resolver(
    name="high_volume_resolver",
    source=high_volume_source,
    resource_group="high-throughput",
    output_features={...},
)

# Runs on the Default streaming server
low_volume_resolver = make_stream_resolver(
    name="low_volume_resolver",
    source=low_volume_source,
    output_features={...},
)

In this setup, the Default streaming server runs low_volume_resolver (and any other resolvers without a resource group), while the high-throughput streaming server runs only high_volume_resolver.