Chalk stores a variety of time series metrics that measure the latency and throughput of resolvers and streaming pipelines. Chalk displays these metrics on the Chalk Dashboard, but can also export them to monitoring systems of your choice via the OpenMetrics standard.
For this purpose, Chalk exposes an endpoint with metrics in a text-based format that systems like Datadog are designed to ingest. Datadog polls this endpoint periodically and ingests gauges, counters, histograms, and summaries from Chalk. Then, these metrics appear in Datadog as custom metrics.
Datadog Agents are processes that run on your servers to collect metrics and send them to Datadog’s online monitoring systems.
You can use an existing agent to collect Chalk metrics, or create an agent specifically for Chalk. Once you have created an agent, you can configure it to ingest metrics from Chalk.
Chalk secures the metrics export endpoint using the same authentication and authorization mechanisms used for accessing other Chalk systems.
After creating a Datadog agent, generate a set of service credentials with the
client_secret handy — you’ll need them in the next step.
Next, configure the Datadog agent to ingest OpenMetrics data from Chalk.
Edit the OpenMetrics configuration file, which
comes standard with Datadog Agent as of version 6.6.0. Datadog Agent configuration files are found in the
agent configuration directory.
In that directory is a file named
Add the following content:
- openmetrics_endpoint: https://api.chalk.ai/v1/metrics/export
## (Optional) A prefix to prepend to all metrics
supply the client ID and secret you generated in
metrics section of the config file,
you can specify any of the metrics found in the
list of supported metrics.
Your Chalk metrics will appear on the Datadog metrics summary page after updating the Datadog Agent configuration and restarting your Datadog Agent.
They should appear like this: