Recording Rules

Starter

Professional

You can find documentation on Prometheus Query Language on the web:

PromQL Documentation

PromQL Cheat Sheet

PromQL Example

Recording Rule creation with PromQL

To create a Recording Rule with PromQL, go to recording rule page and create a new recording rule.

You can write your PromQL in the first part; if the PromQL is error-free, it will display the result in the graph, along with the number of metrics it generates at that moment.

You can add a warning and/or critical threshold and a name; this name will be used as the metric name in the status dashboard and notifications.

Note

If you don’t set thresholds, you can use the metrics generated by the recording rule in dashboards, but there will be no alerts.

Example: Cassandra

We have several Cassandra servers running on our Kubernetes, we want to generate one metric to add it to a dashboard and notify us when the sum of the containers’ CPU usage exceeds 50% (warning) and 60% (critical).

Note

If you have a notification rule that triggers all the problems, then there’s nothing more to do, otherwise you need to create a notification rule with the recording rule.

PromQL:

sum(container_cpu_used{item=~"k8s_cassandra_cassandra.*"})

item=~"k8s_cassandra_cassandra.*" lets you find all the containers whose item name begins with k8s_cassandra_cassandra.