Skip to main content

Notifications

Integrations​

The integration page allow you to create new integration and modify or delete the existing ones.

info

Bleemeo creates default Integrations that allow you to create a notification rule by email, mobile application or webhook.

Create Integration​

You can create new Integration with the + Integration button. The first thing to do on the create new integration is to select the type of integration you need:

Finally you need to write the name of your integration who will be used to create notification rule. If you want to create a new notification rule with this integration go to notifications page. Once done click the Create button. Your new integration is added at the end of the list. It is enabled by default.

Actions on Integration​

You can enable or disable an integration with the toggle button. The 2 small icons on the right allow you to edit or delete an integration.

On this page you can see the notifications that you have configured.

Bleemeo creates a default notification that notify you by email of all errors.

default notification

Create Notification Rule​

You can create new notification with the + Notification button.

1st Step: Scope​

first view

The first thing to do on the create new notification is to select the scope of notification rule, you can:

  • Be notified on any server(s)
  • Be notified on a specific server
  • Be notified on a group of servers (Server group or Tag)

2nd Step: Problem​

Problem with metric​

second view metrics

You have four possibilities of problem with metric:

  • Be notified for any problem on selected server(s)
  • Be notified for specified problem on selected server(s)
  • Be notified with recording rule
  • Be notified when selected server(s) lost connection (This option doesn't exists if you have selected monitor in the previous view)
info

If you select the second option (specific metric), you will be able to save only if this metric has thresholds. It's possible to edit thresholds of selected metric directly in this modal (screenshot above). This is possible only if you have selected a specific agent, or a Server Group in the previous view.

Problem with Recording Rule​

second view promql

To create a recording rule with PromQL, you can use our documentation.

You have a name to choose, this name will be used as the alert name in status dashboard and notifications.

You can add warning and/or critical PromQL and configure the delay. The delay corresponds to the time during which the threshold of the PromQL must be exceeded to change its status.

info

If you have chosen "any servers" it will select all the metrics of the recording rule, if you have chosen a specific server it will select the metrics of the recording rule associated with the selected agent.

3rd Step: Targets​

third view promql

Finally you need to select who will be notified and how. You can choose directly targets and/or contacts group. If you want to create a new integration, you can by clicking on + More.

If you have selected a contacts group that doesn't exist, in the next view, you should edit this contacts group to add targets.

Set a delay for repetition​

You can add a repeat delay to a notification rule. This delay is in minutes, it is 0 by default, which means that the event will be notified only once, without repetition.

If you change it to 10 for example, you will receive a notification every 10 minutes for an event until it becomes "OK" again.

You can stop a repetition of an event from two views, in the list of events and in the status dashboard. If the event returns "OK" and returns in error, the repetition starts again.

Set time constraints on Target​

You can choose the days and times when you want to receive notifications from each target.

For this example, notifications will be sent only Monday to Friday between 8:00 am and 6:00 pm (UTC).

constraint notification

Or this example:

  • Monday to Friday between 8:00 am and 6:00 pm (UTC) on email
  • Week night on mobile application
  • No notifications between Friday 6:00 pm to Monday 8:00 am

complex notifications

Once done click the Save button. Your new notification rule is added at the end of the list. It is enabled by default.

Recording Rule​

You can find documentation on Prometheus Query Language on the web:

PromQL Documentation

PromQL Cheat Sheet

PromQL Example

Recording Rule creation with PromQL​

To create a Recording Rule with PromQL, go to recording rule page and create a new recording rule.

recording rule list

recording rule creation

You can write your promql in the first part, if the promql is error-free, it will display the result in the graph with the number of metric it generates at that moment.

You can add threshold warning and/or critical and a name, this name will be used as metric name in status dashboard and notifications.

info

If you don't set thresholds, you can use the metrics generated by the recording rule in dashboards, but there will be no alerts.

Example: Cassandra​

We have several Cassandra servers running on our Kubernetes, we want to generate one metric to add it to a dashboard and notify us when the sum of the containers' cpu usage exceeds 50% (warning) and 60% (critical).

info

If you have a notification rule that triggers all the problems, then there's nothing more to do, otherwise you need to create a notification rule with the recording rule.

PromQL:

sum(container_cpu_used{item=~"k8s_cassandra_cassandra.*"})

~"k8s_cassandra_cassandra.*" This allows you to find all the containers whose item name begins with k8s_cassandra_cassandra

promql cassandra