Services Metrics

Bleemeo agent can discover services running on your system and automatically monitor specific metrics for such services. For example with Apache HTTP server, the number of requests served is automatically monitored. For each service detected a tag with the service name is created which allows to filter your agents by service running on them.

Agent also checks TCP sockets for each services. By default a simple TCP connection is used to test the service, but some service support specific check. See service details below for supported specific checks. Those checks are executed every minute. They may be run earlier for TCP service: Bleemeo agent keeps a connection open with the service and if that connection is broken the check is executed immediately.

If you have any service not listed on this page, you can define a custom check or define a custom metric.

If you want to disable metrics for a service, you can ignore some services.

Apache HTTP

Agent uses a HTTP check if the service listen on port 80.

To enable metrics gathering, ensure Bleemeo agent could access Apache status on the URL http://server-address/server-status?auto. This should be done by default with Apache on Ubuntu or Debian. If not, this usually means to add the following to your default virtual host (e.g. the first file in /etc/apache2/sites-enabled/):

LoadModule status_module /usr/lib/apache2/modules/mod_status.so
<Location /server-status>
    SetHandler server-status
    # You may want to limit access only to Bleemeo agent
    # require ip IP-OF-BLEEMEO-AGENT/32
</Location>
ExtendedStatus On

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Apache running outside any container
    - id: "apache"
      address: "127.0.0.1"
      port: 80   # HTTP listenner, Agent don't support HTTPS here
    # For a Apache running in a Docker container
    - id: "apache"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 80   # HTTP listenner, Agent don't support HTTPS here

Agent gathers the following metrics:

Apache keeps track of server activity in a structure known as the scoreboard. There is a slot in the scoreboard for each worker and it contains the status of this worker. The size of the scoreboard is the maximum of concurrent users that Apache could server.

Apache do not start all workers when they are not needed (e.g. if there is enough workers waiting, Apache reuse them and don't start a new one).

The sum of all scoreboard items is the maximum of concurrent requests Apache can serve. This sum is calculated and stored in apache_max_workers metric.

Asterisk

Bitbucket

To enable metrics gathering, Bleemeo agent need to be installed with JMX enabled, see Java Metrics for setup details.

In addition Bitbucket need to expose JMX over a TCP port. To enable JMX you can follow Enabling JMX counters for performance monitoring on Atlassian documentation.

Here is a summary for unauthenticated access:

• Add option -Dcom.sun.management.jmxremote.port=3333 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false to the JVM. Default Bitbucket startup-script accept them from JMX_OPTS or JAVA_OPTS environment variable.
• Add jmx.enabled=true to $BITBUCKET_HOME/shared/bitbucket.properties

Warning: this will allow unauthenticated access. Make sure no untrusted access to this port is possible or you should setup authenticated JMX access.

Bleemeo agent should auto-detect the JMX port, if some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Bitbucket running outside any container
    - id: "bitbucket"
      address: "127.0.0.1"
      port: 7990
      jmx_port: 3333
      jmx_username: "monitorRole" # by default, no authentication is done
      jmx_password: "secret"
    # For a Bitbucket running in a Docker container
    - id: "bitbucket"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 7990
      jmx_port: 3333
      jmx_username: "monitorRole" # by default, no authentication is done
      jmx_password: "secret"

Agent gathers the following metrics:

Cassandra

To enable metrics gathering, Bleemeo agent need to be installed with JMX enabled, see Java Metrics for setup details.

In addition Cassandra need to expose JMX over a TCP port. To enable JMX you will need to:

• If using Docker, add environment variable JVM_OPTS=-Dcassandra.jmx.remote.port=7199
• If running Cassandra and Bleemeo agent as native package, nothing should be needed. Cassandra already expose JMX on port 7199 on localhost by default.
• In other case, add option -Dcom.sun.management.jmxremote.port=7199 -Dcom.sun.management.jmxremote.authenticate=false to the JVM.

Warning: this will allow unauthenticated access. Make sure no untrusted access to this port is possible or you should setup authenticated JMX access.

Bleemeo agent should auto-detect the JMX port, if some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Cassandra running outside any container
    - id: "cassandra"
      address: "127.0.0.1"
      port: 9042
      jmx_port: 7199
      jmx_username: "cassandra"  # by default, no authentication is done
      jmx_password: "cassandra"
    # For a Cassandra running in a Docker container
    - id: "cassandra"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 9042
      jmx_port: 7199
      jmx_username: "cassandra"  # by default, no authentication is done
      jmx_password: "cassandra"

Agent gathers the following metrics:

Bleemeo also support for detailed monitoring of specific Cassandra tables. To enable this, add the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Cassandra running outside any container
    - id: "cassandra"
      cassandra_detailed_tables:
      - "keyspace1.table1"
      - "keyspace2.table2"
    [...]
    # For a Cassandra running in a Docker container
    - id: "cassandra"
      instance: "CONTAINER_NAME"
      cassandra_detailed_tables:
      - "keyspace1.table1"
      - "keyspace2.table2"
    [...]

The following per-table metrics will be gathered:

Confluence

To enable metrics gathering, Bleemeo agent need to be installed with JMX enabled, see Java Metrics for setup details.

In addition Confluence need to expose JMX over a TCP port. To enable JMX you can follow Live Monitoring Using the JMX Interface.

Here is a summary for unauthenticated access:

• Add option -Dcom.sun.management.jmxremote.port=3333 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false to the JVM. Default Confluence startup-script accept them from CATALINA_OPTS or JAVA_OPTS environment variable.

Warning: this will allow unauthenticated access. Make sure no untrusted access to this port is possible or you should setup authenticated JMX access.

Bleemeo agent should auto-detect the JMX port, if some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Confluence running outside any container
    - id: "confluence"
      address: "127.0.0.1"
      port: 8090
      jmx_port: 3333
      jmx_username: "monitorRole" # by default, no authentication is done
      jmx_password: "secret"
    # For a Confluence running in a Docker container
    - id: "confluence"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 8090
      jmx_port: 3333
      jmx_username: "monitorRole" # by default, no authentication is done
      jmx_password: "secret"

Agent gathers the following metrics:

BIND

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a BIND running outside any container
    - id: "bind"
      address: "127.0.0.1"
      port: 53
    # For a BIND running in a Docker container
    - id: "bind"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 53

Only the metric from service check is produced:

Dovecot

Agent uses a IMAP check if the service listen on port 143.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Dovecot running outside any container
    - id: "dovecot"
      address: "127.0.0.1"
      port: 143   # IMAP listenner, Agent don't support IMAPS here
    # For a Dovecot running in a Docker container
    - id: "dovecot"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 143   # IMAP listenner, Agent don't support IMAPS here

Only the metric from service check is produced:

ejabberd

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a ejabberd running outside any container
    - id: "ejabberd"
      address: "127.0.0.1"
      port: 5672
    # For a ejabberd running in a Docker container
    - id: "ejabberd"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 5672

Only the metric from service check is produced:

Elasticsearch

Agent uses a HTTP check if the service listen on port 9200.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Elasticsearch running outside any container
    - id: "elasticsearch"
      address: "127.0.0.1"
      port: 9200
    # For a Elasticsearch running in a Docker container
    - id: "elasticsearch"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 9200

Agent gathers the following metrics:

Exim4

Agent uses a SMTP check if the service listen on port 25.

To enable metrics gathering, ensure Bleemeo agent could run mailq command. This usually means to add the following to your exim configuration (e.g. /etc/exim4/conf.d/main/99_local):

queue_list_requires_admin=false

You will then need to fresh configuration (warning: the following will lost any local change in /etc/exim4/exim4.conf.template):

update-exim4.conf.template --run
update-exim4.conf
service exim4 restart

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Exim running outside any container
    - id: "exim"
      address: "127.0.0.1"
      port: 25
    # For a Exim running in a Docker container
    - id: "exim"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 25

Agent gathers the following metrics:

FreeRADIUS

HAProxy

To enable metrics gathering, Bleemeo agent needs access to HAProxy stats page and to known where it may find this page.

On HAProxy side, you will need to enable statistics page on a HTTP(S) frontend, for example:

frontend api-http
    bind 0.0.0.0:80
    stats enable
    stats uri     /statistics

On Bleemeo agent side, you need configure check by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a HAProxy running outside any container
    - id: "haproxy"
      address: "127.0.0.1"
      port: 80
      stats_url: "http://my-server/statistics"
      # For authenticated access, use
      # stats_url: "http://username:password@my-server/statistics"
    # For a HAProxy running in a Docker container
    - id: "haproxy"
      instance: "CONTAINER_NAME"
      address: "127.0.0.1"
      port: 80
      stats_url: "http://my-server/statistics"
      # For authenticated access, use
      # stats_url: "http://username:password@my-server/statistics"

Agent gathers the following metrics:

InfluxDB

Agent uses a HTTP check if the service listen on port 8086.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a InfluxDB running outside any container
    - id: "influxdb"
      address: "127.0.0.1"
      port: 8086
    # For a InfluxDB running in a Docker container
    - id: "influxdb"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 8086

Only the metric from service check is produced:

JIRA

To enable metrics gathering, Bleemeo agent need to be installed with JMX enabled, see Java Metrics for setup details.

In addition Confluence need to expose JMX over a TCP port. To enable JMX you need to:

• Add option -Dcom.sun.management.jmxremote.port=3333 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false to the JVM. Default JIRA startup-script accept them from CATALINA_OPTS or JAVA_OPTS environment variable.

Warning: this will allow unauthenticated access. Make sure no untrusted access to this port is possible or you should setup authenticated JMX access.

Bleemeo agent should auto-detect the JMX port, if some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a JIRA running outside any container
    - id: "jira"
      address: "127.0.0.1"
      port: 8080
      jmx_port: 3333
      jmx_username: "monitorRole" # by default, no authentication is done
      jmx_password: "secret"
    # For a JIRA running in a Docker container
    - id: "jira"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 8080
      jmx_port: 3333
      jmx_username: "monitorRole" # by default, no authentication is done
      jmx_password: "secret"

Agent gathers the following metrics:

libvirtd

Memcached

Agent uses a Memcached check if the service listen on port 11211.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Memcached running outside any container
    - id: "memcached"
      address: "127.0.0.1"
      port: 11211
    # For a Memcached running in a Docker container
    - id: "memcached"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 11211

Agent gathers the following metrics:

MongoDB

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a MongoDB running outside any container
    - id: "mongodb"
      address: "127.0.0.1"
      port: 27017
    # For a MongoDB running in a Docker container
    - id: "mongodb"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 27017

Agent gathers the following metrics:

Mosquitto

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a Mosquitto running outside any container
    - id: "mosquitto"
      address: "127.0.0.1"
      port: 1883  # MQTT listenner, Agent don't support MQTT-SSL here
    # For a Mosquitto running in a Docker container
    - id: "mosquitto"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 1883  # MQTT listenner, Agent don't support MQTT-SSL here

Only the metric from service check is produced:

MySQL

To enable metrics gathering, credentials are required. Bleemeo agent will find the credentials if:

• MySQL and the agent are running on Ubuntu or Debian
• MySQL is running in a Docker container and root password is set through the environment variable MYSQL_ROOT_PASSWORD

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
[...]
    # For a MySQL running outside any container
    - id: "mysql"
      username: "USERNAME"
      password: "PASSWORD"
      address: "127.0.0.1"
      port: 3306
    # For a MySQL running in a Docker container
    - id: "mysql"
      instance: "CONTAINER_NAME"
      username: "USERNAME"
      password: "PASSWORD"
      address: "172.17.0.2"
      port: 3306

Agent gathers the following metrics:

nginx

Agent uses a HTTP check if the service listen on port 80.

To enable metrics gathering, ensure Bleemeo agent could access nginx status on the URL http://server-address/nginx_status. This usually means to add the following to your site definition (e.g. /etc/nginx/sites-enabled/default):

location /nginx_status {
    stub_status on;
}

If your nginx is not build with stub_status or if you need more information about nginx stub_status, see http://nginx.org/en/docs/http/ngx_http_stub_status_module.html

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a nginx running outside any container
    - id: "nginx"
      address: "127.0.0.1"
      port: 80   # HTTP listenner, Agent don't support HTTPS here
    # For a nginx running in a Docker container
    - id: "nginx"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 80   # HTTP listenner, Agent don't support HTTPS here

Agent gathers the following metrics:

NTP

Agent uses a NTP check if the service listen on port 123.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
    [...]
    # For a NTP running outside any container
    - id: "ntp"
      address: "127.0.0.1"
      port: 123
    # For a NTP running in a Docker container
    - id: "ntp"
      instance: "CONTAINER_NAME"
      address: "172.17.0.2"
      port: 123

Only the metric from service check is produced:

OpenLDAP

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
  # For a OpenLDAP running outside any container
  - id: "openldap"
    address: "127.0.0.1"
    port: 389
  # For a OpenLdap running in a Docker container
  - id: "openldap"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 389

Only the metric from service check is produced:

OpenVPN

Only the metric from service check is produced:

PHP-FPM

To enable metrics gathering, Bleemeo agent needs access to PHP-FPM status page and to known where it may find this page.

PHP-FPM needs to expose its status. For example the pool configuration should include:

pm.status_path = /status

By defaults, Bleemeo agent try fcgi://<fpm-address>:<fpm-port>/status. E.g using FCGI over the TCP port on which PHP-FPM listen and use "/status" path.

If it does not match your configuration, you will need to override "stats_url" parameters by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
  # For a PHP-FPM running outside any container
  - id: "php-fpm"
    address: "127.0.0.1"
    port: 9000
    stats_url: "fcgi://127.0.0.1:9000/status"
    # For UNIX socket access, use
    # stats_url: "/var/run/php5-fpm.sock"
    # See below for additional note on UNIX socket permission
  # For a PHP-FPM running in a Docker container
  - id: "php-fpm"
    instance: "CONTAINER_NAME"
    port: 9000
    stats_url: "fcgi://my-server:9000/status"

If using UNIX socket, you should make sure glouton user has permission to access the socket. This usually means that glouton user should be member of the group running php-fpm, which is www-data on Debian/Ubuntu system. Therefor sudo adduser glouton www-data and a restart of Bleemeo agent will grant access.

Agent gathers the following metrics:

Postfix

Agent uses a SMTP check if the service listen on port 25.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
   # For a Postfix running outside any container
  - id: "postfix"
    address: "127.0.0.1"
    port: 25
   # For a Postfix running in a Docker container
  - id: "postfix"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 25

Agent gathers the following metrics:

PostgreSQL

To enable metrics gathering, credentials are required. Bleemeo agent will find the credentials if:

• PostgreSQL is running in a Docker container and username/password are set through the environment variable POSTGRES_USER (with a default to postgres) and POSTGRES_PASSWORD.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
  # For a PostgreSQL running outside any container
  - id: "postgresql"
    username: "USERNAME"
    password: "PASSWORD"
    address: "127.0.0.1"
    port: 5432
  # For a PostgreSQL running in a Docker container
  - id: "postgresql"
    instance: "CONTAINER_NAME"
    username: "USERNAME"
    password: "PASSWORD"
    address: "172.17.0.2"
    port: 5432

Agent gathers the following metrics:

RabbitMQ

Agent uses a AMQP check if the service listen on port 5672.

To enable metrics gathering, credentials are required. By default "guest/guest" is used.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
  # For a RabbitMQ running outside any container
  - id: "rabbitmq"
    username: "USERNAME"
    password: "PASSWORD"
    address: "127.0.0.1"
    port: 5672          # Port of AMQP service
    mgmt_port: 15672    # Port of RabbitMQ management interface
  # For a RabbitMQ running in a Docker container
  - id: "rabbitmq"
    instance: "CONTAINER_NAME"
    username: "USERNAME"
    password: "PASSWORD"
    address: "172.17.0.2"
    port: 5672          # Port of AMQP service
    mgmt_port: 15672    # Port of RabbitMQ management interface

Agent gathers the following metrics:

Redis

Agent uses a Redis check if the service listen on port 6379.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
  # For a Redis running outside any container
  - id: "redis"
    address: "127.0.0.1"
    port: 6379
  # For a Redis running in a Docker container
  - id: "redis"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 6379

Agent gathers the following metrics:

Salt

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
  # For a Salt master running outside any container
  - id: "salt"
    address: "127.0.0.1"
    port: 4505
  # For a Salt master running in a Docker container
  - id: "salt"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 4505

Only the metric from service check is produced:

Squid3

Agent uses a HTTP check if the service listen on port 3128.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
  # For a Squid running outside any container
  - id: "squid"
    address: "127.0.0.1"
    port: 3128
  # For a Squid running in a Docker container
  - id: "squid"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 3128

Only the metric from service check is produced:

uWSGI

Only the metric from service check is produced:

Varnish

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
  # For a Varnish running outside any container
  - id: "varnish"
    address: "127.0.0.1"
    port: 6082
  # For a Varnish running in a Docker container
  - id: "varnish"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 6082

Only the metric from service check is produced:

ZooKeeper

Agent uses a ZooKeeper check if the service listen on port 2181.

To enable metrics gathering, Bleemeo agent need to be installed with JMX enabled, see Java Metrics for setup details.

Also, still for JMX metrics, ZooKeeper must expose JMX over a TCP port. To enable JMX you will need to:

• If using Docker for ZooKeeper, add environment variable JMXPORT=1234
• In other case, add option -Dcom.sun.management.jmxremote.port=1234 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false to the JVM. Usually in JAVA_OPTS in /etc/default/zookeeper.

Warning: in both case, this will allow unauthenticated access. Make sure no untrusted access to this port is possible or you should setup authenticated JMX access.

If some auto-detected parameters are wrong, you can manually override them by adding the following to /etc/glouton/conf.d/99-local.conf:

service:
  [...]
  # For a ZooKeeper running outside any container
  - id: "zookeeper"
    address: "127.0.0.1"
    port: 2181
    jmx_port: "JMX_PORT"
    jmx_username: "monitorRole"  # by default, no authentication is done
    jmx_password: "secret"
  # For a ZooKeeper running in a Docker container
  - id: "zookeeper"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 2181
    jmx_port: "JMX_PORT"
    jmx_username: "monitorRole"  # by default, no authentication is done
    jmx_password: "secret"

Agent gathers the following metrics:

Agent gathers the following metrics through JMX: