Agent Configuration Files

Agent reads its configuration files in YAML format, in the following order:

  • /etc/bleemeo/agent.conf
  • /etc/bleemeo/agent.conf.d/*.conf
  • etc/agent.conf
  • etc/agent.conf.d/*.conf

For directory (like /etc/bleemeo/agent.conf.d) files are read in lexicographic order. We recommend adding a two digits number in front of all filenames to control order. For example “05-system” and “90-custom”.

Configuration files are merged and when a conflict occurs, entry from last file wins.

For example, if you have the following files:

  • /etc/bleemeo/agent.conf
  • /etc/bleemeo/agent.conf.d/05-system.conf
  • /etc/bleemeo/agent.conf.d/90-custom.conf

Where each has an entry “logging.level”, e.g.:

# file /etc/bleemeo/agent.conf
logging:
    level: INFO

# file /etc/bleemeo/agent.conf.d/05-system.conf
logging:
    level: WARNING

# file /etc/bleemeo/agent.conf.d/90-custom.conf
logging:
    level: DEBUG

The result, will be level = DEBUG from 90-custom.conf.

Merge will create the union of dictionary and concatenate list, therefore the following files:

# file /etc/bleemeo/agent.conf
logging:
    level: INFO

network_interface_blacklist:
    - lo
    - docker

# file /etc/bleemeo/agent.conf.d/05-system.conf
logging:
    output: syslog

# file /etc/bleemeo/agent.conf.d/90-custom.conf
network_interface_blacklist:
    - lo
    - vnet

Would be equivalent to one file with:

# keys inside logging are merged
logging:
    level: INFO
    output: syslog

# list are concatenated. No deduplication is done.
network_interface_blacklist:
    - lo
    - docker
    - lo
    - vnet

Environment variables

Bleemeo agent allow to override configuration using environment variables.

For example:

  • logging.level could be override by environment variable BLEEMEO_AGENT_LOGGING_LEVEL
  • bleemeo.account_id could be override by environment variable BLEEMEO_AGENT_BLEEMEO_ACCOUNT_ID
  • agent.upgrade_file could be override by environment variable BLEEMEO_AGENT_AGENT_UPGRADE_FILE

The environment variable name is the configuration name with:

  • dots (.) replaced by underscore (_),
  • all letters converted to uppercase
  • and the prefix BLEEMEO_AGENT_ added.

List and mapping are not supported from environment variable, only simple types (string, integer and boolean) are supported.

For boolean, Bleemeo agent recognize “true”, “yes” and “1” as true and “false”, “no”, “0” as false. This convertion is case insensitive.

Reference

In this reference, option will use with dot (.) in their name. Each dot will be a new indentation level in configuration files. E.g. logging.level is written in configuration files as:

logging:
    level: value

Indentation use whitespace only, no tabulations.

Options available:

agent:
    facts_file: facts.yaml
    installation_format: manual
    netstat_file: netstat.out
    public_ip_indicator: https://myip.bleemeo.com
    state_file: state.json
    upgrade_file: upgrade

tags:
    []

stack: null

logging:
    level: INFO
    output: console

container:
    type: null
    pid_namespace_host: false

bleemeo:
    enabled: True
    account_id: YOUR-ACCOUNT-ID
    registration_key: YOUR-REGISTRATION-KEY
    initial_agent_name: ""
    api_base: https://api.bleemeo.com/
    mqtt:
        host: mqtt.bleemeo.com
        port: 8883
        ssl: True
        cafile: /etc/ssl/certs/ca-certificates.crt
        ssl_insecure: False
    sentry:
        dsn: null

graphite:
    metrics_source: telegraf
    listener:
        address: 127.0.0.1
        port: 2003

telegraf:
    statsd:
        enabled: True
        address: 127.0.0.1
        port: 8125
    config_file: /etc/telegraf/telegraf.d/bleemeo-generated.conf
    docker_name: null
    docker_metrics_enabled: null
    restart_command: sudo --non-interactive service telegraf restart

metric:
    pull:
        {}
    prometheus:
        {}
    softstatus_period_default: 300
    softstatus_period:
        {}

service:
    []

service_ignore_metrics:
    []

service_ignore_check:
    []

web:
    enabled: True
    listener:
        address: 127.0.0.1
        port: 8015

influxdb:
    enabled: False
    host: localhost
    port: 8086
    db_name: metrics

network_interface_blacklist:
    []

disk_monitor:
    []

df:
    path_ignore:
        []
    host_mount_point: null

thresholds:
    {}
agent.facts_file

Path to privileged facts.

The agent does not run as root, but some facts need root privileged, like server serial number. To access those information, during package installation the command bleemeo-agent-gather-facts is run to gather those facts and write them in this file.

Default is “facts.yaml”, e.g. a file named “facts.yaml” in current directory. Our packaging changes this default to “/var/lib/bleemeo/facts.yaml” in /etc/bleemeo/agent.conf.d/05-system.conf.

agent.installation_format

The format used to install Bleemeo Agent. The agent sends a fact “installation_format” with this value.

Default is “manual”. Our packaging changes this default to match the format used:

  • “Package (deb)”
  • “Package (RPM)”
  • “Package (Windows)”
  • “Docker image”
agent.netstat_file

Path to netstat output.

The agent does not run as root, but use information from netstat which requires root privileged. To access those information, a cron generates an netstat output using the command bleemeo-netstat and writes to this file.

Default is “netstat.out”, e.g. a file named “netstat.out” in current directory. Our packaging changes this default to “/var/lib/bleemeo/netstat.out” in /etc/bleemeo/agent.conf.d/05-system.conf.

agent.public_ip_indicator

URL of IP indicator service.

The agent sends a fact “public_ip”, this URL is used to determine public IP of your server.

Default is “https://myip.bleemeo.com”.

agent.state_file

Path to store agent state.

The state must be writable, it stores information like Agent unique identified, credentials for Bleemeo Cloud platform, or currently detected services.

Default is “state.json”, e.g. a file named “state.json” in current directory. Our packaging changes this default to “/var/lib/bleemeo/state.json” in /etc/bleemeo/agent.conf.d/05-system.conf.

agent.upgrade_file

If this file exists, it means that the agent is being updated. In this case the agent avoid generating disconnect/reconnect event.

Default is “upgrade”, e.g. a file named “upgrade” in current directory. Our packaging changes this default to “/var/lib/bleemeo/upgrade” in /etc/bleemeo/agent.conf.d/05-system.conf.

tags

List of tags for the agent. Your agent will be tagged with those tags on Bleemeo Cloud Platform. Tags can be used in dashboards and notifications.

Default is an empty list.

Bleemeo Cloud Platform also automatically add tags by default:

  • one for each services running on the server
  • one for some server information: architecture, OS information (Linux/Windows, version) and server type (physical/virtual).
stack

Default stack for services.

Default is unset.

For more details, see Stack Dashboard.

logging.level

Verbosity level of log messages. Valid values are “DEBUG”, “INFO”, “WARNING”, “ERROR”.

Default is “INFO”.

We recommend to keep “INFO” level. With higher level (warning and error) you may miss useful messages, and info level is already pretty terse. With debug level you will probably get too much messages, this level should be only used for troubleshooting.

This option could be overridden by the environment variable BLEEMEO_AGENT_LOGGING_LEVEL.

logging.output

Output of all log messages. Valid values are “console” or “syslog”.

Default is “console”. Our packaging changes this default to “syslog” in /etc/bleemeo/agent.conf.d/05-system.conf.

This option could be overridden by the environment variable BLEEMEO_AGENT_LOGGING_OUTPUT.

container.type

If set, it’s the name of the containerization solution used for Agent itself. For example “docker” if Bleemeo agent is running inside a Docker container.

Default is unset, meaning Bleemeo Agent is running outside any container.

container.pid_namespace_host

When the agent is running in a container, does it see all PID.

When using Docker, it means that container running the agent is started with --pid=host. It’s needed to gather information about running processes of your server.

Default is false. Our Docker image set this option to true and expect to be started with --pid=host

bleemeo.enabled

Should the agent communicate with Bleemeo Cloud platform.

Default is true.

bleemeo.account_id

The account UUID for Bleemeo Cloud.

The agent needs this information to communicate with Bleemeo Cloud platform. When using standard installation method, this option is wrote in /etc/bleemeo/agent.conf.d/30-install.conf.

This option could be overridden by the environment variable BLEEMEO_AGENT_BLEEMEO_ACCOUNT_ID.

bleemeo.registration_key

The registration key for Bleemeo Cloud.

Like bleemeo.account_id, it’s required for communication with Bleemeo Cloud platform. This secret key allows the agent to registers itself in your account. Like bleemeo.account_id, standard installation method writes this option in /etc/bleemeo/agent.conf.d/30-install.conf.

This option could be overridden by the environment variable BLEEMEO_AGENT_BLEEMEO_REGISTRATION_KEY.

bleemeo.initial_agent_name

The name used when the agent register itself with Bleemeo Cloud.

This variable is only used at registration time, it’s not used for any update.

If unset, the FQDN of the server will be used as name during registration.

bleemeo.api_base

Root URL of Bleemeo Cloud platform API.

Default is “https://api.bleemeo.com/”.

bleemeo.mqtt.host

Address of MQTT server.

Default is “mqtt.bleemeo.com”.

bleemeo.mqtt.port

Port of MQTT server.

Default is 8883.

bleemeo.mqtt.ssl

Should the agent use SSL for MQTT communication.

Bleemeo Cloud Platform only all SSL communication, this option is only useful for Bleemeo agent development.

Default is true.

bleemeo.mqtt.cafile

Path to CAs certificate bundle.

The agent trust CAs present in this bundle when checking trust-chain to MQTT SSL certificate.

Default is “/etc/ssl/certs/ca-certificates.crt”.

bleemeo.mqtt.ssl_insecure

Should the agent ignore MQTT SSL certificate errors.

Default to false, by default MQTT SSL certificate must be valid.

Settings this value to true create a security issue and should only be used in development.

bleemeo.sentry.dsn

Key used for our errors reporting tool.

Default is unset, but our packages configuration set a valid key.

graphite.metrics_source

Main source of metrics. Only supported value is “telegraf”.

Default is “telegraf”. Our packaging configures this value in /etc/bleemeo/agent.conf.d/30-graphite_metrics_source.conf.

graphite.listener.address

Address to binds to for graphite listener.

Metric sources (telegraf and jmxtrans) connect and send data to Bleemeo Agent over this graphite listener. Since both the agent and metric source run on the server, the agent should listen only on loopback interface (127.0.0.1).

Default is “127.0.0.1”.

graphite.listener.port

Port to binds to for graphite listener.

Default is 2003.

telegraf.statsd.enabled

Should the agent configure statsd listener of Telegraf.

Bleemeo agent supports StatsD metrics using Telegraf as StatsD server. If you already had another StatsD server you could disable this feature by changing this option to false.

Default is true.

telegraf.statsd.address

Address to binds to for StatsD listener.

Telegraf will use the following address to bind its StatsD listener.

Default is 127.0.0.1

telegraf.statsd.port

Port to binds to for StatsD listener.

Telegraf will use the following port to bind its StatsD listener.

Default is 8125

telegraf.config_file

Path of the Telegraf configuration file generated by Bleemeo agent.

Due to service discovery done by the agent, it needs to reconfigure Telegraf by writing this configuration file and restarting Telegraf.

Default is “/etc/telegraf/telegraf.d/bleemeo-generated.conf”.

telegraf.docker_name

If set, it means that Telegraf is running inside a Docker container named after this option. It’s used to restart Telegraf.

Default is unset.

telegraf.docker_metrics_enabled

Bleemeo agent require Telegraf 1.0.0 or greater to support metrics for Docker containers. By default agent detects the version of Telegraf. This option allows to force activation or force deactivation of Docker metrics, regardless of version detected by the agent.

Default is unset.

telegraf.restart_command

If Telegraf is not Dockerized (telegraf.docker_name is unset), the agent uses this command to restart Telegraf.

Default is “sudo –non-interactive service telegraf restart”.

metric.pull

Configuration for custom metrics.

For more details on this option, see How to Define a Custom Metric.

metric.prometheus

Configuration for custom metrics from a Prometheus exporter.

For more details on this option, seed Prometheus metrics.

metric.softstatus_period_default

Default duration of the soft-status grace period in seconds.

The Bleemeo agent support a grace period when a metric just crossed a threshold. The metric must stay above the threshold for the duration of the grace period before its status change. It avoid generating an alarm for short spike of the metric (for example CPU usage, that could easily reach 100% during short period).

Default is 5 minutes (300 seconds).

metric.softstatus_period

Per-metric override of the soft-status grace period.

It allow to change the grace period for metric where the default (see above) does not match. For example pending system upgrades, you probably don’t apply upgrades every 5 minutes.

The default don’t define any override, but our packaging changes this to 1 days for pending system upgrades and to instantaneous for time_elapsed_since_last_data since this metric is already a duration since an issue.

service

Configuration for custom services or services overrides.

For more details on custom services, see How to Define a Custom Check.

For more details on services overrides, see What Bleemeo Monitors?

service_ignore_metrics

List of services for which metrics are ignored (not gathered).

See How to Ignore a Service for more details.

service_ignore_check

List of services for which status check is ignored.

See How to Ignore a Service for more details.

web.enabled

Should local web interface be enabled.

This web interface show a quick overview of your server and may be used by a load-balancer to know health status of the server.

Default is true.

web.listener.address

Address to binds to for local web interface.

Default is “127.0.0.1”.

web.listener.port

Port to binds to for local web interface.

Default is 8015.

influxdb.enabled

Should the agent sends metrics to an InfluxDB server.

Default is false.

influxdb.host

Address of the InfluxDB server.

Default is “localhost”.

influxdb.host

Port of the InfluxDB server.

Default is 8086.

influxdb.db_name

Name of the InfluxDB database used to store metrics.

Default is “metrics”

network_interface_blacklist

List of interface name prefixes to ignore.

This list is empty by default.

Default configuration included in our packages ignore some interfaces: “docker*”, “lo*”, “veth*”, “virbr*”, “vnet*”

disk_monitor

List of regular expressions of disk to monitor for IO.

Default is empty list. Our packaging changes this to monitor most physical disks in /etc/bleemeo/agent.conf: sda, sdb, vda, xvda, mmcblk0…

df.path_ignore

List of paths to ignore for disk usage.

For each path in the list, Agent ignores all filesystems at or below this path. For example if “/media” is in the ignore list, no filesystem usage will be reported for a USB key mounted on “/media/usb”.

Telegraf also ignores some filesystem types: tmpfs, procfs, cgroup filesystem…

Default is an empty list.

df.host_mount_point

Path where the root filesystem is.

This is useful when the agent is running in a container. In this case the root (/) for the agent is not the root filesystem of your server. To gather disk usage for all your filesystems you need to expose the real root in the container running the agent (with -v /:/hostroot:ro in Docker). In such case, this option should point the path “/hostroot”.

When set, only filesystem starting with this path are considered and this path fragment is removed before reporting. For example if Bleemeo agent is running in a container and see the following filesystems: “/”, “/hostroot” and “/hostroot/srv”, then only “/hostroot” and “/hostroot/srv” will be reported under the names “/” and “/srv”.

thresholds

Configuration of threshold for each metrics.

Thresholds in configuration file are overridden by thresholds retrieved from Bleemeo Cloud platform. We highly recommend to configure thresholds only using Bleemeo Cloud platform.

No thresholds are hardcoded.

Default configuration included in our packages define thresholds for cpu, disk space usage, memory, network and disk IO utilization in /etc/bleemeo/agent.conf.