Consul

Auto-Detection

Built-in Metrics

Consul is a service networking platform by HashiCorp that provides service discovery, configuration and mesh capabilities.

Auto-Detection

Bleemeo monitors Consul through automatic service detection by process, a service check on port 8500 and built-in metrics read from the agent’s HTTP API from the same port. If the auto-detected parameters are incorrect, you can override them manually.

Linux
Windows

sudo tee /etc/glouton/conf.d/99-consul.conf > /dev/null << 'EOF'
service:
  # For a Consul running outside a container
  - type: "consul"
    address: "127.0.0.1"
    port: 8500

  # For a Consul running in a Docker container
  - type: "consul"
    instance: "CONTAINER_NAME"
    port: 8500
EOF

Create or edit C:\ProgramData\glouton\conf.d\99-consul.conf:

service:
  - type: "consul"
    address: "127.0.0.1"
    port: 8500

Glouton automatically detects configuration changes.

Requirements

Consul needs no telemetry configuration. Glouton reads the agent’s JSON metrics endpoint, /v1/agent/metrics, which every agent serves out of the box.

When ACLs are enabled, give Glouton a token with the agent:read permission. It goes in the password setting — Consul authenticates with the token alone, so username is not used:

service:
  - type: "consul"
    address: "127.0.0.1"
    port: 8500
    password: "REPLACE_WITH_YOUR_TOKEN"

To read an agent that is not where auto-detection looks by default, set stats_url to the agent’s base URL, such as https://consul.example.com:8501. Glouton will append the missing /v1/agent/metrics endpoint by itself.

A Consul agent binds its client interfaces to the loopback address by default. One running in a container therefore has to be started with -client=0.0.0.0 to be reachable at all.

Built-in Metrics

Metric	Description
`service_status`	Status of Consul
`consul_server_isleader`	Whether this Consul server is the leader
`consul_autopilot_healthy`	Consul autopilot health status
`consul_autopilot_failure_tolerance`	Number of Consul servers that can be lost before quorum is lost
`consul_members_servers`	Number of Consul servers in the cluster
`consul_members_clients`	Number of Consul client agents in the cluster
`consul_state_nodes`	Number of nodes in the Consul catalog
`consul_state_services`	Number of services in the Consul catalog
`consul_state_service_instances`	Number of service instances in the Consul catalog
`consul_state_kv_entries`	Number of entries in the Consul key/value store
`consul_raft_committime_mean_seconds`	Average time to commit a Consul Raft log entry
`consul_raft_leader_lastcontact_mean_seconds`	Average time for the Consul Raft leader to contact its followers
`consul_kvs_apply_mean_seconds`	Average time for Consul to apply a key/value write
`consul_rpc_request_rate`	Number of RPC requests served by Consul, per second
`consul_client_rpc_rate`	Number of RPC requests made by the Consul agent, per second

Monitoring Troubleshooting

See Troubleshoot a Service Check or Missing Metrics for what applies to every service: finding the address and port Glouton really uses, what each check message means, and how to read the collection error — which does not appear in the agent logs at the default level.

The Service Check is not OK

The check only opens the API port so it fails when nothing listens there (Connection refused), when a firewall drops the packets (connection timed out after 10 seconds), or when the configured address cannot be parsed (Invalid TCP address). A green check proves the port is open, and nothing more.

Metrics are Missing

Reproduce what Glouton does, from the machine where the agent runs:

curl -s 'http://127.0.0.1:8500/v1/agent/metrics' | head -c 300

Ask for the endpoint without a format parameter, as Glouton does. Adding ?format=prometheus selects a different renderer, one that stays empty unless Consul is configured with prometheus_retention_time — its emptiness says nothing about the metrics Glouton collects.

What you see	Cause	Fix
`403 Permission denied`	ACLs are enabled and no token is configured	Set `password` to a token with `agent:read`
a connection error	The agent binds its client interface to loopback only	Start it with `-client=0.0.0.0`
only cluster-wide metrics missing	The agent is a client, not a server	Raft and autopilot metrics only exist on servers
a latency metric missing on an otherwise complete answer	Consul only reports a timer for operations that happened in the window it is reporting, so `consul_kvs_apply_mean_seconds` is absent from an idle agent	Nothing to fix — it reappears once the traffic does