ZooKeeper

Auto-Detection

Health Check

Built-in Metrics

Bleemeo monitors Apache ZooKeeper through automatic service detection, service checks via the ruok command, and built-in metrics collection including JMX.

Auto-Detection

Glouton automatically detects ZooKeeper instances listening on port 2181.

The service check uses the ruok command. You may need to add it to 4lw.commands.whitelist in your ZooKeeper configuration. See the ZooKeeper documentation for details.

To enable JMX metrics, ZooKeeper must expose JMX over a TCP port and Glouton must be installed with JMX enabled. See Java JMX Metrics for setup details.

To expose JMX:

• If using Docker, add the environment variable JMXPORT=1234.
• Otherwise, add -Dcom.sun.management.jmxremote.port=1234 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false to the JVM options (usually JAVA_OPTS in /etc/default/zookeeper).

Caution

Both methods allow unauthenticated access. Ensure no untrusted access to this port is possible, or set up authenticated JMX access.

If the auto-detected parameters are incorrect, you can override them manually.

Linux

sudo tee /etc/glouton/conf.d/99-zookeeper.conf > /dev/null << 'EOF'
service:
  # For a ZooKeeper running outside a container
  - type: "zookeeper"
    address: "127.0.0.1"
    port: 2181
    jmx_port: "JMX_PORT"
    jmx_username: "monitorRole"  # by default, no authentication is done
    jmx_password: "secret"

  # For an additional ZooKeeper running outside a container
  - type: "zookeeper"
    instance: "NAME_ASSOCIATED_WITH_YOUR_ADDITIONAL_SERVICE"
    address: "127.0.0.1"
    port: 2182
    jmx_port: "JMX_PORT"
    jmx_username: "monitorRole"  # by default, no authentication is done
    jmx_password: "secret"

  # For a ZooKeeper running in a Docker container
  - type: "zookeeper"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 2181
    jmx_port: "JMX_PORT"
    jmx_username: "monitorRole"  # by default, no authentication is done
    jmx_password: "secret"
EOF

Windows

Create or edit C:\ProgramData\glouton\conf.d\99-zookeeper.conf:

service:
  # For a ZooKeeper running outside a container
  - type: "zookeeper"
    address: "127.0.0.1"
    port: 2181
    jmx_port: "JMX_PORT"
    jmx_username: "monitorRole"  # by default, no authentication is done
    jmx_password: "secret"

  # For an additional ZooKeeper running outside a container
  - type: "zookeeper"
    instance: "NAME_ASSOCIATED_WITH_YOUR_ADDITIONAL_SERVICE"
    address: "127.0.0.1"
    port: 2182
    jmx_port: "JMX_PORT"
    jmx_username: "monitorRole"  # by default, no authentication is done
    jmx_password: "secret"

  # For a ZooKeeper running in a Docker container
  - type: "zookeeper"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 2181
    jmx_port: "JMX_PORT"
    jmx_username: "monitorRole"  # by default, no authentication is done
    jmx_password: "secret"

Service Check

Glouton performs a service check using the ZooKeeper ruok command on port 2181.

Metric	Description
service_status	Status of ZooKeeper

Built-in Metrics

Metric	Description
zookeeper_connections	Number of client connections to the ZooKeeper server
zookeeper_packets_received	Number of packets received per second
zookeeper_packets_sent	Number of packets sent per second
zookeeper_ephemerals_count	Number of ephemeral nodes
zookeeper_watch_count	Number of ZooKeeper watches
zookeeper_znode_count	Number of znodes

The following metrics are gathered through JMX:

Metric	Description
zookeeper_jvm_gc	Number of garbage collections per second
zookeeper_jvm_gc_utilization	Garbage collection utilization in percent
zookeeper_jvm_heap_used	Heap memory used in bytes
zookeeper_jvm_non_heap_used	Non-heap memory used in bytes

Monitoring Troubleshooting

See Troubleshoot a Service Check or Missing Metrics for what applies to every service: finding the address and port Glouton really uses, what each check message means, and how to read the collection error — which does not appear in the agent logs at the default level.

The Service Check is not OK

Glouton sends the ruok four-letter command and expects imok. Since ZooKeeper 3.5, four-letter commands are refused unless they are whitelisted, and this is the most common failure:

Status text	Cause	Fix
TCP port 2181, unexpected response "ruok is not executed because it is not in the whitelist."	4lw.commands.whitelist does not include ruok	Set 4lw.commands.whitelist=srvr,ruok,mntr in zoo.cfg and restart
TCP port 2181, Connection refused	ZooKeeper listens on another port or another address	Set address and port
TCP port N, connection timed out after 10 seconds	A firewall drops the packets	Open the port for the agent

Metrics are Missing

The zookeeper_* metrics come from the mntr four-letter command, which has to be whitelisted separately from ruok — a check that passes does not mean mntr is allowed:

Reproduce what Glouton does, from the machine where the agent runs:

echo mntr | nc 127.0.0.1 2181 | head -5

What you see	Cause	Fix
mntr is not executed because it is not in the whitelist.	mntr not whitelisted	Add mntr to 4lw.commands.whitelist
no error	The zookeeper_jvm_* metrics are the ones missing	Those come from JMX, not from mntr: expose a JMX port and set jmx_port as shown above