Kafka

Auto-Detection

Built-in Metrics

Logs

Apache Kafka is a distributed event streaming platform used for building real-time data pipelines and streaming applications.

Auto-Detection

Glouton detects Kafka by its process and listening port (default: 9092).

To enable metrics gathering, Glouton needs JMX access. See Java JMX Metrics for setup details.

Kafka must expose JMX over a TCP port. If you are using Docker, add the environment variable KAFKA_JMX_PORT=1234 – see the Confluent documentation for details.

In other cases, export the following environment variables before running kafka-server-start.sh:

export JMX_PORT=1234
export KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=127.0.0.1"

Caution

This allows unauthenticated JMX access. Ensure no untrusted access to this port is possible, or configure authenticated JMX access.

Glouton auto-detects the JMX port. If auto-detected parameters are incorrect, override them manually:

Linux

sudo tee /etc/glouton/conf.d/99-kafka.conf > /dev/null << 'EOF'
service:
  # For a Kafka running outside a container
  - type: "kafka"
    address: "127.0.0.1"
    port: 9092
    jmx_port: 1099
    jmx_username: "monitorRole"   # by default, no authentication is done
    jmx_password: "secret"

  # For an additional Kafka running outside a container
  - type: "kafka"
    instance: "NAME_ASSOCIATED_WITH_YOUR_ADDITIONAL_SERVICE"
    address: "127.0.0.1"
    port: 9092
    jmx_port: 1099
    jmx_username: "monitorRole"
    jmx_password: "secret"

  # For a Kafka running in a Docker container
  - type: "kafka"
    instance: "CONTAINER_NAME"
    address: "172.17.0.2"
    port: 9092
    jmx_port: 1099
    jmx_username: "monitorRole"
    jmx_password: "secret"
EOF

Windows

Create or edit C:\ProgramData\glouton\conf.d\99-kafka.conf:

service:
  # For a Kafka running outside a container
  - type: "kafka"
    address: "127.0.0.1"
    port: 9092
    jmx_port: 1099
    jmx_username: "monitorRole"   # by default, no authentication is done
    jmx_password: "secret"

  # For an additional Kafka running outside a container
  - type: "kafka"
    instance: "NAME_ASSOCIATED_WITH_YOUR_ADDITIONAL_SERVICE"
    address: "127.0.0.1"
    port: 9092
    jmx_port: 1099
    jmx_username: "monitorRole"
    jmx_password: "secret"

Glouton automatically detects configuration changes.

Built-in Metrics

Metric	Description
service_status	Status of Kafka
kafka_jvm_gc	Number of garbage collections per second
kafka_jvm_gc_utilization	Garbage collection utilization in percent
kafka_jvm_heap_used	Heap memory used in bytes
kafka_jvm_non_heap_used	Non-heap memory used in bytes
kafka_topics_count	Total number of topics
kafka_fetch_requests_sum	Total number of fetch requests per second
kafka_fetch_time_average	Average time to process a fetch request in seconds
kafka_produce_requests_sum	Total number of produce requests per second
kafka_produce_time_average	Average time to process a produce request in seconds

Bleemeo also supports detailed monitoring of specific Kafka topics. To enable this, add detailed_items to the service configuration:

Linux

sudo tee /etc/glouton/conf.d/99-kafka-topics.conf > /dev/null << 'EOF'
service:
  - type: "kafka"
    detailed_items:
      - "topic1"
      - "topic2"
EOF

Windows

Create or edit C:\ProgramData\glouton\conf.d\99-kafka-topics.conf:

service:
  - type: "kafka"
    detailed_items:
      - "topic1"
      - "topic2"

When using Docker, you can set detailed topics via labels:

docker run --label glouton.detailed_items="topic1,topic2" [...]

The following per-topic metrics are gathered:

Metric	Description
kafka_fetch_requests	Number of fetch requests per second
kafka_produce_requests	Number of produce requests per second

To enable additional JMX metrics, you can add custom JMX metrics. A full list of available metrics is available in the Kafka documentation.

Logs

Glouton collects Kafka logs for analysis in Bleemeo.

Monitoring Troubleshooting

See Troubleshoot a Service Check or Missing Metrics for what applies to every service: finding the address and port Glouton really uses, what each check message means, and how to read the collection error — which does not appear in the agent logs at the default level.

The Service Check is not OK

The check only opens the TCP port — it sends no command — so it fails when the port is wrong (Connection refused), when a firewall drops the packets (connection timed out after 10 seconds), or when the configured address cannot be parsed (Invalid TCP address). A green check proves the port is open, and nothing more.

Metrics are Missing

A passing check says nothing about the metrics: they are collected separately.

These metrics travel through JMX, which is a separate pipeline: Glouton writes /var/lib/jmxtrans/glouton-generated.json, jmxtrans connects to the JVM’s JMX port, and pushes the values back to Glouton. Every link has to be in place.

Check that the port is open and answers, from the machine where the agent runs. This only shows a JVM running on the host — for a containerized JVM, compare the address in discovery.txt with the ports the container itself listens on:

sudo ss -lntp | grep java

What you see	Cause	Fix
the bleemeo-agent-jmx package is missing	jmxtrans is not installed	sudo apt-get install bleemeo-agent-jmx (or yum) — see Java Monitoring with JMX
discovery.txt shows no JMX port for the service	No JMX port	Glouton reads it from the JVM command line (-Dcom.sun.management.jmxremote.port); if the JVM does not have it, set jmx_port in the service configuration
connection refused by the JVM	JMX authentication is enabled	Set jmx_username and jmx_password
jmxtrans connects, then times out	JMX answered with an RMI address jmxtrans cannot reach	The JVM must advertise the address discovery.txt shows for the service, with -Djava.rmi.server.hostname
nothing at all, for a JVM in a container	jmxtrans connects to the container address, not to a port published on the host	Publishing the port does not help. The JMX and RMI ports have to answer on the container address, and -Djava.rmi.server.hostname has to be that address

Kafka does not enable JMX by default: KAFKA_JMX_OPTS or JMX_PORT has to be set in the broker’s environment before Glouton has anything to detect.

Per-topic metrics are gathered only for the topics listed in detailed_items; without it, only the broker-wide values are collected.