Skip to content
Bleemeo Bleemeo

Prometheus

Prometheus is an open-source monitoring and alerting toolkit that has become the de facto standard for cloud-native monitoring. Bleemeo is fully compatible with the Prometheus ecosystem, allowing you to leverage your existing knowledge and tools.

What is Prometheus?

Section titled “What is Prometheus?”

Prometheus is a systems monitoring solution that collects metrics from configured targets at given intervals, evaluates rule expressions, displays results, and can trigger alerts when specified conditions are observed.

Key characteristics of Prometheus:

  • Multi-dimensional data model: Metrics are identified by a name and key-value pairs called labels
  • Flexible query language: PromQL allows powerful queries and aggregations
  • Pull-based collection: Prometheus scrapes metrics from HTTP endpoints
  • Service discovery: Automatic discovery of targets to monitor
  • Time series database: Efficient storage of metric data over time

Core Concepts

Section titled “Core Concepts”

Metrics

Section titled “Metrics”

A metric is a numeric measurement that changes over time. In Prometheus, every metric has:

  • A name that identifies what is being measured (e.g., http_requests_total)
  • Optional labels that add dimensions to the metric (e.g., method="GET", status="200")
  • A value at each point in time

Metric Types

Section titled “Metric Types”

Prometheus defines four core metric types:

TypeDescriptionExample
CounterA value that only increases (or resets to zero)http_requests_total
GaugeA value that can go up or downcpu_usage_percent
HistogramSamples observations and counts them in bucketsrequest_duration_seconds
SummarySimilar to histogram, with calculated quantilesrequest_latency_seconds

Labels

Section titled “Labels”

Labels are key-value pairs that identify a specific instance of a metric. They enable Prometheus’s dimensional data model:

http_requests_total{method="GET", endpoint="/api/users", status="200"}
http_requests_total{method="POST", endpoint="/api/users", status="201"}
http_requests_total{method="GET", endpoint="/api/users", status="404"}

The same metric name with different label combinations creates distinct time series.

Time Series

Section titled “Time Series”

A time series is a stream of timestamped values for a single metric with a unique set of labels. Each time series is identified by:

<metric_name>{<label_name>=<label_value>, ...}

For example:

node_cpu_seconds_total{cpu="0", mode="idle"}

Scraping

Section titled “Scraping”

Prometheus collects metrics by scraping HTTP endpoints that expose metrics in the Prometheus text format. Targets are configured either statically or through service discovery.

A typical metrics endpoint returns data like:

# HELP http_requests_total Total number of HTTP requests
# TYPE http_requests_total counter
http_requests_total{method="GET",status="200"} 1234
http_requests_total{method="POST",status="201"} 567

Bleemeo and Prometheus

Section titled “Bleemeo and Prometheus”

Bleemeo integrates with Prometheus in several ways:

PromQL Support

Section titled “PromQL Support”

Bleemeo supports PromQL for querying metrics. You can use the same query language you know from Prometheus to:

  • Build custom dashboards
  • Create alerting rules
  • Analyze metric data

Remote Write

Section titled “Remote Write”

The Bleemeo agent can receive metrics via Prometheus remote write protocol, allowing you to:

  • Forward metrics from existing Prometheus servers
  • Use Prometheus client libraries in your applications
  • Integrate with tools that support remote write

Metric Exposition

Section titled “Metric Exposition”

The Bleemeo agent exposes a Prometheus-compatible /metrics endpoint, enabling:

  • Scraping by Prometheus servers
  • Integration with other Prometheus-compatible tools
  • Custom metric collection

Service Discovery

Section titled “Service Discovery”

Bleemeo automatically discovers and monitors services, similar to Prometheus service discovery. Discovered services are automatically scraped for metrics when they expose a Prometheus endpoint.

Best Practices

Section titled “Best Practices”

Naming Conventions

Section titled “Naming Conventions”

Follow Prometheus naming conventions for consistency:

  • Use lowercase with underscores (snake_case)
  • Include a unit suffix when applicable (_seconds, _bytes, _total)
  • Use _total suffix for counters
  • Be descriptive but concise

Good examples:

  • http_request_duration_seconds
  • process_memory_bytes
  • api_requests_total

Avoid:

  • HTTPRequestTime (wrong case)
  • requests (too vague, missing unit/type)

Label Cardinality

Section titled “Label Cardinality”

Be mindful of label cardinality (the number of unique label combinations):

  • Avoid labels with unbounded values (like user IDs or request IDs)
  • Keep cardinality reasonable (hundreds to thousands, not millions)
  • High cardinality increases storage and query costs

Instrumentation

Section titled “Instrumentation”

When instrumenting your applications:

  • Use the four metric types appropriately
  • Add meaningful labels for filtering and aggregation
  • Document your metrics with HELP and TYPE annotations
  • Expose metrics on a dedicated /metrics endpoint

Next Steps

Section titled “Next Steps”