Skip to content
BleemeoBleemeo

Vault

Auto-Detection
Built-in Metrics
Prometheus Metrics

HashiCorp Vault is a secrets management tool that exposes Prometheus-formatted metrics through its telemetry subsystem.

Glouton can monitor Vault in two ways:

  • Auto-detection — recommended, minimal configuration. Glouton detects Vault and collects a curated set of built-in metrics.
  • Prometheus metrics — scrape Vault’s telemetry endpoint yourself for a fully customizable set of metrics.

Both methods read the same telemetry subsystem, so the telemetry configuration below is required in either case.

Auto-Detection

Section titled “Auto-Detection”

Glouton detects Vault by its process and listening port (default: 8200).

Enable the Prometheus telemetry as shown in Service Configuration below. Because the telemetry endpoint requires authentication, provide a token that can read sys/metrics through the service configuration:

Terminal window
sudo tee /etc/glouton/conf.d/99-vault.conf > /dev/null << 'EOF'
service:
  - type: "vault"
    address: "127.0.0.1"
    port: 8200
    password: "<YOUR_VAULT_TOKEN>"
EOF

To scrape a non-standard endpoint, set stats_url to the full metrics URL instead of address and port. Glouton automatically detects configuration changes.

MetricDescription
service_statusStatus of Vault
vault_core_unsealedWhether the Vault node is unsealed (1) or sealed (0)
vault_core_activeWhether the Vault node is the active node in the cluster
vault_core_in_flight_requestsNumber of requests currently being processed
vault_core_handle_requestsRate of requests handled by the core
vault_core_handle_login_requestsRate of login requests handled by the core
vault_core_check_tokensRate of token checks performed by the core
vault_core_leadership_lossesNumber of times this node lost cluster leadership
vault_expire_num_leasesNumber of currently active leases

Prometheus Metrics

Section titled “Prometheus Metrics”

The auto-detection above collects a curated subset. If you need the full telemetry (runtime, audit, token and lease metrics) or want to choose exactly which metrics to keep, scrape Vault’s endpoint directly instead.

Service Configuration

Section titled “Service Configuration”

Enable Prometheus telemetry in your Vault configuration file (HCL format):

telemetry {
  prometheus_retention_time = "30s"
  disable_hostname = true
}

Restart Vault for the changes to take effect. The metrics endpoint will be available at http://localhost:8200/v1/sys/metrics?format=prometheus.

This endpoint requires a valid Vault token with operator permissions. You can verify it works with:

Terminal window
curl -H "X-Vault-Token: <YOUR_TOKEN>" \
  "http://localhost:8200/v1/sys/metrics?format=prometheus"

Glouton Configuration

Section titled “Glouton Configuration”

Because the endpoint requires authentication, you must include the Vault token in the request headers. Replace <YOUR_VAULT_TOKEN> with a valid token that has permissions to read the sys/metrics endpoint.

Terminal window
sudo tee /etc/glouton/conf.d/99-vault.conf > /dev/null << 'EOF'
metric:
  prometheus:
    targets:
      - url: "http://localhost:8200/v1/sys/metrics?format=prometheus"
        name: "vault"
        headers:
          X-Vault-Token: "<YOUR_VAULT_TOKEN>"
        allow_metrics:
          - "vault_core_unsealed"
          - "vault_runtime_alloc_bytes"
          - "vault_audit_log_request_count"
          - "vault_token_count"
          - "vault_secret_lease_creation"
          - "vault_runtime_num_goroutines"
          - "vault_runtime_sys_bytes"
          - "vault_expire_num_leases"
          - "vault_core_active"
EOF
MetricDescription
vault_core_unsealedWhether the Vault node is unsealed (1) or sealed (0)
vault_core_activeWhether the Vault node is the active node in the cluster
vault_runtime_alloc_bytesNumber of bytes allocated by the Vault process
vault_runtime_num_goroutinesNumber of active goroutines in the Vault process
vault_runtime_sys_bytesTotal bytes of memory obtained from the OS
vault_audit_log_request_countNumber of audit log requests processed
vault_token_countNumber of active tokens in Vault
vault_secret_lease_creationNumber of secret lease creations
vault_expire_num_leasesNumber of currently active leases

Monitoring Troubleshooting

Section titled “Monitoring Troubleshooting”

See Troubleshoot a Service Check or Missing Metrics for what applies to every service: finding the address and port Glouton really uses, what each check message means, and how to read the collection error — which does not appear in the agent logs at the default level.

The Service Check is not OK

Section titled “The Service Check is not OK”

The check only opens the TCP port — it sends no command — so it fails when the port is wrong (Connection refused), when a firewall drops the packets (connection timed out after 10 seconds), or when the configured address cannot be parsed (Invalid TCP address). A green check proves the port is open, and nothing more.

Metrics are Missing

Section titled “Metrics are Missing”

A passing check says nothing about the metrics: they are collected separately.

Glouton reads the Prometheus telemetry endpoint /v1/sys/metrics, using the password from the service configuration as a token.

Reproduce what Glouton does, from the machine where the agent runs:

Terminal window
curl -i -H "X-Vault-Token: TOKEN" "http://127.0.0.1:8200/v1/sys/metrics?format=prometheus"
What you see Cause Fix
404 Telemetry is not enabled Add the telemetry block with prometheus_retention_time and restart
403 permission denied No token, or an expired one Set password to a valid token
403 permission denied The token’s policy does not allow sys/metrics Grant read on sys/metrics to that policy
a TLS error The API is served over HTTPS Glouton builds an http:// URL from the detected address; set stats_url to the https:// URL
503 The server is sealed Unseal it