SquirrelDB Configuration
The configuration is a YAML file. By default, SquirrelDB reads the file
squirreldb.conf in the current directory. You can provide another
configuration file with the flag --config /path/to/file.conf.
All configuration options can be overridden by environment variables. To find the environment variable associated to a setting, you can follow these rules:
- The variable is prefixed by
SQUIRRELDB_ - All letters are converted to uppercase
- Each yaml indentation is converted to an underscore
For example, cassandra.addresses becomes SQUIRRELDB_CASSANDRA_ADDRESSES.
backend
Section titled “backend”The database used to store points, the index and the mutable labels. It serves reads, receives the writes that must succeed and owns the telemetry: this is the source of truth.
Supported values are cassandra and clickhouse. See
storage backends for the differences between the two.
Default: cassandra
The batch block configures batch writes.
For performance reasons, SquirrelDB does not write points immediately in Cassandra. SquirrelDB receives points by timestamp, usually multiple metrics for the same timestamp. But in Cassandra, it writes points by metric — one batch of points for the same metric across multiple timestamps. To perform this conversion, SquirrelDB needs to store pending points in memory and write them to Cassandra periodically.
batch.size
Section titled “batch.size”Configure the time to keep points in memory before writing to Cassandra
Smaller value means:
- less memory usage,
- fewer points in memory (less data loss in case of crash)
- less performance
- more disk usage
Valid time units are ns, us (or µs), ms, s, m, h.
Default: 15m
cassandra
Section titled “cassandra”The cassandra block configures the Cassandra server used by SquirrelDB to
store points. It is used when backend is cassandra.
cassandra.addresses
Section titled “cassandra.addresses”List of Cassandra servers to connect to.
Default: ["127.0.0.1:9042"]
cassandra.username
Section titled “cassandra.username”Username used to connect to Cassandra. When empty, the connection is done without using credentials.
Default: ""
cassandra.password
Section titled “cassandra.password”Password used to connect to Cassandra.
Default: ""
cassandra.cert_path
Section titled “cassandra.cert_path”Path to the TLS certificate used to connect to Cassandra. Optional depending on your server configuration.
Default: ""
cassandra.key_path
Section titled “cassandra.key_path”Path to the TLS key used to connect to Cassandra. Optional depending on your server configuration.
Default: ""
cassandra.ca_path
Section titled “cassandra.ca_path”Path to the TLS certificate authority used to connect to Cassandra. Optional depending on your server configuration.
Default: ""
cassandra.enable_host_verification
Section titled “cassandra.enable_host_verification”Whether to verify the hostname and server certificate.
Default: true
cassandra.keyspace
Section titled “cassandra.keyspace”Cassandra keyspace used to create tables.
Default: squirreldb
cassandra.replication_factor
Section titled “cassandra.replication_factor”Replication factor. This must be 1 for a single server Cassandra. It should be 3 when your Cassandra cluster contains at least 3 servers.
Default: 1
cassandra.aggregate.intended_duration
Section titled “cassandra.aggregate.intended_duration”To improve queries over large period of time, SquirrelDB does some pre-aggregation of the points.
To avoid overloading the system, SquirrelDB lets you throttle the aggregation. This setting configures the target time for aggregating one day of data. The aggregation will take at least this amount of time.
With a long duration, SquirrelDB will take longer to catch-up any aggregation lag (e.g. if SquirrelDB is stopped for multiple days). With a small duration, the aggregation will cause more visible stress on SquirrelDB and Cassandra. If the duration is too short, SquirrelDB will perform the aggregation without throttling.
We recommend keeping the default short duration if the stress caused by the aggregation is not an issue. If you want to increase the duration, 4h is a good value to reduce the stress on Cassandra and still be fast enough to allow some catch-up if needed.
The value is multiplied by the number of SquirrelDB instances in the cluster: the larger the cluster, the slower the pre-aggregation runs on each instance.
Valid time units are ns, us (or µs), ms, s, m, h.
Default: 1m
cassandra.pre_create_shard_duration
Section titled “cassandra.pre_create_shard_duration”The index stores metrics in shards of one week. To avoid a write spike when a new shard starts, SquirrelDB pre-creates the entries of the next shard during this period before it begins.
Valid time units are ns, us (or µs), ms, s, m, h.
Default: 30m
cassandra.pre_create_shard_fraction
Section titled “cassandra.pre_create_shard_fraction”Fraction of the known metrics each SquirrelDB pre-creates in the next shard on every run. A larger value spreads the pre-creation over more, smaller batches.
Default: 27
clickhouse
Section titled “clickhouse”The clickhouse block configures the ClickHouse servers used by SquirrelDB to
store points. It is used when backend or
extra_write_backend is clickhouse.
See storage backends for the supported ClickHouse deployments, and migrating to ClickHouse to move an existing Cassandra deployment.
clickhouse.addresses
Section titled “clickhouse.addresses”List of ClickHouse servers to connect to, using the native protocol (port 9000 by default).
Unlike Cassandra, there is no node discovery: list every ClickHouse server node here. SquirrelDB opens one connection per address, health-checks each of them (reachable, holds the tables and replication not lagging) and routes queries only to healthy nodes. A node that is down, rebuilt empty or still resyncing is excluded automatically and re-included once ready, so no external load balancer is needed.
Default: ["127.0.0.1:9000"]
clickhouse.database
Section titled “clickhouse.database”Name of the ClickHouse database in which SquirrelDB creates its tables.
Default: default
clickhouse.username
Section titled “clickhouse.username”Username used to connect to ClickHouse. When empty, the connection is done without using credentials.
Default: ""
clickhouse.password
Section titled “clickhouse.password”Password used to connect to ClickHouse.
Default: ""
clickhouse.cluster_name
Section titled “clickhouse.cluster_name”When set, SquirrelDB creates its tables with ON CLUSTER <name> and
ReplicatedMergeTree engines (ClickHouse cluster mode). When empty, it targets a
single ClickHouse node with plain MergeTree tables.
The value must match the cluster name declared in the <remote_servers> section
of your ClickHouse server configuration.
Default: ""
clickhouse.max_concurrent_queries
Section titled “clickhouse.max_concurrent_queries”Maximum number of in-flight queries (reads and writes combined) SquirrelDB sends to ClickHouse. Excess queries wait for a slot instead of surfacing a “too many simultaneous queries” error from the server. Reads are capped one below this value, so a write always has a slot available.
When set to 0, SquirrelDB auto-detects the server’s max_concurrent_queries and
keeps a margin. Auto-detection is good enough for a dedicated, one-SquirrelDB
setup; set the value explicitly when ClickHouse is shared, when a per-user limit
applies (max_concurrent_queries_for_user, which auto-detection cannot see), or
when several SquirrelDB fan out to the same servers.
Default: 0 (auto-detect)
clickhouse.wal_flush_delay
Section titled “clickhouse.wal_flush_delay”How often the elected drainer writes the shared write-ahead log to ClickHouse.
Writes are durable as soon as they are acknowledged (they are stored in Redis, see redis.addresses); this setting bounds how long they take to become queryable in ClickHouse.
Valid time units are ns, us (or µs), ms, s, m, h.
Default: 0, which uses the built-in default of 5s
clickhouse.wal_soft_threshold
Section titled “clickhouse.wal_soft_threshold”Number of points buffered in the write-ahead log above which the drainer flushes
early instead of waiting for wal_flush_delay.
The write-ahead log stores one entry per point, and each entry carries the labels of its series (roughly 150 to 200 bytes), so this value also bounds the Redis memory used by the log. The backpressure thresholds are derived from it: SquirrelDB slows the writers down at 3× this value and blocks them at 10×, until the log drains.
Default: 0, which uses the built-in default of 200000 points (about 35 MiB,
blocking at about 2 million points, about 350 MiB)
default_time_to_live
Section titled “default_time_to_live”Metric points are automatically deleted after this delay.
There is no “unlimited” retention, a TTL always applies. For a near-permanent
retention, set a very large value (the cap is about 179 years, e.g. 1500000h).
The TTL is stored as a number of days: a non-integral number of days is rounded
up.
Valid time units are ns, us (or µs), ms, s, m, h.
Default: 8760h (one year)
extra_write_backend
Section titled “extra_write_backend”An optional second backend to which points are also written, best-effort. It must
differ from backend, and the only supported value is clickhouse.
Points are written to this backend without blocking and without failing the request: a write that cannot be done is logged and dropped, so a problem on the extra backend never affects the source of truth or your clients. Dropped ranges are logged so they can be repaired afterwards.
This dual-write mode exists for the
Cassandra to ClickHouse migration. When
empty, SquirrelDB writes to backend only.
Default: ""
listen_address
Section titled “listen_address”The Prometheus API listen address, used for remote storage and PromQL requests. A Prometheus exporter (of SquirrelDB internal metrics) will be exported on /metrics.
Default: localhost:9201
The log block configures logging.
log.level
Section titled “log.level”For debugging, you may lower the log level. Only the messages that have a greater or equal level than the chosen one are shown.
The supported levels are -1 (trace), 0 (debug), 1 (info), 2 (warn), 3 (error)
Default: 1
log.disable_color
Section titled “log.disable_color”If true, don’t use color in logs.
Default: false
max_allowed_time_in_future
Section titled “max_allowed_time_in_future”How far in the future the timestamp of a point may be. Points more than this duration in the future might be rejected.
Valid time units are ns, us (or µs), ms, s, m, h.
Default: 48h
max_request_body_size
Section titled “max_request_body_size”Maximum body size of incoming requests (in MiB). If a body exceeds this limit, the request will be rejected.
Default: 64 (MiB)
promql
Section titled “promql”The promql block configures the behavior of the PromQL endpoint at
/api/v1/query and /api/v1/query_range.
promql.max_evaluated_points
Section titled “promql.max_evaluated_points”Maximum number of points one PromQL may evaluate. Once reached, it will abort the query. 0 means no limit.
Default: 0
promql.max_evaluated_series
Section titled “promql.max_evaluated_series”Maximum number of series one PromQL may evaluate. Once reached, it will abort the query. 0 means no limit.
Default: 0
The redis section lets you configure Redis for the short term storage.
With the Cassandra backend, Redis replaces the default in-memory store of the batches of points. With the ClickHouse backend, Redis holds the write-ahead log and the drainer lease, which makes the log durable and shared between instances.
In both cases, configuring Redis is what makes SquirrelDB instances stateless, and is therefore required for a highly available setup. Without it, points that are buffered but not yet written to the database are lost when SquirrelDB restarts or crashes.
redis.addresses
Section titled “redis.addresses”Addresses of your Redis servers.
If empty, Redis is not used. If set to a single address (e.g. [“localhost:6379”]) then Redis will be used, as a Redis cluster if possible or a simple single-node Redis. If set to more than one address, Redis cluster is used.
Default: []
redis.username
Section titled “redis.username”Username used to connect to Redis. When empty, the connection is done without using credentials.
Default: ""
redis.password
Section titled “redis.password”Password used to connect to Redis.
Default: ""
redis.keyspace
Section titled “redis.keyspace”Prefix used on all Redis keys.
Default: ""
redis.ssl
Section titled “redis.ssl”Enable SSL.
Default: false
redis.ssl_insecure
Section titled “redis.ssl_insecure”Whether SquirrelDB should ignore SSL certificate errors.
Default: false
redis.cert_path
Section titled “redis.cert_path”Path to the TLS certificate used to connect to Redis. Optional depending on your server configuration.
Default: ""
redis.key_path
Section titled “redis.key_path”Path to the TLS key used to connect to Redis. Optional depending on your server configuration.
Default: ""
redis.ca_path
Section titled “redis.ca_path”Path to the TLS certificate authority used to connect to Redis. Optional depending on your server configuration.
Default: ""
remote_storage
Section titled “remote_storage”The remote_storage block configures the Prometheus remote storage API used to
read and write to SquirrelDB. The endpoints are at /api/v1/read and
/api/v1/write. See the
remote storage documentation
for details.
remote_storage.max_concurrent_requests
Section titled “remote_storage.max_concurrent_requests”Maximum number of concurrent remote read and write calls. If set to N, allows
2*N concurrent requests, with N reads and N writes. If set to 0, defaults to
2 * number of CPUs.
Default: 0
require_tenant_header
Section titled “require_tenant_header”You may want to require all requests to set the
tenant header. When this setting is
true, SquirrelDB will return an error to queries and write requests that don’t
provide the tenant header.
Default: false
sentry
Section titled “sentry”sentry.dsn
Section titled “sentry.dsn”Errors and panics are sent to Sentry if the DSN is configured.
Default: ""
telemetry
Section titled “telemetry”telemetry.enabled
Section titled “telemetry.enabled”Enable telemetry.
Default: true
telemetry.address
Section titled “telemetry.address”Address to which the telemetry is sent.
Default: https://telemetry.bleemeo.com/v1/telemetry/
tenant_label_name
Section titled “tenant_label_name”Label used to differentiate metrics from multiple tenants, see multi tenancy for details.
Default: __account_id