System Log Collection

Starter

Professional

Most modern Linux distributions use systemd-journald for system logging. Glouton can collect logs from systemd-journald as well as from Auditd. If your distribution still uses syslog instead of systemd-journald, see the Syslog section below.

systemd-journald

To enable collection of system logs from systemd-journald, set journald_enable to true:

sudo tee /etc/glouton/conf.d/99-logs-journald.conf > /dev/null << 'EOF'
log.opentelemetry.auto_discovery.journald_enable: true
EOF

This option is also enabled when using all_enable.

Once enabled, all system logs from the journal are collected and sent to the Bleemeo Cloud Platform.

Syslog

If your Linux distribution does not have systemd-journald, you can configure Glouton to collect system logs from the /var/log/syslog file instead. Set syslog_enable to true:

sudo tee /etc/glouton/conf.d/99-logs-syslog.conf > /dev/null << 'EOF'
log.opentelemetry.auto_discovery.syslog_enable: true
EOF

This option is also enabled when using all_enable.

Note

If both systemd-journald and syslog are available, prefer systemd-journald — it provides structured metadata and is more reliable.

Auditd

Auditd logs are produced by the auditd daemon and contain security related information such as authentication events, file access, and privilege escalation. The service might not be installed by default on your Linux.

To enable collection of Auditd logs, set auditd_enable to true:

sudo tee /etc/glouton/conf.d/99-logs-auditd.conf > /dev/null << 'EOF'
log.opentelemetry.auto_discovery.auditd_enable: true
EOF

This option is also enabled when using all_enable.

Installing Auditd

For Ubuntu/Debian:

sudo apt-get install auditd

For Fedora, CentOS, Almalinux, RockyLinux or similar:

sudo yum install audit

Log Filters

Filters allow you to control which log entries are kept or dropped based on their content. They can be applied globally to all log sources with global filters.

For example, to only keep logs from a specific program:

log.opentelemetry.global_filters:
  include:
    match_type: strict
    record_attributes:
      - key: 'source_program'
        value: 'sshd-session'

Example: Auditing SSH Logins and sudo Commands

On Debian and Ubuntu, OpenSSH, sudo, su and PAM all write to systemd-journald with the authpriv syslog facility. Enabling the systemd-journald collection is therefore enough to capture every SSH login attempt and every command run through sudo.

The configuration below enables journald and narrows the journal down to those authentication events, so the agent does not ship the whole system journal.

sudo tee /etc/glouton/conf.d/99-logs-ssh-sudo.conf > /dev/null << 'EOF'
log.opentelemetry.auto_discovery.journald_enable: true

log.opentelemetry.global_filters:
  log_record:
    - >-
      resource.attributes["service.name"] == "journald" and
      not IsMatch(attributes["source_program"], "^(sshd|sshd-session|sshd-auth|sudo|su|login|systemd-logind)$")
EOF

How it works:

• log_record is a list of OTTL expressions; a log record is dropped when an expression evaluates to true.
• resource.attributes["service.name"] == "journald" restricts the rule to journal entries. Other log sources — container logs, discovered services, file receivers — are left untouched, which an include filter could not do since its conditions are combined with OR.
• source_program is the attribute Glouton fills from the journal’s SYSLOG_IDENTIFIER field (falling back to _COMM). Everything that is not one of the listed programs is dropped.
• OpenSSH 9.8 split sshd into a listener and a per-session binary, and OpenSSH 10.0 moved the authentication phase to a third one. Depending on the OpenSSH version shipped by your release, login messages appear under sshd, sshd-session or sshd-auth — listing all three keeps the configuration working across Debian and Ubuntu versions.

Collected Events

source_program	Example journal message	Event
sshd-session	Accepted publickey for alice from 192.0.2.10 port 54321 ssh2: ED25519 SHA256:…	Successful SSH login
sshd-session	Failed password for invalid user admin from 203.0.113.5 port 40100 ssh2	Rejected SSH login
sshd-session	Disconnected from user alice 192.0.2.10 port 54321	SSH session closed
sudo	alice : TTY=pts/0 ; PWD=/home/alice ; USER=root ; COMMAND=/usr/bin/apt update	Command run through sudo
sudo	alice : 1 incorrect password attempt ; TTY=pts/0 ; PWD=/home/alice ; USER=root ; COMMAND=/usr/bin/id	Failed sudo authentication
sudo	pam_unix(sudo:session): session opened for user root(uid=0) by alice(uid=1000)	Privilege escalation
systemd-logind	New session 42 of user alice.	Login session created

In the Bleemeo Logs explorer, filter on the source_program attribute to separate SSH activity from sudo activity, and search the log body for Failed password or incorrect password attempt to isolate failures.

Note

Successful and failed authentications are both logged at the info or notice syslog priority, so severity is not a reliable way to tell them apart — match on the message body instead.

Caution

Global filters apply to every log source, and only one log.opentelemetry.global_filters setting is in effect. If you already use global filters, merge the expressions into a single block rather than declaring the setting twice.

Verifying the Source

Check that the journal actually contains the expected entries on the host:

journalctl -t sshd -t sshd-session -t sshd-auth -t sudo -n 20

Going Further

For a complete audit trail — every executed command, file access and privilege change, not just what OpenSSH and sudo choose to log — enable Auditd collection as well.