Skip to content

Glances threshold alerts

You’ll get: an incident when Glances sees memory, load or a disk cross its warning or critical threshold, one per host and check.

  • An ingest key from a generic InfraInbox source, shaped like iik_src_… (see Events API: curl and scripts).
  • Glances running as a service on the host, with its configuration file (/etc/glances/glances.conf).

Glances runs an action as a plain command, not through a shell, so the JSON is built in a script:

/usr/local/bin/infrainbox-glances
#!/bin/sh
# usage: infrainbox-glances <check> <warning|critical> <value> [item]
INFRAINBOX_URL=https://infrainbox.example.com/v1/events
INFRAINBOX_KEY=iik_src_…
host=$(hostname)
what="$1${4:+ $4}"
curl -fsS -m 10 "$INFRAINBOX_URL" \
-H "Authorization: Bearer $INFRAINBOX_KEY" \
-H "Content-Type: application/json" \
-d "{\"title\":\"$host: $what at $3\",\"severity\":\"$2\",\"type\":\"glances.$1\",\"resource\":\"host:$host\",\"dedupKey\":\"glances:$host:$what\"}"
Terminal window
sudo chmod 700 /usr/local/bin/infrainbox-glances

The key is in the script, so keep it readable only by the user Glances runs as.

/etc/glances/glances.conf
[mem]
warning=80
critical=90
warning_action=/usr/local/bin/infrainbox-glances mem warning {{percent}}%
critical_action=/usr/local/bin/infrainbox-glances mem critical {{percent}}%
[load]
critical=5.0
critical_action=/usr/local/bin/infrainbox-glances load critical {{min5}}
[fs]
critical=95
critical_action=/usr/local/bin/infrainbox-glances fs critical {{percent}}% {{mnt_point}}

Restart Glances to load it.

Run the script by hand: /usr/local/bin/infrainbox-glances mem critical 99%. An incident opens on the source.

  • Glances has no action for the return to normal, so these incidents close after 24 hours without a repeat, or when you resolve them. Add critical_action_repeat=true beside a critical_action to report on every refresh while it lasts, which keeps the incident open for as long as the problem does.
  • Glances runs the action on the host, so a host that is down reports nothing. Pair it with a heartbeat.