Anton Lindstrom (about, @twitter, @github)

Standalone checks with Sensu

Published:

The monitoring framework Sensu as I've written about before is being implemented at my new gig. We've been investigating some fun things about it and looked into something that's not very well documented, standalone checks.

A standalone check is a check that's not defined in the server but is defined and executed on the client. The standalone checks can be defined in two ways, either use the Sensu client or send the definition to the Sensu port. What's described below is how we send checks to the server with standard Linux utilities.

Here's the simplest way to send a check to the Sensu server:

echo '{
  "handlers": ["default"],
  "name": "check_name",
  "output": "error message",
  "status": 2
}' | nc -w1 sensu-server.example.com 3030

The command above sends a check that has an error, this means it will error and throw an alert.

To provide some client context of the alert, we can send the following:

echo '{
  "client": {
    "name": "localhost",
    "address": "127.0.0.1"
  },
  "handlers": ["default"],
  "name": "check_name",
  "output": "error message",
  "status": 2
}' | nc -w1 sensu-server.example.com 3030

The alerts can be sent in a blocking or non-blocking way (TCP/UDP). Use nc -w1 -u sensu-server.example.com 3030 for UDP.

Metrics can be sent into Sensu by using the following:

echo '{
  "client": {
    "name": "localhost",
    "address": "127.0.0.1"
  },
  "handlers": ["graphite"],
  "name": "localhost-metrics",
  "output": "localhost.disk_usage.lolcats.used   9000000    1411131484",
  "status": 0,
  "type": "metric"
}' | nc -w1 sensu-server.example.com 3030

All the settings that are used in the configuration and check files can also be used with standalone external checks. This means that it's possible to use the standard Linux toolchain to send alerts or metrics from shell scripts without any extra dependencies. The examples should provide an introduction to use Sensu with nc (1) and echo (1).