Creating collectd data source in JavaScript

Exec Plugin

In the variety of collectd plugins there’s one ‘to rule them all’. If due to some course of events all collectd plugins except for Exec would be taken from you, you’d still be able to restore all its functionality with Exec.

As the name suggests, Exec starts external program or script and interprets its output as source of data. To be specific, it looks for lines that follow this scheme:

What’s interesting, Exec doesn’t specify in what language script should be written, so anything, including JavaScript, might work. In fact, using JavaScript would be beneficial in some scenarios, e.g. when dealing with RESTful services returning JSON.

Before we try JavaScript app as data source for collectd, let’s talk about PUTVAL lines a little bit more.

PUTVAL arguments

PUTVAL data format is heavily influenced by rrdtool, so there’re many similarities in data types, timestamps and even syntax. Its first argument,
hostname/source-instance/datatype-instance , can be anything (except for
datatype component), as long as it uniquely identifies data source.

datatype component is tricky beast.
rrdtool has two groups of types: COUNTER-like types for describing how rapidly value changes over time, and GAUGE-like types for describing numeric values ‘as is’. PUTVAL understands very same types and we can safely use e.g.
gauge for numeric values between 0 and 232.

Moreover, collectd provides custom types, defined in
/usr/share/collectd/types.db, e.g.
temperature (gauge with undefined lower and upper bounds),
count (gauge, 0..undefined), and many others. You can use any of those as well.

The second PUTVAL argument,
interval in seconds, is optional and indicates how often new value will arrive.

Finally,
timestamp can be
N for now or unix time in seconds, and
value – number or
U for undefined.

Connecting JavaScript data source and collectd

Assume the following JavaScript app (let’s call it
sensor.js ) produces meaningful series of data we’d want to feed to collectd. Where would we start?

sensor.js

JavaScript

1

2

3

4

5

constreadSensor=()=>Math.floor(Math.random()*1000);

constcurrentReading=readSensor();

console.log(currentReading);

// profit?

Make sensor.js executable

Firstly, in order to make this script executable, we need to add ‘execute’ permission and a shebang, that specifies which program will be able to interpret
sensor.js .
chmod+xsensor.js will take care of the first problem. As for the second, node.js will be more than happy to interpret the file for us:

sensor.js with hashbang

JavaScript

1

2

3

4

#!/usr/local/bin/node

constreadSensor=()=>Math.floor(Math.random()*1000);

...

Now if we execute script from shell, it’ll proudly produce something:

Execute sensor.js

Shell

1

2

$./sensor.js

# 245

The next step is to produce something that collectd will recognize.

Format output for collectd

For this step we need to build PUTVAL string. I think something like the following is quite OK for very beginning:

However, we’re also using hardcoded hostname and should’ve been producing sensor value more than once. But that’s fixable.

Use collectd’s hostname and update interval

When collectd runs, it introduces two environmental variables: COLLECTD_HOSTNAME and COLLECTD_INTERVAL, which we definitely should use. Moreover, as we’re going to write data regularly, it’s worth telling collectd how often this will happen via
Interval parameter. This is the final look of sensor.js:

So far, so good. However, lack of error messages doesn’t mean that everything is fine. The most reliable way to ensure that
sensor.js’s data gets to collectd is checking if it creates round-robin database files for new data source. Default location for them is usually
/var/lib/collectd/rrd :

Look for RRD with sensor data

Shell

1

2

3

$ls-R/var/lib/collectd/rrd|grepmysensor

#mysensor

#/var/lib/collectd/rrd/e6015696c686/mysensor:

Yup, it definitely works.

Summary

So as you can see, it’s very easy to use JavaScript application as metrics source for collectd. And quite often there’re practical reasons to do so. For instance, when you have to collect metrics from any web service returning JSON, JavaScript is obvious choice. Personally, I used JavaScript (node.js) + collectd + Graphite to build monitoring dashboards for continuous integration pipeline and was very happy with results. GitLab and Google Compute Engine provided JSON APIs for getting data about running builds, pipelines and virtual machines, and combining it all together into collectd data source was a matter of few hours, including Google, coffee and some more coffee.