Blog Articles

Visualizing Smog Sensor Data with the help of Vert.x, Prometheus, and Grafana

Air pollution is a major problem in many cities around the globe. Some people in Stuttgart, Germany have developed cheap smog sensors that people can install on their balconies and other convenient places and then report data to a central site. I have written about that on OpenSource.com. The data is sent to a central server, from where it is visualized on a map. At the time of writing the above article, there was no way of seeing how the value has changed over time. Meanwhile, there is a visualization of the last 24 hours available on the map.

The sensor is also able to send data to an additional target, for example, a local database. Thus, I decided to feed the data to a local server for long-term storage to visualize trends.

As I was playing with Prometheus at this time, I thought that feeding the data into it may be a good use case for further exploration. Now that Prometheus wants to scrape the data and the sensor is only pushing the data, it became clear that I need an intermediate gateway that solves this mismatch and which also translates the data format the sensor sends into the format Prometheus expects.

To implement the gateway, I choose Vert.x a modern day reactive framework that runs on the Java Virtual Machine (JVM). I was first considering other options, but it turned out that Vert.x’ implicit handling of connections and threading made my life easy.

The gateway receives Http POST requests from the sensor with measurement data in InfluxDB format. It exposes the data to GET requests on /metrics in the Prometheus text format.

This first part creates a new verticle and then inside of start() sets the http port it listens on to 10080 and then tells Vert.x that the configuration is ready. Vert.x then starts is accept loop and dispatches incoming requests.

The next step is now to define handlers for the endpoints to do the actual work. In the handlers we need to be careful not to block or implement slow operations; Vert.x may decide to abort the handler in this case.

Now let’s look at the POST handler. This is a tiny bit more complicated as we need to explicitly tell Vert.x to that we want to obtain the body of the request. On top, we also want to get the sensor ID, which is encoded in a Http-Header.

As the target, you enter the IP address and port from the Vert.x gateway. After this is done, you can start Prometheus and pass the path to the configuration to it:

$ prometheus -config.file /usr/local/etc/prometheus.yml

Grafana

Start Grafana and point your browser at port 3000; default login is admin/admin.
To set up Grafana, one needs to define a datasource of type Prometheus that points to the Prometheus server at http://localhost:9090/. Once this is done, we can set up a new dashboard and add graphs to it.

In above screenshot, you see the query editor. As the code in the gateway is set up to put labels on the exported items according to the type (the sensor can also report temperature and humidity), it is possible to query for the values with a label query of {{typ="fs"}}.

Summary

The article shows an end-to-end example of an IoT device sending data in a certain format. A gateway is implemented with the help of the Vert.x framework, which does all the magic of connection handling etc.
Finally, the data is stored in a Prometheus time series database and graphed via Grafana.

Download the Eclipse Vert.x cheat sheet, this cheat sheet provides step by step details to let you create your apps the way you want to.