Blog

A revolution is happening now, as you read this. It’s called the Internet of Things (IoT). An IoT system is an application that collects data from the real world, usually through sensors, and transforms the data into insights for its users. Humans are in control; we are in charge of communicating our desires to the system, which converts our desires into

An Internet of Things system, collecting data from sensors and generating insights for the users. These insights generate desires that are translated into commands that affect the real world.

The Internet of Things is equivalent in scale and impact to the industrial revolution. The increasing ubiquity of sensors and actuators is enabling ordinary people to interact digitally with their environment, and is enabling businesses and institutions to provide on-demand real-world services through digital means. Companies and engineers around the world are building IoT systems to disrupt industries that have not yet been touched by the digital revolution.

A new generation of devices running JavaScript

Year after year, IoT devices are getting cheaper, more powerful, and more power-efficient. For example, the Intel Edison is as tiny as a stamp, while delivering amazing performance with its dual-core atom system on a chip (SoC). The Intel Edison can fully run the Node.js framework, allowing developers to write IoT programs using Node. Devices like the Tessel are pushing the boundaries of the development paradigm by supporting JavaScript on barebone hardware. The Raspberry PI has been one of the cheapest platforms for quite a while now, and it supports Node.js. I believe in using JavaScript as a full-stack technology, from tiny devices to the cloud, and back to people’s laptops and phones. We are just scratching the surface of what a full stack architecture can achieve, reducing the development time and allowing us to develop new Internet of Things applications in a fraction of the time needed before. In particular, isomorphic libraries allow developers to reuse code between devices, in the server and in the browser.

An initial Internet of Things system architecture

A simple architecture of an IoT System, as any software architect would design it in the first step; this architecture is usually built from scratch.

The image above outlines a sample architecture for an Internet of Things application. A standard divide-and-conquer approach would break up the system into the following four major components:

A sensor API, which is called by the sensors to deliver data readings and receive commands.

A public API, which is called by the sensors to retrieve real-time data, historical data, and to manage the devices.

A set of operational services, which are responsible for authentication and authorization, among other things; these services manage their own database.

A set of data services, which are responsible for storing and analyzing the data, either in real time or offline.

How the sensor data and insights flow in the simple architecture.

How the users desires and commands flow in the simple architecture.

In this architecture, sensor data flows through all services before reaching the users. Likewise, commands move through the system in a similar way, touching all internal services before reaching the actuators. However, users want immediate feedback from the real world; real-time delivery of sensor data and commands is a key feature of the Internet of Things. Our simple divide-and-conquer architecture is not well suited to this data flow, as the entire system is involved in each piece of data or command. Another problematic aspect of this approach is its cost. We are handling real-time data in both our APIs, often in two different ways. Could we leverage this to achieve a better architecture?

Behavior of any publish/subscribe system.

A more effective architecture for Internet of Things systems

The core idea of this new architecture (see image above) is to expose a publish/subscribe broker up front to the devices and the users. Usually, these components are part of an enterprise service bus, and are used internally by all sort of organizations. Even popular NoSQL databases like Redis and MongoDB (via capped collection and tailable cursor) offer publish/subscribe functionalities.

How the sensor data and insight flow in a pub-sub driven architecture.

How the users’ desires and commands flow in a pub-sub driven architecture.

Because we have put a publish-subscribe broker in front of the users, we can have it direct the interaction between the sensors and the users, both for data and commands. This approach reduces latency, and allows for offline handling of time-series data and analytics. Moreover, this approach greatly reduces the implementation costs by using the same technology for all real-time communication.

The architecture outlined in the paragraph above can be implemented using the MQTT protocol. MQTT (originally called MQ Telemetry Transport) is a publish-subscribe protocol with an extremely low data overhead (two bytes). It has built-in support for offline messaging and multiple qualities of service levels. Moreover, it can run on both standard TCP/TLS and over WebSocket (ideally HTTPS). It also offers a simple key/value store in the form of retain messages. All these features make it a good protocol to implement IoT systems. I maintain the MQTT.js module. This module allows you to interact with an MQTT broker both from Node.js and from the browser, thanks to Browserify. MQTT.js recently hit version 1.0.0, featuring a massive refactoring and a 2x performance improvement, and the library is now able to receive and send 50k msg/second on my Macbook Pro 2013 (70k msg/second with iojs). Let’s see how we can use MQTT.js to connect to a broker, on any device that can run Node.js. MQTT is also supported on the Arduino, the CC3200, and many more embedded operating systems and development board. Check the related documentation to port these examples to the different development boards. Receiving messages on devices is straightforward: just run “npm install mqtt”, and use the following code:

Keeping our devices configured correctly is usually tricky, because a device can be offline at any time. We therefore use retained messages. A retained message is delivered to anybody when they subscribe, such as after a reconnect. You can send a retained message as follows:

var config = JSON.stringify({

enabled: "false"
})

client.publish("mydevice/config", config, {
retain: true
})

Then you can process your messages as usual. I am also the author of Mosca, a pure-node MQTT broker, that anybody can use to build an Internet of Things system. Mosca is embeddable within any node application, allowing complete customization and the implementation of the pub-sub architecture described above. Now we come to embedding specific behavior inside the broker, for storing each data into a database, having a custom authentication solution, and authorizing each publish and subscribe to the broker. As shown before, every IoT system needs this feature, and adding them to Mosca is extremely easy, as we will see in the following. This is how you process messages to store in a time series database such as InfluxDB:

Lastly, Mosca supports a wide range of configuration options, from clustering on top of Redis or MongoDB to WebSocket support. Check out it’s wiki here for documentation. In this article, we dug deep into how to build an effective IoT System leveraging Node.js, MQTT and Mosca. If you want to know how to deploy a high availability cluster using nscale, our deployer product, check out this article.Credits: Icons from FontAwesome

This blog post is a write-up of my talk “Enterprise IoT Systems with Node and MQTT” at NodeSummit (San Francisco, 10-11 February 2015).