Build an IoT application with AWS IoT [Tutorial]

Developing IoT applications has never been easier thanks to the cloud. All the major cloud vendors provide IoT tools; in this tutorial you’ll learn how to build a complete IoT application with AWS IoT.

End-to-end communication

To get started with AWS IoT, you need an AWS account. If you don’t have an AWS account, you can create one here.

Once you have created your account, you can log in and navigate to the AWS IoT Console.

Setting up the IoT Thing

Once you are on the AWS IoT Console page, make sure you have selected a region that is close to your location. I have selected the US East (N. Virginia) region as shown in the following screenshot:

Now, click on the Get started button in the center of the page. From the side menu, navigate to Manage | Things and you should see a screen as shown here:

Next, click on the Register a thing button and you should see a screen as shown here:

Right now, we are going to onboard only one Thing. So, click on Create a single thing.

On the next screen, we will start filling in the form by naming the device. I have called my device Pi3-DHT11-Node. You can give your Thing any name but do remember to update the code where applicable.

Next, we are going to apply a Type. Since this is our first device, we are going to create a new Type. Click on Create a thing type and fill in the form as shown in the following screenshot:

If we have different types of devices, such as motion sensors, door sensors, or DHT11 sensors, we can create a Type to easily group our nodes.

Click on the Create thing type button and this will create a new type; select that value as the default.

Next, we are going to add this device to a group—a group of Raspberry Pi 3, DHT11 nodes. You can group your devices as per your requirements and classification.

Now, click on Create group and create it with the following values:

We have added two attributes to identify this group easily, as shown in the previous screenshot.

Click on the Create thing group and this will create a new group—select that value as the default. These are the only Things we are going to set up in this step. Your form should look something like this:

At the bottom of the page, click on the Next button.

Now, we need to create a certificate for the Thing. AWS uses certificate-based authentication and authorization to create a secure connection between the device and AWS IoT Core.

Once the activation is successful, click on Attach a policy. Since we did not create any policies, you will see a screen similar to as what is shown here:

No issues with that. We will create a policy manually and associate it with this certificate in a moment.

Finally, click on the Register Thing button and a new Thing named Pi3-DHT11-Node will be created. Click on Pi3-DHT11-Node and you should see something like this:

We are not done with the setup yet. We still need to create a policy and attach it with a certificate to proceed.

Navigate back to the Things page, and from the side menu on this page, select Secure | Policies:

Now, click on Create a policy and fill in the form as demonstrated in the following screenshot:

In the previously demonstrated policy, we are allowing any kind of IoT operation to be performed by the device that uses this policy and on any resource. This is a dangerous setup, mainly for production; however, this is okay for learning purposes.

Click on the Create button and this will create a new policy. Now, we are going to attach this policy to a certificate.

Navigate to Secure | Certificates and, using the options available at the top-right of the certificate we created, we are going to attach the policy:

Click on Attach policy on the previous screen and select the policy we have just created:

Now, click on Attach to complete the setup.

With this, we are done with the setup of a Thing.

In the next section, we are going to use Node.js as a client on Raspberry Pi 3 to send data to the AWS IoT.

Setting up Raspberry Pi 3 on the DHT11 node

Now that we have our Thing set up in AWS IoT, we are going to complete the remaining operation in Raspberry Pi to send data.

Things needed

You will need the following hardware to set up Raspberry Pi 3 on the DHT11 node:

Create a new file named index.js at the root of the AWS-IoT-Thing folder. Next, create a folder named certs at the root of the AWS-IoT-Thing folder and move the four certificates we have downloaded there. Your final folder structure should look something like this:

Open index.js in any text editor and update it as shown in the following code snippet:

In the previous code, we are using awsIot.thingShadow() to connect to our AWS Thing that we have created. To the awsIot.thingShadow(), we pass the following options:

keyPath: This is the location of private.pem.key, which we have downloaded and placed in the certs folder.

certPath: This is the location of certificate.pem.crt, which we have downloaded and placed in the certs folder.

caPath: This is the location of RootCA-VeriSign-Class 3-Public-Primary-Certification-Authority-G5.pem, which we have downloaded and placed in the certs folder.

clientId: This is the name of the Thing we have created in AWS IoT Pi3-DHT11-Node.

host: This is the URL to which the Thing needs to connect to. This URL is different for different Things. To get the host, navigate to your Thing and click the Interact tab as shown in the following screenshot:

debug: This is optional. If you want see some logs rolling out from the module during execution, you can set this property to true.

We will connect to the preceding host with our certificates. In the thingShadow.on('connect') callback, we call thingShadow.register() to register. We need to register only once per connection. Once the registration is completed, we will start to gather the data from the DHT11 sensor and, using thingShadow.update(), we will update the shadow. In the thingShadow.on('status') callback, we will get to know the status of the update.

Save the file and execute the following command:

$ sudo node index.js

We should see something like this:

As you can see from the previous logs on the console screen, the device first gets connected then registers itself. Once the registration is done, we will wait for 15 seconds to transmit the first record. Then we wait for another 30 seconds and continue the process.

We are also listening for status and delta events to make sure that what we have sent has been successfully updated.

Now, if we head back to the AWS IoT Thing page in the AWS Console and click on the Shadow tab, we should see the last record that we have sent update here:

Underneath that, you can see the metadata of the document, which should look something like:

The following example used the MQTTS approach to fetch the shadow data. Whenever we want to fetch the data of a Thing, we publish an empty packet to the $aws/things/Pi3-DHT11-Node/shadow/get topic. Depending on whether the state was accepted or rejected, we will get a response on $aws/things/Pi3-DHT11-Node/shadow/get/accepted or $aws/things/Pi3-DHT11-Node/shadow/get/rejected, respectively.

For testing the data fetch, you can either use the same Raspberry Pi 3 or another computer. I am going to use my MacBook as a client that is interested in the data sent by the Thing.

In my local machine, I am going to create the following setup, which is very similar to what we have done in Raspberry Pi 3:

Create a folder named test_client. Inside the test_client folder, create a folder named certs and get a copy of the same four certificates we have used in Raspberry Pi 3.

Inside the test_client folder, run the following command on the Terminal:

$ npm init -y

Next, install the aws-iot-device-sdk module using the following command:

$ npm install aws-iot-device-sdk --save

Create a file inside the test_client folder named index.js and update it as shown here:

We should see something similar to what is shown in the following console output:

This way, any client that is interested in the data of this Thing can use this approach to get the latest data.

You can also use an MQTT library in the browser itself to fetch the data from a Thing. But do keep in mind this is not advisable as the certificates are exposed. Instead, you can have a backend microservice that can achieve the same for you and then expose the data via HTTPS.

With this, we conclude the section on posting data to AWS IoT and fetching it. In the next section, we are going to work with rules.

Building the dashboard

Now that we have seen how a client can read the data of our Thing on demand, we will move on to building a dashboard, where we show data in real time.

For this, we are going to use Elasticsearch and Kibana.

Elasticsearch

Elasticsearch is a search engine based on Apache Lucene. It provides a distributed, multi-tenant capable, full-text search engine with an HTTP web interface and schema-free JSON documents.

Kibana

Kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster. Users can create bar, line, and scatter plots, or pie charts and maps on top of large volumes of data.

As we have seen in the architecture diagram, we are going to create a rule in AWS IoT. The job of the rule is to listen to an AWS topic and then send the temperature and humidity values from that topic to an Elasticsearch cluster that we are going to create using the AWS Elasticsearch Service (https://aws.amazon.com/elasticsearch-service/).

The cluster we are going to provision on AWS will also have a Kibana setup for easy visualizations.

We are going to use Kibana and build the visualization and then a dashboard from that visualization.

We are going to use Elasticsearch and Kibana for a basic use case.

The reason I have chosen to use Elasticsearch instead of building a simple web application that can display charts is because we can do way more than just building dashboards in Kibana using Elasticsearch. This is where the IoT Analytics comes in.

We are not going to explore IoT analytics per se, but this setup should give you an idea and get you started off.

Setting up Elasticsearch

Before we proceed further, we are going to provision a new Elasticsearch cluster.

To set up Elasticsearch, head over to the Amazon Elasticsearch Service console or use the services menu on the of AWS console page to reach the Amazon Elasticsearch Service console page. You should see a screen similar to what is shown here:

Click on the Create a new domain button and fill in the next screen, as shown in the following screenshot:

Click on the Next button. Under the Node configuration section, update it as illustrated here:

If you are planning to run bigger queries, I would recommend checking Enable dedicated master and setting it up.

Under the Storage configuration section, update it as illustrated in the following screenshot:

Leave the remaining sections as the defaults and click Next to continue.

On the Setup access screen, under Network configuration, select Public access, and for the access policy, select Allow open access to the domain and accept the risks associated with this configuration.

We are using this setup to quickly work with services and to not worry about credentials and security. DO NOT use this setup in production.

Finally, click Next, review the selections we have made, and click Confirm.

Once the process has started, it will take up to 10 minutes for the domain to be provisioned.

Once the domain is provisioned, we should see a similar screen to what is illustrated here:

Here we have our Endpoint, to which data will be sent for indexing. And we also have the URL for Kibana.

When we click on the Kibana URL, after it loads, you will be presented with the following screen:

The previous screen will change once we start indexing data. In the next section, we are going to create an IAM role.

Setting up an IAM Role

Now that we have Elasticsearch and Kibana up and running, we will get started with setting up an IAM role. We will be using this role for the IoT rule and to put data into Elasticsearch:

Select AWS service from the top row and then select IoT. Click Next: Permissions to proceed to the next step:

All the policies needed for AWS IoT access resources across AWS are preselected. The one we are interested in is under AWSIoTRulesActions and Elasticsearch Service. All we need here is the ESHttpPut action.

Finally, click the Next: Review button and fill in the details as shown in the following screenshot:

Once we click Create role, a new role with the name provided will be created.

Now that we have Elasticsearch up and running, as well as the IAM role needed, we will create the IoT Rule to index incoming data into Elasticsearch.

Creating an IoT Rule

To get started, head over to AWS IoT and to the region where we have registered our Thing:

From the menu on the left-hand side, click on Act and then click the Create a rule option. On the Create rule screen, we will fill in the details as shown in the following table:

Field

Value

Name

ES_Indexer

Description

Index AWS IoT topic data to Elasticsearch service

SQL version

2016-03-23

Attribute

cast(state.desired.temp as decimal) as temp, cast(state.desired.humd as decimal) as humd, timestamp() as timestamp

Topic filter

$aws/things/Pi3-DHT11-Node/shadow/update

Condition

Once we fill the form in with the information mentioned in the previous table, we should see the rule query statement as demonstrated here:

SELECT cast(state.desired.temp as decimal) as temp, cast(state.desired.humd as decimal) as humd, timestamp() as timestamp FROM '$aws/things/Pi3-DHT11-Node/shadow/update'

This query selects the temperature and humidity values from the $aws/things/Pi3-DHT11-Node/shadow/update topic and casts them to a decimal or float data type. Along with that, we select the timestamp.

Now, under Select one or more actions, click Add action, and then select Send messages to the Amazon Elastic Service. Click on Configure action.

On the Configure action screen, fill in the details as illustrated here:

Field

Value

Domain Name

pi3-dht11-dashboard

Endpoint

Will get auto selected

ID

${newuuid()}

Index

sensor-data

Type

dht11

IAM role name

iot-rules-role

Once the details mentioned in the table are filled in, click on the Create action button to complete the setup.

Finally, click on Create rule and a new rule should be created.

Elasticsearch configuration

Before we continue, we need to configure Elasticsearch to create a mapping. The timestamp that we generate in AWS IoT is of the type long. So, we are going to create a mapping field named datetime with the type date.