ESP8266: Connecting to MQTT broker

The objective of this post is to explain how to connect the ESP8266 to a MQTT broker, hosted on CloudMQTT.

Introduction

The objective of this post is to explain how to connect the ESP8266 to a MQTT broker. If you are not familiar with the protocol, you can read more about here.

Although this example should work fine with other brokers, we will assume that the broker will be hosted on CloudMQTT.

Since CloudMQTT has a free plan, we can just create an account and test it. Setting an account is really simple and it’s outside the scope of this post. You can check here how to do it and how to create a broker instance.

After completing the procedure, check the instance information page, which should be similar to the one shown in figure 1. The important credentials that we will be using on the ESP8266 code are the server, the user, the password and the port.

Figure 1 – CloudMQTT instance information.

In the ESP8266 side, we will be using an MQTT that supports the ESP8266, called PubSubClient. The library can be installed via Arduino IDE library manager. The coding shown here is based on the examples provided in the library, which I encourage you to try.

Regarding the hardware, the tests shown on this tutorial were performed using a ESP8266 NodeMCU board.

The code

First, we start by including the libraries needed for all the functionality. We need the ESP8266WiFi library, in order to be able to connect the ESP8266 to a WiFi network, and the PubSubClient library, which allows us to connect to a MQTT broker and publish/subscribe messages in topics.

#include <ESP8266WiFi.h>
#include <PubSubClient.h>

Then, we will declare some global variables for our connections. Naturally, we need the WiFi credentials, to connect to the WiFi network. You can check here a previous post explaining in detail how to connect to a WiFi network using the ESP8266.

We will also need the information and credentials of the MQTT server. As explained in the introduction, you will need to know the server address, the port, the username and the password. Use your values in the variables bellow.

After that, we will declare an object of class WiFiClient, which allows to establish a connection to a specific IP and port [1]. We will also declare an object of class PubSubClient, which receives as input of the constructor the previously defined WiFiClient.

The constructor of this class can receive other number of arguments, as can be seen in the API documentation.

WiFiClient espClient;
PubSubClient client(espClient);

Now, moving for the setup function, we open a serial connection, so we can output the result of our program. We also connect to the WiFi network.

Next, we need to specify the address and the port of the MQTT server. To do so, we call the setServer method on the PubSubClient object, passing as first argument the address and as second the port. These variables were defined before, in constant strings.

client.setServer(mqttServer, mqttPort);

Then, we use the setCallback method on the same object to specify a handling function that is executed when a MQTT message is received. We will analyse the code for this function latter.

client.setCallback(callback);

Now, we will connect to the MQTT server, still in the setup function. As we did in the connection to the WiFi network, we connect to the server in a loop until we get success.

So, we do a while loop based on the output of the connected method called on the PubSubClient, which will return true if the connection is established or false otherwise.

To do the actual connection, we call the connect method, which receives as input the unique identifier of our client, which we will call “ESP8266Client”, and the authentication username and password, which we defined early. This will return true on connection success and false otherwise

In case of failure, we can call the state method on the the PubSubClient object, which will return a code with information about why the connection failled [2]. Check here the possible returning values.

To finalize the setup function, we will publish a message on a topic. To do so, we call the publish method, which receives as input arguments the topic and the message. In this case, we will publish a “Hello from ESP8266” message on the “esp/test” topic.

client.publish("esp/test", "Hello from ESP8266");

Then, we will subscribe to that same topic, so we can receive messages from other publishers. To do so, we call the subscribe method, passing as input the name of the topic.

The callback function

After the initialization, we need to specify the callback function, which will handle the incoming messages for the topics subscribed.

The arguments of this function are the name of the topic, the payload (in bytes) and the length of the message. It should return void.

In this function, we will first print the topic name to the serial port, and then print each byte of the message received. Since we also have the length of the message as argument of the function, this can be easily done in a loop.

The main loop

In the main loop function, we will just call the loop method of the PubSubClient. This function should be called regularly to allow the client to process incoming messages and maintain its connection to the server [2].

Testing the code

First, make sure the MQTT server is running. Then, to test the code, just upload it and run it on your ESP8266. Open the Arduino IDE serial console, so the output gets printed.

Upon running, the ESP8266 will send the “Hello from ESP8266” message, which will not be printed on the serial. After that, the ESP8266 subscribes the same topic to which the hello message was sent.

If any other producer sends a message to the “esp/test” topic, then it will be printed in the serial console, as shown in figure 2.

Figure 2 – Messages sent to the “esp/test” topic.

For this tutorial, I used MQTTlens, a Google Chrome application, which connects to a MQTT broker and is able to subscribe and publish to MQTT topics [3]. This is a very useful application that I really recommend for this type of tests.

For the test, MQTTlens was subscribing the “esp/test” topic before connecting the ESP8266. As seen in figure 3, the “Hello from ESP8266” message was printed. After that, two hello messages were sent by MQTTlens, which can be seen in the same figure. The messages sent are the ones shown in figure 2, which were received by the ESP8266, as expected.

Hi!
You have access to the message on the callback function, in the payload argument.
It is an array of bytes, so you need to convert it to the format you are expecting and just apply the switch case to that value.
In my case I’m simply printing each byte of the message, but you can implement the callback function the way you need.
Best regards,
Nuno Santos

Hi!
The documentation for the errors can be seen below:https://pubsubclient.knolleary.net/api.html#state
In your case it seems to be the following:
-2 : MQTT_CONNECT_FAILED – the network connection failed
Which is weird since you should have passed the previous step of connecting to the WiFi network.
Are you using the exact same code of the tutorial? Were you able to connect to the WiFi network?
Best regards,
Nuno Santos

Note however that it is not clear if the library is able to verify the digital certificate of the broker. So, even if it works with SSL, it is kind of not that much useful if it is not able to perform that validation because you will not be able to confirm the identity of the server, which is a security flaw.

Hi! thank you very much 🙂 unfortunately I haven’t yet tried to use SSL yet and I think the library I’ve used on this tutorial doesn’t support it.
You can take a look at this one though, which seems to support:https://github.com/256dpi/arduino-mqtt
Note however that it is not clear if the library is able to verify the digital certificate of the broker. So, even if it works with SSL, it is kind of not that much useful if it is not able to perform that validation because you will not be able to confirm the identity of the server, which is a security flaw.
Let us know if you succeed on making it work.
Best regards,
Nuno Santos

Hellow
i uploaded the full code upthere to my nodemcu, after sucsessfully aploading i can read on the serial monitor that my nodemcu is succesfully connected to the wifi and after that to the mqtt server. “connected” 🙂
after this there is no subscribed message received, my node is also not publishing messages to the server (i check on websocket there is no message received from client!!)
i tried sending message to the client from the web socket, but client (my nodemcu) is not receiving the message!!
i over checked if there is a difference in the spelling but nothing is missed!! so what can you say for me? hope you reply soon!!

Hellow
i uploaded the full code upthere to my nodemcu, after sucsessfully aploading i can read on the serial monitor that my nodemcu is succesfully connected to the wifi and after that to the mqtt server. “connected” 🙂
after this there is no subscribed message received, my node is also not publishing messages to the server (i check on websocket there is no message received from client!!)
i tried sending message to the client from the web socket, but client (my nodemcu) is not receiving the message!!
i over checked if there is a difference in the spelling but nothing is missed!! so what can you say for me? hope you reply soon!!

Hi! Did you test the MQTTLens application I’ve mentioned on the last section of the post?
Maybe there is some issue in your web socket connection, my recommendation is that you first test and confirm with MQTTLens, since it should be working.
If you find a problem at that step, then it is a problem on the ESP side.
If everything works fine with MQTT Lens and then it fails with a custom web socket client, then it may be some problem with the client.
Best regards,
Nuno Santos

At the time I haven’t tested more than one topic, but my guess is that you can call the subscribe method multiple times for different topics and then in the callback function use the topic argument to know from where the message was originated.

Hi!
Sorry for the delay, I’ve been having a lot of work these past weeks.
At the time I haven’t tested more than one topic, but my guess is that you can call the subscribe method multiple times for different topics and then in the callback function use the topic argument to know from where the message was originated.
Let us know if it works, in case you test it 🙂
Best regards,
Nuno Santos

Hi antepher. my name is arafath baig, I am using esp8266 generic module in my project and i am using the example code of esp8266 mqtt and i am getting an error of mqtt not declared.
will you please help me. Give me your email id so that i can send you my code. Thank You.

Hi antepher. my name is arafath baig, I am using esp8266 generic module in my project and i am using the example code of esp8266 mqtt and i am getting an error of mqtt not declared.
will you please help me. Give me your email id so that i can send you my code. Thank You.

Hi!
Did you install the MQTT library needed for the code? It can be installed via the Arduino IDE Library manager.https://github.com/knolleary/pubsubclient
If so, are you using the exact same code from the tutorial or did you adapt something for your application?
Best regards,
Nuno Santos

Hi!
What do you mean by “LED is blinking”? During the process of uploading the code to the ESP?
Once the upload finishes doesn’t it print anything to the serial monitor while trying to connect to WiFi?
You need to make sure your Arduino serial monitor baud rate is the same of the one used on the program (115200).
There should be a dropdown on the serial monitor window to change the baud rate.
Were you able to upload and run programs in your ESP before testing the code here? Could you print content to the serial monitor?
Best regards,
Nuno Santos

I’ve not been using AT commands in a long time, so I’m not sure what you have available these days.

Nonetheless, it seems that you are trying to establish a raw TCP connection to the MQTT broker, which will not work.

MQTT is a protocol built on top of TCP/IP, so you would have to respect the protocol when sending the data.

If you want to send stuff to a MQTT topic using a TCP connection it pretty much means that you would have to implement MQTT on top of it yourself, which I really don’t recommend unless you have a lot of experience with the protocol and how networking works.

On top of that, since you are using AT commands, the task would be even harder.

My recommendation is either to try to find an AT command firmware that supports MQTT (not sure if it exists, I know that the UART OBLOQ,a device based on the ESP8266, has such firmware, so maybe you can try to use it) or program the ESP8266 using, for example, the Arduino core, and use a library that already implements the MQTT protocol.

hello!!
I connected my esp-01 module to my laptop serial port
In the terminal window
AT+CIPSTART=”TCP”,”m14.cloudmqtt.com”,15772
>> CONNECT
AT+CIPSAND=”test1″, jpnwduxq,iObCtLN9cpms
>> ERROR
Can you tell me where it went wrong?
I want to send ‘test123’ to the topic.

Hi!
I’ve not been using AT commands in a long time, so I’m not sure what you have available these days.
Nonetheless, it seems that you are trying to establish a raw TCP connection to the MQTT broker, which will not work.
MQTT is a protocol built on top of TCP/IP, so you would have to respect the protocol when sending the data.
If you want to send stuff to a MQTT topic using a TCP connection it pretty much means that you would have to implement MQTT on top of it yourself, which I really don’t recommend unless you have a lot of experience with the protocol and how networking works.
On top of that, since you are using AT commands, the task would be even harder.
My recommendation is either to try to find an AT command firmware that supports MQTT (not sure if it exists, I know that the UART OBLOQ,a device based on the ESP8266, has such firmware, so maybe you can try to use it) or program the ESP8266 using, for example, the Arduino core, and use a library that already implements the MQTT protocol.
Hope this helps,
Nuno Santos

I am working with MQTT for a while now. It is working very well but I have one small issue that I can not solve. Maybe you have an idea about the following issue.

It is an ESP8266-01 with a button and a LED. Pushing the button would change the state of the LED and send the info to the MQTT broker.
If the device, the ESP8266-01 lost it’s connection with the broker, it try’s to reconnect but if that fails it wait 5 seconds before retrying. If at that time I try to push the button, it will not work. All logic, but when the internet is down for a long time it is impossible to change the state of the LED.

I am using PubSubClient.h with the Arduino IDE.
Just wondering how you would deal with this but if this question is not allowed here, just ignore it please.

That’s a strange issue since for what I’ve understood from your system, sending the data to the broker is a side effect of clicking the button.

So, connecting the LED should be independent of if the Internet connection is up or not, right?

What I would do would be setting an interrupt to detect the click of the button.

Since interrupts need to run fast, I would simply set a flag to true and then in the main loop check the value of the flag.

If it was true, I would turn on the LED immediately and then check if I had internet connection + broker connection. If I had, I would then send the message to the broker.

That way, the LED should keep working independently of internet connection.

Now, one thing you need to figure out in your code is that if the Internet goes down and then goes up again, the ESPis automatically reconnecting. I don’t recall what was the behavior since I’ve not been working with the ESP8266 for a while, but I think that there was a function to set the ESP8266 to auto reconnect when the connection was lost.

On top of that, you need to check if, even if the internet reconnects, the connection to the MQTT broker persists.

Unfortunately I don’t know how the MQTT library behaves, if it can recover automatically from the internet going down or if it needs to be explicitly reconnected.

My recommendation is to run some isolated tests of that behavior and remove the led logic of the equation to figure that out.

For example, you can develop a simple program that sends a mqtt message every 30 seconds, turn off your router for a while and then reconnect it back and see if the messages start getting received again.

In terms of software development, it’s a good idea to test things separately before joining them in the final system.

That way, you can figure out more easily where the problem is during debugging.

Feel free to ask your questions 🙂

Unfortunately due to time constraints I cannot provide implementation code or look into extensive pieces of code, but I always try to give a conceptual help 🙂 Also, if I don’t know the answer, other readers of the blog may be able to help.

Hope this helps getting you in the right track and let us know if you were able to solve the issue 🙂

Hi Nuno Santos,
No worries mate. I am not searching for implementation code (although it may come in handy sometimes, to be honest🙂) or want you to solve this issue just like that. I was just wondering what side to look to or some pointers.

Maybe I have to look at the library or the way it tries to reconnect.
Thanks and I really appreciate your response.
Best regards,
Ray.

Hi antepher,
I am working with MQTT for a while now. It is working very well but I have one small issue that I can not solve. Maybe you have an idea about the following issue.
It is an ESP8266-01 with a button and a LED. Pushing the button would change the state of the LED and send the info to the MQTT broker.
If the device, the ESP8266-01 lost it’s connection with the broker, it try’s to reconnect but if that fails it wait 5 seconds before retrying. If at that time I try to push the button, it will not work. All logic, but when the internet is down for a long time it is impossible to change the state of the LED.
I am using PubSubClient.h with the Arduino IDE.
Just wondering how you would deal with this but if this question is not allowed here, just ignore it please.
Best regards,
Ray.

Hi!
That’s a strange issue since for what I’ve understood from your system, sending the data to the broker is a side effect of clicking the button.
So, connecting the LED should be independent of if the Internet connection is up or not, right?
What I would do would be setting an interrupt to detect the click of the button.
Since interrupts need to run fast, I would simply set a flag to true and then in the main loop check the value of the flag.
If it was true, I would turn on the LED immediately and then check if I had internet connection + broker connection. If I had, I would then send the message to the broker.
That way, the LED should keep working independently of internet connection.
Now, one thing you need to figure out in your code is that if the Internet goes down and then goes up again, the ESPis automatically reconnecting. I don’t recall what was the behavior since I’ve not been working with the ESP8266 for a while, but I think that there was a function to set the ESP8266 to auto reconnect when the connection was lost.
On top of that, you need to check if, even if the internet reconnects, the connection to the MQTT broker persists.
Unfortunately I don’t know how the MQTT library behaves, if it can recover automatically from the internet going down or if it needs to be explicitly reconnected.
My recommendation is to run some isolated tests of that behavior and remove the led logic of the equation to figure that out.
For example, you can develop a simple program that sends a mqtt message every 30 seconds, turn off your router for a while and then reconnect it back and see if the messages start getting received again.
In terms of software development, it’s a good idea to test things separately before joining them in the final system.
That way, you can figure out more easily where the problem is during debugging.
Feel free to ask your questions 🙂
Unfortunately due to time constraints I cannot provide implementation code or look into extensive pieces of code, but I always try to give a conceptual help 🙂 Also, if I don’t know the answer, other readers of the blog may be able to help.
Hope this helps getting you in the right track and let us know if you were able to solve the issue 🙂
Best regards,
Nuno Santos

Hi,
I have not change anything in your code. I am able to send a publish message to the websocket of cloudmqtt, but I am not able to subscribe to the broker. I even tried to subscribe to broker via mosquitto from windows but still no success. I have no idea what might be causing this issue. Thanks in advance. 🙂

Sorry for the late reply. I successfully communicated via MQTT using SIM900. There I used the websocket of cloud MQTT to publish and subscribe. And see live updates.
This could’ve been some temporary issue, I will try it again and let you know if it worked or not.

Hi,
I have not change anything in your code. I am able to send a publish message to the websocket of cloudmqtt, but I am not able to subscribe to the broker. I even tried to subscribe to broker via mosquitto from windows but still no success. I have no idea what might be causing this issue. Thanks in advance. 🙂

Hi,
That’s weird, I have never experienced any problems, although I’ve never tried to do it over websockets.
I think I recall that cloud MQTT had an option to operate over websockets or not. Is that still possible? If so, does it work if not operating over websockets?
Best regards,
Nuno Santos

Hi,
Sorry for the late reply. I successfully communicated via MQTT using SIM900. There I used the websocket of cloud MQTT to publish and subscribe. And see live updates.
This could’ve been some temporary issue, I will try it again and let you know if it worked or not.
Regards,
Abhishek

Hello. Thank for you tutorial, it helped me a lot, but I can suggest one modification.
A lot of people saying that didnt get any message in NODEMCU and here is the why:
client.publish(“esp/test”, “Hello from ESP8266”);
client.subscribe(“esp/test”);
You are posting first and only after subscribing in topic, so, NODEMCU will never get “Hello From ESP8266” because he isn’t listening. So, just subscribe first and after publish in topic.
Sorry, if I undestood wrong and that was your intention.

Hi!
You’re welcome, thanks for the feedback. I’m glad the tutorial was helpful 🙂
Thanks for the suggestion but as indicated in the testing section, the ESP is not supposed to receive the first message precisely because it didn’t subscribe to the topic before sending the message.
That’s why I’ve used MQTTLens to send other messages that the ESP will receive later.
But, as you mention, if we want the ESP to receive its own message, then we should first subscribe to the topic and only then send the message. 🙂
Best regards,
Nuno Santos

After some time the nodemcu stops responding. When I look the LOG in cloudmqtt website, it shows this:
2018-11-09 03:17:37: Client ESP8266Client has exceeded timeout, disconnecting.
2018-11-09 03:17:37: Socket error on client ESP8266Client, disconnecting.
Can you help me?

Hi!
Unfortunately I’m not sure why that is happening, as I’ve never experienced the issue.
My suggestion is to open a issue in the GitHub page of the libraries, so someone there can most likely help.
If you happen to find the reason / solution for your problem, please let us know as it will be useful for others 🙂
Best regards,
Nuno Santos

I’m having just one problem: with the MQTTLens.
I can’t connect it. Can you please tell what information I need to fill in each space of the MQTTLens (Connection name, Hostname, tcp/ws, Port…)?
Thanks for the help!

As to rewriting the sentence, here’s what I’d do. My approach adds more information and uses “you” instead of “we.” Feel free to adapt this any way you choose.

“After you include the four lines shown above, you will declare an object of the class WiFiClient, as shown below. That object lets you connect to a specific Internet provider (IP) and a specific port. Then you will declare a second object of the class PubSubClient that can receive information from the espClient object.

Code here…

The C++ constructor of the class PubSubClient provides overloaded functions in C++. Basically that means the function can receive several arrangements of arguments passed to it. For example, int publish (topic, payload) and int publish (topic, payload, length). For more information about function overloading, see: (add reference here)