If it helps someone.

Menu

As part of series on Arduino development, this article will give you and an Introduction to Arduino hardware and software. Subsequent posts will cover these topics in detail

What is Arduino

Arduino is an open source project, company that designs and manufacturers micro controller based kits for various purposes. Most of the boards designed by Arduino are open source. So, you will find various manufacturer producing these boards. There are different types of boards from Arduino. The company also builds tools for programming these micro controllers. This with the support of strong community makes the development easy for everyone.

Arduino has different types of boards. They are all good for development work before you actually use the corresponding micro controller on a production board.

In this article I will explain how to send HTTP requests with ESP8266 module. We will see how to send an HTTP GET and a POST request. I will also introduce a new library for processing JSON. Lets start.

Setup wifi

Include the following header. This library has all the required APIs for connecting to a WiFi network.

HTTP Server

I have created a simple servlet application. It has just two end points. Details of the end points will be explained in the following sections. But you can use any other as you wish. Servlet code will also be attached with this post.

HTTP Requests – Get

ESP8266 for Arduino also comes with a HTTP client library ESP8266HTTPClient. Add the following include in your code

Thanks to few wonderful people out there. Now we can run Arduino programs on ESP8266. Arduino IDE for ESP8266 is nothing but your favorite IDE with additional support for ESP8266 boards. In this article I will explain how did I run my first Arduino program on ESP826 module. This is a NodeMCU module. But we can use any other model.

Download the latest Arduino (version > 1.6.x). I have used 1.6.8

Run the Arduino IDE and open preferences IDE. In Additional Board Manager URL, enter the URL http://arduino.esp8266.com/stable/package_esp8266com_index.json

Open the Board Manager, you will see an option esp8266 by ESP8266 Community as shown in the following picture.

Click install after selecting this option. Download and install will take approximately 2 minutes.

Once upload is done select the board NodeMCU 1.0(ESP-1E Module) or whatever is appropriate for your board.

We will also get example programs to run. Lets try a simple Blink example. Open the example by selecting Examples > ESP8266 > Blink and click Upload. This example will blink an onboard LED on NodeMCU board. You can also connect an LED to the PIN

Now lets do some programming on the device. You will find lot of examples on internet. In this demo, we will setup Wifi AP with ESP8266 and wait for connections. NodeMCU firmware runs init.lua when the module is rebooted. That is why you see a message when you connect ESP8266 with NodeMCU firmware just flashed.

lua: cannot open init.lua

Open the ESPlorer IDE and on the left side, start entering the following piece of code.

This will just setup the ESP8266 in AP mode. Click Save and save the file as init.lua. ESPLorer will now transfer the file to the module and run it. You can scan for wifi network using your mobile or any other device. It will show a wifi network with SSID ESDEMO. You can connect to the network use the password welcome1234

Some tips

1. If you have a large program, you might see error messagaes for insuffecient memory. In such cases, split your program into multiple files. These additional files should have functions only. Then you can load these files only when it is required.
2. Use a custom build of NodeMCU. You can do it at http://nodemcu-build.com/ instead of having a build setup of your own. You will be able to select the modules that you want and exclude others. This saves a few KBs for your program.

ESPlorer IDE for ESP8266

If you are familiar with LUA programming language, then there is an IDE to work with ESP8266, ESPlorer IDE. It is written using java with launcher available for MS Windows platform as well as for Linux. It has lot of features useful when you are developing with ESP8266. This IDE uses serial port to talk to ESP8266. So, you can use the connection that I mentioned in my previous articlewith ESP8266 or use module like NodeMCU board with ESP8266. In this article I am using one such board. See the picture below.

Download the ESPlorer IDE from http://esp8266.ru/esplorer/ mentioned above and launch the executable ESPlorer.bat or directly run the jar file ESPlorer.jar. This will open a nice UI shown below.

If you have connected your ESP8266 to a serial port, you will see it in list of ports at the top. Click the ‘Open’ button. You will see a message like

PORT OPEN 9600

But no other message. Now Reset the ESP8266, you will see the familiar welcome message from ESP8266.

That means, your ESP8266 is ready for your programs.
The firmware present on the device can accept any LUA code on the serial port. Lets try it. There is a small editable dropdown at the bottom which has a button that says Send. It will have list of available functions already defined. Lets try one of them. Select the item =node.heap() and press the Send button. This function will return the free heap space. You will see something like

ESP8266 wifi module is a low cost (less than $5) , self contained module with integrated TCP/IP stack. The module comes with firmware using which the module can be controlled with simple AT commands. The module has its own application processor. So, instead of using AT commands, if we have a mechanism by which we can program the device itself, it will help in offloading some work which otherwise would be done by an additional microcontroller. ESP8266 firmware SDK is available and anybody can write their own firmware. To make the job easier there exists a firmware by NodeMCU, using which we can program ESP8266 in LUA programming language. In this article I will explain about using ESP8266 with NodeMCU firmware. This will be the basis of all other ESP8266 tutorial in my blog where I use NodeMCU.

Websocket protocol uses a server, which is a simple TCP application (like a webserver) and a client, which may be a browser. There are lot of examples on how to use websockts in javascripts. In this tutorial, I will explain how I wrote a simple Websocket server application. All I had to do was to follow the protocol specification.

Websocket server has two steps when communicating with a client.

Handshake

Data communication

Websocket Handshake

Messaging used for handshake will look similar to HTTP with headers and syntax. Here is the sample request header

Every header will enter with a carriage return and new line. Don’t forget to add carriage return and new line. What is important in this handshake is the Sec-WebSocket-Key in request and Sec-WebSocket-Accept. Rest of the items can be hard coded. As a response to the key sent by client, the server prepares a Sec-WebSocket-Accept using the following logic.

This is a continuation of my previous article Moving data between HBase and MySQL. In this article we are adding one more database, which is mongoDB. I will be using the same base framework and interfaces. The example shows how to export mysql to mongodb.

1. Introduction

mongoDB is a leading noSQL document database. It stores data in document format. If you are familiar with JSON structure, then you will understand exactly how data is represented in mongoDB. For example

{
id : "1234",
name :
{
first_name : "ABC",
last_name : "xyz",
}
}

mongoDB uses a type of JSON called BSON. There is no concept of tables in mongoDB. Instead it uses collections. In mongoDB, you need not create objects like collections, db, explicitely. First usage of the same creates it. For installation details please refer getting started from mongoDB.

For using examples, from this article, you need java driver for mongoDB. This can be donloaded from here .

1. Introduction

In this article I will explain about moving data from HBase to MySQL. HBase an unstructured database and MySQL, a structured database. With very little changes, it can be used to move data from MySQL to HBase

HBase is a Hadoop database, that stores data as key value pair. HBase can store billions of rows X millions of columns. It is built based on Google BigTable model. Hbase has its own API to retrieve data and write data. For more details on Hbase have a look at http://hbase.apache.org/.

MySQL is a most popular open source database. It is a RDBMS similar to Oracle or Microsoft SQL Server. MySQL provides a JDBC access to data. You can get more details http://www.mysql.com/.

Because the interface to access the two databases are different, it makes the process of moving data from Hbase to MySQL a little bit complex. I have designed everything in such a way that, the application does not identify if it is JDBC based access or custom API. That makes it easy to add more and more types of databases without breaking applications. SO, moving from Hbase to MySQL or MySQL to HBase does not too much of code change.Continue reading ‘Move data from Hbase to MySQL’ »

I was not able to install hadoop without spending at least two days and re-install of OS few times. It may not be the case always. I just wanted to run a few MR jobs and access HBase and HDFS. Instructions available online for hadoop hbase installation were mostly about installing hadoop on a production or clustered environment. I have never found a way to install all these three components for testing purpose. I was using Cloudera on ubuntu. Instructions may be useful for a production environment. But was never easy for me. So, I decided to download and install Hadoop and Hbase and configure them. Here is what I have done.