Break

Mysql To HDFS : How To Import Data Using Sqoop

In this post, Hadoop developers will make you learn about the procedure to perform Sqoop Installation and import data from MySql to HDFS. Make sure you understand every point discussed here.

If you have any doubt, ask developers. They will respond to your query soon. Since last few years most of the organizations have made a decision to use the BigData as a component to perform operations on the data.

As the data is growing exponentially it has become essential to transform the Traditional applications to BigData compitable application.

The traditional applications uses the relational database and RDBMS for interacting with the database, In order to analyse the database using Bigdata components, it is required that the data should be available to the components like HDFS, Hbase, Hive, Pig.

In this article I am going to explain one of the BigData component i.e Sqoop, which is used for Import data from RDBMS to HDFS.

After downloading, extract the package using following command in terminal. I am assuming that the package was downloaded and saved in your Downloads folder.

tar -xvf ~/Downloads/sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz

In next step, create a folder for Java JDK files and folder using following command.

sudomkdir -p /usr/local/sqoop/

In next step, move all the files and folders to new location by running following command.

sudo mv /usr/local/sqoop/sqoop-1.4.4.bin__hadoop-2.0.4-alpha

In next step, export SQOOP_HOME and PATH variable by adding following lines to the end of $HOME/.bashrc file of user. If you use shell other than bash, you should update its appropriate files instead of .bashrc.

I used gksugedit $HOME/.bashrccommand to edit file and appended following lines.

So once the command is fired, it will begin the MapReduce job for Bulk import and at the end of the execution you can find the datafile present in HDFS,

My final output file in HDFS looks like,

Now, your hadoop-sqoop is running and you can import as well as export data using Sqoop.
Follow every instruction shared by proficient hadoop developers in this post. Once you understand the procedure to perform Sqoop Installation and import data from MySql to HDFS, perform it and share your experience with the developers.

This article has been written by Ethan Millar who are working with Aegis Softtech as a technical writer since more than 5 years. The main object to write this article is to Perform Sqoop installation And Import Data from Mysql to HDFS.