Experienced as Hadoop, expertise in providing end to end solutions for real time big data problems by implementing distributed processing concepts such as map reduce on Hadoop frameworks such as HDFS and Hadoop Ecosystem components.

Experienced in writing live Real-time Processing and core jobs using Spark Streaming with Kafka as a data pipe-line system.

Used Kafka Streams to Configure Spark streaming to get information and then store it in HDFS.

Worked extensively with Sqoop for importing metadata from MySQL and assisted in exporting analysed data to relational databases using Sqoop.

Created Hive tables as per requirement, internal and external tables are defined with appropriate static and dynamic partitions, intended for efficiency and bucket to and write HQL scripts to perform data analysis.

Worked on Hive optimization techniques using joins, sub queries and used various functions to improve the performance of long running jobs.

JavaScript Requires
Sorry this site will not function properly without the use of scripts.
The scripts are safe and will not harm your computer in anyway.
Adjust your settings to allow scripts for this site and reload the site.