Elasticsearch-Hadoop

Best of Two Worlds for Real-Time Analysis

Connect the massive data storage and deep processing power of Hadoop with the real-time search and analytics of Elasticsearch. The Elasticsearch-Hadoop (ES-Hadoop) connector lets you get quick insight from your big data and makes working in the Hadoop ecosystem even better.

NewSecure your sensitive settings with a keystore and leverage Elasticsearch as a dead letter handler destination. Read More

Interactive Analytics on Your Hadoop Data

Hadoop shines as a batch processing system, but serving real-time results can be challenging. For truly interactive data discovery, ES-Hadoop lets you index Hadoop data into the Elastic Stack to take full advantage of the speedy Elasticsearch engine and beautiful Kibana visualizations.

With ES-Hadoop, you can easily build dynamic, embedded search applications to serve your Hadoop data or perform deep, low-latency analytics using full-text, geospatial queries and aggregations. From product recommendations to genomic sequencing, ES-Hadoop opens up a new world of broad applications.

Seamlessly Move Data Between Elasticsearch and Hadoop

Live decision making only happens with lightning fast data movement. With dynamic extensions to existing Hadoop APIs, ES-Hadoop lets you easily move data bi-directionally between Elasticsearch and Hadoop while exposing HDFS as a repository for long-term archival. Partition awareness, failure handling, type conversions, and co-location are all done transparently.

Natively Interface with Spark and Friends

ES-Hadoop offers full support for Spark, Spark Streaming, and SparkSQL. Additionally, whether you are using Hive, Pig, Storm, Cascading, or standard MapReduce, ES-Hadoop offers a native interface allowing you to index to and query from Elasticsearch. No matter what you use, the absolute power of Elasticsearch is at your disposal.

Your Data is Secure Everywhere

ES-Hadoop ships with all the security features you'll need, including HTTP authentication and SSL/TLS support, to securely move your data between your Hadoop and Elasticsearch clusters. It also works with Kerberos-enabled Hadoop deployments.

Works with Any Flavor of Hadoop

We are official partners with Cloudera, MapR, Hortonworks, and Databricks, so whether you’re using vanilla Hadoop or any other distribution, we’ve got you covered. ES-Hadoop has been certified with CDH, MapR, and HDP.

Get Started with ES-Hadoop

ES-Hadoop is a single binary with no extra dependencies, so distributing it within your cluster is simple and fast.