Start a KeyLines trial

Elasticsearch is part of the open source portfolio known as the Elastic Stack. It’s a fast and scalable search engine built on the Apache Lucene software library. Lucene is a high-performance technology for searching and indexing data, but it can be very complex to use. Elasticsearch makes the power of Lucene more accessible by pre-selecting sensible defaults and providing a more intuitive REST API.

The power and simplicity of Elasticsearch makes it popular with organizations that need to search very large volumes of data, including Facebook, Wikimedia and Stack Exchange. It supports near real-time data searching on a petabyte scale, using a system of sharding and routing to scale outwards from the beginning.

In this tutorial, we walk through the steps of connecting KeyLines to Elasticsearch. This guide assumes a basic understanding of Javascript and explains how to use Elasticsearch as the backend for your KeyLines graph visualization application. We’ll use two technologies from the Elastic Stack:

Logstash – a tool for streaming, munging and loading data into Elasticsearch

Visualization architecture

KeyLines is a JavaScript file that’s deployed to a web server. A chart element (powered by either HTML5 Canvas or WebGL) is then embedded into a web page for the user to access. As the user interacts with the chart, events – like click, double-click, drag – are raised. You can then customize the response to these events, creating a custom user experience.

KeyLines’ integration architecture is very simple. Elasticsearch provides a REST API and works with the JSON data structure:

The KeyLines – Elasticsearch visualization architecture

In this scenario, KeyLines runs in the web browser. Every time a user clicks, hovers or interacts with the graph data on a chart, it raises an event. When the user requests data, the browser application sends an AJAX request to the Elasticsearch REST API. Elasticsearch returns the data as a JSON object that KeyLines styles and displays on the chart.

Step 1: Download prerequisites

To build a KeyLines-Elasticsearch integration, download the following:

Keylines.js contains components to build the visualization part of our application. Make sure to include the assets folder from the KeyLines download.

Index.htm contains the KeyLines chart and some customization code to describe the general UI.

Step 3: Load data into Elasticsearch

If your Elasticsearch instance is populated already, you should skip this step.

For this example, we used a dataset of movies and actors taken from IMDB.com and saved as JSON files.

Denormalize the data to make graph queries faster and more efficient. We’ve done this for you already. The denormalized data is in the JSON files you downloaded with the Elasticsearch.zip file. For more information, see Denormalizing Your Data.

Load the data into your Elasticsearch instance.

Write configuration files to run Logstash and import our dataset. They look like this:

Next steps: Extending the UI

As you worked through the steps, you’ll have noticed some additional KeyLines controls such as running automatic layouts. This is just one way you can extend the functionality of KeyLines to help users explore and understand their data.

KeyLines offers a wide range of ways to customize your final application – far too many to outline here. Instead we recommend taking a detailed look through the KeyLines SDK documentation, especially the API reference and sample demos.