Bringing time-series data to life with KeyLines

In the beginning there were relational databases. Then came a wide variety of NoSQL databases. And most recently, time series databases (TSDBs) are making an impact.

TSDBs are purpose-built to manage high volumes of indexed, chronological data points. With the rise of IoT and the cloud, the quantity of real-time data that exists has dramatically increased. This includes anything from stock market prices to meteorological records, bank transfers to IT server memory usage, telephone calls to air quality levels.

Organizations are recognizing the value in monitoring and analyzing their data points to help understand the present and predict the future. It’s not surprising that DB-Engines ranked TSDB as the fastest growing database type for the last two years running.

The best way to find deeper insight in time-based data is by visualizing it using KeyLines’ advanced time bar features. In this blog post, we’ll cover two useful topics:

how to visualize a telecoms network of SMS traffic to quickly identify issues that need attention. This is for anyone who wants to visualize time-based graph data, no matter what kind of database it’s stored in.

Why use a TSDB?

The main advantages of TSDBs is their ability to handle data that’s constantly scaling. In real-time monitoring, new data is continuously being added – sometimes at rates of millions of data points every second. Relational and NoSQL databases aren’t designed for that.

It’s easy to carry out data aggregation, so raw data can be routinely downsampled or summarized for analysis purposes.

They support flexible data retention policies, so it’s easy to store valuable, downsampled data for long periods, and automatically delete details about every data point more frequently.

Working with InfluxDB

We’ve chosen to integrate KeyLines with InfluxDB, the open source time series database from InfluxData. This popular, open source TSDB has a built-in HTTP API, and uses SQL-like queries to interact with the data.

We’ll go through the integration details later, but first let’s understand our dataset.

Visualizing the Nodobo dataset

We’re analyzing Nodobo data, featuring the anonymized text message records of 27 high school students over a five month period.

The time series database contains over 11,000 SMS events, with a sender, receiver and timestamp for each data point. For the purpose of this example, we’ve added a ‘slow’ flag for messages that take over three seconds to arrive:

Styling items in KeyLines

Cellphone nodes are sized according to the number of slow text messages they’re associated with. KeyLines achieves this by applying weightings by degree, a useful Social Network Analysis (SNA) centrality measure.

Let’s get back to the time elements of our visualization, and what it reveals about our data.

Identifying patterns with selection lines

The time bar histogram shows the total value of events that occurred at specific times. It’s an easy way to spot changes in the volume of activity, something that’s useful when you’re trying to understand the nature of your time-based data.

Overlay the histogram with selection lines and you can compare values for specific chart items against total values. They’re ideal for showing trends and identifying outliers or unusual patterns.

A spike in the selection line shows the exact time that a slow SMS occurred

In our telecoms example, knowing the precise time an error occurred is key. Cross-referencing this data with details of network upgrades or known outages might be useful. It can also contribute useful insight to help with future network performance management.

If you’re keen to know what’s happening under the hood, we’ll walk you through the integration basics.

Integrating KeyLines with InfluxDB

First we need to send an AJAX request to InfluxDB’s /query HTTP endpoint for the time period we’re focussing on.

You’ll notice this code will add some nodes more than once, but that doesn’t mean our chart will show duplications. KeyLines is clever enough to discard duplicate nodes that have the same id.

That’s it. The array of nodes and links can now be loaded into the KeyLines chart, and the same data loaded into the time bar.

You can then bind any chart event to the time bar, allowing you to support the filtering and selection actions described in this example.

Every time we filter this way, KeyLines visualizes the data points between two specific timestamps. For example:

SELECT * FROM “text_msg” WHERE time >= ‘2010-09-19T21:07:08Z’ AND time <= ‘2010-11-15T01:00:14Z’

To optimize performance, queries will only bring in new data that doesn’t already exist in the chart – it isn’t loading the entire dataset every time. This is particularly important when you’re dealing with the potentially huge datasets typical of TSDBs.

Take time out to try KeyLines

Whether you’re an experienced TSDB user, or someone who’s keen to interact with their time-based graph data, KeyLines has the time bar features you need.

We’ve covered some of them here, but there’s also the ability to integrate with advanced geospatial features, use multiple time bars at different levels of granularity, and use continuous animation to demonstrate how networks evolve.