Author: Jeremiah Evans, Senior Applications Developer

Eating the Elephant - Velocity

So far in this series we have introduced the “3 V’s” of Big Data: Volume, Velocity, and Variety, focusing on Volume. Recall that in the introduction we encouraged you to ask yourself, “What decisions could I be making better if I was seeing the current state of the system, and not the state from yesterday, or a week ago?”

The landscape of data-generating devices is growing exponentially. Some of these sources include:

While we will talk of the increased variety of these feeds in the next installment, today our attention is on the velocity with which we can now collect data. Not only is data coming in more quickly as more users interact with more systems, more often, but that data can also be moved from the source to our data stores in almost real time.

Chasing the Lead

Journalists are always chasing leads. What tip is going to get them that exclusive scoop? What random meeting is going to lead to the breaking headline? Yes, reporters have to be good writers, and need good editors, but it doesn’t matter if your news isn’t relevant or up to date.

The same thing is true of the reporting you do internally. Are the reports driving your decision making process based on the latest data and insights, or are they aggregations that are already a month old? You may have models and forecasts based on solid analysis of your historical data, but how much more helpful would those predictions be if you were able to respond to the current state of your business, not the state of it a month ago, or 6 months ago?

What will you do to make sure your reports are driving business value?

Know the Market

Knowing the market and where you fit in it is vital to a thriving business. Traditional marketing campaigns involve a lot of guesswork and crossed fingers. Buy an ad, wait 6 months, see if any of your new customers say they saw your billboard. With modern digital ad campaigns you don’t need to wait half a year to see your data, so why should you wait half a year to get business value out of it?

Get the Most Out of Your Data, NOW

We are in the middle of a paradigm shift in the way we process data. Data is being generated faster, it’s moving faster, and we need to process it faster. But velocity isn’t just about the raw stats like Tweets per minute. The data you’re collecting is getting closer and closer to real-time, and that real situation is changing every bit as fast as the data comes in.

Nightly loads and morning reports aren’t going anywhere overnight, but we are seeing more and more businesses move towards streaming data models that allow them to respond to new information before it’s even landed into their data store. What business decisions could you be making right now, that with your current platform you won’t be making until next week, next month, or even next quarter?

What's Next?

Data is being generated and collected faster than ever before, and that is a trend we expect to see continue. It’s time to start implementing a strategy to respond to the data quickly. Be sure to stay tuned for our next Big Data “3 V” article where we will take a look at the Variety of data, and how our phased approach to a Big Data implementation can help you get from where you are to where you want to be. Data is changing, and you can change with it - one piece at a time. Bon appetit!