In Big Data era, insight is power

As we stand at the dawn of a new era of computing, it's time for a smarter approach to storage.

There is no magic bullet to
solving the myriad challenges brought on by the data deluge. No single
answer or technology will enable organizations to achieve the bevy of
business goals we all seek—greater efficiency, higher performance,
improved management, and lower costs. The information volumes are
growing too large, too fast, and will only continue climbing.

Each of the most common storage technologies available today—disk, solid state drives (SSD), and tape systems—can play individual, yet complimentary strategic roles in any storage infrastructure. It is possible to deploy two or even one of the three technologies exclusively—and let’s face it, many organizations do just that (and some manufacturers actually encourage it). But it is not the most strategic of approaches. No, selecting a single technology system for all your storage needs would only limit the potential benefits and require tradeoffs and compromise.

Conversely, a storage infrastructure that leveraged all the available systems strategically, such as, SSDs for fast data access, disk for enterprise storage, and tape for back-up and recovery, would begin to reap efficiency and performance gains almost immediately. Add to that mix advanced technologies such as those that automatically move information to the most appropriate drive—be it disk, SSD, or tape—and the infrastructure starts becoming strategic.

Additional technologies such as real-time compression of active data, that automatically compresses data before storing, increasing capacity on-the-fly by up to five times, and the storage infrastructure becomes a critical business ecosystem unto itself. When finished the ecosystem is self-sustaining, highly efficient, affordable and capable of providing a foundation for analytics and insight. That’s smarter storage.

Albert Einstein’s view of information was as perceptive as it was prophetic. Einstein lived until 1955, just three years after the first mainframe computer, the IBM 701, shipped with the first magnetic tape storage system, the IBM 726. That solution launched the modern digital computing age and subsequently, the information revolution. Today, almost 60 years after Einstein’s passing, man is still wrestling with the challenges posed and provided by more readily available and accessible information.

And as we stand at the dawn of this new era of computing, it’s time for a smarter approach to storage. To be smarter about what information we store, where we store it, how we store it and how we retrieve what we need, when we need it. The advances are available today and growing in sophistication all the time. Whether we choose to adopt approaches like these is up to us. But the sooner we do, the sooner we can leverage what is, instead of spending resources searching for what we think should be.

Brian Truskowski is the general manager of IBM System Storage and Networking, IBM Systems and Technology Group.

Ignoring the fact that there are more people alive today than the total of all humans ever living on planet Earth ever, and of course that means more data ...
Whenever I read about "big data," I can't help but think that there's always been "big data." The only difference is, since this "big data" was not stored electronically, nor was the vast majority of it obsessively kept in safe storage, no one ever worried about accessing most of it.
I mean, did people always save all their personal letters before? Not me, for sure. And yet now, if your personal letters sit in your hard drive, most people probably feel compelled to move them to long term storage, along with all their other files, for safekeeping. In case their hard drive crashes. How many people go through every single file, to see whether it makes sense to keep it?
And once you have these stored electronically, you feel you should be able to find anything you're looking for, even though no one would have obsessed over this previously.
The "personal file" example is, of course, just an example. All you have to do is look at your typical enterprise shared drive to see that there is a huge amount of "who cares" material in there. You know, a purchase order from 20 years ago. That presentation you never actually gave, about stuff that is totally obsolete today anyway. Back when, when you moved your office, most of that stuff had to be tossed. Now, instead, it gets meticulously saved in some long term storage facility.
Not saying this is bad, not saying we shouldn't be looking to ways to sort through all this stuff, but what I am saying is, it's really not a new problem. It's a problem that always existed, only now we're worrying about it.

Big Data is more than a trend, it is a new way forward. It allows organizations and governments to operate more efficiently than ever before. It allows them to make better predictions through better analysis. None of this replaces the human. We will still be needed to make the risky call based on human intuition that a computer just can't make. But Big Data allows us to be wiser and better prepared as we make those risky calls.
As Brian pointed out at the core of Big Data is the storage infrastructure. Big Data will not be a one-size fits all infrastructure but a cast of storage components all tuned to perform a certain function. Unlike other initiatives that storage supports Big Data requires "everything" capacity, performance and economical long term retention.
Like a symphony what is needed is a conductor to manage data flow and bring order. It should also automate as much of the work as possible so that human intervention is kept to a minimum. This requires a company with a broad portfolio of storage products and a history of automation through analytics.

In his book "Reinventing Discovery" Michael Nielsen defines data-driven intelligence as the ability of computers to extract meaning from data. Now he compares data-driven intelligence to human and artificial intelligence, but for our purposes we can contrast it to other types of IT software intelligences.
Among those are application-driven intelligence which codify business processes such as ERP and financial applications that are the bedrock of modern IT, infrastructure intelligence, such as operating systems and middleware, and communicating intelligence, such as e-mail, tweeting, texting, and FaceBook. All are vital and growing, but the one that is now attracting our intention more and more is data-driven intelligence.
Data-driven intelligence applications, of which Big Data is a focal point,are created and managed to fit the needs of the data which may be (and likely are) independent of the application that created the data. No, this is not new, but the growth rate and the value of the analyses that are associated with the data surely are.
And where there is data there is storage. Managing that storage for performance, affordability, and as Brian points out most importantly insight is going to become more critical to all enterprises, both public and private. That presents a challenge to storage in a number of ways as storage is inextricably intertwined with data-driven intelligence . Those that make the connection and do it right will reap the benefits. Those locked into a price per GB mentality will not. It should be fun watching what happens and who the winners and losers are.