I'm the founder of SiliconANGLE.com, a Silicon Valley tech blog covering the intersection of Computer Science and Social Science. It's also the home of SiliconANGLE.tv and #theCUBE video broadcasting at top tech events. We love the enterprise tech marketplace.
I live in the heart of Silicon Valley (Palo Alto) where I research and report on the top trends in technology entrepreneurship, venture capital, consumerization of enterprise and emerging technology innovation.

Big Data Is Big Market & Big Business - $50 Billion Market by 2017

We all know that Big Data is the hottest sector in IT at the moment. I’ve been covering this market since it started so I’m so excited by the recent new market size numbers. But the question on everyone’s mind is just how big is the Big Data market? Open source analyst firm Wikibon provided the answer this week in a groundbreaking report that pegs the current Big Data market at just over $5 billion in total factory revenue.

Further, Wikibon forecast the Big Data market will grow at an astounding CAGR of 58% between now and 2017, hitting the $50 billion within five years. Well, that should quiet the doubters that claim Big Data is all hype and no substance.

The fact is, as Wikibon’s report confirms, vendors from whales like IBMIBM and HP to pure-plays like Vertica and Cloudera are bringing in significant revenue today helping enterprises, governments and healthcare organizations process and make sense of the torrents of unstructured data flowing from mobile devices, sensors, social media and other sources. And the business of Big Data is only going to grow as the volume of data increases.

Consider that today Big Data technologies like Hadoop are mostly in production at Web and online gaming companies, large financial services firms and banks, and online retailers. But Big Data will and already is making its way to the “traditional” enterprise, both large and small. Even a small, independent retailer, for example, can and should make use of Big Data technologies to mine social media and other data to better understand what customers are saying about it, to keep tabs on the competition and to stay ahead of emerging trends.

“Big Data is the new definitive source of competitive advantage across all industries,” writes Jeff Kelly, Wikibon’s Big Data analyst and lead author of the report. “For those organizations that understand and embrace the new reality of Big Data, the possibilities for new innovation, improved agility, and increased profitability are nearly endless.”

Another important point from the report to highlight is that, while Hadoop may be the poster child of Big Data, there are other important technologies at play. In addition to Hadoop, the open source framework for distributing data processing across multiple nodes, these include massively parallel data warehouses “that deliver lightening fast data loading and real-time analytic capabilities,” as the report states; analytic platforms and applications that allow Data Scientists and business analysts to manipulate Big Data; and data visualization tools that bring insights from Big Data analysis alive for end-users.

Where is all this innovation coming from? According to the report:

Of the current market, Big Data pure-play vendors account for $300 million in Big Data-related revenue. Despite their relatively small percentage of current overall revenue (approximately 5%), Big Data pure-play vendors – such as Vertica, Splunk and Cloudera — are responsible for the vast majority of new innovations and modern approaches to data management and analytics that have emerged over the last several years and made Big Data the hottest sector in IT.

Wikibon even broke down Big Data pure-play revenue by vendor, which provides some interesting insights on which vendors are really driving the market:

Then there’s services, perhaps more important in the Big Data space than in most other IT sectors. The reality is that deploying and managing technologies like Hadoop take highly skilled engineers for which the talent pool is lacking. Same goes for the Data Scientists with a combination of math, statistics and programming skills needed to analyze Big Data. As a result, there is a huge opening for vendors that can help enterprises fill the Big Data skills gap with training, technical and professional services.

Post Your Comment

Post Your Reply

Forbes writers have the ability to call out member comments they find particularly interesting. Called-out comments are highlighted across the Forbes network. You'll be notified if your comment is called out.

Comments

Wikibon have got the percentage wrong for Think Big Analytics, even the big data numbers look wrong for SAP , at times when the press is buzzing that they have already done $100 revenues for SAP HANA, their big data offering.

We don’t consider 100% of HANA as big data…not even close. Jeff Kelly can provide more insights on the assumptions and you should follow up with him on Think Big – we always welcome input to fine-tune the figures.

Great and timely post. I want to reiterate the importance of a key point you made “talent pool is lacking. Same goes for the Data Scientists with a combination of math, statistics and programming skills needed to analyze Big Data”.

Big Data offers companies of all sizes a unique opportunity to reshape the future of work by empowering the edge and the middle. It can give ‘working smart’ a whole new definition. But, the skills to leverage this technology are lacking in organizations.

Vendors must incorporate into their solutions the UIs and interpretation tools to enable adoption and, along the way, teach users the underlying methodologies. It not enough to leave it to ‘training ….and professional services”. To do so will only repeat the painful history of BI and analytics.

I think the skills gap is a major adoption barrier and one that the industry needs to focus on. Jeff Kelly and I did this video right after EMC announced its data science certification program. It’s a big issue that more vendors need to address:

Thanks Christine: I agree with you 2000%. What’s so compelling about Big Data is that anyone can innovate with it. You don’t have to be a alpha geek programmer to create value around data. Even though the market is currently in this “alpha geek” phase it will soon be moving to a model of programming “made simple” via abstractions so folks with domain expertise in areas can innovate and program great valuable solutions. So exciting.

For me it’s a great time since my blog SiliconANGLE.com was the first tech blog to have the editorial focus of cloud mobile and social in 2008-09; it continues today. Cloud mobile and social is the hottest areas of investment and innovation.

While Big Data is indeed important, it seems the most important aspect of using the data is asking the right questions and implementing them in meaningful ways to best serve the client. The Internet itself is a sea of information and I would suspect that few (if any) companies have actually seen tremendous advances for their efforts moving into the Online space mainly because: 1.) the way the data is aggregated, 2.) the actual relevance and usefulness of said data, and 3.) because of the depth of actual thinking done by the data miners.

So even if this becomes a $50 billion industry (which is probably will due to the companies invested in developing such a service) I would still wonder how useful such information will truly be. I guess only time will tell.

While Big Data is indeed important, it seems the most important aspect of using the data is asking the right questions and implementing them in meaningful ways to best serve the client. The Internet itself is a sea of information and I would suspect that few (if any) companies have actually seen tremendous advances for their efforts moving into the Online space mainly because: 1.) the way the data is aggregated, 2.) the actual relevance and usefulness of said data, and 3.) because of the depth of actual thinking done by the data miners.

So even if this becomes a $50 billion industry (which it probably will due to the companies invested in developing such a service) I would still wonder how useful such information will truly be. I guess only time will tell.

Hi Steve: with all due respect the list of vendors are listd there. Due dilligence was over a year of work by Wikibon and SiliconAngle. We cover the details of each vendor at SiliconANGLE.com and Wikibon.org.

For more detailed info on the market sizing read this post (see link below)

http://wikibon.org/wiki/v/Big_Data_Market_Size_and_Vendor_Revenues

On due diligence it’s the best research report on the market for big data market sizing. Go look around then come back. Very groundbreaking. Wikibon will add to it as the market matures and changes even more.

I just started my company WayZen Inc., a business application design and development company. I am currently working on a Content Management System focused on Big Data management, Apache Hadoop, MapReduce, Zookeeper, Hive, HBase, Pig, Cassandra, etc. I started my company out of my own pocket and on a shoestring budget. If I told you the rest of how this company is starting, you wouldn’t believe me at this time, so I kindly ask you to wait for the movie and book.

I am waiting to complete the two applications that I am building to seek funding, in the mean time, I am working and doing web design on the side to put money into building WayZen Inc.

Bootstrapping may not be easy, but at least I know what I can do, and I find work-arounds to accomplish the task before me and set milestones as I go. I believe this is making me a better owner and president of my company, because I know how to work with an extremely small budget. For example I purchase software, hardware, and take full advantage of Cloud Service Provider (CSP) services, such as Google’s – Google Apps for Business, Yahoo! Small Business suite, Microsoft Bizspark, Microsoft’s Windows Live, Box.net, etc.., in addition, I use a lot of Open Source software, IDE’s, and SDK’s.

I once seen a YouTube video of a man in Eastern Europe, who apparently had very little money, but he was so determined to build a computer, and teach himself programming that he put together an ad-hoc computer without a case and it worked! I felt sorry for him and wanted to send him one of my laptops because anyone with that drive and determination, coupled with innocation and creativity should be given all the assistance they can to achieve their goals, and expolore their creative potential, think of the accomplishments he could achieve.

We are all born with the greatest computers and software on the planet, the human brain and mind. All we have to do is use them in a positive and creative way, and enhance them with education and training. If one cannot afford to go to a college or university, there is nothing stopping them from going to the library and from purchasing the things they need to teach themselves, after all, look at Mr. Warren Buffet.

I take my inspiration from the Wright Brothers, they failed many times, were laughed at, were ignored, and then some even fought with them, but in the end, they won! So not matter what, never give up on yourself, your goals, your dreams and your hopes. Keep getting up when you are knocked down and keep moving forward, and I believe you will WIN!

Kindest regards, Von’Victor Rosenchild Founder, President and CEO WayZen Inc. Brooklyn, New York

Great to see some numbers since more than a few people have said big data is all hyped up. But these numbers show it is serious. Your point about expecting it will be accessible by normal folks with a reasonably short time is one I have heard a couple of times, and one I have also hear a backer of Mu Sigma (which I wrote about last week) disagree with.

Great post and our investors and prospective investors were delighted too. This is excellent research and I appreciate the length of time and effort that wen into this. What is seemingly missing from every post on big data is how important it is to interpret data for customers. Our clients are unable to interpret insights and identify actionable tactics. We have been hand-holding for years, and moving data to insights to ideas to dollars. Our belief is that data needs a vertical solution; one that leverages a practice skill in interpreting big data with domain expertise. Until data is soloed, it’s not contextual, relevant, or useful in most cases. We view the data world in a series of verticals rather than one big horizontal playing field.

I am expecting a lot of mindless overinvestment in capacity followed by a bubble burst. Which means the more interesting number is the true value of the investment after the building boom settles. I suspect it will be much less than $50 billion to the companies that buy, but the overall economy will bank a lot of value that amounts to a lot more than $50 billion.

I would love to see a follow up article to this suggested $50b suggestion and Ventaktesh’s opinion I bet, a year later, is exactly right – lots of over investment in a bubble market that has most probably burst. To find out definitively what the market value is would be a great insight!

Hmmm, it seems odd that Vertica is characterized as the leading pure play vendor, even though it is wholly owned by HP and Aster Data is owned by Teradata. Yet Netezza (bought in 2010 by IBM), which invented the Data Warehouse Appliance space and clearly has the largest market share and revenues over all other DWA players is not even mentioned.

Telecom Austria in Bulgaria (Mtel) has threatened me with cutting off my fingers if I do not participate in their cartel in the telecommunication industry in Bulgaria. It have been done through involuntary implanted small device similar to hearing apparatus in my sinuses, connected to nerves, controlled wireless with their network. Many politicians and Government servants in Bulgaria﻿ are controlled in a similar manner. -Detsche telecom was fined 100 mill for bribing a Balkan country regulator, where the other palyer is Telekom Austria. -The corruption scanal in Telekom Austria led to the resignation of a former Chacellor. http://convergingtechnology.eu/threats-for-developing-countries/