Introduction

The hype surrounding Big Data, which showed no signs of abating in 2012, now has big dollars backing it up. Factory revenue generated by the sale of Big Data-related hardware, software and services took a major step forward in 2012, growing by 59% over 2011(a).

The total Big Data market reached $11.59 billion in 2012, ahead of Wikibon’s 2011 forecast. The Big Data market is projected to reach $18.1 billion in 2013, an annual growth of 61%. This puts it on pace to exceed $47 billion by 2017. That translates to a 31% compound annual growth rate over the five year period 2012-2017.

Growth Drivers and Adoption Barriers

The growth rate of Big Data revenue in 2012 was due to a number of factors, including:

An increased awareness of the benefits of Big Data as applied to industries beyond the Web, most notably financial services, pharmaceuticals, and retail;

Increasingly sophisticated professional services practices that assist enterprises in practically applying Big Data hardware and software to business use cases;

Increased investment in Big Data infrastructure by massive Web properties – most notable Google, Facebook, and Amazon – and government agencies for intelligence and counter-terrorism purposes.

In the enterprise space in particular, the combination of a better understanding of the use cases for Big Data and more mature product and service offerings resulted in a significant percentage of Big Data early adopters graduating from small, proof-of-concept projects to large-scale, production-level deployments. This evolution naturally required increased investment in Big Data hardware, software, and services. Feedback from the Wikibon community included multiple reports of $100 million+ deals from both government and commercial buyers.

Additionally, a number of enterprises previously reluctant to undertake Big Data projects due to fuzzy ROI, lack of specific business use cases and/or concerns over product and services maturity, began exploring Big Data in their organizations with small pilot projects, their concerns assuaged by the market potential underscored by the growth factors listed above.

The Big Data market is still within the confines of the early adopter phase and is poised for significant growth. For the Big Data market to reach its full potential, enterprises and vendors must overcome several obstacles. While a detailed discussion of these obstacles is outside the purview of this report, they are worth noting. They include:

The well-publicized lack of analytic specialists and Data Scientists armed with both the technical skill and business acumen to derive insights from large, multi-structured data sets merged from disparate sources.

A lack of understanding among enterprises on how to organize Big Data staff to best identify business requirements for Big Data projects and effectively communicate insights gleaned from Big Data to the business.

Vendor marketing overly focused on “speeds-and-feeds,” product features and “Big Data-washing” rather than laying out a vision for Big Data in the enterprise, articulating a path to achieve this vision, and maximizing the potential for Big Data to disrupt well-established vertical markets.

Development of Big Data platforms and tools by vendors that eschew open frameworks in favor of closed, locked-down solutions. This will limit interoperability with competing and complimentary products and reduce customer choice.

A lack of best practices and related technologies for managing Big Data as a corporate asset, including data quality, data governance, and security platforms and tools.

A dearth of Big Data application development tools and services that allow existing developers to build and customize Big Data applications using common and popular application development languages and processes.

Big Data Vendor Revenue

As part of its market-sizing efforts, Wikibon tracked and/or modeled the 2012 Big Data revenue of more than 60 vendors. This list includes both Big Data pure-plays – those vendors that derive close to if not all their revenue from the sale of Big Data products and services – and vendors for whom Big Data sales is just one of multiple revenue streams.

Methodology

Regarding methodology, the Big Data market size, forecast, and related market-share data was determined based on extensive research of public revenue figures, media reports, interviews with vendors, venture capitalists and resellers regarding customer pipelines, product roadmaps, and feedback from the Wikibon community of IT practitioners.

Many vendors were not able or willing to provide exact figures regarding their Big Data revenue, and because many of the vendors are privately held, Wikibon had to triangulate many types of information to determine our final figures. We also held extensive discussions with former employees of Big Data companies to further calibrate our models.

Information types used to estimate revenue of private Big Data vendors included supply-side data collection, number of employees, number of customers, size of average customer engagement, amount of venture capital raised, and age of vendor.

Big Data Definitions

It is critically important to understand how Wikibon defines Big Data as it relates to the market size overall and to revenue estimates for specific vendors in particular. Wikibon’s definition of Big Data contains two equally important parts.

First, from a technology perspective, Wikibon defines Big Data as those data sets whose size, type, and speed-of-creation make them impractical to process and analyze with traditional database technologies and related tools in a cost- or time-effective way.

Second, Wikibon believes Big Data requires practitioners to embrace an exploratory and experimental mindset regarding data and analytics, one that replaces gut instinct with data-driven decision-making, and exchanges stubbornness for a willingness to question long-held assumptions. Projects whose processes are informed by this mindset meet Wikibon’s definition of Big Data, even in cases where some of the tools and technology involved may not.

Based on the above definition, Wikibon includes the following products and services under the umbrella of Big Data:

Application development platforms and tools as applied to Big Data use cases;

Business intelligence and data visualization platforms and tools as applied to Big Data use cases;

Analytic and transactional applications as applied to Big Data use cases;

Big Data support, training, and professional services.

Key Findings: 2012 Big Data Market Highlights and Trends

Following are key findings, market highlights and trends for the Big Data market in 2012.

Market-leader IBM offers by far the largest product and services portfolio by both breadth and depth. The company also supports its Big Data practice with a well-crafted, high-level marketing campaign focused around its Smarter Planet initiative that often includes illustrations of real-world Big Data deployments. The biggest criticism of IBM from practitioners is that the company’s portfolio is so wide and deep it causes confusion. IBM combats this confusion by initiating many Big Data customer engagements through its professional services division. A challenge and area of focus for IBM moving forward is to continue to articulate its Big Data vision in a way that focuses on industry solutions and not point products.

HP achieved second-place status in the overall Big Data market by revenue in 2012. It did so mostly thanks to revenue derived from Big Data-related services, followed by sales of hardware to support Big Data deployments. HP by its sheer size is in a position to impact and participate in a number of Big Data deployments.

Professional services was the largest segment of the Big Data market in 2012. Among firms that derive 100% of their Big Data revenue from professional and/or cloud-based services, which accounts for 44% of the overall Big Data market, the leader by total Big Data revenue was Accenture with $194 million. Consolidated across vendors, professional and cloud services revenue accounted for $5 billion of total 2012 Big Data revenue.

Amazon continued and Google kicked off increasingly aggressive moves into the Big Data market. Each introduced new products and services to allow enterprises to leverage Big Data analytics and storage-as-a-service with the usual benefits associated with public Cloud services (elasticity, pay-by-the-drink, trading upfront CAPEX for monthly OPEX, etc.) Specifically, Amazon introduced RedShift, an analytic-database-as-a-service, to its portfolio and struck a deal with MapR to allow customers to run its Hadoop distribution on Amazon Web Service, among other announcements. Amazon also continued to build its Elastic MapReduce business. Google finally got into the Big Data game by productizing Big Data tools and technologies, such as BigQuery, it has long used internally, and likewise introduced MapR as a service via Google Compute Engine.

While M&A activity was relatively tepid, two important acquisitions took place in 2012 that have the potential to impact the long-term Big Data market. The first was VMware’s acquisition of analytics firm CETAS. VMware had already begun efforts to apply virtualization technology to Hadoop, and the acquisition of CETAS gives the vendor a more comprehensive Big Data portfolio. The creation of the Pivotal Initiative further indicates that VMware and EMC are continuing to invest in Big Data for the long-term.

The second deal worth noting was WANdisco’s acquisition of Hadoop provider AltoStor. WANdisco specializes in data replication across the WAN, which it applies to Hadoop (both its own distribution as well as Cloudera’s and Hortonworks’ distributions) with the aim of making the open source Big Data framework reliable enough to support mission critical applications.

Microsoft officially entered the Hadoop market in 2012 with the release of an on-premise Hadoop product - HDInsight Server for Windows – and a cloud-based Hadoop service - Windows Azure HDInsight Service. Both are based on Hortonworks’ open source Hadoop distribution. Microsoft also announced PolyBase, which aims to allow the SQL Server Parallel Data Warehouse to execute SQL queries against data stored in Hadoop.

A movement to bring SQL and NoSQL together in a unified platform was firmly established in 2012. Hadapt and Teradata Aster, which kicked off this movement in 2011 continued to lead the charge but were joined by competitors Cloudera, Microsoft and others in 2012.

Facebook, Google, and Amazon as well several three-letter government agencies continued to invest heavily in commodity hardware to build out massive internal Big Data infrastructures. Facebook alone spent close to $800 million on infrastructure in just three quarters in 2012. This spending is reflected in Big Data revenue for the original device manufacturer (ODM) category that appears at the bottom of the table. Specifically, Facebook and others like it purchase, configure, and deploy off-the-shelf hardware from ODM’s such as Quanta, rather then purchasing commodity machines from vendors such as Dell or HP, to support the majority of their operations.

Spotlight on Hadoop and NoSQL Market Sub-Segment

As mentioned in the introduction of this report, Hadoop-related software and services matured rapidly in 2012, leading to increased adoption of enterprise-level products by companies in industries beyond the Web. In many cases, companies that had previously deployed community (read: free) versions of vendor Big Data software bundles for proof-of-concept projects began upgrading to paid software and services to support production-level deployments.

As a result, leading Hadoop distribution vendors Cloudera and MapR enjoyed significant revenue growth last year. Cloudera grew revenue to $56 million in 2012 from $18 million in 2011. MapR grew revenue to $23 million in 2012 from $7 million in 2011. Hortonworks, in its first full year of existence, did $18 million in revenue in 2012.

Likewise, in the related NoSQL space a handful of vendors that offer commercial versions of popular open source databases enjoyed significant revenue growth as pilot projects blossomed into production deployments supporting real-time, Web-scale applications and services.

Among these vendors is 10gen, which offers a commercial version of the open source, document-oriented MongoDB; Aerospike, whose NoSQL database supports very low-latency online transactional applications; and DataStax, the company behind commercial Cassandra that counts Netflix among its marquee customers.

Leading the way in terms of revenue in the Hadoop/NoSQL subsegment of the Big Data market in 2012 was a 10-year-old firm, MarkLogic. The company’s NoSQL document store is in use at Bank Of America, the Defense Intelligence Agency and Warner Brothers, among other household names in the media and financial services industries.

Ultimately, however, the NoSQL market is largely up for grabs. Each NoSQL database has its related strengths and weaknesses, and no one NoSQL database currently “does it all.” Big Data practitioners must take a number of factors into consideration when selecting a NoSQL database to facilitate large-scale transactional workloads, including scalability, performance, security, and ease-of-development.

Below is a cut out of Big Data revenue associated with those vendors specializing in Hadoop and NoSQL software and services. Note that these vendors account for total Big Data revenue of $272 million and are growing at a faster percentage rate than the rest of the Big Data market.

Figure 1 - Source: Wikibon 2013

Big Data Revenue by Market Segment

Below is a segmentation of the Big Data market by hardware, software and services.

Figure 2 - Source: Wikibon 2013

Wikibon further dissected Big Data revenue by type down to a more granular level.

Wikibon’s Big Data Forecast

Wikibon projects the Big Data market to top $18 billion in 2013, a growth rate of 61%. Looking beyond 2013, Wikibon forecasts the total Big Data market to approach $50 billion by 2017, which translates to a 31% compound annual growth rate over the five-year period 2012-2017. While the global economic outlook is for slow to stagnant growth over this period, Wikibon believes the Big Data market will not be severely impacted and may, in fact, benefit from enterprises needing “to do more with less,” which effective Big Data analytics facilitates.

Wikibon further expects the balance of revenue generation and value to shift from Big Data infrastructure and middleware to value-add services and software over the next five years. As noted, hardware revenue accounts for 37% of Big Data revenue and a large portion of software and services revenue is associated with infrastructure software and technical services that tie Big Data platforms and data together.

Wikibon believes Big Data infrastructure, middleware, and technical services will become increasingly commoditized as they mature and common standards are adopted. Practitioners will increasingly look to NoSQL and in-memory database software, streaming analytic platforms, vertically focused analytical and transactional applications and application development platforms (both on-premise and Cloud-based) and associated consulting and professional services to address specific, high-value business problems and opportunities.

Action Item: While Big Data vendor revenue is forecast to grow significantly over the next five years, Wikibon believes that Big Data practitioners will create much more value than technology and service providers in the long-term. When selecting vendors to support Big Data initiatives, therefore, CIOs and Big Data practitioners must evaluate the products and services on offer in the context of how best to monetize Big Data to achieve competitive advantage. This includes evaluating “speeds and feeds” and other product features but should also include evaluating how well vendors can assist enterprises in adopting a sustainable culture of data-driven decision-making.

Footnotes: (a) Wikibon revised its 2011 Big Data market size estimate to $7.2 billion from $5.1 billion. Upon further review and extensive feedback from the Wikibon community, it was decided that the original figure underestimated the level of revenue generated by original device manufacturers.

Based on what I know about the data science services industry, your estimates of % of big data revenue for Opera Solutions, Fractal Analytics and Mu Sigma are off significantly. I think <10% of revenues for each of these companies comes from Big Data solutions.

I am curious regarding the validity of the numbers for the global Big Data Market. A Big Data Market Report by Transparency MR, states that the global big data market was worth USD 6.3 billion in 2012 and is expected to reach USD 48.3 billion by 2018, at a CAGR of 40.5% from 2012 to 2018. Could you please enlightenment me on that?Reference: http://www.transparencymarketresearch.com/big-data-market.html

Hello Vaishbhandary. 1/ hard to say as their report is behind a very expensive and not so "transparent" paywall. 2/ From the Transparency Market Research website there is this methodological statement: • "The market structure and forecasts are developed on the basis of secondary research..."

My guess is there is the difference. The report from Wikibon is based on primary research with specific data by vendor that adds up to the market total in a "transparent" way. I don't see any reference to market share data in the Transparent Research report description so it's not clear where there baseline comes from.

Best pay the $5k and compare with the free Wikibon study to get your answer.

Hi Vinod. Good points. They were wrapped into the other category, but I think it may be wise to add them. We will work on adding them. Any insights on these companies and specifically their Big Data practices you could share?

Thanks.
Not much insights,other than in news. I do believe the India pureplays have much to gain and will be investing in differentiating themselves with Big Data Analytics & Cloud. They have doubled their revenue & improved market share in last 6 years and are giving stiff competition to global MNCs.

Cognizant- In last 3 years has made some 4 acquisitions in consulting space and could look at improving its consulting share..Tied up as 3rd party integrators with Amazon for RedShift. Amazon low cost Cloud offering could have major market influence
Infosys- Recently launched BD Edge platform- Integrated Cloud-Big Data offering. They seem to have won a new customer recently for this platform
Wipro- launched the Wipro AssuredHealth platform in partnership with Microsoft. Maybe focussing on building target offerings with partners for various verticals. They are building a super computer, something they could leverage for BD & A work (like IBM Watson)
CSC- Believe a recent gartner report has identified them as Leaders in the BI & A space. Considering BD&A services to grow faster, they could be a key player

Thank you very much for this analysis. It is yet another reason, and even more proof, that WikiBon is such a virtuous organization. This is exactly the kind of info I need for context and I know many in large enterprises need this too.

I found the Big Data market forecast you put together very helpful. I just had a couple of questions, to which I would really appreciate your response.

1. There seems to be a growing number of services (e.g., Datasift, Grepsr) focused on online data collection. In which category of your market forecast would you place these data collection services - perhaps SaaS, app software, professional services, or a mix?

2. To what extent is government-sourced revenue included in your market forecast?

Thanks in advance for your help. Looking forward to more of your posts.

Hi Jonathan. We would place data content providers/aggregators in the services bucket. Its a good point you raise. We plan to dig into this market more for our next market sizing. Services that make data sets available to enterprises and other service providers are a key enabler to the Data Economy.

Hi Jeff,
I came across this study thanks to a 5 pages article in Le Monde that cited Wikibon as one of their source for market analysis. It is a great study and we can finally see some real figures on the "Big Data" business. However I did not see Sinequa in the list. Probably because we are mainly a European vendor, but we start to have some customer in the US too. Our revenue in 2012 was $9,5M, 100% on Big data search. Our competitor in the list are Attivio, Marklogic, Lucidworks, Digital Reasoning, ... Could it be possible to add us? Thanks!

Jeff, came across this again this week as it was posted on LinkedIn and it got me thinking. I appreciated the analysis and it is great to see this kind of data in the public domain vs. sold by research firms and selectively doled out by vendors. Also I think your insights and assessments were spot on. Assuming you are going to refresh this for 2013 I have something for you to consider: There is a 4th leg to this stool. In addition to the traditional view of HW, SW and Service components I would argue that data providers/enrichers are an important element in the sizing of this market and the revenue/value from this leg of the stool will be greater than the others. I also believe we are going to see a blurring of the lines across the ecosystem as vendors offer a blend of Software, SaaS, Analytics as a service which include data.