Busted: The Business Intelligence Industry's Biggest Myth

Self-service is one of the hottest terms in business intelligence. Unfortunately, it's also one of the most misunderstood.

Normally, when I hear the term self-service, it conjures up images of do-it-yourself activities such as self-serve gas stations, car washes, and vending machines. If you tell me something is self-service, I think "OK, this should be easy." Yet, if you ask me to compare the idea of getting self-serve yogurt at my local fro-yo shop with building a business dashboard using a vendor-proclaimed self-service-BI tool, it's hard not to chuckle. One of those kids is not like the others.

The Promise and the Pain

I've used BI tools for over a decade. I first logged in to Crystal Reports (acquired by SAP via BusinessObjects) back in 2002. I've built some cool analyses and experienced the thrill of figuring out a tough data-related question here and there, but I've also suffered through spirit-crushing frustration as well. Although the BI industry has created powerful technology with huge potential for business insights, ease of use has never been its strong suit.

For most executives, that's been fine, because if your BI tool can help your marketing team figure out what promotion was responsible for your sales of women's handbags going up 12 percent in the Southeast, and then show you where to replicate it across other regions, it can be the difference between hitting your forecast and missing it. Spread that impact across companies and industries, and the potential for value creation is enormous. It's why companies spend $14 billion on BI software and $55 billion on software services every year. Unfortunately, for me and millions of others trying to self-serve, there's a comparable amount of pain and missed opportunity.

Self-Service: Easy for All?

Self-service should mean "easy for anyone, no training required." Unfortunately, in the world of software, the term has slowly come to represent anything that can be done, as long as it doesn't involve writing code. Sure, you can build a dashboard to analyze your business, no problem. Just go to a training class for three days, then spend two weeks chained to your desk trying to figure it out with your new best friend, Mr. User Manual, and voilà—self service!

This definition of self-service technically makes sense. You can do it yourself. All you need is a good head on your shoulders and the tenacity of a honey badger, and yes, you will eventually figure out all of these applications. However, if we've learned anything from Apple, Facebook, and Google, it's that the easier you make technology, the more people will use it and actually self-serve. Unfortunately, everyday business people, who are on the front lines and need access to their data most, are out of luck because today's tools are still too complex. The result is that we're living in a world where BI has devolved into two classes of citizens: the creators and the consumers (aka the haves and the have-nots).

Two Classes of BI Citizens: Creators and Consumers

Today's modern BI tools have become easier to use than the tools of decades past, not to mention a lot prettier (which helps—ask any marketer). It's fair for vendors to market that improvement. A trained person can now do in two days what used to take six weeks. That's great, but how many business executives do you know with two free days to build an analysis? When I hear vendors evangelizing self-service, I can't help but ask "self-service for whom?"

Who is doing all of these analyses? Enter the dramatic rise of the role of business analysts and data scientists. These professionals have become hugely in demand over the last few years. McKinsey estimates that by 2018, we'll have a 165,000-person shortage of these skills in the workforce. Today, these individuals are the "creators" of BI while the rest of your business leaders act as "consumers." Even though the tools are easier to use, they still require a significant amount of training, and making time for three days of training isn't enough—you need to invest in these tools to stay current.

If you're only logging-in a couple times a week to do ad hoc analysis, it's easy to forget what you learned in a three-day course months earlier. It only takes 15 minutes of frustration when you're under deadline pressure and can't figure out your BI tool to pick up the phone to call in business analyst reinforcements. That's why we've resorted to paying analysts to go to training classes and perform analyses all day, every day, for the rest of us. It's either that or fall back on common sense and intuition.

The Ultimate Measure: Adoption

If you agree in principle that self-service should be easy, then user adoption is the ultimate way to measure it. After all, no one ever created a brilliant BI insight without logging in first. Interestingly, if we look at adoption in absolute terms, the BI industry has made huge strides. What used to take five people six weeks to complete now takes one person three days. That's major progress!

As a result, companies have been willing to buy more software licenses in hopes of putting the power of insight into more hands. In the '90s, a company with 1,000 employees might buy 20 BI licenses, with only 5 IT people ever figuring out how to use them. Today, that same company will buy 200 licenses with 50 business analysts logging in regularly. Although 50 users is a huge bump from five, the adoption rate remains constant around 25 percent.

Gartner's Cindi Howson recently documented this trend, noting a BI adoption rate of 22 percent in 2013, down from 24 percent the prior year. We're going the wrong way, and therein lies the problem. Despite all the promises, BI adoption has plateaued or declined for decades, so how is it that the industry is growing at 9 percent per year? Although the promise remains, there's still a big disconnect between vendor claims of easy self-service and how users vote with their logins. The vast majority of the employees are still going without access because the tools are too complex to use.

More Bells and Whistles, Still the Same Old Bottlenecks

In some cases, adoption rates of so called self-service tools are improving because BI vendors are making it easier for BI "consumers" to login, view dashboards, and twist a few pre-determined knobs and dials—even from their mobile devices. Compared to printouts and e-mailed reports of old, this is a big improvement. For an executive who commands an analyst and has them configure everything just right, it can be very productive and even gamified fun to slide a few dials on a phone. The trick is getting everything just right—which we all know rarely happens—and even then you're still only able to go an inch deep.

Meanwhile, the majority of employees who don't have the benefit of a dedicated analyst are left to pick up the scraps of someone else's dashboard, which normally generates more questions than answers. If pre-configured dashboards worked for the masses, the industry would be 1/50th its current size and the 4:1 ratio of services to software would be just the inverse. Instead, we're still playing the same game of telephone and fighting over bottlenecks that have plagued the BI industry for decades. Imagine how much productivity would be unlocked if these tools truly were self-service for all.

Where Are We Headed?

The BI industry's biggest challenge is adoption. Watch a demo of how to build a dashboard from any business analyst and you'll be blown away by the sophistication, blur of rapid-fire clicks, and eventually the beautiful visualizations. It's amazing, but clearly difficult, so if you where would you begin to make the experience easier?

Typically, when the enterprise software world struggles, it looks for inspiration in consumer tech. We've seen the enterprise successfully borrow cloud, social, and mobile—so surely there's an idea to be borrowed for BI.

What's the easiest, most-adopted consumer technology? It's search. Google has taught billions of people how to get answers without a training manual. It makes sense to look there. The trick is that the whole experience needs to be extremely simple. Using search to find a pre-built report or dashboard, then enter "edit mode" and start the same complex knob-twisting won't work. We need a search engine for numbers that can calculate answers as fast as Google returns Web pages. If you are a sporting goods retailer and want to know how many men in Sacramento bought golf clubs last week resulting from the new free putter promotion, you should be able to just type "sales in Sacramento last week for putter promotion" and instantly get an answer with a chart.

Now, I know this is hard from a technical standpoint—for all of Google's simple elegance, there's a ton of brilliance under the hood. The point is, it's doable. It has to be. It took Google over a decade to evolve its search to the amazing all-knowing, type-ahead sophistication you see today, but I have faith we'll get there in the BI industry, too. We've all suffered long enough and BI designed for anyone to self-serve has more promise to create value than ever.