The Next Revolution for Finance -- Embedded Analytics

Everyone in finance knows the term “business intelligence” (BI) and when most people think of BI, they think of software that’s purchased and bolted on to their business systems to extract and analyze data for reporting on business performance. Yet while this bolt-on approach to BI has served a very good purpose for a long time, it is on its way out and a new and better way has emerged, called embedded analytics.

To understand how traditional BI came about—and why it’s on its way out—we have to look back more than 30 years. One of the biggest problems facing finance departments at that time was the difficulty of keeping up with growing transaction volumes using manual processes. This problem was even comically captured in the prologue of the 1983 film “Monty Python’s The Meaning of Life.”

In the film, dozens of accountants from a fictional assurance company are hunched over their desks, frantically keying in transactions in unison, and are then shown rowing in a ship’s galley under the supervision of whip-cracking bosses. In real life, there were a lot of finance departments drowning in transactions. The computer industry saw an opportunity, and began designing financial systems that would solve the biggest pain point of the era: transaction processing.

Technology at the time was well-suited to transaction and data processing. Even though a gigabyte of memory cost more than $200,000 back then, it wasn’t an issue, as not much data storage was required to efficiently and accurately automate transactions for GAAP. So with these new financial management systems, the world of finance was off to the races.

Yet as is human nature, the solution to this burning problem kept everyone happy for about 45 seconds. Business leaders began thinking about what was the next problem to be solved. Everyone realized that where there was data, there was power, and began to ask questions about their business data that transactional systems couldn’t answer. Their systems were built upon flat-file, relational database models, yet true analytics required a multi-dimensional model. Data had to be moved for multi-dimensional analysis to occur.

During the next 10 to 20 years, the traditional BI system and industry emerged to solve this next problem. Developers created the star schema, or data cube, and provided the foundation for the BI system as we know it.

So why do I call these systems “bolted-on” BI? Because they truly are separate entities and there’s nothing seamless about them. Software programs were written to extract, transform, and load data from ERP systems into BI systems that would then produce business reports. It required that a company’s data live in two places—ERP and BI. That meant, and still means, multiple licenses and security, integration between the two systems, a lot of data clean-up and infrastructure, and overall, considerable cost and complexity. Also, by the time data is moved from ERP to BI, it’s no longer current.

Once the infrastructure for BI is put in place, it’s difficult to change. The VP of marketing can’t easily start tracking marketing campaign spend, for example, if the system wasn’t set up to do that in the first place. To add that capability adds more time and cost, so the VP finds a workaround and tracks spend using a spreadsheet. Once again, company data resides in multiple places.

Enter Embedded Analytics

Over the past 10 years the demand for analytics has continued to grow, as has the money and time companies have spent on it. Yet a lot of other things have happened.

The Web revolutionized technology for the masses, creating rich user experiences that continue to evolve at a dizzying pace. Business technology vendors can create interfaces that are powerful yet familiar to users, similar to the consumer applications they access from their home PCs and mobile devices every day. This provides a way to make business insights accessible to more types of business managers who aren’t necessarily IT or finance experts.

In the development world, object orientation has matured to make it possible to build modern business systems based on an object model and not a relational one. Things worth understanding—an employee, product, business unit, or customer—can be defined as objects, making it possible to form richer relationships among those objects and the data associated with them.

A business system built on an object model (instead of thousands of flat-file relational tables) is multi-dimensional from the get-go—there’s no need for a bolted-on BI system to come in and save the day. A modern, object-oriented business system should be able to quickly deliver an analytics report, and the authorized business user should be able to easily drill down and view the report by multiple dimensions.

The key here is quickly and easily, and that’s where in-memory technology plays a critical role. It is now possible to build systems that allow transactional data to run in the main memory. Low memory costs have made this practical. Remember that gigabyte of storage that cost $200,000 in the 1980s? Now you can stop by a vendor booth at a trade show and grab a handful of one gigabyte thumb drives out of a fishbowl.

So how does an organization recognize true, embedded analytics? First rule: You should not have to move your information to learn more about your business. You should not have to buy a BI system that must be bolted on. The marketplace has a way of confusing this difference, but here’s one red flag: a price list. A supplier that presents 10 different price options to make something work is offering a bolted-on product. It doesn’t matter if the box is the same color as the business system, and the holes match up nicely—it’ll still be bolted on.

Final Thoughts

In the ending of the prologue to “Monty Python’s The Meaning of Life,” the transaction-burdened accountants stage a mutiny against their cruel bosses and take command of their building-turned-pirate-ship. While I certainly don’t condone mutinies, I believe embedded analytics is the beginning of a revolution for finance departments and their businesses. In 30 years, we’ve moved from manual processes to automated transaction processing, and then to analytical processing using bolted-on BI. The next phase, where finance departments can participate in offering the most value to the business—and be champions of real change—is embedded analytics.

Mark Nittler has been the vice president of application strategy at Workday since 2005. He's held marketing and product strategy roles at Symantec, Peoplesoft, and Commerce one and began his career as an accountant at Deloitte.

3 Comments

Embedded analytics is a natural extension to any transaction system, since the original source data is there for the using. It makes since to migrate from 'bolt-on' to 'built-in' as a matter of efficiency and integration.

However, external analytics is unlikely to be going away anytime soon. The natural boundary (or barrier if you prefer) for embedded analytics is that it has access primarily or solely to the transaction system upon which it is built. There is no doubt that import features will be, and are available, but the system doing the importing looks just like a 'bolt-on' to the system doing the exporting.

So this means that the more traditional enterprise information warehouse will continue to be that neutral ground for truly enterprise-wide information analytics needs for some time to come.

Over time improvement in the handling of decentralized federated data models may open the door for each system that has embedded analytics to reach beyond its own transaction platform in a more seamless manner. As that happens, the respective and multiple embedded analytics platforms will need to be evaluated for their primary function: Is it to provide analytics for the transaction platform upon which they sit, or is it to provide enterprisewide analytics? If it is the latter, then a rather compelling case can begin to be made to once again separate the analytics engine from the transaction processing engine.

Nicely stated but not novel. I would add that many vendors now have and have had embedded analytics for several years now. So this is not at all unique. Star schema (or "cube") based enterprise data marts are for previous generation ERP solutions that were designed around a GL and chart of accounts and did basic financial reporting. However analytics have evolved to the point where a) many transactional systems now have embedded BI and b)you still need an enterprise data warehouse (not necessarily built on relational star schemas) to combine data from multiple types of systems and data sources (persistent and streaming) to run more sophisticated analytics.

What's also worth mentioning here is that now most analytics run in memory so the "legacy way" of describing star schemas and cubes is a bit dated.

Mark interesting article but I disagree with (if I am interpreting it correctly)"Developers created the star schema, or data cube, and provided the foundation for the BI system as we know it."
A star schema is not a Cube. As star schema is a relational model consisting of one or more fact tables connected to (usually) conformed dimension tables and queried via SQL just like any other relational DB. A Cube (more correctly an OLAP Hypercube) is a different thing altogether. The data storage is not relational and rather than SQL, queries are executed via MDX or similar. Cubes may be built on top of Star Schemas but they are not the same types of objects.