There has been much written in the industry about the convergence of transactions and analytics in database management systems (DBMS). There are many names for this convergence from multiple sources. Many of these sources have given proper credit for the concept; however, some have not, allowing readers to believe they originated the idea. I have no issue with anyone with the desire to create their own names or acronyms and to write about this. I do have an issue when they want readers to believe they originated the idea. One perfect example of how to do it right is Hybrid Operational / Analytic Processing (HOAP) from 451 Research. They created that name and acronym, and in their research, they credited Gartner, Inc. with the original concept (thank you 451 Research).

Gartner introduced Hybrid Transaction / Analytical Processing (HTAP) first in a Hype Cycle, “Hype Cycle for In-Memory Computing Technology, 2013” dated July 2013. In that document, we defined HTAP as: “An in-memory computing (IMC)-enabled hybrid transaction/analytical processing (HTAP) architecture leverages IMC techniques and technologies to enable analytical processing on the same (in-memory) data store that performs transaction processing”. Since that time, a search of Gartner’s body of research produces 186 specific documents pertaining to or about HTAP. Since our introduction of HTAP, we defined two forms of HTAP – Point-of-Decision HTAP & In-Process HTAP. Point-of-Decision HTAP is an architecture using in-memory computing (IMC) techniques and technologies to enable concurrent analytical and transaction processing on the same in-memory data store. In-Process HTAP is an application architecture whereby, in the context of a given application, analytical and transaction processing techniques are weaved together as needed to accomplish the business task.

Another misconception is that HTAP is a separate market, product or function within a product; this not true. HTAP is an architecture used for specific application use cases. You can purchase HTAP enabled applications but you cannot purchase HTAP as a product or “turn-on” HTAP in a product. DBMS products capable of supporting HTAP applications will have features such as builtin analytics and will normally be in-memory computing (IMC) enabled. Point-of-Decision HTAP is possible with DBMS products not supporting IMC but of course the application or transaction latency will be longer. In-Process HTAP really does require IMC as the latency requirements of the process or transaction is necessary higher and although “stepping out” of the process to perform analytics is possible, the SLAs of the transaction will normally not be met.

Finally, I want to set the record straight about operational database management systems (OPDBMS). The portion of the DBMS market involved with transactions has been called On-Line Transaction Processing (OLTP) for greater than 40 years, since the release of the IBM CICS software in 1969. Today, OLTP no longer describes transactions as all transactions are on-line. Further, we do have new types of transactions using new types of data. For example, IoT brings lighter-weight, streaming transactions that may or not be OLTP. Hence, in May, 2013, my colleague Merv Adrian and I published the first Gartner research renaming OLTP to OPDBMS – “The OLTP DBMS Market Becomes the Operational DBMS Market“. We believe this extends the OLTP market to these new types or data and transactions.

There is more change coming to this very fragmented market called Database Management Systems! Today we see eight or 10 major use cases, and 100’s if not 1000’s of DBMS products. There are relational, graph, document, XML, wide-column, key-value, time-series, columnar, in-memory, object-oriented, pick, prerelational and more types of DBMS products. Something has to give! Stay tuned here, soon.

Donald Feinberg
VP Distinguished Analyst 25 years at Gartner 50 years IT industry

Donald Feinberg is a vice president and distinguished analyst in Gartner Intelligence in the Information Infrastructure group. Mr. Feinberg is responsible for Gartner's research on database management systems and data warehousing infrastructure. Read Full Bio

Thoughts on Setting the Record Straight – HTAP & OPDBMS

I share your principle Don: don’t take credit for others work. Though perhaps, as in other industries, parallel invention and differing competitive solutions perhaps may cause such contention. However, your call for clear terminology, is to me, what is inarguably necessary for comprehension of the problems to be solved, and proper evaluation of the solutions available. There’s so much more data and ways of using it, perhaps the plethora of dbms for now is a good thing, for a while.

It is indeed important to give credit where it is due. Gartner has done a lot to share the idea of a single architecture for transactional and analytic workloads. But you seem to credit yourselves for the original concept. Would it not be appropriate to acknowledge, for example, Professor Hasso Plattner’s 2009 Sigmod paper, “A Common Database Approach for OLTP and OLAP Using an In-Memory Column Database”, which resulted in commercial database products as early as 2010? (SAPHANA – and other vendors have made similar claims) http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.175.8831&rep=rep1&type=pdf

Temo,
You are correct. Hasso did publish a paper on in-memory DBMS (IMDBMS) technology in 2009. And yes, it did become SAP HANA. However, Hasso’s paper does not discuss transactions and analytics in the same transaction. It does discuss an IMDBMS for storing both transaction and analytics data in the same database. The definition of HTAP does not require all data in one database, in-memory or not. Microsoft SQL Server uses two IMDBMS types, a row store for transactions and column-store for analytics. IBM Db2 and Oracle, today, have an IMDBMS column-store for analytics and a traditional disk-based row store for transactions. HTAP is still possible with Db2, Oracle & SQL Server. Hasso was one of if the first to suggest how to create an IMDBMS for storing both OLTP & OLAP data; however, his paper does not describe HTAP transactions.

About

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.