Article

Breaking the Bounds of Traditional Data Management

Harness the Power of Extreme Transaction Processing to Drive Growth in Your Organization

by Terry Orsborn | SAPinsider, Spotlight-Articles, Volume 14, Issue 4

October 1, 2013

As the amount of data and information that organizations hold grows, traditional methods of storage and performance must change to accommodate it. One way IT teams have been keeping up with the information explosion is extreme transaction processing (XTP). XTP is a transformation of the traditional online transaction processing (OLTP). Learn about the rise of XTP and what business implications it brings.

IT organizations have had decades to develop strategies for gathering, maintaining, and using information generated by traditional enterprise applications. However, the amount and variety of information that companies must store, process, and deliver is growing and becoming more complex, while the time available to process it is shrinking just as quickly. These changes are transforming the previously well-understood world of online transaction processing (OLTP) into something more formidable: extreme transaction processing (XTP). It is difficult for IT teams to keep up and find ways to deliver additional performance and scalability in response to growing workloads.

What factors are behind the rise of XTP and what are the business implications?

Moving Past Traditional Approaches to Data Management

Almost every enterprise either already considers itself a “data-driven business” — one that can quickly and securely provide comprehensive, accurate information to any user, any time, on any device — or is rapidly trying to pivot in that direction. To achieve this goal, IT must prepare for fundamental expansions that will transform traditional OLTP into XTP. While it’s tempting to think of the latter as just “OLTP on steroids,” there are defining characteristics that merit considering it in a separate light, such as:

Transaction loads. These are now measured in millions per hour — or billions per day — which is far greater than the number encountered in traditional transactional scenarios.

Speed requirements. In the not-too-distant past, applications that used databases were expected to return information in seconds or less. Now, transactions are measured in microseconds and milliseconds. Your users are more sophisticated and demanding than ever before.

Workload variability. Thanks to cost-control and efficiency efforts, the same hardware must now service a variety of processing loads, including transactions, analytics, and business intelligence. These assignments are changeable based on time of day, year, and so on.

Diversity of data. Conventional relational data is still a big part of the picture, but you must also contend with vastly larger amounts of unstructured and semi-structured information, such as images, video, text, XML, and JSON. Machine-to-machine streaming communications, as well as devices like smartphones, sensors, and RFID tags, are generating much of this data.

Data volumes. Transactional applications and business intelligence are helping to turn gigabytes into terabytes, and terabytes into petabytes. In fact, in 2013, the total of all digital data created and replicated will reach four zetabytes, almost 50% more than 2012 volumes and nearly quadruple 2010 volumes.

Taken together, these transformations are driving new types of solutions. These “extreme” applications used to be confined to a handful of specialized environments. For example, Wall Street — where trillions of dollars in trades are settled every day — has long served as the poster child for extreme applications. However, XTP goes far beyond the narrow confines of finance. Enterprises in every industry need to make critical decisions in real time while factoring in massive amounts of information. Meanwhile, their users, customers, and other information consumers expect fast turnaround, high availability, and access to data on the device of their choice. These requests aren’t limited to large organizations; instead, they’re becoming a fact of life in enterprises of all sizes. Cloud computing, commodity hardware, and larger and more diverse storage options are all helping to nurture these trends.

A company that learns to apply the right blend of technology and business processes will reap abundant benefits. Its infrastructure will be capable of supporting the obligations listed earlier, often by better leveraging existing assets. This lays the groundwork for an organization that’s agile, responsive, and exceeds customer expectations. The final result is a strengthened competitive position against rivals that have bet on simply maintaining the status quo.

With that said, it’s a good idea to start preparing for the XTP uses that are coming your way.

Preparing for XTP

If you haven’t yet experienced the XTP requirements I’ve described, it can be tempting to just wait for them to appear and then take action, particularly when your IT budgets aren’t growing at the same pace as your responsibilities. But procrastinating means that you’ll have to scramble when XTP-style scenarios arrive. While there’s a lot of variability among different IT organizations, each should be able to take basic steps in anticipation of XTP.

Your primary goal should be to design a comprehensive information management architecture that recognizes diverse data types and can cope with unpredictable usage patterns. At the same time, you must never lose sight of the fact that your relational database management system (RDBMS) will continue to serve as the system of record, and that transactional information management will remain at the heart of your company’s IT assets. This means that you should continue to devote considerable time and attention to your RDBMS.

Regardless of the exact mix of your selected technology portfolio, it should be able to:

Meet highly demanding performance expectations

Handle massive transaction spikes

Enable new usages and applications

Make the most of existing IT assets

Save time on administration

Capture and manage huge data volumes

Provide continuous availability

Support a mobile workforce

Finally, total cost of ownership (TCO) is a particularly important facet of any XTP environment. Remember that TCO is composed of many factors, such as software costs, efficiency, availability, and time to market.

A comprehensive platform such as the SAP Real-Time Data Platform is ideal for XTP. It employs a unified, holistic approach that pairs the speed and scalability of in-memory computing with a unified framework that cleverly fuses data regardless of where it has been persisted, whether in the cloud, in memory, or on disk. The SAP Real-Time Data Platform is made up of:

SAP Sybase Replication Server for high availability and data replication

SAP Sybase Event Stream Processor for streams

SAP Sybase SQL Anywhere for mobile

Conclusion

By proactively planning for XTP, you have a better chance of successfully supporting the response times, data types, workloads, and information volumes that are the hallmarks of XTP, but only if you acknowledge that your relational database will remain at the center of your IT portfolio.

To maximize your benefits, consider adopting an infrastructure that offers an array of specialized solutions for the full range of XTP uses. The SAP Real-Time Data Platform incorporates all of the necessary components to construct a solid XTP foundation. Learn more at www.sap.com/rtdp.

Note: This article is derived from “The Unstoppable Rise of Extreme Transaction Processing (XTP)” by Robert D. Schneider of WiseClouds, LLC.

Terry Orsborn

Terry Orsborn (terry.orsborn@sap.com) is a Director at SAP, where she manages product marketing for SAP Sybase Adaptive Server Enterprise (SAP Sybase ASE). She develops the marketing strategy and directs programs to drive awareness and demand in the marketplace, including go-to-market strategy, field enablement, marketing campaigns, and demand-generation programs.

In a digital economy that runs on speed, agility, interconnectedness, and massive amounts of data, organizations are increasingly looking to the SAP HANA platform to provide the capabilities they need...