With trillions of dollars worth of business being traded every day in the financial markets, there can be no doubt that both the financial system and global businesses rely on the efficient operation of our major exchanges... they are truly mission-critical to the economy in every sense.

At the heart of these exchanges is a trade-management system which accepts trade data in real-time from the trading platform and provides several key services:

Validation of the trades

Execution of complex rules to convert the trades into a granular standardized form

Management of a datastore of daily trades

Provision of messaging services for exchange members, enabling them to query and update trade information

Publishing of trade history for auditing purposes

Daily trade clearing

Undertaking to replace such an important system can never be taken lightly. Indeed, one of the world’s main exchanges had been enhancing and maintaining their system for about 20 years. Although they had known for some time that it needed replacing, the cost and risk of a rewrite in a low-level language such as C++ or Java had always been prohibitive. And high-level languages could not provide the necessary performance or scalability. Eventually, however, driven by the need to replace obsolete underlying hardware and software, combined with the need to overcome important scaling and flexibility constraints, the exchange decided it was time to reimplement their trade-management system using new technology.

FIGURING OUT THE POSSIBILITIES

Naturally, an extensive evaluation of technologies was undertaken. To provide confidence in the decision and to help build the business case, the exchange set Ab Initio the challenge of replicating some of the key elements of the trade-management system in a Proof of Concept (POC). The aim was to determine if the technology could:

Keep up with peak trading volumes, both current and estimated far into the future

Provide a resilient architecture for 24×7 operation with no missed messages, ever

Handle the proprietary TCP/IP-based messaging protocol used for all communication

Provide a large increase in productivity, a requirement for the business case

Many POCs have clear-cut goals, such as: “We must get modules A, B, and C implemented in no more than 3 weeks; the system has to process X gigabytes per hour with turnaround for individual messages of no more than 80 milliseconds; it has to run on less than 10 CPUs; and it has to meet all operability and robustness requirements.” The POC with the exchange had all of these and one additional, critical one: the existing development team, steeped in traditional programming languages, had to want to switch to Ab Initio®.

WORKING TOGETHER

The joint Ab Initio/exchange team developed and implemented a number of new ideas and approaches during the POC. They figured out the sizing of the hardware platforms needed to meet the response and latency times. While a number of server configurations were tested, the efficiency of the Ab Initio infrastructure meant that, in the end, only 3 cores of a modern Linux blade were needed to handle a critical part of this international market infrastructure. One can imagine the faces at the exchange when they learned that a small server was sufficient to run it!

The joint customer/Ab Initio team built an architecture that could support all the flexibility needed to handle changing market circumstances: new exchange products, different regulatory frameworks, and other industry-wide initiatives. Toward this end, a modular service-based architecture using Ab Initio’s publish/subscribe capabilities was adopted. This not only allowed the application to be decomposed in a logical manner, but also created a resilient architecture with processing distributed across multiple servers. As one would hope, all aspects of the system were designed to ensure transactional integrity, and specific effort went into ensuring that the application provides the real-time performance required of a modern exchange.

All the numerical and architectural requirements of the POC were met in a matter of just a month. The toughest requirement, getting the exchange team on board with Ab Initio, became the easiest to meet: the development team leader declared that he would be ten times more productive in Ab Initio than in a standard third-generation language; and he had the backing of his team, based on their own first-hand experience.

IMPLEMENTATION

With the POC finished, the key business owners agreed upon an internal project plan, including schedules and costs. The development team provided conservative estimates for the development and test activities – at least 40% faster with Ab Initio for the whole effort – to make sure they would be successful and leave room for the unforeseen. They were secretly salivating at the prospect of getting the project done even faster and under budget, and being hailed as heroes.

The system went live in 2010 and is now being deployed to further exchanges around the world. And best of all, the business owners have a new relationship with their IT development team. They can count on systems being done faster than they could have hoped for previously, and these systems have a degree of robustness and flexibility that will allow the business to grow without impediments.