Featured in
Architecture & Design

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Featured in
Operations & Infrastructure

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Featured in
Enterprise Architecture

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Sponsored Content

Bio

Uri Cohen is responsible for everything product related at GigaSpaces, and leads the definition and execution of the company’s product strategy. He has been with the company for the past four and a half years, serving in a variety of customer facing and technology roles from solutions architect through technology evangelist.

Software is changing the world; QCon aims to empower software development by facilitating the spread of knowledge and innovation in the enterprise software development community; to achieve this, QCon is organized as a practitioner-driven conference designed for people influencing innovation in their teams: team leads, architects, project managers, engineering directors.

Interesting. Uri names the NYSE as the example of how large scale and high throughput these systems are, then later he claims that database technology can't keep up. To set the record straight, NYSE uses database technology to store every order and every trade -- it is the system of record for their real time market engines. I guess that means that database technology can keep up?

There are obviously other markets (besides the NYSE) that do use different technology for their market engines. The last I heard, for example, the largest and the third largest foreign exchange markets did not rely on database technology for their real time systems of record, and instead used a combination of in-memory technology and a more custom form of durable storage.

I was trying to stay away from a vendor-specific response for two reasons: First, I don't want to disclose any confidential customer information; and second, I was trying to clarify an incorrect statement on technology, and not make it into a vendor versus vendor conversation.

Obviously I have been known to skirmish a bit with the guys from Gigaspaces, but it's as friendly and as respectful as two long-time and fierce competitors can be ;-)

I will try to find out what information is public, so I can respond with more detail.

I find that the level of one's surety is at times inversely proportional to the level of one's knowledge.

(Needless to say, I am quite surprised that you would state such a thing unequivocally.)

Most of the hedge funds are using kx KDB to process and store taq data from NYSE, but not sure if NYSE is using KDB

There are many, many hedge funds. I've only personally worked with a few dozen of them, so I cannot speak authoritatively. I do know that KDB is a well-regarded solution in specific use cases in financial services, and even though I have not personally seen it in use at hedge funds, I am not surprised by the likelihood that it would be used. Here's a number of products in the general OLTP database arena that I've seen in use at hedge funds:

I haven't seen any Sybase in use at hedge funds, even though it is very common in the more established financial services businesses (because they all purchased unlimited perpetual licenses back in the early 1990s for almost nothing.)

For analysis (OLAP etc.), I've seen a number of other more specialized solutions.

For order books, trading systems, algo systems, etc., I've seen the use of in-memory data grids. (That's not surprising, since that is probably why I was there talking to them in the first place. ;-)

The danger of quoting numbers without reading properly
by
Fernando Racca

Uri says that calling a method in java would take 300 ms, do that with two threads contending for a lock, and you'll need a quarter of a second. Hold on a sec... a quarter of a second??? I work in a market maker, but not even the oldest computer i've ever used would take that amount of time!

Oh, maybe if you actually go to the paper he's quoting, on the Disruptor, you can actually see that they are actually trying to increment a counter.... 500 million times!