Programming Cradle

Wednesday, March 26, 2014

Let's continue our journey of capturing data from InteractiveBrokers. In this blog, we discuss how to capture real time bars. A bar here means a summary of prices over the last 5-second interval. Currently you can choose one of the following 4 types of prices that occurred in the interval to generate bars:

TRADES: actual trade prices;

BID: bid prices;

ASK: ask prices;

MIDPOINT: mid prices.

For the type of prices you chose, the information in a bar includes OHLC (open-high-low-close), volume, WAP (weighted average price), and count (the number of trades that occurred, which is applicable to TRADES only). Compared to ad hoc data like tick prices or market depth, real time bars are sent over to you on a regular basis -- you receive an update every 5 seconds.
The code example here is to capture real time bars on TRADES. As in the previous two blog posts, the first thing we need to do is to specify the stock we are interested in. Again, here I chose Infosys listed in Mumbai.

Tuesday, March 4, 2014

In the last blog post, we discussed how to capture real-time tick data from InteractiveBrokers. Now let's see how we can capture real-time market depth data from InteractiveBrokers. The example code in this post is very similar to the one in the previous post. As far as the code is concerned, the main difference is just that now we subscribe to and process market depth events instead of tick price events. However, to interpret market depth events is slightly more complicated than to interpret tick data events, since an order book is basically a table with multiple entries.

Before we talk about how to process market depth events to maintain an order book, let's first take a look at an example below, which is a snapshot of the order book for the stock "Infosys" listed in Mumbai. The snapshot was taken at some time on 24th Feb 2014. You can see that the depth of this order book is 5, i.e., it shows the best 5 entries for both bid side and ask side, so there are 10 entries in total. Moreover, all entries are sorted by price, so that the entries for bid are in descending order and those for ask are in ascending order. By observing the market depth data, it's not difficult to see that the first bid/ask entry of an order book should be always the same as the tick price data. As such, subscribing to tick price events is equivalent to subscribing to market depth events with depth 1.

bid

ask

position

price

size

price

size

1

3748.70

14

3749.00

71

2

3748.50

290

3749.40

10

3

3748.45

20

3749.45

1

4

3748.15

25

3749.85

50

5

3748.10

125

3749.90

15

So, how do we subscribe to market depth events? It's almost as easy as to subscribe to tick price events. The following code snippet creates an contract object specifying the stock we are interested in. In this case, it's Infosys listed in Mumbai.

Please note that though one can specify an arbitrarily large number for market depth, the effective market depth will still be subject to the exchange and your data subscription with InteractiveBrokers. More often than not, market depth data are not free, but there are some exceptions, like NSE in Mumbai. Once we have invoked the reqMktDepthEx method, we immediately receive the initial batch of market depth events, which are meant to initialize a complete snapshot of the order book. After the initial batch, we will receive subsequent market depth events on an ad hoc basis. These subsequent events represent increment changes to the order book. Basically, a market depth event consists of the following properties:

id: ticker ID, which is specified by us via the reqMktDepthEx method;

side: 0 = ask, 1 = bid;

operation: 0 = insert, 1 = update, 2 = delete;

position: indicate which position in the order book to update;

price: bid/ask price;

size: bid/ask size.

For example, the snapshot example above was constructed by the following initial batch of events:

event sequence

id

side

operation

position

price

size

1

1

1

0

0

3748.7

14

2

1

1

0

1

3748.5

290

3

1

1

0

2

3748.45

20

4

1

1

0

3

3748.15

25

5

1

1

0

4

3748.1

125

6

1

0

0

0

3749

71

7

1

0

0

1

3749.4

10

8

1

0

0

2

3749.45

1

9

1

0

0

3

3749.85

50

10

1

0

0

4

3749.9

15

From the 11th event onward, we receive increment changes. For example, the 11th event was as follows:

event sequence

id

side

operation

position

price

size

11

1

1

1

0

3748.7

17

Based on this event, we should update the first bid entry of the order book with price 3748.7 and size 17. So I hope this is clear enough on how to interpret market depth events.

As for to unsubscribe, we just need invoke the cancelMktDepth method with the ticker ID for which we want to cancel the subscription.

tws1.cancelMktDepth(1)

Finally, below is a simple demo program, which captures market depth events for 10 minutes and simply prints them out. This program does not present an order book visually.

Tuesday, January 21, 2014

In the last blog post, we saw how to set up connectivity to InteractiveBrokers. Let's move on to see how we can capture real-time tick price data from InteractiveBrokers. Before I show the complete example code, let me first go over some key steps.
The first thing is to specify for which asset you want to capture tick prices. We do that by filling in an IContract object. In the code snippet below, tws1 represents the connection to InteractiveBrokers and we use the factory method, createContract(), to create a blank IContract object. If the asset we are interested in is S&P 500 ETF Trust, we can fill in the IContract object as follows:

Then we can submit the IContract object via the reqMktDataEx method so as to subscribe to prices information of S&P 500 ETF Trust. The reqMktDataEx method takes 4 parameters. Here we need to focus on only the first 2: ticker ID and contract. Ticker ID is an integer, which we can choose any number as long as we assign a unique number to each asset we subscribe to. Below is how we invoke the reqMktDataEx method to submit the IContract object we created.

tws1.reqMktDataEx(1, contract, "", 0)

So now we let ticker ID 1 represent S&P 500 ETF Trust. If later on we no longer want to listen to the prices of S&P 500 ETF Trust, we can pass ticker ID 1 to the cancelMktData method to stop receiving any further updates.

tws1.cancelMktData(1)
After subscribing via the reqMktDataEx method, any tick price updates will be sent to us via events. As such, we write the event handler, processTickPriceEvent, to process tick price events. The handler is simply just to distinguish different types of tick prices and print them out. (As for the code of the handler, please refer to the complete code at the end of this blog post.) Then we register the handler with tick price events as follows:

tws1.tickPrice.AddHandler( new TickPriceEventHandler(processTickPriceEvent))

After registering with the events, we have to invoke Application.Run() to enter the event loop to start capturing tick prices. My intention here is to capture tick prices for 10 seconds and then quit. To end the event loop after 10 seconds, I set up the following asynchronous workflow.

Sunday, January 5, 2014

In this blog post we show how to write a F# console program which runs on Windows and connect to InteractiveBrokers (IB). There are a few different options of programmatic connectivity provided by IB. Here we use the ActiveX control (Tws.ocx) and Trade Workstation (TWS), so the end-to-end communication between our console program and IB goes like this:

F# code <=> Tws.ocx <=> TWS <=> IB server.

Before we begin, you need to have an account with IB, which can be either a real account or a paper account. Once you have the account ready, here is the step-by-step guide to the console program.

Install TWS and API:

Trade Workstation: please download and install a standalone version from here. You need to configure TWS to accept ActiveX clients by doing the following:

Select Edit->Global Configuration

Select API->Settings

Check "Enable ActiveX and Socket Clients" and note down the port number specified at "Socket Port". By default, the port number is 7496.

To make use of the ActiveX control, we need to add references to these 2 DLLs in the Visual Studio project. (Note that as the only thing we are trying to do here is connecting to and disconnecting from TWS, we need only AxTWSLib.dll. However, in following few blog posts, we will try to capture market data and submit trade orders, which will requires both DLLs.)

Write F# code to connect and disconnect:
Below you can see the code of our console program. There are some remarks about the code I would like to make:

Tws.ocx is an ActiveX control which is meant to be added to a GUI container. To get the control initialized properly, a dummy Windows Form is created to host it, as you can see at the beginning of the main function. As our intention here is a console program, we don't display the form.

Friday, January 25, 2013

Thanks to Tomas Petricek's invitation, I am honored by being able to write a chapter for an upcoming book - F# Deep Dives. The chapter I am responsible for is Chapter 4: Numerical computing in financial domain. The first draft of the chapter is now available online via Manning's early access program.

Friday, September 7, 2012

In the previous post, I mentioned that I chose System.Random, because it’s faster than any other random number generators available in Math.NET Numerics. And I also said that I have a minor concern about System.Random, because someone found a problem in the implementation of System.Random and raised this issue to the .NET team in early 2011. And the team admits it’s a problem. However, this issue is still there with .NET Framework 4.5. And it seems to me that the .NET team is not planning to take any immediate action.

As I know little about the algorithm of System.Random, I’m not sure if it’s worth the efforts to pick up the details and quantify the impact. As such, I decided to take an easy way out, i.e., close my eyes to this issue do Kolmogrov-Smirnov test (KS test) on only the final prices distribution. If the simulated distribution is statistically indistinguishable from the true, theoretic distribution in terms of shape, I will be fine with System.Random. At the end of the day, I think what really matters to me is whether System.Random can ultimately give me the correct shape of the final prices distribution I want.

Basically the F# code is pretty much the same as in the previous post except the following:

I add a run_stat_test function to run the statistical test and generate a Q-Q plot in Matlab.

I change the Monte Carlo parameter N to 1 so that I simulate only one data point, i.e., final price, for each simulated path.

You can see that I take the logarithms of all final prices and test them against a normal distribution with mean = (log S0)+(r-0.5*sigma*sigma)*T and standard deviation = sigma*sqrt(T) using KS test. And the output from the test is as follows:

h = false, p = 0.7243765247

And here is the Q-Q plot:

p-value is about 0.72, which means it’s quite likely that the simulated prices were indeed sampled from the normal distribution, and h=false means that we can’t reject the null hypothesis. Moreover, the Q-Q plot does not show any obvious outliers off the diagonal line. Therefore, it’s ok for me to continue to use System.Random.

If any expert happens to know about the issue of System.Random and is willing to shed some light on it, it will be appreciated.

Sunday, September 2, 2012

In all the Monte Carlo simulation codes I have done in previous posts, I simulate only the underlying prices at expiry, because European Calls and Puts are dependent on only the final price at expiry and we knew that the final prices follow a lognormal distribution, for which there's a nice analytic expression and hence we can easily simulate. However, if we move further out of the safe zone of European options, it's quite often that we need to simulate entire GBM (Geometric Brownian Motion) paths. Before we really get into dealing with path-dependent options, let's first take a look at my implementation for generating GBM paths based on the Euler method.

In this post, I started using a open-source numerical library, Math.NET Numerics. Don Syme wrote a blog on how to install and use it. Basically I use it in my code for two purposes:

using the Normal class to draw normal random numbers. Internally the Normal class also uses the Box Muller transform to generate samples, but it uses the polar form of the transform, which is faster than the basic form I had been using in previous posts. Another good thing about the polar form is that it does not suffer the infinity issue I mentioned in a previous post.

using some handy statistical functions, like mean, standard deviation, and inverse CDF, because I like to compute standard error and confidence interval.

Below is the code snippet to construct a function which generates Brownian Motion increments. You can see that I use System.Random as the random source, because it's a few times faster than all other uniform random sources provided by Math.NET Numerics. (I actually have a minor concern about System.Random. I might talk about it later in another post.)

// risk factor dW

let rnd = new System.Random(1)

let get_dW dt =

let dW = Normal.WithMeanVariance(0.0, dt)

dW.RandomSource <- rnd

(fun () -> dW.Sample())

The below function, generate_GBM_paths, is the one that actually simulate all the entire paths. Here I use an array of arrays to keep all the paths simulated. Storing every complete simulated path is memory consuming, but it's convenient for me to do post-simulation analysis. I think this code snippet is a good example to show that F#'s built-in array processing functions are really expressive and handy, compared to other major general-purpose languages.

As you can see from the main function of the code, after generation of GBM paths, I extracted final prices out of all the paths and passed them to:

the statistical functions provided by Math.NET Numerics to compute

mean, i.e., the forward price of the underlying

standard deviation of the simulated final price distribution

standard error of the final price simulation

confidence interval

Matlab to plot KDE (Kernel Density Estimation) to see if the simulated final price distribution looks like a lognormal distribution. I recently just learnt KDE from the book, "Data Analysis with Open Source Tools," written by Philipp K. Janert. In case you haven't heard about KDE, it's something like an advanced version of histograms. In a KDE, the graph will be smoothed by a density function and you don't need to find out what bin size you should use. If you don't like to run this KDE part, you could simply comment it out.