"When the tide goes out, you can tell who's been swimming naked" {W.Buffet}

R for Metatrader

Post navigation

The Shared-Memory between multiple R sessions mentioned in my previous post got me thinking … quite some potential indeed. As a result, I investigated further using (calling) multiple R sessions from the same MT4 script. Specifically, I wanted to have a clearer understanding of the time required to performed lightning fast & dead slow processing, while being in the same MT4 script and without preventing real-time tick analysis (that is HFT potential at least on the receiving+processing side).

I drafted the following scenario to compare performances:

Single R session called at every tick-update, displaying Bid/Ask @tick-update and using ChartSeries to plot either 500 bars or 10,000 bars – Bars are generated in the Init() function.

2 R sessions called sequentially, first R session displaying the Bid/Ask @tick-update, the second using ChartSeries to plot either 500 bars or 10,000 as previously

2 R sessions, but where the second R session is executed asynchronously, the first session plotting the Bid/Ask in @tick-update, the second using ChartSeries to plot 500bars/10,000bars.

The 10,000bars chart is used to further amplify potential anticipated observations made with the 500bars (that is a delay, or even a loss, of ticks due to the time required to “sequentially” render the chart). The two major area of concerns I had were the cooperation of 2 R sessions within the same script and the second the time required to perform the asynchronous call.

RESULTS are presented thereafter; y-axis is the number of ticks (GbpUsd) and x-axis is the time required to perform a complete MT4 Start-loop in [ms].

CONCLUSION: I have been very surprised by the behavior of the Asynchronous mode of MT4R. The cost of checking if an Asynchronous call is pending of finished prior re-issuing another Asynch call is about 1ms. This, added to the 1-2ms required to display Bid/Ask @tick-update depicted an extremely fast data processing system.

Although I’m not too happy about the set-up as I think a c++ multithreaded (or mpi) would have been better (through Rcpp or else), I can’t really discard the obvious that based on these simple tests, the R Asynch multisession set-up seems perfectly feasible for a Live Trading System. I even think I will push the limits to testing HFT trading, for which I will probably have to recompile Blas and Atlas anyways.

Finally, upcoming strategies should be design so as to separate real-time processing from asynch processing. As well, I did not yet evaluate the time required to send trade signals from R -> MT4, neither did I evaluate the Broker time to accept such signal (what’s the point of processing data at 2-5ms if trade signals are executed at 250ms)

Considering forwarding tick data from MT4 to R requires less than 2ms, but that charting 4 different time-frame (1min, 15min, 30min and 1hour) at each tick-update may require more than 250ms (depending on the number of bars in history), I think it is fundamental to investigate further in different types of non sequential programming and R, specifically multithreading or parallel programming.

Indeed, although charting is perfectly useless for our purpose of algorithmic trading, it is a valuable example to demonstrate a main loop that still receives tick updates at a rate of few milliseconds, while another “thread” would chart different time-frame every 1/2 seconds, without impacting the real-time gathering of ticks.

Along with the evaluation of MT4->R data transfer investigation, this is another major area to clarify prior stepping into the system design world (the fun part), considering our goal is to achieve a high-quality trading platform with Open Source Softwares.

Currently, I’m investigating:

Parallel computing in R through Rmpi, Snow. I found that particular example applied to optimizing an algorithmic system on backtested data at Revolutionalytics. This comforts me in saying that I’m probably following the right path. A list of High performance R computing can be found here.

On the Shared-Memory between multiple R sessions, I wanted to add that this strategy may support the development of multicurrency trading systems. Each currency would have its own Metatrader EA that at least reports the Bid/Ask continuously for that given PAIR, in a Shared-Matrix space …. actually quite a lot to think about but i shall not yet go that way 😉

The more I get into this, the more I adjust my objectives. Currently, I’m willing to make the most of the Quantmod and Blotter R-packages, while eventually building a MT4 equivalent of the InteractiveBroker-R_API. As a first step, I was willing to Chart ticks from MT4 in R using the Quantmod’s chartSeries().

There are different options to export ticks from MT4:

Using the DDE – the major drawback, if I’m not mistaken, is this is an export-ONLY mechanism. Considering I’m willing to use R to generate the trade signals, the DDE is not an option.

Using sockets on 127.0.0.0 – the major issue is the extra-work required to design the communication protocol. However, there are some interesting sockets ideas. I believe this is a similar, apparently efficient, COM/DCOM server design.

Through a database interface such as MySQL.dll for MT4 – on the bright side, it allows for the easy management of tick data (using timestamps as main key), but I believe this would bring quite some extra-complexity and speed issue. Here is an example.

Thereafter is a screenshot of my R-Chart from MT4 tick, along with a video of the overall performance regarding R-tick “real-time” performance (although my code is not yet designed for speed).

I have uploaded a video of both MT4 next to R-chart so as to have a sense of the R-charting delay: Megaupload link ( .swf playable in any browser). As you may notice, at some specific time R is lagging behind MT4. However, one should keep in mind that, at every tick, I pass the last 500 OHLC+Volume quotes from MT4 to R, whereas a more logical approach would simply append that last new tick. Nonetheless, this gives some insight in R’s performance once we’ll start performing real processing through R, to eventually lead to a trade signal.

From that first test, I was relatively concern by the ability to quickly send MT4 ticks to R in real-time. I decided to write the simplest transfer logger in that I simply send Bid/Ask prices @everytick from MT4 to R (assigning 2 double [variables] in R) while measuring execution time. Surprisingly I found out only 1-4 ms are required. Here is my logged-video Megaupload link. I quickly added a line to pass a Matrix[1000] that I initialized in the Init() that resulted in an average increase of 2ms (Megaupload link). I will further investigate different processing scenario to have a clear picture of MT4->R->MT4 live-trading set-ups.

I have designed and developed Automated Trading Systems (ATS) for few years now. Since trading live, I have been fairly profitable (that is, I achieved my goals). However, I had the unsatisfactory feeling that I was missing some information to increase the level of trust in my systems … I was missing the proper tools to closely examine, and follow-up on the Real Equity Curve of an ATS.

There are quite a bit of information on the web for such but I did not find any Free Open Source tools to perform such assessment, specifically for Metatrader. Indeed, I do use MT4 for trading my strategies, and it’s fine with me; no issue nor any concerns. You might think otherwise but I will not go there … that’s far, far from my current objective.

As a first start, thereafter I present 2 examples of my programming kick-start: (1) The aggregation of multiple StrategyTester reports and, (2) the visual follow-up of an optimized MT4 Moving Average from backtesting to paper-trading. Every information and material needed can be found at http://code.google.com/p/rscripts4metatrader/

(1) Using: MT4_StratRep_Analyzer()

(2) Using: MT4_RealEquityCurve_Analysis ()

Metatrader Optimization on the Moving Average EA for the period 2008.01 -> 2009.01

Post navigation

Disclaimer

Your access to, and use of, this Site is subject to the following Terms Of Use. By accessing and using this Website, you are deemed to have read and accepted these Terms Of Use and the regulatory restrictions governing the use of this Website.