Thursday, September 27, 2007

How Long Should I Backtest An Online Daytrading System?

I am frequently asked how long one should backtest a online daytrading system. Though there's no easy answer, I will provide you with some guidelines. There are a few factors that you need to consider when determining the period for backtesting your online daytrading system:

Trade frequency

How many trades per day does your daytrading system generate? It's not important how long you backtest a daytrading system; it's important that you receive enough trades to make statistically valid assumptions*: If your online daytrading system generates three trades per day, i.e. 600 trades per year, then a year of testing gives you enough data to make reliable assumptions*. But if your trading system generates only three trades per month, i.e. 36 trades per year, then you should backtest a couple of years to receive reliable data.

Underlying contract

You must consider the characteristics of the underlying contract. The chart below shows the average daily volume of the e-mini S&P:

It doesn't make sense to backtest a trading system for the e-mini S&P before 1999, because the contract simply didn't exist! In my opinion it doesn't make sense to backtest an e-mini trading system before 2002 because at that time the market was completely different; less liquidity and different market participants. I believe that a reliable testing period for the e-mini S&P are the years 2002 - 2004.

The problem is that many traders over-use the functions provided by the different backtesting software packages and think more is better. Many so-called system developers try to imply that the longer you backtest the better and more robust your system will be. That's not always true.

Conclusion

When backtesting you need to know these things. It's not enough to just run a system on as much data as possible; it's important to know the underlying market conditions. In non-trending markets like the e-mini S&P you need to use trend-fading systems, and in trending markets like commodities you should use trend-following methods.