In this reading assignment we will dive deeper into the most important issue when it comes to building a profitable trading strategy - Backtesting. We will look at some of the common mistakes, over-optimization and curve-fitting and how you can avoid them. Read through this blog post and answer the following questions in this forum thread. Use the knowledge you have learned so far as well.

What is so dangerous about over-optimization?

How long should a testing period be if you are serious about building a profitable trading strategy?

curve fitting; it maybe over adjusted or tailored to past events (back testing) and may not be suitable for the future markets or new future data sets.

snapshot 6 months; with a comparison to the longest possible (>5 years ideally, 9 to 11 years) data set humanly possible!

Focus on the development of a trading system aiming for adaptability, broad optimizations, robust profitability and large periods of testing data increasing better chance or probability of achieving a Ferrari.

Asymmetric system; Asymmetric information can lead to adverse selection, incomplete markets and is a type of market failure.

What is so dangerous about over-optimization? Over-optimization is dangerous because it trasforms the strategy in a curve-fitting of past data that will be never identic to the future data. So the very good results on the past data will not reply in the future.

How long should a testing period be if you are serious about building a profitable trading strategy? In the traditional market a testing period of 9-11 years should be used.

Why should you avoid asymmetric trading signals? Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

1. What is so dangerous about over-optimisation?
It will only be optimised for the specific data set. This is not a clear indication of what will happen in the future. Your algorithm becomes more of a curve-fitting program; this will mean losses in the future as it eliminates the probabilistic nature of the market.

3. Why should you avoid asymmetric trading signals?
These trading signals should be avoided as they are generally based around the economic variables at that time period for the particular currency (interest rates etc). A good example is over optimisation of previous data sets. This can create two separate sets of criteria in the market for “shorts” and “longs”. This strategy becomes excessively prone to curve-fitting.

A strategy performing extremely well on a given dataset is likely to be “unready” to other market situations and conditions. Meaning, it is not general enough. Which might lead to missed profits or even loss of the funds.

How long should a testing period be if you are serious about building a profitable trading strategy?

About 10 years to train the strategy

About 2 years outside that period to check how it performs on new unknown data.

Why should you avoid asymmetric trading signals?

More probability of making a mistake. And more space for over-optimizing due to “more parameters to tweak”.

Over-optimization yields a result that is not generally applicable, but only optimized for that particular dataset. Therefore, it can’t be reliably used for other sets of data, which is directly the goal of optimization in the context of trading.

The testing period should be on the order of a decade, because this provides enough market data to account for both micro- and macroeconomic cycles, and other cyclic events in the economy. In the context of cryptocurrency trading, since the market is less than a decade old, one must test through a least one bull and bear market, and we’ve seen a couple of those already.

Asymmetric trading signals create bias toward a certain trend. This partially invalidates the model, since it makes an assumption that the general behavior of the tested dataset will remain the same in the future.

(1) What is so dangerous about over-optimization? This can lead to curve fitting which is dangerous because it can give a false sense of predictability of future data.(2) How long should a testing period be if you are serious about building a profitable trading strategy? Time frame should be greater than 30 minutes. Ideally 9 to 11 years of data should be used for the process in order to ensure that a large amount of market conditions become available.(3) Why should you avoid asymmetric trading signals? This can increase the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

If you over-optimize you are fitting your algorythm to specific circumstances that have happened in one section in the past. Here you risk making it too specific to those and it could make it miss fire as the data in the future is unlikely to be exactly the same.

About 9-11 years from the post… However this could be difficult with bitcoin trading as its newish.

Asymmetric systems rely on macro economic variables that change throughout the ecconomic cycle.

Over-optimization can lead to the curve-fitting issue in which your algorithm gonna lost predictive power. Remember you want to develop an algo to predict the future, not to perfectly fit with past data.

The testing period should be long enough. In crypto world, it should be more than 4 years.

We should avoid asymmetric trading signals because the asymmetric can eventually lead to curve-fitting in the long run.