Monday, February 16, 2015

Some conclusions about the behavior of markets

After a couple of months of attempting to apply my interpretation of Kuperin et al's method to the prediction of forex market trends with neural networks, I've come to a few of realizations.

I've concluded that neural networks are, in fact, insufficient for predicting market trends. I've also come to believe that technical analysis is, as a whole, just a pseudoscience. I've come to believe these things because I've noticed several recurrent flaws in the studies that presume to demonstrate the efficacy of these forecasting methods.

Take neural networks, for example. The way you test the efficacy of a neural network in predicting market trends is by taking a series of steps:

1. First, you have to choose a network architecture.
2. Then, you acquire some large dataset to train and test the neural network you picked.
3. You train your neural network using a section of this dataset.
4. You "tell" the neural network to make a prediction, and see how close the network's prediction is to the actual data, taken from the same dataset.

One problem lies in the fact that, in step 1, you can choose from an enormous set of different types of neural networks. There are infinitely many different neural networks to choose from, and some of them will perform better than others. Does finding one which produces statistically significant results for a specific dataset really mean anything?

If you choose some neural network architecture, set it's number of neurons, you tune the sigmoid function for it's neurons, and then run your tests, and it turns out that the quality of the predictions is unimpressive, what do you do? You just change the parameters, and try again. You try a different architecture, different number of neurons, alter the sigmoid, and let's say that, again, you find nothing interesting. So you keep changing the parameters until you come up with a neural network that seems to work.

But what's the actual extent of that neural network's efficacy? Maybe it only works for the dataset you're using. Maybe if you used the data of 10 years from now, the same neural network would produce bad predictions.

So you ask yourself these questions, and you decide to make another test. You train and test your neural network using data from a specific period, say, the usd/euro fluctuations from year 2006. And then you test the same network architecture, by training and testing it using data from 2007, 2008, 2009...

But even if you find a neural network that works along that whole time line, wouldn't you be falling in the same survivorship bias as before? That is: it makes no difference whether you test your network with 2006 data first, and then with 2007, 2008, 2009... data, if you're going to throw it away if it turns out that it doesn't work along the whole time-line. That's basically the same thing you did before! It seems to me that this would just be cherry-picking the "best" neural network, out of the infinite set of possible different neural networks. There's still no guarantee that, 10 years from now, that same network you picked will produce good predictions.

This is no way to do science.

In science, you should have some degree of control over the "independent variables" which affect the outcome you're attempting to predict. Technical analysis assumes that future fluctuations can be predicted, to a degree, from the information of past fluctuations in price. From technical analysis, you could consider the "independent variables" to be a set of past price fluctuations. And the "dependent variable" would be the future fluctuations, which you are trying to predict.

The assumption is that there exists some mathematical formula which takes in it's input a set of past fluctuations, and spits to it's output a set of future fluctuations. An example of this is the MACD indicator. But MACD suffers from the same survivorship bias problem I laid out before. If you intend to use MACD to predict price fluctuations, you should know that MACD uses a set of parameters. The MACD is actually "tunable". So, if your MACD starts to fail, you can always tune it to "fix it". But then, does the MACD really work at all? Or are you just fooling yourself?

Tuning the MACD, or a neural network architecture for that matter, to fit the data, is the same as changing your theory every time you're confronted with contradicting results. What if there's really no mathematical relation between the past of price fluctuations, and the future of them? I mean, you keep scrambling your formula to try to come up with one that produces good predictions. But if the fluctuations are random, your formula may just be some locally optimal formula that works for your particular dataset. Again: no guarantee that it'll work for future datasets.

You really have no control over the behavior of markets in order to do the kinds of experiments required to extract underlying principles for their behavior. You may think that using different datasets to test your formula is a sort of "control" over the input parameters for the formula. That is: you can use a different set of inputs and outputs, to see if your formula works there too. But you're not controlling the underlying causes that made the data be the way it is. You're just hoping to find some periodicity in a time series. This is not at all the same as stimulating a system (i.e., the market) to find causal relationships.

Maybe there IS a formula that can predict market fluctuations. But I think the correct way to construct this formula, is not by scrambling the weights in a neural network, or the parameters in a MACD, or any other technical analysis indicator. The correct formula should be based on principles extracted from the behavior of economic agents. From an accurate and tested model of the behavior of investors.

I think the correct approach would be to think of markets in the way we think about the weather. People who study the behavior of weather apply the principles of the dynamics of fluids to the complex system that is the weather. That is: principles which have been extracted from controlled lab experiments, where we can actually control the independent variables which affect the physical behaviors of simple components of the weather, such as the air, or water.

The way to construct a correct mathematical model for the behavior of markets would be to extract the principles which govern their simplest components. In the case of markets, the components would be behavior of individual investors, the behavior of companies, and societies, and even the behavior of natural catastrophes or events which might influence an investor's decision to buy or sell. The market is, in fact, an enormously complex system, which involves all these things, and maybe more.

Using something like a neural network with 200 neurons to try to predict the fluctuations produced by such a beast seems to me like a long shot. The correct model would have to be, from my point of view, in all likelihood, much, much more complex than that. Not to mention that the very methodology of using past fluctuation data to try to predict future fluctuations is just wrong. Trying to find periodicity is not the same as modeling.

An example of causal relationships in markets can be learned, for example, from "front running". Front running is a kind of market fraud that takes advantage of a particular market inefficiency which guarantees a rise or fall in price. So people who invest based on front-running actually KNOW that the price is going to fall or rise. The fact that it's documented as a (mostly) certain way of predicting the market which, because of this, is illegal, is proof that there exist basic principles which govern the markets, and allow people to make accurate price predictions. I'm not saying that we should start a front-running scheme. I'm just saying that the underlying principles exist, and they've even been documented and classified as "cheating". Maybe it's possible to extract some other principles, through observation or experiments, in order to produce a better predictive model for price fluctuations.

ForexTrendy is a state of the art software capable of determining the most profitable continuation chart patterns. It scans through all the charts, on all time frames and analyzes every possible breakout.