Wednesday, September 02, 2009

Author Malcolm Gladwell, in his fascinating bestseller "Outliers: The Story of Success", cites neurological research showing that "10,000 hours of practice is required to achieve the level of mastery associated with being a world-class expert." This seems to apply across many different types of experts, whether they are "writers, ice skaters, concert pianists, chess players ... Even Mozart ... couldn't hit his stride until he had his ten thousand hours in".

Reflecting on my own experience, I have become consistently profitable only after 4 years of actual trading (research alone doesn't count -- real money need to be at risk.) So while the number of hours may not be exactly 10,000, the order of magnitude is about right.

So if your trading has not been profitable, ask yourself this: "Have I traded 10,000 hours yet?"

20 comments:

Anonymous
said...

interesting assertion. i think the 10,000 hours might be, under certain conditions and assumptions, a necessary condition. but it probably is not a sufficient condition. so for all of those people who put in five years of real effort ( 8 hours a day for 250 hours a year )maybe half might do "well" (whatever that means) and the other 50% will wonder why they ever bothered

I heard the general gist of this book. It's pretty interesting.. But for the 10k hours, I'm sure is just a generalization.

What if the person make the same mistake repeatedly and never stop to find out how why the mistake was made or learn from it or even have new insight..all for over 10k hours? Then there is no improvement and thus equal to zero hours.

It's just like someone claims to have 5 years of programming experience when all they do is just type in "if then else statements" in a heavy duty language like C++, but never be interested in find out if there is any more to "if then else", such as Object Orient Programming, and the traps and pitfalls. (C++ has many dark corners). Then these people is equal to someone who has only 100 hours of experience when they spent 10k hours. The opposite is true.. they can accelerate the learning process by going into advance stuff and thus the 100 hours becomes like 500 hours.

Same goes for trading. I know someone who 'trades' daily since before the dotcom boom, but when I asked them what method they use, they say they just buy when stock is cheap or the trend is in the direction and double up if it dips from the general direction. This is the same guy who lost 300k previous due to margin call and because of the recent market, got back 200k.. meaning so far a net loss of 100k.. over a 10 year period.

Gladwell’s point applies for specific domains. However, it’s out of scope when it comes to trading because it fails to characterize random and dynamic nature of markets. In most domains the rules are clearly stated and hardly change, but when they do the participants are notified. The markets however don’t clearly state the rules and certainly don’t tell the participants when they change.

Under the domains of “writers, ice skaters, pianists, and chess players” 10,000 hours allows one to take advantage of the law of large numbers. These activities are static, they don’t change (or at least not significantly), so practice builds upon practice and your skills increase. After 10,000 hours you’ve made many mistakes and have had the opportunity to correct your mistakes.

It is more difficult to apply the law of large numbers to trading because the markets are dynamic. Regime shifts, fundamental changes in the way the system works, and randomness can destroy prior work. The old trader’s saying, “the system works until it doesn’t” captures the difficulty of identifying regime shifts and fundamental changes in the market. You might have a profitable mean reversion trading system (any strategy will work for this example) that constantly makes money. Then one day you start losing money when your positions don’t follow the pattern you predict. The losses can came as either a “once in a lifetime” unforeseen event or a large string of small, consecutive losses that, on the aggregate, kill you. At what point do you abandon ship and scrap the system?

Imagine that the rules of chess change at random--random rule changes at random times. Say you play for 10,000 hours and developed a good idea that to win you must capture your opponent’s king. Then suddenly the rules change, capturing your opponent’s king no longer makes you the winner, rather you win by capturing the opponent’s rook and the pieces move in different patterns. Pawns move like knights and knights move like pawns. You’ve spent 10,000 hours and became a master at a certain type of chess but those rules don’t apply any more. How many hours would it take to become a master at this new kind of chess?

It would be interesting to expand Gladwell’s theory to include traders. While 10,000 hours may be necessary for certain disciplines, I’d like to know of disciplines where participants spent 10,000 hours yet failed to achieve mastery. I’d also like to know of participants who spent 10,000 hours and failed to achieve mastery. Gladwell’s work suffers from a confirmation bias because he looks at all the successful people and sees that what they all have in common is 10,000 hours. He then infers that 10,000 hours makes a master. Reading “Outliers” I found no effort to find evidence counter to his thesis. The proper way to understand the situation is to look at the people who spent the time but didn’t become a master.

A lot of what you say is true. The markets are forever changing. But the people who run the markets do not change. Human behaviour is the same as it was 5000 years ago when the Indians and Chinese were trading rice. Boom will follow bust.If you can smell greed in the air then pause for thought. Antony Bolton, the british warren buffet (!), published an article in early 2007 in the Times warning of cove lite loans, meaning covenant lite. He went on to say he, in his 30yr career, had never seen these products and that we were in a credit bubble. Greed and fear will never change. If you are trying to find technical trading rules - then yes, they change all the time - just as the ways to win a game of chess changes. 10,000 is just a number and its meaningless.

Trading is indeed harder than many other disciplines because the rules are constantly changing. However, there are still successful algorithmic traders who have been profitable throughout their long career. (Jim Simons? David Shaw? Ed Thorp?) Sure, some of their individual strategies have failed. That happen constantly, to them, to me, and to every trader. But good traders constantly come up with new strategies, and they also make sure that the demise of old strategies is graceful, gentle, and non-violent.

I read the comments and read how 10,000 hours is not enough because trading is harder and does not involve rules.

I would add 10,000 hours is the minimum if you have the ability to be a trader.

Take the case of me trying to be a professional hockey player. I can try all I want I will not make the cut. I can be athletic, but I am not athletic enough.

Or how about musician I will not make the cut...

But with respect to an engineer, or trader, or even painting I would completely agree. Since I already have the disposition to be an engineer, artist, and trader.

So while people say the market is random, I say it is not random. In fact it is anything but random. It just appears random to those that don't have a disposition to be a trader. I don't mean this comment to be cruel, but then again you are not going to see me playing center in the NHL...

Hi Ernie, a little out of the topic question, but can you answer me this, at what point did you start managing people money and at what point did they become interested in investing in your strategy ( pair trading )? Did you show the backtested results? Did you show the forward tested trades?

I started to manage a joint account with my partner about 14 months ago, and started managing other investors' money since March 09. We show investors the actual trading track record, not the backtest results.

Any advice on how to start quant trading without a programming background. I have been trading the last two years as an independent trader. But I have been very interested in quant and the innovation around the technique. I've read your interview on how you got started but would like more details. We can exchange emails if that is the prefer way to discuss this topic. Thanks

The thing was that i am also a "at home " quant trader but my capital is very small. ( Enough only for trading 2 contracts on each sides of a pair trade ). Now i was wondering whether to continue on this path to generate profits out of pair trading and go on increasing the size of the contracts and thus profits or to look out for people to invest in the strategy.

I think, I have to second bwc2000.Absolutely, on an absolute scale, 10k is a plausible number, often very necessary number. But there is a method to madness. And thats by immersion. So when you immerse yourself in trading or in any art, you are basically doing it, even when tickers are not running.

Its in those times, when you think, and think incisively you gather deep insights, some fundamental perspective, and that has/will serve to reduce your real time to maybe 3000 or 5000hours. So you are actually mentally trading for 10k hours, but in reality perhaps 5000. Which I would say is a blessing for beginners. Since, they might not have the staying power to take an exposure to the market of 10k hours, or even 5000 hours. So, I would say, take a simulator, Run it on real historical quotes, take ample breaks, dont rush in things, and do immense deep thinking. At one level, I dont know, if the quality of deep thinking is different from the deep thinking you might have when you suffer reverses in the market, but I would like to believe the practice you get from such an exercise will prolong your staying power in the market enough to actually give you good real live insights, which will then again better your performance.

Hi, Ernie, I don't understand the 10000 hr before consistently profitable claim. Ain't algo trading suppose to be rules? Where is the human factor that require the 10000 hours or any 'training' time? And you removed time spent on algo development in the same comment.

Hi anon,The 10,000 hours include the time we spend researching and programming an automated strategy, and time spent on observing its live executions and revising/improving the strategy as a result of those observations.Ernie