give an explanation of this. If the market is controlled by fear and greed and operates from the herd mentality, then why do so many people in the herd lose money?

More...

Interesting question, and I wish that I had a good answer, which I don't.

You may be right that fear and greed control the markets in a very short term perspective. In a longer time frame, I believe that the markets are more controlled by a future anticipation of something, such as a company's earnings or the future economic health of a country or global economy, and so on.

Every trader has their own story to tell, but there may be some similarities in all the trading stories. Being the imperfect beings that we are, we have a tendency to become overconfident and impulsive after a string of wins, and overcautious and risk adverse after losses. We also become too bullish at market highs and too bearish at market lows. Even though I have traded the markets for several decades, I still have a tendency to make these mistakes from time to time.

What has helped me to become a better trader is to learn to analyze the markets first on a long term time frame chart, and then work my way to shorter time frame trading opportunities ( similar to what Dr. Alexander Elder and others do). I have learned the hard way to understand normal market cycles, long term market sentiment, and short term profit potentials. This has helped me to develop objective trading rules, and rely less and less on emotional trading.

So if there are 100 retail traders in the herd who are selling on panic then how is the one person who is not trading with the herd going to make money by going long when the majority of the force is down ? IE this is similar to you buying 10 futures contracts while 1,000 are being sold by those in panic.