Hedge fund managers live and die by their performance. Their profit and loss numbers show whether they made or lost money. But are performance figures an accurate measure of success?

Doug Hirschhorn, an author and trading psychology coach who coaches portfolio managers and principals at hedge funds, uses a sports analogy to show why looking at an outcome-based analysis of trading success can be misleading. He describes a tennis player who loses a match in three sets: 6-0, 6-0, 6-0. “If I asked you if this guy was a good tennis player, you would conclude that he wasn’t. But what if I told you that his opponent was Roger Federer, and that every game had gone to deuce? Then you would rethink.”

A hedge fund manager’s performance figures don’t show the mechanics of how he or she reached those gains or losses. A behavioural bias called the self-serving attribution bias suggests there is a tendency for managers who do well to attribute their success to their own skill, while those who trip up tend to blame external factors. The role that luck plays is underestimated. And looking at why traders have made money is just as important as looking at why they have lost money.

Clare Flynn Levy, founder and chief executive of Essentia Analytics, which has developed cloud-based software to capture performance data, said: “Performance data is not really a measure of your skill. It’s a measure of an outcome. Your skill is a contributor to that outcome but so is luck. Without being able to discern between the two, you end up taking credit and blame for stuff that isn’t you.”

Simon Savage, co-head of European and global long/short strategies at GLG Partners, a hedge fund manager owned by Man Group, said: “The perennial challenge with financial markets is that so much is driven by a noisy data set. You might be completely right for the wrong reasons.”

Improvements in technology and data analytics enable managers to dissect individual decisions so they can systematically understand their own attributes and work out where their “edge” lies. Is the trader good at running winners and cutting losers? Are the positions sized correctly? Was a trader correct for the right or wrong reasons? Was there a good outcome for the wrong reason? Often managers don’t have a good grasp of why they are making money.

Hedge funds use the Sharpe ratio to show whether their outperformance is skill or luck. This ratio, which measures risk-adjusted returns, sets out to tell an investor in a hedge fund whether historical profits have been generated through accumulated smart trades or a single jackpot win – which could have been luck. But this new data analysis goes deeper because it focuses on the process and thinking behind trades. The decision matrix, taken from coaching materials at a large hedge fund, shows that this fund is looking deeper than just whether a trade made a profit.

What the data reveals

Data analytics can shed light on a trader’s hit rate – the percentage of investments that are right or wrong – or their win/loss ratio. This is the average profit from a winner divided by the average loss from a losing position. A high number demonstrates that a manager has an innate ability to find investments with more upside than downside. Capturing data can also show whether a trader makes money from trading around positions or from a buy-and-hold strategy. It can compare the effectiveness of jumping into a position as opposed to scaling in.

Coaches encourage managers to keep an online trading notebook or diary to avoid relying on their memory, which is “amazingly inaccurate” and “dangerous to rely on”, according to Savage. This has two benefits. It is an audit of decisions and within that audit, traders and coaches can look for patterns, with a view to practising more of the good things and less of the bad.

Savage said: “Now there is more evidence within the process of decision making. It’s more evidence at a level at which you think skill will show itself. Outcome is completely ineffectual.”

Hirschhorn said that just as a tennis player might repeatedly watch a video of their match, a trading journal forces managers to look at a replay of how they think about trading the markets, to hone in on what the trade is, why they’re in it, anything that has changed, the risk/reward profile and the sizing of the trade.

Hirschhorn said: “Process is the core element. Process leads to profits.” Typically managers have established conviction based on intangibles so Hirschhorn uses process to match the conviction level with the sizing.

Hirschhorn looks at the data with his clients in basis points and percentage points, rather than dollars. He said: “You have to be able to separate the money from the trading decisions because the market doesn’t know if you’re up or down, long or short. The market doesn’t know your individual trading situation.”