Summary: The Tyranny of Metrics

My book summaries are designed as captures for what I’ve read, and aren’t necessarily great standalone resources for those who have not read the book.

Their purpose is to ensure that I capture what I learn from any given text, so as to avoid realizing years later that I have no idea what it was about or how I benefited from it.

My One-Sentence Summary

While metrics can and do offer extraordinary benefits when they’re used carefully and properly, there is significant chance of them being chosen incorrectly, being gamed and corrupted by various parties, and ultimately becoming toxic to the very cause they were created to help.

Content Extraction

Empower others, but be willing to step in with micromanagement temporarily if things get out of hand

Don’t be so dominating and intimidating that your leaders can’t step up and lead themselves

Discipline is great, but too much leads to a lack of creativity

Too much creativity and not enough discipline leads to sloppiness and mistakes

The more a quantitative metric is visible and used to make important decisions, the more it will be gamed—which will distort and corrupt the exact processes it was meant to monitor.

An adaption of Campbell’s Law

And the second is a similar, more simplified version of the same, by Goodhart:

Anything that be measured and rewarded will be gamed.

An adaption of Goodhart’s Law

All quotes here are from the book itself unless otherwise indicated.

“The three components of great training are realism, fundamentals, and repetition.”

Metrics can be and often are useful, but the thing to avoid is Metrics Fixation, which is where you replace judgement with numeric indicators, you think making metrics public will solve everything through motivation, and thinking that the best way to motivate people is by giving them money or ordinal rankings.

I’m not convinced this is true in theory, but it’s definitely true in practical terms given current society and technology.

Not everything that matters is measurable

Not everything that’s measurable matters

Common flaws

The most characteristic feature of metric fixation is the aspiration to replace judgment based on experience with standardized measurement.

Takeaways

Metrics are a good way to align a team around a goal.

The more numeric, visible, and reward-tied a metric is, the more likely it is to be gamed and turn toxic to its original purpose.

When you use a metrics program, be sure to periodically ensure that undesired externalities have not emerged as a result. And be prepared to go digging, since the negative effects could be well-hidden.

Moderation is key. Use metrics, but don’t let them control you or become a substitute for judgment.

Remember to constantly revisit the spirit of what you’re trying to attain, and continuously ask yourself whether the tangible things you’re tracking are high-signal proxies for those goals.

Notes

There is a previous book by the same two guys, called Extreme Ownership, and while it was good, it did emphasize the extremes of each point that was made. This book corrects that by focusing everything on the balances that have to be constantly adjusted for the situation. This is basically the better version of the first book, but you can still benefit from the first one as well.