Why you should care

Because it’s a numbers game.

By Gary Smith

The Daily DoseNOV 22 2016

Gary Smith is the author of What the Luck?: The Surprising Role of Luck in Our Everyday Lives (The Overlook Press).

A frustrated woman sought help from the advice column Ask Amy: “I keep meeting men who appear to have it all together,” she wrote, noting how it wasn’t too long before she realized she had them all wrong. “Oftentimes they live with relatives, they are hung up on an ex-wife or girlfriend, they are often financially irresponsible and/or have serious emotional issues.”

She’s hardly alone. We have all been attracted to glitter that turns out to be gloss — the false smile, the tiresome self-absorption, damaged goods. Believe it or not, our frustration and disappointment may just be another example of a pervasive source of confusion that statisticians call regression.

The bad news? Being let down by regression is normal. The sobering truth? Others probably feel the same way about us.

Teachers don’t know who will be their best students, doctors don’t know which treatments will be the most effective and investors don’t know which stocks will soar. Companies don’t know which job candidates will be the best employees, just like voters don’t know which politicians will be the most worthy. So it’s only natural that we don’t know which prospective soul mates will be genuine and which will be duds.

When we try to assess anything, whether it’s a person, place or thing, it’s always possible to overrate or underrate it. But the regression insight is that those things we rate the highest are more likely to be overrated than underrated. When a student scores 90 on a test, how likely is it that this student had a bad test and would normally score even higher? Not likely at all. When we see a student do well, we assume she will always do well. But there is luck in the specific questions asked and in the answers given. It’s unlikely that someone will have bad luck and still get the highest score, so she probably had good luck, which means that she is good but perhaps not as extraordinary as her score suggests. Same goes for job candidates. If one aces an interview, how likely is it that this person had a bad job interview? When a company’s earnings increase by 30 percent, how likely is it that the company had a bad year?

The student, job candidate and company, in other words, all regress. The student will probably get a somewhat lower score on the next test; the job candidate will probably not perform as well on the job as expected; and the company’s earnings will probably increase by less than 30 percent next year. The regression principle says that whatever seems to be far above average probably is above average, but not as far above average as it seems.

The same is true in our search for love. Everyone looks for different things when it comes to finding a soul mate, so let’s just call it “pizzazz.” You may be at work or play when you see someone who seems to have plenty of pizzazz. But when you get to know the person better, there is usually disappointment.

It is unlikely that someone would be underrated and still seem to have the most pizzazz. So the person who seems to have the most pizzazz is probably overrated. This does not mean that we shouldn’t choose those who appear to have the most pizzazz. But the regression principle shows us that we shouldn’t be surprised if our love-at-first-sight has less going for him or her than we thought. The bad news? Being let down by regression is normal. The sobering truth? Others probably feel the same way about us.

This also explains why the grass always seems greener on the other side and why familiarity breeds contempt. Many get fooled into thinking that they’ve found true love outside their marriage … only to be disappointed later. Regression predicts that most affairs will end badly. It also cautions against giving up on what you have — because what you covet is probably not as great as it seems.