Data-themed articles, essays, and studies

Tyranny Of The Median

expected value = (value of an event) x (probability of the event) +

(value of no event) x (probability of no event).

Casinos use this formula to calculate the expected payout for games like roulette (using a negative “value” for a lost bet). In the long run we’ll lose, but in the short run, we might win, or win big – that’s the thing with probabilities.

The same essential formula is applicable in general decision making, although unlike the casinos we might have to deal with uncertaintities in our estimates of the probabilities and values involved, and deal with the possibility the expected value is zero within those uncertainties. That’s OK, it just says in that case we don’t have a numerical basis for a decision.

What if we were to look at things another way, which competes with the risk assessment approach:

value = (value of median outcome) x (median outcome)

A “median” value is the one you get if you line up all of the outcomes from smallest to biggest and pick the one in the middle. (In casino roulette, you would roll the wheel many times, and find out that the median outcome is that you lost money.) I deliberately removed the word “expected” from the left-hand value in this second formula – in this formula we’re only looking at one outcome, and that’s the median outcome. If you apply this approach to roulette, you’re never going to play, because this formula (wrongly) says you will always lose. It’s tyranny of the median – if I go this route I won’t lose money, but I also won’t be any fun.

Applied to gambling that logic is clearly off, but the second formula is also applied often in decision-making. I select a median outcome, and based on my estimate of whether that does or doesn’t have value, I will or won’t proceed. A lot of the time my reasoning won’t be too distorted – I travel to the grocery to get food (and since I need to eat that presumably has a positive value), but don’t worry excessively about the remote possibility that I might have an accident and rack up a big car-repair bill. I might hire a new employee with the expectation they’ll stick around long enough to become fully productive, ignoring the possibility that they’ll leave after a short time. I might promote someone presuming that they will work out in their new position, ignoring the Peter-Principled option that they do not. It’s just being practical – who has time to figure out the chances and costs of every possible outcome, even if we could?

Often this median approach isn’t too bad, and when occasionally things don’t work out, we take our lumps and move on. However, the second formula really breaks down when a (hopefully) unlikely outcome has a very high cost. By the lights of the second formula, we shouldn’t spend a dime to prevent nuclear war, because nuclear war is unlikely. Few would agree that this is a good approach, and even given considerable uncertainties, an differential analogue of the first formula is prudent policy:

value = (reduced probability of nuclear war) x (unimaginable costs)

– (spending on nuclear diplomacy and deterrence)

The value of our policy is to reduce the chance of a horrendous event. Talking of “value” might seem a little cold, but the decision is undoubtedly the right one. While reasonable people might differ on how funds are dispersed to prevent nuclear war, few would say that no money should be spent simply because we aren’t on the brink of Armageddon. That would be an an extraordinarily anarchic policy. Spending funds prudently is part of the government’s job – based on the best evidence of what a nuclear war would be like, we’ve invested in protecting ourselves from a terrible outcome.

If we agree that investment against very bad outcomes – even if they are unlikely – is a matter of governmental sensibility and prudence, perhaps we should be grateful that our current cast of governmental anarchists wasn’t in office in the mid-twentieth century. It’s reasonable to wonder if we would have invested in any nuclear policy, simply because nuclear war hadn’t yet taken place.

Let’s look at the closely-parallel case of global warming, doubtless far from most people’s safe-topic list, and one perhaps best avoided in polite company. But here we are, neither particularly safe, nor all that polite. Governmental representatives wishing to do nothing are applying a tyranny-of-the-median formulation, based on the idea that global warming damage is unlikely or impossible, so a median outcome is “there is no problem.” In that view, the value of spending on research is, well, negative:

cost = (spending on research that we don’t really need)

Like the cases above, that doesn’t take all factors into account. (Ironically, Keynesian economics says that raw spending is a lousy metric for true cost, and there might be a net positive income through the mechanism of increased economic growth and increased tax revenues. But that’s not the immediate point.)

Accurate risk assessment requires that even unlikely scenarios are taken into account when they have huge costs. Given even a modest chance of astronomical global warming damage, a realistic formula for the value of research spending would be like the one above:

(decreased probability of severe global warming)

– (research spending)

Even if the probability of severe global warming damage were only ten percent, even one percent – the cost of major coastal flood damage, of crop failures, of more extreme weather – would justify research expenditures of many billions of dollars without question. There isn’t anything to dicuss, unless we wish to claim that predictions of global warming are truly a hoax, or the (probability of severe global warming) is exactly zero, or research can have no impact on the issue. We can take that Flat-Earther approach – but when the majority of responsible professionals agree that severe global warming damage is quite possible without urgent action, the burden of proof rests with those wishing to demonstrate otherwise. Those in government who aren’t quite schooled enough or bright enough to understand this, or are simply afraid or unwilling to spend money for any reason, would ideally move back to whatever endeavor prepared them so woefully for the responsibilities they now hold.

Evidently, I’m all for prudent action in this area – is there really a responsible option? That said, I do confess impatience with some the language we use to describe the impact of government policies, such as carbon taxes that might be required to combat global warming. And high on my list of toxic terminology is “dislocation.” Policy analysts talk about dislocation as if their birthday cakes were a little off-center on the party plates, rather than the dehumanizing concept that people can become irrelevant at any point in their careers. I detect an unhappy taint of social prejudice in this terminology – that somehow a skilled electrician or tool-and-die maker is less valuable and easier to “re-train” than a database administrator, an analyst, or an accountant. I’ve worked with and respect all of the above – and I wouldn’t care to take that bet, personally.

I don’t see that anyone today is entirely immune from the possibility of “dislocation.” Imagine if governmental anarchists achieved electoral majorities, and publicly-funded universities were curtailed or shuttered. Impossible? I can’t be sure, myself. The tyranny of the median only sees the middle, and despises what is seen to be unusual, elitist, or subpar with equal and evangelical fervor. If curtailing the non-normal is just a matter of failing to spend, any level of governmental non-performance is within the realm of possibility. Buyer, beware.