Absolute numbers generally give little insight to anything, but even percentages, averages and ratios must sometimes be treated with caution

Earlier this week, the British advertising regulator ASA banned an Amazon advert that promised one-day delivery for Amazon Prime members. The Independent newspaper reported that they did so “after receiving hundreds of complaints from customers”. 280 people had complained, most of them because they had not received their goods within one day one day of placing their order, around Christmas 2017. Is 280 a lot?

In any case it’s a good example of how the media often use (large) absolute numbers in support of a story. 280 does sound significant, but without knowing how many shipments were made in total, it doesn’t really tell us all that much. Worldwide, Amazon sent 5 billion parcels ordered via Prime in 2017. Actual UK shipments are not reported, but using their turnover in the UK (about £8.8B or $11.4B) and globally ($178B), a first approximation suggests 6.5% of the Prime deliveries were made in Britain – or about 320 million, so maybe 40 million in December. That puts 280 complaints (0.0007%) into some perspective.

Without any reference, we can easily be misled into ascribing some spurious significance to a number, but once a number is expressed as a relevant percentage, it makes more sense (and is often not remotely as sensational).

However, percentages themselves are not necessarily all that enlightening and may themselves need more context. A couple of years ago I already wrote about the capacity of the “%”-sign to fool us in the shops, and in opinion polls and predictions. Here are some more situations where extra care may be needed before drawing a conclusion.

A cancer epidemic, or what?

A few weeks ago, the Independent newspaper reported that, globally, the number of cancer cases had risen by a third in the last 10 years. What could be behind this dramatic epidemic? Was this the reckoning for our carcinogenic lifestyle choices (smoking and diet, exposure to the sun)? In part, for sure, but there is more to the story. Cancer is, by and large, a condition that mostly afflicts older people: half the cancers diagnosed in the UK are in people aged 70 and above. Only just over 10% of cases are in people below the age of 50. So, as the population ages, it is hardly surprising that the total number of cases will go up. A “cancer up by 33%” headline is more informative than “new cancer diagnoses up by 120,000”, but even the ratio of current incidence over historical incidence doesn’t give us the full picture.

Economics statistics can also be subject to a strange, and potentially misleading effect. Household income is a common metric to examine the distribution of economic resources in a society. By looking at which share of income goes to which slice of the household population we can judge whether a society is, and is becoming more or less equal. But could it be, for example, that everyone gets richer, but that the middle-class household income nevertheless drops?

In a superb video, economist and host of the Econtalk podcast Russ Roberts shows us, by means of a hypothetical and simplified example, how such a surprising result can come about. Imagine a perfectly egalitarian society, with ten 2-person households. Every citizen earns $50,000, so the average household income in each quintile (a slice of 20%) is $100,000, exactly 1/5 of the total. The economy performs well, and 30 years later, every citizen’s income has doubled. However, through divorce on the one hand, and younger people (replacing those who died in that period) hooking up later on the other, the population, still 10 people, now forms 15 households: 10 singletons and 5 couples.

When these 15 households are divided into quintiles again, we now find that the “richest” fifth represents 30% of the total income, whereas for the ‘middle class’ quintile that is just 15%. We can also see that the richest 20% households have doubled their income and increased their share of the total by 50%; the middle class households’ income, in contrast, has stagnated and their share is down by 25%.

Now this is of course a hypothetical situation, but you can see how changes in demographics can influence the figures, and distort our understanding. In real life, consider the effect of an ageing population. Pensioners generally have a lower income than people in work. As everyone lives longer, the proportion of pensioners in society increases, and so will the proportion of low-income households. Similarly, lesser-educated adults tend to have relatively lower incomes than people with a degree, and as the former category is more likely to live in single adult households, they ‘pull the middle down’. Over time, the distribution can therefore show what looks like a shift of income from the poor towards the rich, just because of an increase or decrease of a given type of household.

Hiring bias

Something similar can happen in a variety of other situations. Imagine a hospital that wants to avoid earlier criticism that women are less likely to be offered a job than men. In the preparation of the latest annual report, both the head of the clinical staff and the head of the admin staff show that women did better than men in the hiring process. Nearly 87% of female candidates were hired for clinical posts, and more than 45% for the administrative posts; higher figures than those for men (80% and 40% respectively). So everything is aggregated, but to the horror of the CEO, for the hospital as a whole, it turns out that nearly 64% of the women were hired, but that the men were almost 5 percentage points more likely to be offered a post, at close to 69%.

This phenomenon is known as Simpson’s paradox, after the British statistician Edward Simpson, who described it in a classic paper in 1951. (Trivia: this was not the first reference to it, though: it was identified earlier by Karl Pearson in 1899 and Udny Yule in 1903, but it was Simpson’s name that stuck.)

The intriguing story of the hospital shows that we need to be careful in interpreting percentages. Are women less likely to be hired (as the aggregate data suggest) or more likely (according to the departmental data)? As so often, we need to try and untangle correlation and causation. Is the cause of the apparent discrimination against women the hiring policy? When we look at the individual data, we see that women apply disproportionately for an administrative role (where there are fewer vacancies, and where the success rate across the board is half that for clinical posts). But just try to capture that nuance in a headline…

Finally, here’s an example how you can use statistics to satisfy your boss with minimum effort. Zita is the manager of two teams of translators who make subtitles for film and TV. Team A consists of four people, Alice, Bob, Chris and Dan; in Team B are five: Erik, Fran, Gerry, Hank and Iris. Their long-run daily averages (number of words translated per day) are shown in the table below. Team B clearly performs significantly better than Team A, and her boss instructs her to crack the whip on Team A, so that their average gets at least to the industry norm of 2500 words per day.

Zita ponders for a moment, and then has a flash of insight: she decides to move Fran to Team A. Hey presto: not only does the average of team A hit the target 2500, but her boss will no doubt be pleased to hear that Team B’s performance is also up!