All posts tagged complexity quotient

Mathematics is not primarily a matter of plugging numbers into formulas and performing rote computations. It is a way of thinking and questioning that may be unfamiliar to many of us, but is available to almost all of us. (p.3)

John Allen Paulos

John Allen Paulos is an American professor of mathematics who came to wider fame with publication of his short (130-page) primer, Innumeracy: Mathematical Illiteracy and its Consequences, published in 1988.

It was followed by Beyond Numeracy: Ruminations of a Numbers Man in 1991 and this book, A Mathematician Reads the Newspaper in 1995.

Structure

The book is made up of about 50 short chapters. He explains that each one of them will take a topic in the news in 1993 and 1994 and show how it can be analysed and understood better using mathematical tools.

The subjects of the essays are laid out under the same broad headings that you’d encounter in a newspaper, with big political stories at the front, giving way to:

Local, business and social issues

Lifestyle, spin and soft news

Science, medicine and the environment

Food, book reviews, sports and obituaries

Response

The book is disappointing in all kinds of ways.

First and foremost, he does not look at specific stories. All the headlines are invented. Each 4 or 5-page essay may or may not call in aspects of various topics in the news, but they do not look at one major news story and carefully deconstruct how it has been created and publicised in disregard of basic mathematics and probability and statistics. (This alone is highly suggestive of the possibility that, despite all his complaints to the contrary, specific newspaper stories where specific mathematical howlers are made and can be corrected are, in fact surprisingly rare.)

The second disappointment is that, even though these essays are very short, they cannot stay focused on one idea or example for much more than a page. I hate to say it and I don’t mean to be rude, but Paulos’s text has some kind of attention deficit disorder: the essays skitter all over the place, quickly losing whatever thread they ever had in a blizzard of references to politics, baseball, pseudoscience and a steady stream of bad jokes. He is so fond of digressions, inserts, afterthoughts and tangents that it is often difficult to say what any given essay is about.

I was hoping that each essay would take a specific news story and show how journalists had misunderstood the relevant data and maths to get it wrong, and would then show the correct way to analyse and interpret it. I was hoping that the 50 or so examples would have been carefully chosen to build up for the reader an armoury of techniques of arithmetic, probability, calculus, logarithms and whatever else is necessary to immediately spot, deconstruct and correct articles with bad maths in them.

Nope. Not at all.

Lani ‘Quota Queen’ Guinier

Take the very first piece, Lani ‘Quota Queen’ Guinier. For a start he doesn’t tell us who Lani ‘Quota Queen’ Guinier is. I deduce from his introduction that she was President Clinton’s nomination for the post of assistant attorney general for civil rights. We can guess, then, that the nickname ‘quota queen’ implies she was a proponent of quotas, though whether for black people, women or what is not explained.

Why not?

Paulos introduces us to the Banzhaf power index, devised in 1965 by lawyer John F. Banzhaf.

The Banzhaf power index of a group, party or person is defined to be the number of ways in which that group, party or person can change a losing coalition into a winning coalition or vice versa. (p.10)

He gives examples of companies where three or four shareholders hold different percentages of voting rights and shows how some coalitions of shareholders will always have decisive voting rights, whereas others never will (these are called the dummy) while even quite small shareholders can hold disproportionate power. For example in a situation where three shareholders hold 45%, 45% and 10% of the shares, the 10% party can often have the decisive say. In 45%, 45%, 8% and 2% the 2% is the dummy.

He then moves on to consider voting systems in some American states, including: cumulative voting, systems where votes don’t count as 1 but are proportionate to population, Borda counts (where voters rank the candidates and award progressively more points to those higher up the rankings), approval voting (where voters have as many votes as they want and can vote for as many candidates as they approve of), before going on to conclude that all voting systems have their drawbacks.

The essay ends with a typical afterthought, one-paragraph coda suggesting how the Supreme Court could end up being run by a cabal of just three judges. There are nine judges on the U.S. Supreme Court. Imagine (key word for Paulos), imagine a group of five judges agree to always discuss issues among themselves first, before the vote of the entire nine, and imagine they decide to always vote according to whatever the majority (3) decide. Then imagine that a sub-group of just three judges go away and secretly decide, that in the group of five, they will always agree. Thus they will dictate the outcome of every Supreme Court decision.

So:

1. I had no idea who Lani ‘Quota Queen’ Guinier was or, more precisely, I had to do a bit of detective work to figure it out, and still wasn’t utterly sure.

2. This is a very sketchy introduction to the issue of democratic voting systems. This is a vast subject, which Paulos skates over quickly and thinly.

Thus, in these four and a bit pages you have the characteristic Paulos experience of feeling you are wandering all over the place, not quite at random, but certainly not in a carefully planned sequential way designed to explore a topic thoroughly and reach a conclusion. You are introduced to a number of interesting ideas, with some maths formulae, but not in enough detail or at sufficient length to really understand them. And because he’s not addressing any particular newspaper report or article, there are no particular misconceptions to clear up: the essay is a brief musing, a corralling of thoughts on an interesting topic.

This scattergun approach characterises the whole book.

Psychological availability and anchoring effects

The second essay is titled Psychological availability and anchoring effects. He explains what the availability error, the anchor effect and the halo effect are. If this is the first time you’ve come across these notions, they’re powerful new ideas. But I recently reread Irrationality by Stuart Sutherland which came out three years before Paulos’s book and spends over three hundred pages investigating these and all the other cognitive biases which afflict mankind in vastly more depth than Paulos, with many more examples. Next to it, Paulos’s three-minute essay seemed sketchy and superficial.

General points

Rather than take all 50 essays to pieces, here are notes on what I actually did learn. Note that almost none of it was about maths, but general-purpose cautions about how the news media work, and how to counter its errors of logic. In fact, all of it could have come from a media studies course without any maths at all:

almost all ‘news’ reinforces conventional wisdom

because they’re so brief, almost all headlines must rely on readers’ existing assumptions and prejudices

almost all news stories relate something new back to similar examples from the past, even when the comparison is inappropriate, again reinforcing conventional wisdom and failing to recognise the genuinely new

all economic forecasts are rubbish: this is because economics (like the weather and many other aspects of everyday life) is a non-linear system. Chaos theory shows that non-linear systems are highly sensitive to even minuscule differences in starting conditions, which has been translated into pop culture as the Butterfly Effect

and also with ‘futurologists’: the further ahead they look, the less reliable their predictions

the news is deeply biased by always assuming human agency is at work in any outcome: if any disaster happens anywhere the newspapers always go searching for a culprit; in the present Brexit crisis lots of news outlets are agreeing to blame Theresa May. But often things happen at random or as an accumulation of unpredictable factors. Humans are not good at acknowledging the role of chance and randomness.

There is a tendency to look primarily for culpability and conflicts of human will rather than at the dynamics of a natural process. (p.160)

Hence so many newspapers endlessly playing the blame game. The Grenfell Tower disaster was, first and foremost, an accident in the literal sense of ‘an unfortunate incident that happens unexpectedly and unintentionally, typically resulting in damage or injury’ – but you won’t find anybody who doesn’t fall in with the prevailing view that someone must be to blame. There is always someone to blame. We live in a Blame Society.

personalising beats stats, data or probability: nothing beats ‘the power of dramatic anecdote’ among the innumerate: ‘we all tend to be unduly swayed by the dramatic, the graphic, the visceral’ (p.82)

if you combine human beings’ tendency to personalise everything, and to look for someone to blame, you come up with Donald Trump, who dominates every day’s news

so much is happening all the time, in a world with more people and incidents than ever before, in which we are bombarded with more information via more media than ever before – that it would be extraordinary if all manner or extraordinary coincidences, correspondences and correlations didn’t happen all the time

random events can sometimes present a surprisingly ordered appearance

because people imbue meaning into absolutely everything, then the huge number of coincidences and correlations are wrongfully interpreted as meaningful

Tips and advice

I was dismayed at the poor quality of many of the little warnings which each chapter ends with. Although Paulos warns against truisms (on page 54) his book is full of them.

Local is not what it used to be, and we shouldn’t be surprised at how closely we’re linked. (p.55)

In the public realm, often the best we can do is to stand by and see how events unfold. (p.125)

Chapter three warns us that predictions about complex systems (the weather, the economy, big wars) are likely to be more reliable the simpler the system they’re predicting, and the shorter period they cover. Later he says we should be sceptical about all long-term predictions by politicians, economists and generals.

It didn’t need a mathematician to tell us that.

A lot of it just sounds like a grumpy old man complaining about society going to the dogs:

We need look no further than the perennial appeal of pseudoscientific garbage, now being presented in increasingly mainstream forums… (p.145)

The fashion pages have always puzzled me. In my smugly ignorant view, they appear to be so full of fluff and nonsense as to make the astrology columns insightful by comparison. (p.173)

Another aspect of articles in the society pages or in the stories about political and entertainment figures is the suggestion that ‘everybody’ knows everybody else. (p.189)

Sometimes his liberal earnestness topples into self-help book touchy-feeliness.

Achieving personal integration and a sense of self is for the benefit of ourselves and those we’re close to. (p.112)

But just occasionally he does say something unexpected:

The attention span created by television isn’t short; it’s long, but very, very shallow. (p.27)

That struck me as an interesting insight but, as with all his interesting comments, no maths was involved. You or I could have come up with it from general observation.

Complexity horizon

The notion that the interaction of human laws, conventions, events, politics, and general information overlap and interplay at ever-increasing speeds to eventually produce situations so complex as to appear unfathomable. Individuals, and groups and societies, have limits of complexity beyond which they cannot cope, but have to stand back and watch. Reading this made me think of Brexit.

He doesn’t mention it, but a logical spin-off would be that every individual has a complexity quotient like an intelligence quotient or IQ. Everyone could take a test in which they are faced with situations of slowly increasing complexity – or presented with increasingly complex sets of information – to find out where their understanding breaks off – which would become their CQ.

Social history

The book was published in 1995 and refers back to stories current in the news in 1993 and 1994. The run of domestic political subjects he covers in the book’s second quarter powerfully support my repeated conviction that it is surprising how little some issues have changed, how little movement there has been on them, and how they have just become a settled steady part of the social landscape of our era.

Thus Paulos has essays on:

gender bias in hiring

homophobia

accusations of racism arising from lack of ethnic minorities in top jobs (the problem of race crops up numerous times (pp.59-62, p.118)

the decline in educational standards

the appallingly high incidence of gun deaths, especially in black and minority communities

the fight over abortion

I feel increasingly disconnected from contemporary politics, not because it is addressing new issues I don’t understand, but for the opposite reason: it seems to be banging on about the same issues which I found old and tiresome twenty-five years ago.

The one topic which stood out as having changed is AIDS. In Innumeracy and in this book he mentions the prevalence or infection rates of AIDS and is obviously responding to numerous news stories which, he takes it for granted, report it in scary and alarmist terms. Reading these repeated references to AIDS made me realise how completely and utterly it has fallen off the news radar in the past decade or so.

In the section about political correctness he makes several good anti-PC points:

democracy is about individuals, the notion that everyone votes according to their conscience and best judgement; as soon as you start making it about groups (Muslims, blacks, women, gays) you start undermining democracy

racism and sexism and homophobia are common enough already without making them the standard go-to explanations for social phenomena which often have more complex causes; continually attributing all aspects of society to just a handful of inflammatory issues, keeps the issues inflammatory

members of groups often vie with each other to assert their loyalty, to proclaim their commitment to the party line and this suggests a powerful idea: that the more opinions are expressed, the more extreme these opinions will tend to become. This is a very relevant idea to our times when the ubiquity of social media has a) brought about a wonderful spirit of harmony and consensus, or b) divided society into evermore polarised and angry groupings

Something bad is coming

I learned to fear several phrases which indicate that a long, possibly incomprehensible and frivolously hypothetical example is about to appear:

‘Imagine…’

Imagine flipping a penny one thousand times in succession and obtaining some sequence of heads and tails… (p.75)

Imagine a supercomputer, the Delphic-Cray 1A, into which has been programmed the most complete and up-to-date scientific knowledge, the initial condition of all particles, and sophisticated mathematical techniques and formulas. Assume further that… Let’s assume for argument’s sake that… (p.115)

Imagine if a computer were able to generate a random sequence S more complex than itself. (p.124)

Imagine the toast moistened, folded, and compressed into a cubical piece of white dough… (p.174)

Imagine a factory that produces, say, diet food. Let’s suppose that it is run by a sadistic nutritionist… (p.179)

‘Assume that…’

Let’s assume that each of these sequences is a billion bits long… (p.121)

Assume the earth’s oceans contain pristinely pure water… (p.141)

Assume that there are three competing healthcare proposals before the senate… (p.155)

Assume that the probability of your winning the coin flip, thereby obtaining one point, is 25 percent. (p.177)

Assume that these packages come off the assembly line in random order and are packed in boxes of thirty-six. (p.179)

Jokes and Yanks

All the examples are taken from American politics (President Clinton), sports (baseball) and wars (Vietnam, First Gulf War) and from precisely 25 years ago (on page 77, he says he is writing in March 1994), both of which emphasise the sense of disconnect and irrelevance with a British reader in 2019.

As my kids know, I love corny, bad old jokes. But not as bad as the ones the book is littered with:

And then there was the man who answered a matchmaking company’s computerised personals ad in the paper. He expressed his desire for a partner who enjoys company, is comfortable in formal wear, likes winter sports, and is very short. The company matched him with a penguin. (pp.43-44)

The moronic inferno and the liberal fallacy

The net effect of reading this book carefully is something that the average person on the street knew long ago: don’t believe anything you read in the papers.

And especially don’t believe any story in a newspaper which involves numbers, statistics, percentages, data or probabilities. It will always be wrong.

More broadly his book simply fails to take account of the fact that most people are stupid and can’t think straight, even very, very educated people. All the bankers whose collective efforts brought about the 2008 crash. All the diplomats, strategists and military authorities who supported the Iraq War. All the well-meaning liberals who supported the Arab Spring in Egypt and Libya and Syria. Everyone who voted Trump. Everyone who voted Brexit.

Most books of this genre predicate readers who are white, university-educated, liberal middle class and interested in news and current affairs, the arts etc and – in my opinion – grotesquely over-estimate both their value and their relevance to the rest of the population. Because this section of the population – the liberal, university-educated elite – is demonstrably in a minority.

Over half of Americans believe in ghosts, and a similar number believes in alien abductions. A third of Americans believe the earth is flat, and that the theory of evolution is a lie. About a fifth of British adults are functionally illiterate and innumerate. This is what Saul Bellow referred to as ‘the moronic inferno’.

On a recent Radio 4 documentary about Brexit, one contributor who worked in David Cameron’s Number Ten commented that he and colleagues went out to do focus groups around the country to ask people whether we should leave the EU and that most people didn’t know what they were talking about. Many people they spoke to had never heard of the European Union.

On page 175 he says the purpose of reading a newspaper is to stretch the mind, to help us envision distant events, different people and unusual situations, and broaden our mental landscape.

Is that really why he thinks people read newspapers? As opposed to checking the sports results, catching up with celebrity gossip, checking what’s happening in the soaps, reading interviews with movie and pop stars, looking at fashion spreads, reading about health fads and, if you’re one of the minority who bother with political news, having all your prejudices about how wicked and stupid the government, the poor, the rich or foreigners etc are, and despising everyone who disagrees with you (Guardian readers hating Daily Mail readers; Daily Mail readers hating Guardian readers; Times readers feeling smugly superior to both).

This is a fairly entertaining, if very dated, book – although all the genuinely useful bits are generalisations about human nature which could have come from any media studies course.

But if it was intended as any kind of attempt to tackle the illogical thinking and profound innumeracy of Western societies, it is pissing in the wind. The problem is vastly bigger than this chatty, scattergun and occasionally impenetrable book can hope to scratch. On page 165 he says that a proper understanding of mathematics is vital to the creation of ‘an informed and effective citizenry’.