Intellectuals hate progress. Intellectuals who call themselves “progressive” really hate progress. It’s not that they hate the fruits of progress, mind you: most pundits, critics, and their bien-pensant readers use computers rather than quills and inkwells, and they prefer to have surgery with anaesthesia rather than without it. It’s the idea of progress that rankles the chattering class – the Enlightenment belief that by understanding the world we can improve the human condition.

An entire lexicon of abuse has grown up to express their scorn. If you think knowledge can help solve problems, then you have a “blind faith” and a “quasi-religious belief” in the “outmoded superstition” and “false promise” of the “myth” of the “onward march” of “inevitable progress”.

Yes, it’s not just those who intellectualise for a living who think the world is going to hell in a handcart. It’s ordinary people when they switch into intellectualising mode. Psychologists have long known that people tend to see their own lives through rose-coloured glasses: they think they’re less likely than the average person to become the victim of a divorce, layoff, accident, illness, or crime. But change the question from the people’s lives to their society, and they transform from Pollyanna to Eeyore.

Are they right? Is pessimism correct? Could the state of the world, like the stripes on a barbershop pole, keep sinking lower and lower? It’s easy to see why people feel that way: every day the news is filled with stories about war, terrorism, crime, pollution, inequality, drug abuse, and oppression. And it’s not just the headlines we’re talking about; it’s the op-eds and the long-form stories as well. Magazine covers warn us of coming anarchies, plagues, epidemics, collapses, and so many “crises” (farm, health, retirement, welfare, energy, deficit) that copywriters have to escalate to the redundant “serious crisis”.

Whether or not the world really is getting worse, the nature of news will interact with the nature of cognition to make us think that it is. News is about things that happen, not things that don’t happen. We never see a journalist saying to the camera, “I’m reporting live from a country where a war has not broken out” – or a city that has not been bombed, or a school that has not been shot up. As long as bad things have not vanished from the face of the Earth, there will always be enough incidents to fill the news, especially when billions of smartphones turn most of the world’s population into crime reporters and war correspondents. And among the things that do happen, the positive and negative ones unfold on different timelines. The news, far from being a “first draft of history”, is closer to play-by-play sports commentary.

Whether or not the world really is getting worse, the nature of news will interact with the nature of cognition to make us think that it is

The answer is to count. How many people are victims of violence as a proportion of the number of people alive? How many are sick, how many starving, how many poor, how many oppressed, how many illiterate, how many unhappy? And are those numbers going up or down? A quantitative mindset, despite its nerdy aura, is in fact the morally enlightened one, because it treats every human life as having equal value rather than privileging the people who are closest to us or most photogenic. And it holds out the hope that we might identify the causes of suffering and thereby know which measures are most likely to reduce it.

Pessimism, to be sure, has a bright side. The expanding circle of sympathy makes us concerned about harms that would have passed unnoticed in more callous times. Today we recognise the Syrian civil war as a humanitarian tragedy. The wars of earlier decades, such as the Chinese civil war, the partition of India, and the Korean war, are seldom remembered that way, though they killed and displaced more people.

Steven Pinker, photographed at Harvard University by Rose Lincoln

When I grew up, bullying was considered a natural part of boyhood. It would have strained belief to think that some day the president of the United States would deliver a speech about its evils, as Barack Obama did in 2011. As we care about more of humanity, we’re apt to mistake the harms around us for signs of how low the world has sunk rather than how high our standards have risen.

We’re apt to mistake the harms around us for signs of how low the world has sunk rather than how high our standards have risen

The financial writer Morgan Housel has observed that while pessimists sound like they’re trying to help you, optimists sound like they’re trying to sell you something. Whenever someone offers a solution to a problem, critics will be quick to point out that it is not a panacea, a silver bullet, or a one-size-fits-all-solution; it’s just a Band-Aid or a quick technological fix that fails to get at the root causes and will blow back with side effects and unintended consequences. Of course, since nothing is a panacea and everything has side effects (you can’t do just one thing), these common tropes are little more than a refusal to entertain the possibility that anything can ever be improved.

Adapted from Enlightenment Now: The Case for Reason, Science, Humanism, and Progress by Steven Pinker, published by Viking