Early on New Year’s Day, I began scrolling through the messages people had left on social media. Usually you find a note of hope among popping corks and exploding fireworks. Not this year. All I found were posts like “2018 was a terrible year. Don’t expect more from 2019” or “I dread the year to come”.

I started to suspect that those I followed on social media were all just a bit depressing. But that theory evaporated when a new batch of articles trying to capture the spirit of our age appeared. “It’s all over,” one piece declared. “All that’s left to us is making the best of a bad situation,” another announced. A fascinating new book bore the title The Worst is Yet to Come.

It seems that 2019 is the dawn of an age of deep pessimism. According to some, things are going down quickly. All we can do is try to survive. The editors of n+1, one of the hippest intellectual journals on the planet, claim that our only hope seems to be that “our arrow-slinging children will bear us on their backs out of the civilization we ruined for them”.

Upbeatsters rubbish such dark resignation. Things aren’t actually that bad, they claim. Life spans are expanding, GDP is rising, female empowerment is growing, poverty is declining. Even if there are a few pesky problems, such as falling life expectancy among poor people in rich nations due to deaths of despair, that’s no reason for negativity. If we are to believe we can do something about these big problems, we are told that we need to think positively. After all, there is a significant body of work showing that people with an optimistic outlook tend to suffer much less distress when faced with big life events such as childbirth, starting a business or facing a significant illness.

In recent years, we have started to recognise the limits of being relentlessly upbeat. There is a growing movement of people prompting us to harness the power of pessimism. Pessimism is experiencing a strange revival in philosophy. Eugene Thacker reminds us that it will all inevitably end in ruin one day. Accepting that insight can give a strange sense of consolation and can free us to live.

In self-help circles, people are beginning to embrace negative thinking by turning to stoic philosophers such as Seneca. Instead of closing their eyes and imagining the perfect future, they are sitting back and trying to envisage the worst-case scenario. Rather than envisaging themselves living in a luxury minimalist house in Malibu, these postmodern stoics try to imagine themselves sleeping in a cardboard box outside the downtown Los Angeles bus station.

These exercises are not just perverse forms of psychological masochism. There is increasing evidence that positive thinking can impede action. In an experiment, the psychologist Gabriele Oettingen found that thirsty people in the lab who were asked to imagine a glass of icy water showed less energy, which could be put into getting hold of an actual glass of water. In another study, Oettingen found that while people who developed clear and defined goals for the future fared well, others who simply nurtured positive fantasies seemed to flounder. Psychologists have also found that pessimism can actually motivate us. For instance, one study by Julie Norem and Nancy Cantor found that people often adopted a pessimistic stance to steel themselves against future disappointments. This helped to motivate them to undertake what would otherwise seem like insurmountable tasks. More recent work suggests that pessimism and optimism are not polar opposites, but separate systems in our brains. You are not either pessimistic or optimistic. You can be both at the same time, or either.

Perhaps the pessimism that infuses our age is not something we should recoil from or wallow in. Maybe pessimism could force us to realistically consider the worst-case scenario. Pessimism could help steel us against the inevitable anxieties that the future brings. A good dose of pessimism may actually motivate us in our attempts to address the problems we face. Pessimism could console and even free us. When mixed with some optimism, pessimism may help us to think more soberly and realistically about challenges that we face. Although being pessimistic is painful, it is certainly better than harbouring delusional fantasies about sunny uplands of the future.

• André Spicer is professor of organisational behaviour at the Cass Business School at City, University of London