At prime minister’s questions on Wednesday, Keir Starmer said he had added up a total of 27,241 coronavirus deaths so far, leaving the UK “possibly on track to have the worst death rate in Europe”.

Is he right? Unfortunately, measuring the impact of the virus is a fiendishly complex task. It’s nothing like keeping score in a game. Starmer, as a lawyer, would know that we have to define our terms carefully. And so, assuming we want to make a comparison based on death rates, we first need to decide what a death rate is.

You would think it would be easy for a bean-counting statistician to count deaths – the one certain thing (apart from taxes). But it is remarkably difficult. I have stopped taking much notice of the number given out at the daily press conferences, as it is only based on reports from hospitals, oscillates wildly around weekends, and recently included deaths that occurred a month ago. And this week the number of UK deaths jumped up by nearly 5,000 to 26,097 in one day – rather close to Starmer’s count – by retrospectively including non-hospital deaths that had tested positive for the virus.

But even this is too low, as it does not include the many deaths of people who were not tested. The Office for National Statistics data on death registrations is the last word, although inevitably delayed by around 10 days, and these figures would be expected to take the current total to significantly more than 30,000. But we should be very cautious in comparing even this uncertain total with those of other countries.

Why are coronavirus mortality rates so different? – video explainer

Every country has different ways of recording Covid-19 deaths: the large number of untested deaths in care homes have not featured in Spain’s statistics – which, like the UK’s require a positive test result. The numbers may be useful for looking at trends, but they are not reliable indicators for comparing the absolute levels. If we were naive enough to take the counts at face value, the new figures propelled the UK past France and Spain into second place in Europe behind Italy, which is not encouraging because we are behind Italy in terms of what stage of the epidemic we are at. A more equitable metric might be Covid-19 deaths per million. Ignoring tiny countries, our current score of 388 puts us fourth, behind Belgium (632), Spain (509) and Italy (452).

People are not so interested in the numbers themselves – they want to say why they are so high and ascribe blame

But these are still deeply unreliable numbers, as it is not clear if we should just be looking at Covid-19-labelled deaths anyway. The effects of seasonal flu are not based on tests or death certificates, but at looking at the total number of deaths over the winter, seeing how many extra there are than a baseline, allowing for climate, and assuming these excess deaths were linked to flu. On average, over the last 10 years this has come to about 8,000 flu-related deaths, rising to 26,400 in 2017-2018 and 28,300 in 2014-15.

Many feel that excess deaths give a truer picture of the impact of an epidemic. The latest ONS figures reveal that in the week ending 17 April, the week after Easter, around 22,500 deaths were registered in England and Wales, compared with an average of around 10,500 a week at this time of year. Nearly 9,000 of these had Covid-19 on their death certificate, but that leaves 3,000 extra deaths – more than 400 a day – essentially unaccounted for. Many of these will be indirect effects from the disrupted health service: routine treatments have been largely abandoned, A&E attendances and non-Covid-19 hospital admissions have plummeted.

But, of course, people are not so interested in the numbers themselves – they want to say why they are so high, and ascribe blame. But if it’s difficult to rank this country, it’s even trickier to give reasons for our position. Covid-19 mainly harms the elderly, with the average age of deaths above 80, and its fatality rate doubles every seven years as a person ages. Italy’s population is elderly (it has a median age of 47), while Ireland’s is much younger (a median age of 37), so we would expect different effects. And Covid-19 is a disease of crowded areas – New York is rather different from Reykjavik. An obsessive comparison is being made between Norway and Sweden: Sweden’s more relaxed social distancing policies may or may not have been instrumental in their current death rate being 233 per million, compared with Norway’s 38.

Even if we reach some sort of stable situation – if we can imagine it – will we ever know the direct and indirect health effects of the epidemic, taking into account reduced road accidents, the benefits of reduced pollution, the effects of recession and so on? Many studies will try to disentangle all these, but my cold, statistical approach is to wait until the end of the year, and the years after that, when we can count the excess deaths. Until then, this grim contest won’t produce any league tables we can rely on.

• David Spiegelhalter is a statistician and professor of the public understanding of risk.

• This article was amended on 1 & 8 May 2020. The UK has the fourth-highest number of Covid-19 deaths per million, not the third-highest number as an earlier version stated. And it is untested care home deaths which do not feature in Spain’s statistics, rather than all care home deaths.