The Tories turn the tables on Labour today over the highly contentious area of inheritance tax by claiming that more than 4 million people in Britain now face an average liability of £60,000.

Labour has attempted to portray David Cameron as the champion of privilege after he pledged to raise the individual inheritance tax (IHT) threshold to £1m.

Gordon Brown, who faced accusations that he was sparking a fresh class war when he claimed that Tory economic policy was dreamed up on the playing fields of Eton, says Cameron's plans would benefit the richest 3,000 estates in the country.

The Tories hit back today, saying that the government's decision to abandon plans to raise the individual IHT threshold from £325,000 to £350,000 in the pre-budget report means that more than 4 million people will be hit. Analysis by the Tories of new figures from the Wealth in Britain report, released by the Office for National Statistics, shows that:

• About 5m households have wealth outside their pensions above the individual IHT threshold of £325,000.

• More than 4 million adults – one in 10 of the total – live in households where the average wealth outside their pensions is more than £325,000. This figure is reached after allowing for the number of people aged over 16 in each household.

• The average liability for the 4.3 million people in this bracket is £60,000 for every person.

Philip Hammond, the shadow chief secretary to the Treasury, said: "These figures show that inheritance tax remains a threat for millions of families under Labour. Coming on top of the government's planned tax increase on anyone earning over £20,000 this reveals that the real victims of Labour's class war are middle-income families.

"If you aspire to save for your future and pass something on to your children then Labour is no longer the party for you. The Conservatives will restore inheritance tax to what it was designed to be – a tax on the very rich – and ensure that it is only paid by millionaires."

The Treasury, which says that fewer than 3% of estates pay IHT, will claim that the Tory figures give an unrealistic assessment of how the tax is levied. The Tories cannot guarantee that all 4 million people identified in their analysis would be liable. Many will live as married couples, allowing them to pool their wealth to double the IHT threshold.

The attempt by the Tories to show that inheritance tax hits people on middle incomes displays their determination to fight Brown's move to appeal to core Labour voters by portraying Cameron as the defender of the rich. The prime minister said last month that the beneficiaries of the Tories' tax plans, which would see the individual IHT threshold raised to £1m, "resemble the leader of the opposition's Christmas card list".

A Treasury source said: "The Tories' claim of 4 million is utter nonsense. This is just another Tory attempt to distract people from the fact they are cutting tax credits for families on modest incomes while giving a big tax cut to the 3,000 richest estates."

Since then the Tories have sensed weakness after Brown alarmed many members of the cabinet by appearing to resort to class war tactics with his jibe about Eton. Jack Straw, the justice secretary, and Tessa Jowell, the cabinet office minister, warned Brown over the weekend against pursuing a class war.