However, Angela Rayner, the deputy Labour leader, described his comments as “poor” and said MPs had been working hard during lockdown.

We have all really tried to keep cross party working on #Coronavirus calling fellow MPs “Lazy” and “Work-shy” is poor Henry, many colleagues from all sides of the house and their staff have been working very hard for our constituents during the lockdown, no need for this nonsense https://t.co/L5OABaMvnl

Henry this is an appalling thing to say, would you like to compare cases dealt with in this crisis? Hours helping with food, PPE, testing? Number of questions put down to ministers? Number of bill amendments written? Also calling people working at home workshy is quite something https://t.co/RROVCxbG7O

Rees-Mogg said on Wednesday that the remote working arrangement, which saw a handful of MPs keeping their distance from each other in the House of Commons with others dialling in with video messaging, should be changed.

“We have to recognise that if we persist with the present arrangement it will become harder to make progress in a timely fashion,” he said.

“That is why, in line with government advice for those who cannot do their jobs from home, I am asking members to return to their place of work after Whitsun (recess).”

He insisted that when MPs return, parliament would be a “COVID-19 secure workplace” and would not be a return “to the crowded, bustling chamber of old”, with members practising social distancing.

A spokesman for Labour leader Sir Keir Starmer said on Wednesday: “The government’s own advice, its own public health advice, is those who can work from home should.

“Parliament has developed a successful system using technology to ensure the scrutiny of government, while allowing people to work remotely.

“The government has yet to provide an honest explanation as to why they want to bring this virtual system to an end.”

SNP Cabinet Office spokesperson Tommy Sheppard described the plans as “reckless, cavalier and downright dangerous” and were forcing MPs “to make a choice between standing up for those who elected them and putting their own health and the health of others at risk”.

Liberal Democrat Alistair Carmichael, who represents Orkney and Shetland, said the trip MPs make to go back to London would be “non-essential journeys”.