Around 2,400 senior public officials are being paid their salary off the government payroll, which may have allowed the avoidance of tax, the chief secretary to the Treasury, Danny Alexander, disclosed on Wednesday.

The announcement follows a Whitehall review that also discovered around 1,400 appointees - 60% - have been engaged this way since the coalition government came to power. Seventy of the senior public officials had been paid this way for more than 10 years.

The disclosure raises question marks about why Tory, Liberal Democrat and Labour ministers tolerated the payments for so long and how much it may have cost the Treasury.

Departments pay intermediaries, such as employment agencies, in around 85% of cases, personal service companies in around 10% and the self-employed in less than 5%.

More than 1,000 of those identified are IT contractors, the review found. The majority of those identified are paid on a daily basis, with around 70% costing the appointing department more than £400 a day.

Since January, 350 contracts using off-payroll measures have been ended – with 10% of the people staying with departments under new terms and conditions.

Under new rules, the most senior staff must be on payroll unless there are exceptional temporary circumstances, departments must seek formal assurance from contractors they are paying full income tax and national insurance, and if assurances cannot be given, contracts will be terminated.

Alexander said the government would consult on a new law to require people controlling an organisation to be on its payroll.

"It's clear that off-payroll engagement without sufficient transparency has been endemic in the public sector for too many years.

"The solution to this problem is not to turn a blind eye or brush it under the carpet. We have to bring an end to the 'don't ask, don't tell' approach to this issue," he said.

Alexander said his review had looked at all government departments, their arm's-length bodies, and the boards of NHS trusts.

Devolved administrations, local authorities and the BBC were not included in the review.

Alexander said his review had not sought to identify tax avoidance but only off-payroll contracts; he said these were "opaque", and tax avoidance could have taken place.

There are circumstances where an employer may appoint an individual off payroll, the review said – the fact that an individual is engaged in this way does not mean that they are not paying the correct amount of tax.

"There are circumstances where it may be necessary and appropriate - for instance where a department needs to employ specialists to carry out short-term roles and where there is no available civil service expertise," he said.

Departments will be able to seek formal assurance from contractors with off-payroll arrangements lasting more than six months and costing over £220 per day that income tax and national insurance obligations are being met. Departments should consider terminating the contract if that assurance is not provided.

This will be monitored carefully with financial sanctions for departments that do not comply, he said, and the recommendations will be implemented within three months.

Sir Bob Kerslake, the head of the civil service, said it was "completely inappropriate" for any senior permanent position in the civil service to be held by a contractor instead of a civil servant paying tax in the ordinary way.

"These new measures are an important step in addressing the relatively small number of cases that breach this principle," he said.

Sign up for the Guardian Today

Our editors' picks for the day's top news and commentary delivered to your inbox each morning.