In classic Thatcherite fashion, the coalition government has decided that what Whitehall really needs is a bunch of business chiefs to drip ruthless corporate wisdom through the ranks of the civil service. Who could possibly be better to supervise swingeing cutbacks in the public sector than battle-hardened industrial and commercial titans?

The cabinet office minister, Francis Maude, last week produced an impressive line-up of business figures who have agreed to become non-executive directors of government departments. Among those answering the call to serve Queen and country are GlaxoSmithKline's boss Andrew Witty, the Rio Tinto chief Jim Leng, former Tate & Lyle head Iain Ferguson, Centrica's Sam Laidlaw and Sara Weller of Argos.

This is a pretty straightforward ideological initiative: it's down to a belief that all things private are superior to the public sector. The government wants to recreate the structure of corporate boardrooms in Whitehall.

The exact role of the new non-executives (or Neds, as they are widely known) is unclear. They will advise and support government ministers for a stipend of £15,000 annually, and if they think that a top civil servant isn't up to the job, they can recommend dismissal. But they'll only pop in for the occasional board meeting and certainly won't be giving up their day jobs.

It's a bit counterintuitive to add to the ranks of civil service leadership at a time when the rank and file is facing 400,000 job losses. And it's deeply debatable whether a Whitehall recreation of the clubbable atmosphere of Britain's boardrooms is really a positive.

Non-execs have a mixed reputation in the City. They're supposed to be an independent safeguard against executive excess, but at their worst they're disengaged members of the great and the good who pick up a fee to supplement their pensions, but lack the expertise or patience to dig deeply into the books. The late Tiny Rowland once dismissed part-time directors as Christmas tree baubles. And the supermarket boss Sir Ken Morrison quipped that for the price of a Ned, he could hire two checkout girls – who would provide much better value.

Peter Hahn, a corporate governance expert at Cass business school, points out that over the past two decades, as non-executives have taken a pre-eminent role, they have singularly failed to prevent an explosion in executive pay. And there are plenty of examples of failure. Where were the non-execs when RBS and Lloyds went on their binges of reckless lending?

Particularly puzzling is the criteria for selecting these government directors. The cabinet office says it advertised the roles and headhunters helped identify candidates. But the search was spearheaded by the former BP chief executive Lord Browne, who has been thumbing through FTSE 100 index in search of credible characters. The coalition's fixation with Browne is remarkable. Let's remember that the former oil boss, who drew up government policy on tuition fees, has a spotty ethical record – he was slammed by US authorities for presiding over a lax safety culture at BP and subsequently lost his job for lying to a court about his private life.

The FDA union, which represents senior civil servants, says that, in principle, it welcomes a degree of business expertise at the big spending ministries. But there's concern that Maude may be too keen on big-name signings. Witty, for example, is widely considered to have done a good job at Glaxo, where he presides over a drugs company of 99,000 people in 100 countries, generating turnover of £28bn. No doubt he'll turn up for quarterly board meetings at Vince Cable's department of business.

But is he really going to have time to scrutinise whether the permanent secretary's latest cost-cutting wheeze is extracting all the fat that it ought to be? Or is he just an impressive name to add to the government's monogrammed letterhead?