Thursday 5 September 2013 15.30 EDT
First published on Thursday 5 September 2013 15.30 EDT

What do the following recent news stories have in common? IT failings over the introduction of new welfare payments; the never-ending saga of BBC executives paying each other silly money; defence procurements coming in billions of pounds over budget; the recklessness of the bankers? Throw in dozens of other cases from the private and public sectors and there emerges a clear pattern: of decisions taken by individuals or groups that constitute failure or dereliction of duty but which go unpunished.

The word "punishment" is enticingly loaded. In international relations it is in vogue. Should Bashar al-Assad be punished if it is clear his government used chemical weapons? From the former Yugoslavia to Rwanda, attempts are made to punish world leaders and their henchmen. Occasionally, businessmen are punished too – think Bernie Madoff and his Ponzi schemes. He received 150 years in jail. But his case is totemic because it is so rare.

Where there is incontrovertible evidence of fraud, courts usually convict. The individual has a criminal record. It is hard, although not impossible, for that person's career and reputation to recover. Justice is done.

But far more difficult are the many cases in which senior public figures are culpable in decisions that have led to huge financial loss, in some cases ruining peoples' lives, but criminality cannot be proven. The bar for a trial is necessarily set high and can be insurmountable.

So what possible punishments are left? Summary dismissal is used against a shopfloor worker for nicking a few products from the assembly line, or a middle manager for sexual harassment. The weapon is almost never deployed against top executives. Part of the reason is financial – companies would rather pay them off than endure the publicity of a tribunal. The more pernicious reason is cultural: as a member of the board or senior executive you never know when you might bump into that person. Why leave yourself susceptible to a quiet act of revenge in the future when you don't have to?

It is only when the public bays for blood that extra measures are taken. The story of plain Fred Goodwin is brilliantly told in Iain Martin's new book, Making it Happen: Fred Goodwin, RBS and the Men Who Blew Up the British Economy. Aggressive, obsessed by the baubles of wealth, Fred the Shred is so determined for RBS to take over the banking world that he omits to find out what his wheeler-dealer teams are up to. At least as culpable are the board members who are quite happy to take the money for their non-exec non-labours and forget to ask questions.

Goodwin – friend of the royal family, prime ministers, chancellors and the Scottish political class – is stripped of his knighthood. He retains an enormous pension and is to be found polishing his vintage cars, the pantomime villain. It makes us feel better and the corporate and political worlds can "move on".

But the odd case of ritual humiliation is no substitute for better governance. That will not improve until proper systems of accountability for failure are introduced. In the private sector, when shareholders incur losses, it is up to them to complain – but almost invariably they don't, as institutional investors account for most holdings. Why would they want to rock the boat?

When public money is spent, the case for action is even clearer. It beggars belief that during the bank bailouts of 2007 to 2008, ministers did not – even as they took urgent decisions – do more to punish those whose hubristic decisions led to the crisis.

At the BBC, although the money lost has been tiny in comparison to the banks, the sense of injustice at the largesse shown by management towards its own is felt just as strongly. A few dozen people paid each other ridiculous sums as they moved from one job to another or began to enjoy lucrative early retirement. They did so believing (correctly) that they would get away with it, and convinced themselves they deserved it.

After inquiries by a Tory MP, the Crown Prosecution Service probed whether crimes had been committed and concluded that they hadn't. To prove criminal intent, if there had been, would have been too hard. To prove malpractice might have been easier, but there is no effective mechanism.

We need to devise a process whereby serious action can be taken against egregious acts of back-scratching, waste and lack of rigour in governance. It is surely a win-win for any political party with the courage and tenacity to introduce such a system. Some models already exist. Professional bodies for doctors, lawyers and accountants serve this purpose. Are they robust enough? A new public body could be created, perhaps including representatives of the CBI and TUC. Or if that's too cumbersome, maybe the Commons public accounts committee – which is good at haranguing and exposing but has little powers besides – could play a part.

Transparency is key. Legislation must be introduced to override confidentiality and data protection clauses in specific cases under investigation. Checks and balances would be needed to protect those who feel wrongfully accused. Those found to have played fast and loose with others' money could be put on a blacklist of public appointments for a specified number of years. There may be other ways too; but this is a debate which needs to be started.

Responsible executives, non-executives and civil servants have nothing to fear in exposing and punishing the bad apples. Bringing out the stocks serves little purpose. But, in order to begin the herculean task of improving confidence in public life, we need far smarter forms of redress.