The Comedy of Errors – with apologies to William Shakespeare in the week of his 450th birthday

There are two ways that mistakes can happen when you are carrying out an experiment to test a hypothesis. Experiments usually have two possible outcomes: accepting a “null” hypothesis, which means concluding that the experiment does not challenge its truth, and rejecting a null hypothesis, which means concluding that the experiment does provide sufficient evidence to do so.

Type 1 errors, otherwise known as “false positives” are when you think there is evidence for rejecting the null hypothesis (eg deciding there actually is something wrong with a smear test) when there isn’t. Type 2 errors, otherwise known as “false negatives”, are when you accept the null hypothesis but you really shouldn’t (eg telling someone they are all clear when they are not).

Saddam Hussain once famously said “I would rather kill my friends in error, than allow my enemies to live”. This suggests that he was really very much more concerned about Type 2 errors than Type 1 errors.

He is not alone in this.

A recent widely reported academic paper published in Nature claimed to have a test that “predicted phenoconversion to either amnestic mild cognitive impairment or Alzheimer’s disease within a 2–3 year timeframe with over 90% accuracy”.

The latest statistics from the Alzheimer’s Society suggest that around 1 in 14 or 7% of over 65s will develop Alzheimer’s. Probably not all of these people will contract the disease within 3 years, but let’s assume for the sake of argument that they will. Even so this means that, out of 1,000 people over 65, 930 people will not get Alzheimer’s within 3 years.

Applying the 90% accuracy rate allows us to detect 63 out of 70 people who actually will get Alzheimer’s. There will be 7 cases not picked up where people go on to develop Alzheimer’s. However the bigger problem, the Type 1 error that Saddam Hussain was not so bothered about, is that 10% of the people who do not and will not get Alzheimer’s will be told that they will. That is 93 people scared unnecessarily.

So 63 + 93 = 156 people will test positive, of which only 63 (ie 40%) will develop Alzheimer’s within three years. The “over 90%” accuracy rate becomes only a 40% accuracy rate amongst all the people testing positive.

In statistical tests more generally, if the likelihood of a false positive is less than 5%, the evidence that the hypothesis is true is commonly described as “statistically significant”. In 2005 John Ioannidis, an epidemiologist from Stanford University, published a paper arguing that most published research findings are probably false. This was because of three things often not highlighted in the reporting of research: the statistical power of the study (ie the probability of not making a type 2 error or false negative), how unlikely the hypothesis is being tested and the bias in favour of testing new hypotheses over replicating previous results.

As an example, if we test 1,000 hypotheses of which 100 are actually true but with a 5% test of significance, a study with power of 0.8 will find 80 of them, missing 20 because of false negatives. Of the 900 hypotheses that are wrong, up to 5% – ie, 45 of them – could be accepted as right because of the permissible level of type 1 errors or false positives. So you have 80 + 45 = 125 positive results, of which 36% are incorrect. If the statistical power is closer to the level which some research findings have suggested of around 0.4, you would have 40 + 45 = 85 positive results, of which 53% would be incorrect, supporting Professor Ioannidis’ claim even before you get onto the other problems he mentions.

We would have got much more reliable results if we had just focused on the negative in these examples. With a power of 0.8, we would get 20 false negatives and 855 true negatives, ie 2% of the negative results are incorrect. With a power of 0.4, we would get 60 false negatives and 855 true negatives, ie still less than 7% of the negative results are incorrect. Unfortunately negative results account for just 10-30% of published scientific literature, depending on the discipline. This bias may be growing. A study of 4,600 papers from across the sciences conducted by Daniele Fanelli of the University of Edinburgh found that the proportion of positive results increased by over 22% between 1990 and 2007.

So, if you are looking to the scientific literature to support an argument you want to advance, be careful. It may not be as positive as it seems.

My father used to regularly paraphrase Benjamin Franklin at me about nothing being certain except death and taxes when I was growing up. However, having spent the turn of the century advising members of small self-administered schemes how to navigate the 6 (some claimed there were in fact up to 13) different tax regimes for pensions which then applied so as to get the maximum possible benefit from them, I was a cheerleader of the tax simplification which the 2004 Finance Act brought in and which demolished all that.

Now it seems that actuaries are no longer going to be necessarily required for members of defined contribution (DC) schemes to get at their savings. In an age of increasing uncertainty about both death and taxes, I find myself cheering this too.

With the right consumer guidance, advice and support, people should be able to make their own choices about how to finance their retirement. Everybody’s circumstances are unique and it should not be for the State to dictate how someone should have to spend their savings.

It then adds:

Those who want the security of an annuity will still be able to purchase one. Equally, those who want greater control over their finances in the short term will be able to extract all their pension savings in a lump sum. And those who do not want to purchase an annuity or withdraw their money in one go, but would prefer to keep it invested and access it over time, will be able to purchase a drawdown product.

So the question has to be asked: why are these freedoms and choices not to be extended to defined benefit (DB) members as well?

The reasons the Government have advanced for the change are equally compelling when applied to DB:

There is a lack of choice for people at retirement, which has become more of an urgent concern now that auto enrolment is boosting DC membership. This is even more the case for DB members who are already numerous (although getting less so daily), as their only choices are how much cash to take up to the 25% tax free limit and (up to a point) when to retire. The other freedom DB members have, of course, is to transfer out, although this freedom makes everybody feel very nervous and is possibly about (see below) to be snuffed out altogether.

Current regulations deter innovation. This is, of course, why defined ambition as an idea has been so slow to get off the ground.

Restrictions on cash commutation imply a lack of trust of members to be able to decide how they spend their savings.

The concern that the annuity market has not maximised income for scheme members. This is mirrored by the high cost of de-risking via bulk annuities, which is the ultimate “flight path” for most DB pension schemes, and which many argue has resulted in a big drag on the growth of UK PLC.

All that would be required to extend these freedoms would be to allow DB members to commute as much of their benefits at retirement, whether for cash or income drawdown, as they wanted, with the rest taken as pension as now.

To be fair to the Government, they do acknowledge the logic of extending the freedoms set out in the consultation to DB members in section 6. But then something strange happens.

Firstly, for public sector schemes, as they are mostly unfunded, the Government says it is concerned about the negative cashflows of members transferring out. If 1% of public service workers did so, the joint Treasury/HMRC analysis is that the net cost would be £200 million. This, I think, provides a revealing peak into the world of state funding, where taking on the Royal Mail Pension Plan was seen as positive for Government finances and off balance sheet private finance initiative (PFI) contracts continue to be negotiated offering doubtful value to the state. It doesn’t matter how much things cost over all, it seems, as long as you are only paying out a bit at a time. The Government often behaves in this respect like the victim of a pay day loan shark. Depending on the commutation terms offered, extended commutation has the potential to solve the public sector pension crisis in a way that Hutton’s Pensions Commission didn’t quite manage to.

Not even considering the option of allowing greater commutation from the schemes themselves, the Government has already decided to ban such transfers from public sector to DC. There is to be no consultation on this.

For private DB schemes, the Government says the decision is “finely balanced”. They are worried about all of those currently captive DB pension investments being spent on Lamborghinis. This rather contradicts the earlier declaration of trust in pensioners to make appropriate decisions about their retirement – after all appropriate investment in support of regular income in retirement (which would presumably be recommended by the “guaranteed guidance” to be offered to DC members) should not differ markedly from the equivalent investments in DB schemes. Whether DB schemes invest on a longer-term basis than individuals is, as the Kay Review made clear, uncertain.

However the Government is very concerned about financial markets – they have section 6 of the consultation devoted to nothing else. It is almost as if individuals can be trusted to look after themselves, with a slightly bigger safety net and a bit of advice, but financial markets cannot.

Again, the Government is not consulting on extending commutation of benefits, but solely on the transfer issue. And apparently removing the current right of all members of defined benefit schemes, except in exceptional circumstances, as proposed with public service defined benefit schemes…must be the government’s starting point, unless the issues and risks around other options can be shown to be manageable.

Even if the Government does manage to stop people pouring out of the exits before April next year, this has to be bad policy. To provide more freedom and choice to one group of pensioners and at the same time to remove a longstanding freedom (and one available at the point members joined the schemes) from the other groups is clearly unfair. What is worse, with an election looming, it is likely to be unpopular.

By the way, one of the things that stands out for me in this whole consultation is the use of State with a big S and government with a small g. It is as if typography alone could portray the “State” as big and bad and “government” as on the side of the little guy. I have done the reverse here.

So, if you DB members want to stop the flickering light of Freedom and Choice dying before it even got going, I advise you not to go gentle but to rage, rage and respond in large numbers to questions 9 and 10 of the consultation in particular. You have until 11 June.

In all the talk about annuities and the poor value they currently offer, nearly all of it has been based on standard annuity rates, ie where there is nothing sufficiently medically wrong with you to affect your life expectancy. However this is almost certainly not the rate you should be looking at.

Go to any of the annuity provider or broker websites, sometimes buried away a little, and you will find a link explaining what they can offer in the way of “enhanced” or “impaired lives” annuities. Legal & General’s web page on this looks like the kind of warning notice you find on the wall of your doctor’s surgery waiting room, with headings like Smoking, Type 2 Diabetes and High Blood Pressure. But in the upside-down world of buying annuities these become good things to do or have.

Just Retirement give some handy illustrations of what various conditions could mean for your income: up 20% for minor conditions like obesity and hypertension, up 30% for “moderate” ones like being a heart attack survivor with a bypass and 40% for serious medical conditions like stage 2 bowel cancer one year in. However, you don’t need to get anywhere near the frankly frightening conditions in the moderate and serious boxes to make a big difference to the income you can receive. annuitydiscount.co.uk provide a very long list of medications (covering every letter in the alphabet except J and Y) which could lead to an impaired life annuity if disclosed to the annuity provider.

As the BBC article from 2012 posted by the Better Retirement Group on enhanced annuities says: “At its simplest an annuity is a bet with the insurance company about how long you will live.”

They studied 2,000 people (mean age 63.3 years, range 45.7 to 96.7) from the population-based Rotterdam Study in whom high-resolution, structural brain MRI scans had been carried out. Asymptomatic brain infarcts (more commonly known as strokes) were present in 145 people (7.2%). Among other findings, aneurysms (1.8%) were the most frequent. Benign brain tumors also turned up reasonably often (1.6%). The most extreme case was someone with a large, chronic subdural haematoma, who was subsequently found to have had a minor head trauma 4 weeks before the MRI scan. Some of the scans are shown below.

But the really amazing thing is this: only 2 of the 2,000 people scanned (the subdural haemotoma mentioned above and another who had a 12 mm aneurysm of the medial cerebral artery) had any idea that there was anything wrong with them!

Another huge area of undiagnosed disease (and on the annuity.co.uk list for enhanced annuities) is prostate cancer. According to a systematic review of prostate cancer biopsy schemes by the University of York in 2005, where they quoted from the NHS Centre for Reviews and Dissemination publication on screening for prostate cancer, Effectiveness Matters:

Post mortem studies show that 30% of men over 50, who had no symptoms of prostate cancer whilst alive, had histological evidence of prostate cancer at the time of death. This percentage rises to 60-70% in men over 80 years of age. In other words, most men with prostate cancer die with, rather than from, the disease.

The main reason these studies have been carried out is to determine whether screening for prostate cancer, which kills 3.8% of men with the disease, has saved many lives. The Prostate Specific Antigen (PSA) test that is commonly used to detect prostate cancer in the absence of symptoms is not only prone to false positives and negatives (ie telling you you have it when you don’t and don’t have it when you do – something all screening suffers from to some extent), but can lead to you being offered treatment which may well be worse than the disease. This is discussed further in the excellent The Norm Chronicles, by Michael Blastland and David Spiegelhalter, which questions whether, overall, screening is particularly effective in saving lives.

The latest Association of British Insurers (ABI) facts and figures on the UK annuity market suggest that enhanced annuities have grown in popularity, to 24% in 2012 from 2% in 2003. There is scope to make further large increases in these figures if more people can be persuaded to have themselves screened for some of the most common undiagnosed conditions before they retire.

Have you, as a result of your frenetic activity since Christmas, got a bit of a peer review backlog? I can help. Let me be the scheme actuary you’re temporarily short of. With a 10% discount on the rates shown here until the end of the UK 2013/14 tax year, and a further 10% reduction for type 2 peer reviews.

Imagine a ship tossed around in a rough sea. The waves throw the vessel in all directions, before the sea level plummets sharply. Whirlpools have formed to the south of them and there are fears these will spread north and swallow the ship. The only means of escape is a rope ladder dangled above the ship, from an airship desperately being inflated above their heads. However the only place on the ship where this can be reached is the ship’s bridge, which is reserved for the ship’s officers.

Strangely the crew do not attempt to storm the bridge but seem resigned to their fate. Instead the captain orders the airship to dump several tons of ballast into the ship, pulling it even further down into the water. To all pleas for mercy from the crew his reply is the same: row harder. The captain has had the sail removed and passed up to the airship crew, on the understanding that the material will be made into a tow rope that will pull the ship to safety. But that seems like a long time ago. The crew have been left with no choice but to row, their daily rations gradually dwindling.

Question: if the sea level rises again, as of course it will eventually, so that the rope ladder moves into reach for everyone on the ship not too weak to take advantage of it, despite everything the ship’s officers have done to make the crew’s lives more hopeless, should we congratulate the ship’s captain on his stewardship? I and many others think not.

Meanwhile the head of a pin is drawing perilously close to the fabric of the airship as the crew pile heedlessly up the rope ladder…..

Germany has surprised the European Commission (EC) by suddenly insisting that stiffer data protection controls are incorporated into the negotiations for the Transatlantic Trade and Investment Partnership (TTIP), which began earlier this year, and for which the second round has started this week. For those of you who have not heard of it before (understandable, as the negotiations so far have had a deliberately low profile), the purpose of the TTIP is to create a single transatlantic market, in which all regulatory differences between the United States US and the EU are gradually removed. The EC calls it “the biggest trade deal in the world”.

As the EC goes on to say:

On top of cutting tariffs across all sectors, the EU and the US want to tackle barriers behind the customs border – such as differences in technical regulations, standards and approval procedures. The TTIP negotiations will also look at opening both markets for services, investment, and public procurement. They could also shape global rules on trade.

Concerns have started to emerge about the massive transfer of power from governments to corporations that the final deal might allow. However Germany’s intervention on data protection is just the latest of a list of reasons that have been advanced for why the TTIP talks are unlikely to go anywhere. From the legislative schlerosis of the US, to protectionist instincts on both sides recently strengthened by austerity, to French paternalism towards their film industry, to European fears about an influx of GM foods, the TTIPing point will never be reached, they say. So nothing to worry about then.

Or is there? A document published last year by the US Chamber of Commerce and BusinessEurope explains how it would be able to overturn existing legislation which got in its way. And if the long tortuous progress of Solvency 2’s implementation date, the bureaucratic equivalent of the man with the end is nigh sandwich board on his back, has taught us anything, it is that unimplemented regulatory frameworks can still have massive impacts. Just this month it was revealed that the best funded pension schemes in the FTSE 100 are insurers, precisely because of the impact of those schemes on insurers’ solvency capital requirements under Pillar 1 of Solvency 2. And the clear rebuff to EIOPA from exporting these requirements to occupational pension schemes has not prevented the work to develop a framework for imposing them from continuing.

So what would TTIP mean for defined benefit (DB) pension schemes? Well, at first sight, not very much. US DB schemes tend to have funding targets equivalent to FRS17 levels, which would be seen as at the weak end of UK funding targets. However, as we have seen with the process of market harmonisation in the EU, horse trading may lead to the US being stuck with stiffer requirements imported from the EU on pensions in order to maintain subsidies for US farmers, say.

And there are two features of the US DB landscape which would be an issue for many UK DB schemes.

The first is the recovery plan length, which typically does not exceed 7 years in the US. Possibly not too onerous in many cases, if coupled with a FRS17-type funding target, but the EIOPA caravan has surely travelled too far for any dilution of funding target to be allowed at this stage. A 7 year recovery plan would however represent a considerable increase in contribution requirements for many schemes within the UK’s current funding environment.

The second is the restrictions placed on US pension schemes which fall below prescribed funding levels. If the funding level falls below 80%, no scheme amendments are allowed which would increase benefits until the funding level has first been restored to 80% or above, and certain types of benefit payments are restricted. These restrictions become much more stringent below 60% funding, when benefit accrual must cease and the range of benefits which cannot be paid out is extended to cover “unpredictable” contingent events.

We may not be out of the woods of Solvency 2 yet as far as DB pension schemes are concerned. But even if we do manage to break out of EIOPA’s grip, it may be only to find ourselves surrounded by a larger forest.

Mention the Phoenix Four in Birmingham and you are likely to get a strong reaction. Most people knew someone who worked at MG Rover’s Longbridge plant, and many local families supplied workers for generation after generation. A huge rally brought tens of thousands onto the streets in 2000 when BMW put MG Rover up for sale, protesting against what had appeared at the time to be the most likely outcome of Alchemy Partners buying it and turning it into a low volume car manufacturer with only 2,000-3,000 of the 6,500 jobs there remaining. So there was jubilation when the ‘Phoenix Four’ group of businessmen (John Towers, John Edwards, Peter Beale and Nick Stephenson) stepped in to take the business off BMW’s hands for £10 with a further £500 million accompanying the business from BMW to sweeten the deal. By 2005 all the jobs had been lost.

A Government inquiry into Phoenix Venture Holdings (PVH – the Four’s company) reported in 2009 that The Four had managed to extract £42 million in salaries and pensions by this time. The inquiry spent 4 years and £16 million getting to grips with the convoluted machinations by which this was achieved. No criminal charges resulted. The Four were not even disqualified from being company directors. Instead, in 2011, they belatedly agreed voluntarily not to serve as directors for 3 (Edwards), 5 (Towers and Stephenson) and 6 (Beale) years respectively.

In January last year, the Executive Counsel to the Financial Reporting Council (FRC) finally turned to the advice The Four had been receiving during the whole saga, from Deloitte and specifically their head of UK corporate practice Maghsoud Einollahi, alleging that their conduct fell short of the standards reasonably expected of them in relation to Project Platinum (the project to put a deal together) and Project Aircraft (the specific deal to transfer MG Rover Group’s (MGRG’s) accumulated tax losses to a subsidiary of PVH). The tribunal ruled on these allegations last month. It makes entertaining reading unless you happen to be a former MG Rover employee.

One of the issues was that Deloitte had muddied the waters about who they were representing (MGRG or PVH) in order to mask a massive conflict of interest. As the tribunal states:

If the identity of the client is not known it is not possible to identify and consider whether there is any conflict existing or potential. That is the real importance of identifying the client. Here the client was known to the Respondents (ie Deloitte and Einollahi) a substantial time before the final existence of a letter of engagement and nothing was done about it.

The Phoenix Four were always the client. Deloitte were at all times acting on their behalf. We know too that the Respondents were represented at an MG Rover Group Limited Board Meeting and made a presentation to the Board thus suggesting that they were acting for MG Rover and not the Phoenix Four.

But my favourite bit is the extract of Einollahi’s testimony on who his client was:

Q: (reading his previous testimony) “…you did not think you had a client…” A: (Pause) I think that is fair, that I didn’t believe I contractually had a client. Q: Exactly A: But Q: And the problem is the one that I have alluded to already, that you would be holding yourself out to third parties as acting for, in this case, the group (ie MG Rover) A: (Nods)

Following this Pinteresque dialogue, the tribunal moved on to Deloitte’s fee of £7.5 million. Part of the defence case had been that £7.5 million was not a very large fee within the context of Deloitte’s annual fee income, that contingency fees (ie which were paid only if a given result was achieved) were common and that clients were not prepared to accept different arrangements. The tribunal was not impressed:

It seems to us that Mr Einollahi would charge a contingency fee of a size he thought that he would be paid by the client without considering whether it was appropriate or not. Again when he gave evidence he was cross-examined and we refer to one question and answer. Q: …you did not like to negotiate fees downward? A: I didn’t – I didn’t act for people who wanted to negotiate my fees downward. I didn’t need to.

The tribunal concluded:

He wanted that fee of £7.5 million and realised that his best prospects of achieving that fee were by a deal between the Phoenix Four and HBOS rather than between MGRG and First National Finance or MGRG and HBOS

Project Aircraft, the scheme involving moving around MG Rover tax losses, had been attempted before under the title Project Salt/Slag and rejected by the Inland Revenue. Aircraft succeeded where Slag failed largely because the Revenue believed this time that MG Rover would benefit from the profits generated by the scheme.

Mr Towers said “frankly, for us, what mattered was there was a possibility here of creating cash, additional cash for the group and most particularly, for the cash-consuming part of the group which was the car company”. Mr Beale’s evidence was to the effect that MGRG benefited from the transaction because “it gave the group additional cash reserves which it could lend to MG Rover as and when required”. The Inspectors (from the Government inquiry) said at Chapter XI paragraph 17 “in practice, much of the money which the group generated from Project Aircraft was used to fund a payment to the Guernsey Trust”. (The beneficiaries of which included Messrs Beale, Edwards, Stephenson and Towers.) The Inspectors continued “immediately before Barclays Bank made its £121 million loan (which also paid off a previous loan and some other creditors), PVH had credit balances on its bank accounts totalling £2,184,083. The loan increased the credit balances to £14,736,629, enabling the company on 26 June 2002, without having received any money from any outside source in the interim, to pay £7,905,125 to the Guernsey trust (as well as paying £2,261,875 to Deloitte in respect of fees for Project Aircraft). No payment was made by PVH to MGRG at this stage, or in fact at any time before November 2003.

The tribunal continued:

Mr Einollahi undoubtedly played a significant part in Project Aircraft. He must have been aware, and admits that he was so aware, that the Phoenix Four were on holiday in Portugal in 2001 and while on holiday agreed between themselves to pay themselves very substantial bonuses. They in fact paid themselves collectively about £7 million after the conclusion of the Project Aircraft transaction. These sums came essentially from assets of MGRG and were used to make these very substantial payments to the Phoenix Four. They received the whole of the proceeds and MGRG received none.

In conclusion, the tribunal said:

They (ie Deloitte and Einollahi) placed their own interests ahead of that of the public and compromised their own objectivity. This was a flagrant disregard of the professional standards expected and required and was in each individual case, and of its own, serious misconduct.

The Executive Counsel, who had made the complaints, asked for a severe reprimand and a fine of between £15 million and £20 million. They also requested that Einollahi be excluded from membership of the Institute of Chartered Accountants in England and wales (ICAEW) for 6 months and fined an amount based on an assessment of his financial resources. Deloitte suggested instead that the fine should only be £1 million and Einollahi should not be fined at all.

At this point, in my view, the tribunal lost its way a little. They decided on a severe reprimand and a fine of £14 million for Deloitte. This was calculated as follows:

We have assessed the financial gain from the fees attributable to both Project Platinum and Project Aircraft with a deduction for the total amount of recorded costs against these projects. We have added interest at 1% over base rate to deny Deloitte any financial gain from the misconduct.

This raises an interesting question about what calculations other firms might make in the future about the chances of ending up in a tribunal like this and the likely consequences against the rewards of the deals themselves. If worst case scenario is that they won’t make a profit, I remain unconvinced that this will prove much of a deterrent.

They added:

We have borne very much in mind that Deloitte is not insured against the imposition of a fine and has undertaken to indemnify Mr Einollahi against any fine imposed upon him.

It is heartwarming to see them looking after their errant employee in this way, but their insurance arrangements should be of no interest to anyone.

Einollahi himself was excluded for 3 years rather than the 6 months requested by the Counsel, but only because he was not prepared to voluntarily relinquish his practising certificate. He also refused to cooperate with the assessment of his financial resources, leading to the tribunal to put a bit of a finger in the air and opt for a fine of £250,000.

So what now? The tribunal made much of the public interest in the hearings:

It was particularly important in the case of both Project Platinum and Project Aircraft that the public interest be considered because of the concern of inter alia the Government, employees, other employers, particularly in the West Midlands, creditors and the general public about the continuation of large scale car manufacturing in the West Midlands.

The importance of considering the public interest is further emphasised because both the Projects resulted in very large sums of money that might have been utilised for the benefit of the MG Rover Group in the running of its business instead, being used for the benefit of individuals, including the Phoenix Four.

But what is the public interest? My assumption would have been that it must primarily be about the portion of the general public which was most damaged by all this, namely the MG Rover workers who lost their jobs and their communities. The local MP, Richard Burden, agrees. The Trust Fund for former MG Rover workers, which John Towers had at one point said would have over £50 million in it, was finally wound up earlier this year when the £23,000 actually available was donated by the workers to a local hospice.

The £14.25 million awarded in fines would normally go to the Consultative Committee of Accounting Bodies (CCAB), an umbrella group for several professional bodies, which pays the costs of FRC disciplinary cases. However in this case the costs of the proceedings of just under £4 million have already been charged to Deloitte on top. Is the case for meeting the costs of future disciplined accountants really greater than the public interest in making some contribution to the communities that the FRC’s members have facilitated into the ground?

There will be some time to make this decision in. Depressingly, Deloitte and Einollahi filed formal notice on 1 October that they are appealing the decision, as indeed they have contested everything that wasn’t nailed down throughout the process. Their joint statement read as follows:

“We recognise the general desire to move on from this case but do not agree with the main conclusions of the tribunal which we feel could create significant uncertainty for individual members and member firms of the ICAEW.”

After all, if it ever became accepted that consultants had any responsibility to the most vulnerable people affected by their less-than-professional manoeuvrings, where might it end? There is no time limit on the tribunal member hearing the appeal to make a decision on whether an appeal can go forward.

Enough is enough. Deloitte should do the right thing and drop their appeal now.

A man is sentenced to 7 years in prison for selling bomb detectors which had no hope of detecting bombs. The contrast with the fate of those who have continued to sell complex mathematical models to both large financial institutions and their regulators over 20 years, which have no hope of protecting them from massive losses at the precise point when they are required, is illuminating.

The devices made by Gary Bolton were simply boxes with handles and antennae. The “black boxes” used by banks and insurers to determine their worst loss in a 1 in 200 probability scenario (the Value at Risk or “VaR” approach) are instead filled with mathematical models primed with rather a lot of assumptions.

The prosecution said Gary Bolton sold his boxes for up to £10,000 each, claiming they could detect explosives. Towers Watson’s RiskAgility (the dominant model in the UK insurance market) by contrast is difficult to price, as it is “bespoke” for each client. However, according to Insurance ERM magazine in October 2011, for Igloo, their other financial modelling platform, “software solutions range from £50,000 to £500,000 but there is no upper limit as you can keep adding to your solution”.

Gary Bolton’s prosecutors claimed that “soldiers, police officers, customs officers and many others put their trust in a device which worked no better than random chance”. Similar things could be said about bankers during 2008 about a device which worked worse the further the financial variables being modelled strayed from the normal distribution.

As he passed sentence, Judge Richard Hone QC described the equipment as “useless” and “dross” and said Bolton had damaged the reputation of British trade abroad. By contrast, despite a brief consideration of alternatives to the VaR approach by the Basel Committee on Banking Supervision in 2012, it remains firmly in place as the statutory measure of solvency for both banks and insurers.

The court was told Bolton knew the devices – which were also alleged to be able to detect drugs, tobacco, ivory and cash – did not work, but continued to supply them to be sold to overseas businesses. In Value at Risk: Any Lessons from the Crash of Long-Term Capital Management (LTCM)? Mete Feridun of Loughborough University in Spring 2005 set out to analyse the failure of the Long Term Capital Management (LTCM) hedge fund in 1998 from a risk management perspective, aiming at deriving implications for the managers of financial institutions and for the regulating authorities. This study concluded that the LTCM’s failure could be attributed primarily to its VaR system, which failed to estimate the fund’s potential risk exposure correctly. Many other studies agreed.

“You were determined to bolster the illusion that the devices worked and you knew there was a spurious science to produce that end,” Judge Hone said to Bolton. This brings to mind the actions of Philippe Jorion, Professor of Finance at the Graduate School of Management at the University of California at Irvine, who, by the winter of 2009 was already proclaiming that “VaR itself was not the culprit, however. Rather it was the way this risk management tool was employed.” He also helpfully pointed out that LTCM were very profitable in 2005 and 2006. He and others have been muddying the waters ever since.

“They had a random detection rate. They were useless.” concluded Judge Hone. Whereas VaR had a protective effect only within what were regarded as “possible” market environments, ie something similar to what had been seen before during relatively calm market conditions. In fact, VaR became less helpful the more people adopted it, as everyone using it ended up with similar trading positions, which they then attempted to exit at the same time. This meant that buyers could not be found when they were needed and the positions of the hapless VaR customers tanked even further.

Gary Bolton’s jurors concluded that, if you sell people a box that tells them they are safe when they are not, it is morally reprehensible. I think I agree with them.