‘Won’t somebody think of the spads?!’ said one wag following the recent reshuffle. We here at the Constitution Unit (and Hull) have been. We’ve just written a book on spads, gloriously entitled Special Advisers: Who they are, what they do and why they matter. We’ve spent 18 months looking at special advisers between 1979 and 2013: all 626 of them. We interviewed over 100 people, including almost 40 spads and 30 ministers (both former and current).

As part of this we’ve been looking at the tenure and distribution of spads over time, both within a government and over successive parliamentary terms. So here we present an interim analysis of the last spads reshuffle.*

The first point is turnover. Of the 63 Spads who began in 2010, only 31 remain. Half have left. The majority of the initial batch who remain are connected to ‘the big beasts’ of the government (David Cameron, George Osborne, Nick Clegg etc)—or at least, those whose ministers have not been reshuffled out.

But this misses the bigger story. The total number of spads employed by the Coalition between 2010 and 2014 is around 175. In fact, the number of spads who leave has been increasing as time goes on. In 2010 five left; in 2013 around 30 did. This makes sense: spads leave because of reshuffles, exhaustion, wanting to do something new—and getting out while the going is still good. But they must be replaced.

Special advisers (spads) are in the newsagain. The Coalition government has finally (and belatedly) released its annual report on numbers and cost: there are now 98 spads in post—30 more than when the Coalition started, breaking the Coalition pledge to keep numbers down and topping the former high of 83 (in 2001) under Tony Blair.

But so what? Why the obsession with the number and cost of spads anyway? They have a peculiar reputation: they are spinners, intriguers, bag carriers, policy wonks; they are exercise great power, being more influential than officials and junior ministers. They are young, inexperienced, politicians-in-waiting. We’ve all seen The Thick of It. Contrary to their stereotype as political mischief makers, we have found strong support for our conclusion that spads have become accepted as indispensable to government.

The Constitution Unit has been carrying out an 18 month project on special advisers. It involves the construction of a database of all known spads between 1979 and 2012; surveys of former spads; and we have carried out over 100 interviews with former and current spads, ministers, officials and external stakeholders (interest groups, political parties) in order to understand what it is spads do and how they go about it.

The first obvious point is that if numbers are rising, it is because ministers keep appointing them. That’s a banal statement, but it needs to be said. Ministers appoint spads, so the real question is, why do ministers need them? A key reason is ministerial overload: ministers are now simply too busy, and the governing of Britain too complex. And it is not just about the volume and complexity of work, but also the pace at which ministers are expected to respond. Ministers need support, and they must often act through others.

That doesn’t answer the question entirely. After all, ministers have hundreds, sometimes thousands, of officials to support them. But there are various reasons why spads are still necessary. Part of it is about the civil service: Ministers can choose very few of the career officials with whom they work; career officials are often generalists; and they are expected to remain neutral. Part of it is about British government, which is fragmented, being divided into departmental silos.

Ministers require support from outside the civil service to help them achieve their objectives. Being personal appointments, spads serve the political interest of their ministers first, and that of their party in government next (in that order). They are not as bound by departments as officials are, and can take account the overall strategy of the government. The Coalition has also complicated matters: with two parties in government, spads are also needed to ensure intra-government or bipartisan support within government.

Spads have become indispensable. They began as a useful resource supplementing the work of the civil service, but as one former Cabinet minister said to us, ‘I couldn’t have done the job without them.’ Almost all of those we interviewed—ministers, officials and external stakeholders—saw spads as an established institution within British government. Parliamentary committees in various reports—from the Treasury and Civil Service Committee in 1985 to the Public Administration Committee in 2012—have consistently acknowledged the importance of spads to the process of governing. Spads, along with the permanent civil service, are now recognised in legislation.

Yet spads receive little support. They have no job description of merit. They receive minimal induction or training; and their performance is not systematically evaluated. There is little time for them to stop and reflect because of the pace of the job and the long gruelling hours; and there is zero job security—their contract terminates when their minister leaves, when there is an election, or at the whim of their minister. Generally speaking a spad’s time in government is short-lived and frenetic. About half of all spads between 1979 and 2010 have a tenure of about three years or less in government (unsurprising, given the low tenures of ministers); but a quarter of them have a tenure of six or more years (indeed, some spads commented that they had effectively became the institutional memory of the department because of the rapid turnaround within the senior civil service).

If this description of a spad’s life sounds ad hoc and chaotic, that’s not surprising. What they are able to do depends almost entirely on their minister. There is no job description for ministers or MPs either (see Lord O’Donnell on this, here and here). Politics is still thought best for amateurs—and professionalisation discouraged. Given these conditions it is surprising there are not more blunders involving spads. But the vast majority (there have been over 600 since 1979) do their job competently, and without fuss.

So it is time to start taking spads seriously: they are an established part of government. There needs to be greater professionalisation: more systematic recruitment, better induction and training, at the very least. The current job description of special advisers dates from the 1970s; there needs to be more about what spads can do rather than listing prohibitions on their conduct. Professionalisation means greater control from the centre; but the alternative is to keep ‘muddling through’.

The focus on numbers is misguided. Spads are here to stay. We need to ask how they can be more effective in government. These are issues we develop further in the book we are writing about special advisers, as the main output of our 18 month project. It is to be called Special Advisers: Who they are, what they do and why theymatter and will be published next summer.

It would appear absurd and self-defeating to remove a sitting Prime Minister less than 3 months before a general election and return a leader who had himself been removed from office only 3 years previously. After all divided parties do not win elections. The Rudd-Gillard soap opera may be a personal battle for supremacy of a dysfunctional Australian Labor party, but its roots lie in systematic and elite driven party politics. The simple answer as to why the Australian Labor party ousted Gillard this week and Rudd in 2010 is that they could and they had previous. Since 1945, there have been several challenges to sitting Prime Ministers in the party room and numerous examples of party leaders being turned out of office in both main parties. Liberal Prime Minister John Gorton was challenged shortly after winning the 1969 election and again in 1971 when a tied vote famously saw him casting the deciding vote against himself. Andrew Peacock failed to unseat Liberal Prime Minister Malcolm Fraser in 1981. Under Labor Bob Hawke managed to see off Paul Keating’s first party room challenge in 1991 but after a destabilising 6 month backbench campaign from Keating, lost the premiership. Rudd’s comeback is not so unusual. As Pat Weller observed the vanquished in Australian politics are reluctant to leave the stage. Challengers regroup and fight again as Keating did in 1991, but also leaders can hang around to fight to regain the crown as John Howard did successfully and Peacock unsuccessfully in the Liberal party.

The political culture of party leadership in Australia is ‘brutal’. The end for Gillard’s leadership of the party and country was swift, as Rudd’s first effort had been. A leadership ‘spill’ can be organised a short notice and defenestration is swift and ruthless. The oligarchic nature of party organisation ensures that party leaders need to satisfy, placate, manipulate or cajole their peers to survive in post. Research into leadership selection has shown that leadership selection and ejection, concentrated as it is within the Federal Parliamentary Party marks the ALP and Liberal party out. Furthermore the institutionalisation of the ALP’s factions is not only more entrenched than any other Australian party, but arguably any other social democratic party in the Western world. Australian party politics has managed to resist the trend towards expanding leadership selection beyond the parliamentary party to the membership. The short three year electoral cycle is cited as the most common reason for maintaining the status quo. Both main parties cannot afford to indulge in extended leadership selection and be ‘leaderless’. So the power to select the party leader remains firmly in the hands of the parliamentary caucus with this concentration of elite power exacerbating the role of factions within the ALP. Former party leader Mark Latham called Labor a ‘virtual party controlled by a handful of machine men’.

As Rudd and Gillard found out, once leadership speculation gets going in Canberra a cocktail of party power brokers and political journalists can easily destabilise an incumbent Prime Minister. The devastating critique of Rudd by journalist David Marr in early June 2010 represented a tipping point in Rudd’s fortunes giving rise to the concerted internal party opposition. Gillard, with the opinion polls tanking for some time, suffered from Rudd’s constant sniping and a strain of virulent misogyny peddled from the Opposition and media. Running a minority government, fighting Rudd within the ALP and facing an aggressive centre-right bully in Liberal party leader Tony Abbott as well as the constant media attacks meant she had little chance. Once the speculation is set in motion it becomes a ‘self-fulfilling prophecy’, as leadership consolidation is an elusive commodity in Australian politics and Federal MPs only see self-preservation.

Unedifying and undemocratic it may be but the parliamentary caucus dynamic and machine politics create an Australian leadership setting in which hypocrisy, deceit and plotting are endemic.

The Constitution Unit this week held a seminar on the IPPR’s new report on Civil Service reform. Guy Lodge from the IPPR outlined the report and former Permanent Secretary Sir Leigh Lewis made various observations, all pertinent. Numerous good points were made in the question and answer session that followed. Everyone agreed – sincerely and indeed correctly – that the report was not only excellent but also very important.

But I wonder. Not about the excellence but about the importance. Whitehall watchers, myself included, find fascinating the detail of how Permanent Secretaries are appointed and how many people should serve in a Minister’s private office. But it is hard to argue that those issues matter a row of beans in considering how to avoid episodes like the West Coast mainline saga or whether or not it is sensible to reduce the number of specialist heart units in hospitals. And it is those issues which are far more likely to matter to the ordinary citizen than whether the Prime Minister or the head of the civil service should appoint the next Sir Humphrey.

There is certainly no doubt that there are big issues about the civil service which are worth serious study. What is the right balance between Ministers deciding policy and public officials who are tasked to deliver that policy? Should the civil service have direct links to Parliamentarians in the way that local Government officials have direct contacts to Councillors, regardless of their political party? Should we have much more wholesale turnover of the civil service with a change of Government, following the US model. And should we perhaps more openly recognise that not all Ministers are actually competent to run large Departments when they have no prior relevant experience.

The IPPR could have been asked to look at those issues but they weren’t. Indeed, even within the broad frame of what they were asked to look at, it was all a bit selective. For example the IPPR were asked to look at how Permanent Secretaries were appointed but not being asked to look at how they were dismissed – even though it is an open Whitehall secret that quite a few recent Permanent Secretary departures were not entirely voluntary.

So we are back into the old game of tinkering with the system. Who knows how much emphasis the Government will now put on this but no doubt we will have some a long debate followed by some modest reforms, probably with Francis Maude forecasting Nirvana and some former Permanent Secretaries forecasting Armageddon.

There are times, of course, when tinkering is the only way of making progress. The differences of view on how the House of Lords should be reformed, for example, should not be allowed to prevent sensible interim reforms such as the Lord Steel proposals. But it would be nice to see a slightly more ambitious attempt to look at the issues with a little more seriousness. We are not perhaps fiddling while Rome burns as London isn’t burning in quite the same way. But surely we could do better.

It is occasionally suggested by Whitehall veterans that Ed Balls began as a spad and ended as a civil servant. We have no such evidence that this happened. The confusion seems to lie in the fact that the previous person with the title ‘Chief Economic Adviser’ was a civil servant—Sir Alan Budd), as is the current one (David Ramsden).

But in the period that Balls was ‘Chief Economic Adviser’ he was also a special adviser. He was clearly stated as such in parliamentary questions between 1997 and 2001.[1] The records are not clear for 2001-3 (Balls is not named in the records we have seen),[2] but the Chief Economic Adviser in this period is specified as a special adviser in Hansard. We’re presuming that person is still Ed Balls. So the story seems to be that Gordon Brown as Chancellor decided to appoint Balls to a role which was conventionally held by a civil servant or formal employment terms. But Balls remained a spad.[3] That is our understanding. But we would welcome—indeed, encourage—corrections.[4]

This is nerdystuff, but it’s important. This is what research is all about: grappling with imperfect information. It’s assumed that everything in government is always perfectly recorded, but it is not. For instance, there is an expectation of regular, annual data releases on numbers and names of special advisers. And for the first two years of the Coalition numbers and names were released at regular intervals—roughly, every four months. There has not been a new release since October 2012—seven months.[5]

Does this matter? It matters to researchers like us. Imperfect or faulty information means we may make faulty inferences. So for instance the lack of up to date data on special advisers means it is difficult to determine tenure of spads with consistency. Data releases on spad numbers never talk about spads leaving, only entering government—so we have to infer from their absence that they have left.[6] Tenure is important because it would help us understand the nature of the work that spads do: for example, if tenure is short, it may suggest short-termism; if tenure is changing, it may suggest the nature of the job is changing.

But there is a deeper point here: government transparency.[7] Some people have already taken office in the period since October 2012 as special advisers, but they are not listed anywhere. Would it not be advisable to list their names and details so that people with relevant interests and concerns might be in contact with these advisers? More generally, if spad numbers have dropped or risen, that might also be useful so that the public to know, so that they can appraise the current government’s use of special advisers, and their numbers.

[2] See, eg., HC Deb 16 July 2003, cc328-9W. In fact there is generally very little Hansard coverage of special advisers in that period.

[3] Brown had a way of doing such things: he had a ‘Council of Economic Advisers’ which was for all intents and purposes simply another group of special advisers—if one looks at the annual data releases on special advisers, those ‘sitting’ on the Council of Economic advisers were included, oddly, as a footnote, as if to suggest they were not special advisers. Needless to say the Cabinet Office continues to follow this obfuscatory practice.

[5] To be exact: there was a (poorly edited) release on 19 October 2012—see Matt Honeyman’s spellbinding post on this. But there has since been a revised version of those special advisers employed as of October 19 2012 published 17 February 2013. But all that did was revise the earlier release: it does not tell us who was a spad as of 17 February 2013.

[6] Of course we can work this out in other ways, but they are less authoritative than government statements.

[7] Liz Fisher from Oxford University is similarly critical of the transparency agenda: see

The press has reported today that No 10 has rejected the potential appointment of David Kennedy, currently CEO of the Climate Change committee as Permanent Secretary at DECC. Yet again this is seen as a sign of a coalition split and of meddling by No 10 in civil service appointments.

There is no reason to doubt the underlying accuracy of the story, though we can rest assured that nothing will be disclosed on the record. But does that mean either the coalition is yet again in crisis or that No 10 is throwing its weight around? Possibly but probably not.

From the outside, a strong case could be made for two types of appointment to DECC at present.

First there is the case for someone with business credibility in the energy industry given the need, as Ed Davey said in Parliament yesterday, for £110 billion private sector investment in our energy infrastructure by 2020. This is a huge challenge – and the Department will need to be strongly business facing in the next few years. Moreover, Ministers always start by trying to attract suitable individuals with a business background, though are seldom successful.

Second, usually as fallback, there is a case for a Whitehall insider, someone who knows how to manage a Whitehall Department at a time of significant downsizing and structural reform. And who will know how to keep the show on the road.

But David Kennedy did not really tick either box. Kennedy is hugely respected for his knowledge of climate change and if the Department had been recruiting a head of policy development, then he would have been an outstanding candidate. But his background is neither a business one, nor a Whitehall insider. And the role of Permanent Secretary is no longer, if it ever was, about being the principal policy adviser to Ministers. It is primarily a managerial role, making sure the Department delivers what Ministers want.

All this is, of course, speculation on my part. But I do recall one appointment in my civil service career where my Department was on the verge of appointing a totally “outside the box” candidate to a senior role – but we were saved from doing so by a wise soul in No 10 telling us to think again. Six months later we were all grateful to No 10 for their response. This case is not remotely like that one and Kennedy might well have been successful in the role. But a No 10 veto can always be exercised wisely as well as wilfully.

The political commentariat love nothing more than predicting the end of the coalition, driven by splits and crises. And we have seen a rash of such stories this week over both energy policy and Leveson.

But the truth is a bit more complicated – and more interesting – than that.

Energy first. Until the middle of this year, DECC had been a beacon of coalition harmony in that both a Lib Dem Secretary of State and a full team of Tory Ministers were happily pursuing a common policy with no obvious internal rows. There were admittedly some well reported disagreements with both George Osborne and Vince Cable – but no one in Whitehall would regard those as anything other than conventional Departmental tensions not coalition ones.

But then Cameron sacked the mild mannered and coalition friendly Charles Hendry and replaced him with the much more populist and known climate change sceptic John Hayes. The only way to interpret this change was that the Tories were aware of growing unrest on their own backbenches – mainly on windfarms but also on energy costs – and decided to throw them a little red meat.

And Hayes immediately decided that his new role entitled him not only to be a flagship for those restive backbenchers but to use his new role to attack his own Department’s policy in public. He hasn’t exactly been successful in that in that Ed Davey has first rebutted him and then proceeded to issue an Energy Bill which is far closer to existing policy than anything Hayes would support. DECC hasn’t got everything it wanted in the Bill – but what is missing can be put down to the traditional funding concerns of the Treasury.

So is this a coalition split? No, for the very good reason is that it is actually a clear case of our old friend, the Tory split. Certainly the other leading Tory in DECC, Greg Barker, seems to have not the slightest problem with the policy he had been pursuing first with Huhne and now Davey.

Similar considerations apply to Leveson – though it is early to predict exactly how that will pan out when people have actually read it. The interesting thing here is that over 50 Tory backbenchers have already come out in favour of statutory back-up to the press regulatory system. So again, it seems that the Tories are split rather more than the coalition is split.

But there are interesting lessons to be learnt from both energy and Leveson. We at the Constitution Unit have long believed that there will be no coalition split because it is highly unlikely to benefit either partner from ending it before 2015. But what we are now seeing is the impact of the coalition on the way that politics is conducted in the UK. That means a junior minister can attack his own Departmental policy and not be sacked. Government backbenchers don’t wait until their Prime Minister has announced his response to a report before announcing how they will vote. And the Prime Minister and the Deputy Prime Minister can argue their different cases at the dispatch box on the same day.

To my mind that is nothing but healthy. As a former Whitehall civil servant, I know there are numerous issues on which there is genuine disagreements behind closed doors. If coalition means we see a few more of those disagreements out in the open, then three cheers!

Post navigation

The Constitution Unit in the Department of Political Science at University College London is the UK's leading research body on constitutional change.

This blog features regular posts from academics and practitioners covering a wide range of constitutional issues in the UK and overseas. You can navigate by theme and contributor using the menus at the top of this page, and subscribe to receive new posts to your inbox below.

Follow blog via e-mail

Enter your e-mail address to follow this blog and receive notifications of new posts by e-mail.