A couple of minutes before 10pm on Thursday 8th June, Nick Timothy and Fiona Hill went into a side room at CCHQ. Out in the main, open-plan, campaign office, assembled politicians, strategists and staff were glued to the BBC, awaiting the exit poll that would give at least a general idea of how the General Election had gone. But Timothy and Hill left the hubbub to receive advance sight of the result, a privilege afforded to them as the joint Chiefs of Staff to the Prime Minister.

Expectations were quite high, even after a tough few weeks on the campaign trail. The early polls and local election results had set many Tories dreaming of a huge, even triple-figure majority. But by election day itself, few seriously thought that was still within reach. There was, however, general agreement that the Conservatives would still secure an increased majority. From Labour HQ to CCHQ, from Downing Street to the offices of its hired campaign gurus, no-one was predicting a net loss of seats.

And yet that was the very news delivered to the Prime Minister’s two closest advisers – the exit poll was predicting a hung Parliament. It hit them like a punch in the gut – spelling disaster for their boss, whom they had helped to propel into Downing Street, for their plans and ambitions, carefully cultivated over years, and for their Party. Perhaps it even imperilled Brexit, a duty which they knew the Conservatives must fulfil, or risk a fate that didn’t bear imagining.

When the news broke publicly a few minutes later, the impact was no less severe out on the campaign floor, and for Conservative candidates and activists around the country. Hopes were crushed, hard work appeared to have gone to waste, dreams of finally ascending to the green benches were ruined and the much-heralded opportunity to defeat socialism for a generation had been fumbled. It will be a long time before any Tories who were watching forget the sinking feeling as the graphic flashed up on the screen, as David Dimbleby repeated that “they don’t have an overall majority”.

Once the initial shock passed, the question that sprang swiftly to many lips was simple: how did it all go so wrong?

Despite months of coverage and debate – and with ongoing briefing wars between ‘friends’ of various people involved – that question is still only partially answered. Questions of personality and policy have already been explored in great detail – understandably, given their central role in the outcome.

But so far there has only been limited discussion about the functioning – or otherwise – of the Conservative campaign machine, a factor important enough to make the difference between having a majority and not, even after other factors had taken their toll. Over this new series, ConservativeHome sets out to study the successes and failures of the campaign machine in detail.

This piece is the first of three long-reads which will explore what worked, what didn’t and how the problems might be fixed, drawing on interviews and statements from insiders at every level of the Conservative election effort.

Today we look at the national campaign organisation: how it was structured at the top, who was making the key decisions, and how these factors contributed to the outcome.

Inevitably, the dominant factor in how the Conservative campaign was constructed was the need for speed. Though some doubted the claims at the time that the Prime Minister had only decided to call the snap election over her Easter holiday, immediately preceding the announcement, all the senior figures I have spoken to, and all the evidence I have seen, confirm that this is indeed what happened. People who had been involved with the 2015 campaign were recalled to CCHQ less than 24 hours before she announced the election on the steps of Downing Street.

That flurry of calls at the end of the Easter Bank Holiday weekend was crucial. In 2015, the Conservative campaign team had secured a majority against all expectations. As I recounted shortly afterwards, the 2015 campaign hadn’t all gone to plan, and the end result rested on just 901 voters across the most marginal seats, but it had worked nonetheless. There was no doubt in the leadership’s mind that they needed, in the words of one senior participant, “to put the band back together, to repeat the trick”.

But after the 2015 election, many of the key specialists who made up “the band” had scattered to the four winds. Lynton Crosby and Mark Textor, who did the messaging, day-to-day management and polling, were still on retainer to CCHQ but were in Australia. Jim Messina, who crunched the data, and his team had returned to the United States. Craig Elder and Tom Edmonds, who had cut their teeth in digital campaigning as part of the in-house CCHQ team in the run-up to 2010, had gone back to running their consultancy – dipping into politics to work for Stronger In in the referendum, but not staying on as part of the Conservative Party operation.

A rusty machine and a hollowed-out operation

They all answered the Prime Minister’s call, dropping a variety of other jobs to return to CCHQ as consultants once more. But immediately there were signs that “repeating the trick” wouldn’t be as easy as simply getting everyone in the same room again.

For a start, to quote one of Crosby’s favourite sayings, “you can’t fatten a pig on market day”. While the 2015 election campaign undoubtedly took Labour by surprise, it had been a long time in the making. The top team planning and preparing for it had been in place for almost two years before polling day – gathering data, developing and testing messages, training up staff and establishing what they intended to do and how they intended to do it. By contrast, their swift reunion in 2017 provided no time to prepare. They were pitched straight into running a campaign that became live only a few hours after they had agreed to take part in it (I’m told that in some cases, contracts were still being discussed after Theresa May’s announcement in Downing Street).

Nor was the machine they returned to in the same state as they had left it in 2015. Senior, mid-ranking and junior staff with election experience had variously been signed up as Special Advisers, moved to the private sector, or been let go as part of the regular cost-cutting that tends to take place after costly election campaigns. “They had let things wither on the vine, starving [CCHQ of] both campaigning platforms and people,” one senior participant says, “The groundwork three months out [from the election] was not ready, but that question wasn’t asked.”

“Crosby was basically chief executive of CCHQ for 18 months to two years [before 2015]” another insider recalls, with a senior team reporting directly into him, including teams responsible for Voter ID, Digital, Field Campaigns, the Conservative Research Department (CRD), Press, and the oversight of 120 local campaign managers, but “by the start of 2017, lots of those roles had gone.” Darren Mott, the former Field Director, had stayed on as Director of Campaigning, and the staff responsible for each of these roles now reported to him, but he was not given the power which Crosby had previously wielded. “The CRD became degraded, the Press Office became degraded,” and around 100 of the 120 campaign managers had been let go. They lost their jobs just after they had gained experience of their first General Election, and after a mammoth effort to train them up and build relationships with their local parties – a “bloody catastrophe”, in the words of one senior Party official.

Institutional knowledge and expertise had been lost, and other assets had been mothballed or lost entirely. The effect was to leave them fighting a General Election with an operation which had lost its edge. Some veterans of the EU referendum battle who saw CCHQ in action were troubled by comparisons with the energetic Vote Leave campaign – “it felt sleepy…We had more money, more resources but less energy.”

Faded data

The shrinking of CCHQ had involved closing down some functions entirely, such as the ShareTheFacts website which had provided accessible messages and rebuttal for activists to share online.

Furthermore, data degrades over time – people move house, change email address, die, marry, change their minds, gain new concerns or interests and so on.

For example, after the 2010 election, the Conservative Party held about 500,000 email addresses, which had shrunk to about 300,000 which were still usable by the time planning began again in 2013. By the end of the 2015 General Election, that list had grown to around 1.4 million through proactive gathering of addresses through online campaigns – but, by April 2017, almost two years of leakage had again drastically reduced the list.

The same went for data gathered through canvassing. CCHQ’s Voter ID and Get Out The Vote operation in 2015 had worked well, particularly in its ruthless targeting of voters in what were then Lib Dem seats. But that data was now out of date, and even the proportion of it which was still usable was now far less relevant. The 2017 battlegrounds largely weren’t in those former Lib Dem seats, and the potential swing vote mostly wasn’t Lib-Con. Data on the disillusioned former Labour and UKIP voters in Midlands and Northern seats was in short supply. The Conservative Party had played no formal role in the EU referendum, and so had no Leave/Remain canvassing data of its own. And the intervening local and mayoral elections had not yielded enough data to overcome either problem. The advent of Brexit had certainly fractured old loyalties, but the Conservative Party only had a limited idea of where this had happened, and who it had happened to.

Consider these problems, and we can start to see why it simply wasn’t possible for the old team to walk through the door and immediately return to their former effectiveness. “Drive the car as we build it,” became an unofficial mantra – what had taken them two years to construct in peacetime, now had to be cobbled back together over days or weeks under the full-on, all-consuming pressure of a General Election campaign.

The message and the chain of command

Having been brought back together, and increasingly aware of the challenges that faced them, the team had to get going immediately. Even though the Fixed Term Parliaments Act technically meant waiting for the approval of Parliament, there was little doubt that that approval would be granted. The campaign was already underway.

After informing the full Cabinet of her decision at 9am on that Tuesday morning, May had fired the starting gun on the campaign shortly after 11, and her chosen message was duly delivered to the nation. She cited the threats by Labour, the Liberal Democrats, the SNP, and assorted Lords to disrupt and jeopardise the Brexit process – setting out her stall as offering “strong and stable” leadership to get the country through the process of leaving the European Union. She had chosen to fight on Brexit.

Even on the day, there was concern among some of those close to the Prime Minister about this message. “It was about a month late,” says one, citing the fact that the Opposition had signally failed to actually defeat the Government on any Brexit votes, “The framing was just wrong. It was a Brexit election without anything to debate.”

Whatever the unease, the words had been said out loud on national television. Most of the group who would sit at the top table throughout the campaign were not on board at the time that messaging decision had been taken – somewhere between Timothy, Hill, Crosby, and the Prime Minister herself, the play had been chosen and the rest of them would have to run with it.

Once the election had begun, the Prime Minister’s tight circle expanded somewhat. At the daily meetings, which effectively ran the campaign, the usual attendees were Lord (Stephen) Gilbert, Crosby, Textor, Alan Mabbutt, Timothy and Hill. Initially, meetings were chaired by Gilbert, with Crosby dialling in, but once the Australian arrived in the UK he took over the role of chairman.

That was a notably less clear chain of command compared to the previous election. Having been bruised by falling short in 2010, David Cameron and Lord Feldman had granted Crosby effectively executive control of the day-to-day running of the 2015 campaign. This time, while he was chairing proceedings and undoubtedly of great power in the room, he was not the undisputed source of authority on what they should do. The Prime Minister had, for the past year, vocally set out a different form of Conservatism to that pursued by either Cameron or Crosby, and the general view was that her power was vested in her Chiefs of Staff. If there was a disagreement, who had the final say?

Since election day, the lack of an answer to that question has allowed just about every side to suggest that one or more of the others was responsible for errors that were made. The impact of that lack of clear command structure could be seen during the campaign itself, not least in mixed and conflicting messages. How did “strong and stable” fit with the radicalism that May had previously espoused? The answer, as discovered with the so-called ‘Dementia Tax’, was that it didn’t – “the change narrative and ‘strong and stable’ collided in the manifesto”, notes one Tory insider.

Around the table sat people who wanted to present different and conflicting versions of the Prime Minister’s agenda. Because none of them had a truly final say, they each got their own way to some extent, with disastrous results. One route or the other might work – a mixture of both (choosing to sit out the leaders’ debates, but also choosing to send the Home Secretary as a stand-in, for example) produced the worst of both worlds.

Finding the target

So, with old data, a campaign headquarters which had been hollowed out, no time to prepare, an unclear chain of command and a starting message about which some of them had reservations, the old band had to find the fashionable way to present its best tunes.

Given the limited data, and limited relevance of the data that they did have, they had to rely on swift polling and mass purchase of consumer data in order to try to apply the poll findings to the populace at large and gain some idea of which voters to target.

This was a reasonable approach in the circumstances, and offered the best chance of plastering over the cracks. Something similar had worked for Vote Leave – as I reported at the time, having started with no data at all, Vote Leave was able to bridge the gap in a few short months through savvy use of canvassing data to extrapolate the identification of likely target voters from more general polling and consumer data.

But those close to the leadership knew it was risky, particularly given the degree to which politics was in flux after the EU referendum and with the rise of Corbyn. “With a volatile electorate, you need a good run-up to canvass and and get lots of data,” one told me, “but with the snap election we didn’t have time.” Vote Leave had been fighting a campaign on a binary issue, without the added complexity of multi-party politics, too. In the event, while those involved in the Leave campaign estimated their software identified targets with a rough accuracy of 75 per cent, the CCHQ team trying to do the same found the task far harder and their hit rate lower.

What they came up with – principally via work by Messina and Textor – was a calculation that focused on winning over two groups of voters: former UKIP supporters, and direct switchers from Labour, particularly those who had voted Leave and were often part of Labour’s traditional working class vote, many of whom were to be found in seats in the Midlands and the North. Their plan, dubbed “Targeted Voter Turnout” (TVT), sat at the centre of everything that was to follow with the ground campaign (of which, more tomorrow). It dictated who the potential switchers were, it extrapolated from the consumer data to determine where they probably lived, down to individuals within households, and by looking at how many such people lived in each constituency, it drove the choice of which seats to target. While ordinarily, other specialised aspects of the campaign – such as digital – would develop their own models, the lack of time meant the whole operation rested on this analysis.

At the same time, messages were being developed and tested to sharpen the campaign’s appeal to those groups. One fateful development at this early stage was the decision to shift from a more traditional, team-based model of campaigning, to a highly personalised obsession with May herself as the figurehead – an approach we now know was recommended by Crosby and Textor in April, shortly before the campaign. We all remember the banners proclaiming “Theresa May’s team”, and anyone involved in the canvassing operation will recall the scripts that spoke endlessly of “Theresa May’s candidate”. The die had to be cast quickly, on the basis of less than ideal information – so it was.

The manifesto, and the tragedy in Manchester

Initially, indications were that these decisions were on the money. The opinion polls were strong, and the Conservative campaign passed its first test – the local elections – on 4th May, just over two weeks after the General Election was first announced. Beating Labour by 11 points, it seemed that May had made the right decision and her campaign team’s approach was working. Most encouragingly for those in CCHQ, in many (but not all) places, the former UKIP vote appeared to be breaking heavily in the Conservatives’ favour – something publicly acknowledged at the time by Labour and Liberal Democrat strategists, too. This was a key assumption of the TVT number-crunchers, and the sight of it actually happening at the ballot box bolstered their faith in their assumptions.

But the General Election was still another five weeks away – time in which anything could happen. When the Prime Minister called the snap election, many in her Party were perplexed at why she had chosen 8th June as the date – what was so “snap” about a campaign of almost two months? The view inside Downing Street was that “five weeks was the probable minimum”, according to someone privy to the discussions, but two factors intervened. First, as John Rentoul has reported, they miscalculated by expecting Labour to be obstructive, and therefore built in time to legislate to overcome the Fixed Term Parliaments Act. Second, I’m told that Sue Gray, the Cabinet Office Director General of Propriety and Ethics, who advises on such things, “insisted” that under the FTPA an election really ought to last seven weeks at minimum. In the event, both views combined to produce an overlong campaign, maximising the time during which things could go wrong.

They came to regret it. “Even if it had been the 1st of June, and not the 8th, I think we’d have a majority,” one adviser lamented. Because, of course, things did go wrong.

Most famously, when the manifesto launched, it contained a controversial policy for which no preparatory work had been done among voters, and which raised more questions than it answered. Even at high levels of the Conservative Party and Government, it came like a bolt from the blue. The manifesto was developed in the Conservative Policy Unit office on the fourth floor of CCHQ in Matthew Parker Street, overseen by Nick Timothy, Ben Gummer (the then-MP and Cabinet Office minister), and John Godfrey (then the Prime Minister’s Head of Policy). Even for an activity which is necessarily and traditionally secretive, the circle was extremely small. Ministers were consulted on various policies relevant to their brief, but were mostly unaware of the wider contents. George Freeman, who chaired the Prime Minister’s Backbench Policy Board, did not see it before publication.

The 1922 Committee was not consulted, either. It had announced the timetable for restaffing its policy committees the day before the election was called, but did have policy chairs in place for most departments, who I’m told were not brought into the process.

The final draft of the manifesto was finished just over 24 hours before its launch, and was completed by Gummer and Timothy. So when it was revealed, it contained a policy about which MPs, had they been consulted, would undoubtedly have expressed grave doubts, communicated in a way that invited questions about total costs. Almost none of those likely to be asked about it by reporters or voters had any forewarning of the idea’s existence, still less the reasoning that lay behind it or the practicalities of its implementation. Candidates had received no briefing information, either – “[I] got roasted at a hustings the day the manifesto was released”, one recalled.

It swiftly became clear that they had a problem – the press took a hearty interest in it, not least because they had been starved of news by a repetitious campaign, and Labour seized on the term “dementia tax”. But with a flood of feedback from people who hadn’t previously been consulted, immediate concern from canvassers noticing the impact on the doorstep, and a division of authority at the top table, it took most of the weekend to develop an answer. Even then, the Prime Minister’s announcement on the following Monday that there would be an “absolute limit” on total costs for any individual lacked a specific number, thus failing to end the row.

That night, their self-inflicted problem was compounded by a tragedy beyond their control when a suicide bomber attacked concert-goers in Manchester, slaughtering innocent civilians including children. There was no doubt that the campaign had to be suspended in order to focus on matters of mourning and national security. The political impact, understandably not uppermost in anyone’s minds at the time, was inadvertent but inevitable: the debate about the manifesto was frozen where it stood, the attempt to clarify was effectively cancelled, and voters were left with their own doubts and the Opposition’s message in their minds as the defining and permanent view of the manifesto.

The target shifts, and is lost

That was unavoidable – and they were right not to try to avoid it, given the appalling circumstances. But when campaign recommenced, the vulnerabilities of the rickety machine – the car they had had to build while driving it – began to take their toll.

After racing early on to produce some kind of sticking plaster for their shortage of data and on-the-ground insight into voters’ minds, those findings from the start of the campaign were no longer accurate. The manifesto, the criticisms of the Prime Minister’s perceived lack of openness to the electorate, and the impact of a major terrorist attack had all contributed to some proportion of voters changing their minds about the election. At the same time, Labour’s own messaging and targeting was reaching more people, more effectively, than even the Opposition’s own planners had expected. Quite who was changing – UKIP voters, Labour switchers, the Tory core vote? – and what they were changing their view about – May personally, their policy priorities, which party they trusted with their finances? – was unclear.

Put simply, those carefully segmented TVT cohorts, and the targeted messages designed for each of them, were now off the mark. Some of the messages no longer worked with their target audience, some were actively counter-productive, and some previously non-target voters had become swing-voters but were not being addressed.

At this point, we need to consider how a campaign in this situation might learn that it has such a problem. It could get it from the polls – but with a time lag, with lots of noise, and with the caveat that the Conservative experience in 2015 and 2016 taught them not to trust pollsters. It could hear it from its own campaign data – but if there are movements taking place among people whom you aren’t canvassing, then you won’t hear it. Or it could hear it from its candidates and activists – but CCHQ, as we will see in tomorrow’s article, has a somewhat sceptical view of its own colleagues on the ground.

As it was, the Tory campaign suffered from a combination of these problems. After the positive early results on the doorstep, in polling and at the local elections, many former UKIP voters were now assumed to be coming across – so “they were taken out of the targeting. They slipped away after the manifesto, but we missed it due to the narrow focus.” Similarly, with a campaign which increasingly sought direct switchers from Labour, the Conservative core vote was not a target group – they were assumed to be in the bag given the threat of Corbyn, but many of them, as ageing homeowners, were precisely the people most worried by the social care policy. Again, little data was flowing in from doorstep conversations with them, so their shift to not voting, or even voting for another party, was missed.

CCHQ’s own polling revealed some issues, but only after a time lag. Fatefully, I’m told that a refresh of the modelling was planned for the weekend after the manifesto launch. But, amid the furore over the social care policy, the half-completed attempt to clarify the policy and then the total shut-down necessitated by the Manchester attack, they found it impossible to gather reliable data successfully. “[The refresh] was contaminated…and we were never able to get a clean set of data,” according to one of those inside CCHQ during that series of crises.

All this meant that those running the campaign were slow to adjust and correct their messages and their targeting. One senior Conservative found that “change did take place, but in weeks, not in days”. Had the data been better-founded, the response might have been swifter. Had the campaign been shorter, polling day might have come before quite so much damage was done. Instead, the response was slow and the long campaign left plenty time for votes to be lost.

The red tide online

Much has been made in the last few months of the impact of the online campaign on the election result. The Conservatives appeared in 2015 to have won the online war through canny targeting, and heavy use of Facebook, sparking a somewhat paranoid series of articles in the left-wing press this time round implying there were Tory dark arts at play. This time, though, the Labour Party seemed to have got the better of the digital war.

Those involved on the Conservative side certainly agree that Labour had upped their game drastically on the previous election. The resources the Opposition put into digital campaigning were far greater than before, and they had learned from and built upon the tactics of their opponents last time round – though it should be noted that simply judging their effectiveness on numbers of views conceals the fact that they were still in some cases delivering messages that were far from perfect.

The Tory digital operation was well-resourced – CCHQ’s financiers were able to provide the team with just about everything they asked for. And, as those allegations of dark arts suggest, Edmonds and Elder were again pursuing a strategy that made innovative use of social media’s potential for targeting adverts to very precise cohorts.

However, the effectiveness of that operation was blunted by two factors. The first was technical – as explained above, the digital gurus were brought in, like everyone else, at very short notice, there wasn’t time to do extensive new modelling and testing, and so they ended up reliant on the central targeting model that was rendered increasingly obsolete as the campaign progressed. That modelling also informed decisions about which platforms to focus on – if the national strategy assumed young non-voters weren’t a priority, then the digital campaign followed suit. The second factor, which is more concerning, was that while advertising can do a lot, Labour benefited from a “red tide” of support from third party groups who provided massive extra organic and paid-for reach for Corbyn’s messages online.

How many disparate groups can you think of who were allied to or sympathetic to Labour in the course of the election? Count unions, charities, pressure groups, campaigning social media communities, and crowdfunded advertising campaigns, and you’ll realise there are a lot. Two separate senior Tory campaigners who have been keeping tally estimated to me that the number was in triple figures – not including the many other local groups and even individuals active in specific seats. Conservative policy decisions certainly exacerbated that problem – in particular, the pledge of a free vote on fox hunting and the failure to clearly confirm support for a ban on ivory, neither of which were central issues to the national campaign, nonetheless acted as recruiting sergeants for Labour online.

In the online war, that gave Labour two things: reach and trust. Between them, these third-party allies were able and willing to repeat Labour-supporting messages to many millions more people than were subscribed to or targeted by the official Labour Party outlets. Compounding the impact was the fact that people are more likely to trust a message from a group which is (at least nominally) outside the orbit of a party HQ, and which they’re engaged with in their regular life outside election time.

That was a huge boost to the Opposition, and created a sense of (small-m) momentum which a primarily paid-for Tory campaign struggled to match. “We didn’t see the tide coming,” admitted one source close to the Prime Minister, and the failure of the Conservative Party to nurture a wider movement beyond its own membership proved to be costly. Of course, that “red tide”, and the lack of a blue equivalent, didn’t just exist online – it was there in the real world, too, as I’ll explore tomorrow.

A broken machine

While personality and policy no doubt lay at the heart of the problems in the election campaign, the serious flaws in the national Conservative Party campaign machine cannot be ignored. All the evidence I have seen and heard in the last three months leaves me in no doubt that those flaws contributed to and exacerbated other problems, delayed detection of the harm they were doing, and then made it harder to correct them once they became evident.

A hollowed-out CCHQ, reliant on hiring in outside leadership as well as expertise, simply wasn’t prepared for the snap General Election that the Prime Minister chose to call. Once it was hastily restaffed, it was still forced to rely on insufficient data and was controlled by an unclear chain of command. Under the pressure of a troubled campaign and outside events beyond its control, it was slow to adapt and had few if any external allies lined up to fight its corner.

Whether they over-estimated the ease with which the 2015 General Election strategy had been developed and implemented, or knew it needed a run-up but wrongly estimated that Corbyn could easily be beaten without having everything in place, the failure to ensure the machine was ready to do the task it was set proved disastrous.

This is not a question of recrimination, but rather of blunt scrutiny to ensure that it does not happen again – something which must be done now. One senior official, close to the action throughout, put it to me starkly: “We should ask a question of each CCHQ team – are you ready to fight another election in two years? If the answer is yes, then it’s a lie.”

> The second article in this series, a study of the performance of the campaign machine in the ground war across the country, can be found here.