Sir Peter Housden served as Scotland's top official for five years, stepping down from the civil service last year after more than a decade at the helm of big government organisations. Here, he tells CSW's Matt Foster why he thinks its time ministers dropped their "sword-in-hand" approach to public sector reform – and what Brexit is likely to mean for Scotland

Michael Gove battled “The Blob”. David Cameron went toe-to-toe with the “enemies of enterprise”. And Tony Blair could show you the scars on his back. All three heavyweight politicians went to great lengths to show voters that they were restless reformers of Britain’s public services – and were unafraid to go on the record to criticise the workforce when parts of their reform plans ran into trouble.

But Sir Peter Housden, until last year the most senior civil servant in the Scottish government, thinks the new crop of politicians getting used to their ministerial red boxes might benefit from a more constructive approach.

“I think that sort of gladiatorial, sword-in-hand leadership model, as a default in public services, is a real problem, because it demotivates hundreds and thousands of men and women who work in public service,” Housden tells CSW, as we sit down to discuss the thinking behind his new report, Rethinking Public Services, over a coffee in Westminster.

“People tire of it. They don't engage with it. It sounds, frankly, self-serving when it's put in that way. The most effective ministers that I've worked with have actually been outstandingly good at engaging with practitioners and reflecting back to them the importance and value of what they do and learning from that, incorporating it into their practice.”

Housden has spent his time since stepping down as permanent secretary of the Scottish government last year – after more than a decade in top tier civil service jobs – doing some serious thinking about what actually works in government.

"The most effective ministers that I've worked with have actually been outstandingly good at engaging with practitioners" – Sir Peter Housden

And he uses his new report – published this month by the Centre for Public Impact – to make it clear that he believes New Public Management, the private sector-inspired model of public service reform in vogue with successive generations of ministers and managers since Margaret Thatcher’s time in office, has had its day.

The report argues that while New Public Management and its variants have brought "important successes and the creation of lasting assets" – including a stronger sense of accountability and a "new architecture and business model based on choice and contestability" – it is beset by "signal weaknesses and structural flaws".

“There have been huge steps forward in my working lifetime,” Housden says.

“We’ve gone from a public services environment in the late 1970s where, for example, inequality was regarded as inevitable, failing schools were akin to the weather – they just happened – and there was very little confidence and drive towards improvement. I think the strategies for reform that have evolved over that period have been hugely successful. But like all things, they lose their impact over time.”

Instead, Housden is a big fan of the “co-production” approach to service design that’s been championed by the Scottish government.

This, he explains, is not just a buzzword. It’s an attempt to involve citizens, voluntary organisations, private providers, and frontline practitioners in the design, commissioning and delivery of services. The aim is to avoid top-down reorganisations – and build systems around need, rather than centralised target-setting that can all too quickly degenerate into box-ticking.

It means, he argues, no more “passing down stone tablets of wisdom to the willing, passive recipient”.

“I think this is very strongly borne out in something like medicine, where there's a huge amount of science and evidence and research that goes into clinical practice – but doctors would be the first people to tell you that you treat patients, not symptoms,” he says. “There is no simple relationship between knowledge and what you should do for this individual – the outcomes you get from any given treatment are so varied, you know. And it's a question of exploration – that's the notion of co-production.”

But this more flexible model of service design cannot, according to Housden, get off the ground without a significant shift in the tone used by politicians who’ve grown used to defining themselves against the public sector.

"I think part of the problem with the way we have thought about public services is that practitioners – be they doctors, nurses, teachers, social workers, whoever – are often regarded as the problem. You know, they are tied up in bureaucracy, they are not sufficiently innovative,” he says.

“All these charges are laid at their door. But, in co-production, you actually start to value these two sides of the equation equally and importantly. And you think about what you can do to nurture the quality of that moment of co-production.”

“Obstinacy and vision”

Housden, a former teacher and council chief executive who brought an outsider’s eye to Whitehall when he joined the Department for Communities and Local Government in the early 2000s, is keen to stress that his report is not simply a call for politicians to stop pressing for reform.

“So many of the real advances in public services over this period only happened because of political will, obstinacy, vision and determination to see things through,” he says. “But it's a question of the mix of messages and approaches that you adopt as a politician.”

The former Scottish perm sec is clear that doing more to draw on the experience of frontline staff – making them feel they have a stake in the policies they’re being asked to deliver – will be more important than ever at a time when conditions continue to be “very tough for rank-and-file workers”.

“They've had a pay freeze, promotion opportunities have been restricted – the environment's been very challenging, with lots of churn,” he says.

“And I think if you want to keep those organisations working effectively, the kind of places where people want to come to work, where people want to stay, so you can recruit, train and attract good people, then actually the modern workforce demands engagement.”

That’s not just a message for politicians, either. Housden is clear that public sector leaders have a duty to get the basics right for their staff – ensuring HR processes are up to scratch, providing “structured opportunities” for personal development and, crucially, building a cadre of switched-on line managers willing to challenge and support their direct reports.

"The modern workforce demands engagement" – Sir Peter Housden

It's an approach Housden says he tried to foster in the Scottish government, which opted not to implement the UK civil service’s performance management system for its own staff.

That system’s call for “guided distribution” – requiring civil service managers to regularly assign a set proportion of their staff as poor performers – has been sharply criticised by trade unions, who see it as heavy-handed, and deeply damaging to morale.

Housden explains that while civil servants working for the Scottish government have a “recognisable civil service performance management system with objectives, a mid-year review and all the rest”, he was more interested in focusing on “the texture of relationships”, and setting objectives for senior staff “on a much more frequent, flexible basis”.

“Instead of our conversations beginning with, 'why haven't you done this?', 'where's that consultation document?', or 'why is minister X so frustrated about Y?', we might often begin by saying 'talk to me about your people.' We'd say: 'What's happening with Derek?' or 'We were talking about Angela the other week. How's that gone?'

“We said to ourselves that we were really going to prioritise this. We were going to help those outstanding practitioners develop their ability to work with others – and we're not going to have an option whereby you can opt out of being an effective manager of people.”

"There is a bit of a game that goes on here"

It’s an approach to staff development that Housden says has been taken on “with force and vigour” by his “fantastic” successor as perm sec Leslie Evans, who now leads the 17,000-strong team of officials owing their loyalty to the Scottish government, although remaining part of a unified, UK-wide civil service.

Evans certainly has a lot on her plate in the wake of Britain’s vote to leave the European Union, a decision which has massive implications for Scotland given the nation’s overwhelming backing – 62% – for remaining in the bloc.

CSW speaks to Housden just after his old boss – first minister Nicola Sturgeon (pictured) – uses a key speech to set out a formidable list of the benefits of EU membership that she expects Scotland to retain after Brexit, including continued free movement of labour and access to, as well as influence over, the single market.

When asked about the vote, Housden appears keen not to step on his successor’s toes, although he says he was not surprised to see Sturgeon move so quickly to put a second Scottish independence referendum on the table after the result.

“I think the position of the governing party in Scotland on this has been very clear, that continuing membership of the European Union would be a core pillar of the case for independence,” he says.

“Nicola Sturgeon, after the first independence referendum, had always talked in terms of a change in material circumstances as being one of the factors that would weigh in her calculations about a second referendum. And of course, a decision to leave the European Union in a referendum is certainly a material factor. So no, I don't think anyone was surprised.”

“That's what political arguments are like. You throw everything at the enemy and if the civil service is one of those bricks you can throw, then they'll do it" – Sir Peter Housden

The run-up to the European Union vote was also marked by frequent accusations on the part of the Brexit campaign that top civil servants, bound to support the policy of the elected government, were biased in favour of the Remain side. It’s familiar territory for Housden, whose own time in office saw him take flak from pro-Union newspapers and politicians accusing him of having “gone native” in favour of independence.

That kind of pressure, he says, “goes with the territory”.

“The referendum campaign replicated exactly the situation that we faced in Scotland. The civil service in the UK and in Scotland works for the government of the day. And the government of the day here had taken a position on continued membership of the European Union, just as the government of the day in Scotland had taken a policy decision on independence.

"The civil service has a statutory duty to propagate and support those processes of government in that direction. Inevitably, people will seek to oppose that. But there is a bit of a game that goes on here, and all experienced and mature politicians on the other side of the argument know perfectly well that that's what the role of the civil service is – and if they were in government they would deploy the civil service in the same way."

“Civil servants will need strategic decisions about what Brexit actually means to be set by ministers on the basis of advice and discussion" – Sir Peter Housden​

Despite the heated rhetoric from some Leave campaigners about “Europhile mandarins” being put in charge of implementing the referendum result, Housden says he is “very confident” that the civil service will get on with the job “incredibly well” – provided ministers are clear about the precise manner of Britain’s exit from the EU.

“What they'll need is for strategic decisions about the direction of travel and the objectives of what Brexit actually means to be set by ministers on the basis of advice and discussion. A clear course will have to be set. That's what civil servants are good at, in my experience, helping with the analysis of options, the risk management, all of those kind of issues. Ministers will then have to make their decision and the civil service will be able to implement it.

“That's not to say that people on the other side of the argument who would wish that either the Brexit decision were to be reversed, or believe that the objectives are wrong and want a harder Brexit won't continue to complain about the civil service. They'll be complaining about everything else they can find.

“That's what political arguments are like. You throw everything at the enemy and if the civil service is one of those bricks you can throw, then they'll do it. But it doesn't actually impact upon on the effectiveness, or indeed the morale, of the organisation.”

"It was a wonderful passage of life"

For now, Scotland's former top official has plenty to be getting on with – he’s in the middle of buying a house for his move back to England, and, as well as trying to “start a conversation” with his critique of public sector reform, he’s been taking on a range of consultancy gigs.

But with the entire constitutional future of the UK once again up in the air, it’s undoubtedly a fascinating time to be working north of the border. Does Housden ever regret his decision to remove himself from the thick of it?

“Just occasionally,” he says with a smile. “I was walking up the stairs in Stormont castle to go and see [Northern Ireland permanent secretary] Malcolm McKibbin the other day – and I thought 'I miss this'.

“But, you know, it was a wonderful passage of life, and you don't go through life missing things. It was tremendously interesting, but for me, its moment passed. It was the right time to draw a line under working in large, politically-directed organisations and to find another way of contributing in public service. It was a fantastic period – but if you said to me, ‘would you like to go back and do it again?’ I’d say no.”

“Yes and no. I think civil servants often undervalue the unique skills that they acquire in government. In an ideal world you've got a mix. You can come into this through a variety of different doors. You meet some outstanding permanent secretaries who have come in through the Fast Stream and worked through the civil service, who have often had very rich experience, not only in terms of departments, but through secondments into business or internationally – so these are very well-rounded and developed people.

"You get other people, I was one of a number, who came in through other routes. I found the people who came in from outside and enjoyed it and succeeded had a deep respect for the civil service and what it did. They didn't come in to say 'I'm going to sort this out, they make it all very confusing and difficult, really it's quite simple'. Those kind of people are never happy because government is a rich, contested, unsettled place – and I always used to look for people who had a twinkle in their eye about that. People who are curious, respectful and wanted to be part of what, inevitably, in the best traditions of the word, is a mess.”

Whether he set out to be a permanent secretary

“Well, I'd been a local authority chief executive when I joined the civil service, so I knew what a permanent secretary was and a little bit about how government worked. They looked to me like they were really interesting jobs. So yes, I'm ashamed to say I went into the civil service thinking, ‘I'd like to do that.’

“In those days I had a line manager and mentor in David Normington in the department. And under Andrew Turnbull, who was then the permanent secretary, there was a very conscious process, which is now accepted practice, of identifying a group of people who had the potential to become permanent secretaries and to provide them with opportunities and support so that potential could be explored.”

Hear from innovative colleagues and peers on the latest developments in digital transformation and how it is changing central government; from engagement with users, building and maintaining infrastructures, sharing data and securing our systems against the latest cyber threats.