‘NHS England is a dysfunctional organisation that really isn’t fit for any purpose I can see.’ Photograph: John Robertson for the Guardian

John Middleton is England's longest-serving public health director. He retires at the end of the month after 26 years in the job in Sandwell, in the West Midlands. He has presided over striking improvement in many, though not all, of the borough's health indicators. But he leaves deeply worried about the prospects for continued improvement, in particular among young people, who have suffered years of joblessness since the economy crashed.

"It became clear in the early 2000s that middle-aged Sandwell men were dying prematurely because their lives had been robbed of work, activity and hope by the Thatcher recession," says Middleton. "Now that pattern will be repeated for the nearly 1 million under-25s who are unemployed and without hope today."

One may infer from this that Middleton – "Dr Johnny" as he is known locally – is an unashamed campaigner for public health and against those who undermine it. He believes that the last Labour government did much to boost public health with policies such as Sure Start for young children, the minimum wage and the National Service Framework for reducing heart disease, and he fears those gains are at risk as the NHS struggles with reform and spending curbs, and as income inequality remains stubbornly high. "It's not who your parents are, but how deep their pockets are, that will determine how healthy you can be," he says.

One good thing that the coalition government has done, in Middleton's opinion, is to return public health to local government – "back where we belong". This is not a view universally shared: a survey by the British Medical Association last week suggested that, since the transfer last April, more than one in two public health consultants had considered quitting the specialism out of frustration. But Middleton, vice-president of the Faculty of Public Health, which covers the UK and Northern Ireland, sees enormous advantages in such places as Sandwell, where local councils have seized the initiative and started to weave public health into the local government tapestry.

Driving around Sandwell, which is encircled by Birmingham, Walsall, Wolverhampton and Dudley, he points out continuing investment in leisure centres, social housing improvement and market gardens that help to educate people about (and supply) healthy food – projects typically delivered by the solidly Labour council in partnership with either the private or voluntary sector.

At Neptune Health Park, built on the site of a forge that made heavy chains for ocean-going liners, you can arrive by narrowboat for a GP appointment, blood test or X-ray, or to join a health-promoting guided walk. Some people do. "They used to say we had more canals than Venice and more derelict land than the Bronx," says Middleton. "Neither was probably ever true, mind you."

He admits that some of the £20m in public health money given by the NHS last year is being used by Sandwell to avert cuts in other services that have a public health dimension. "I am not aware of any council that isn't using some of the money to prop up budgets," he says, "but there are ways of doing it. You can do it as we do here through some kind of negotiation, and a creative description of what councils can do for the public's health, but then there are some councils just saying: 'We'll take £2m off [the public health budget] and that's it.' That's really short-sighted."

Public health money, he argues, could justifiably be diverted into food safety inspections if those inspections also looked at the salt or trans fat content of what was being served. "With the number of cheap fast-food takeaways around schools, the problem now is not food deserts but fat swamps," he says.

Similarly, funding could reasonably go into leisure services if it was conditional on them making a more active contribution to people's fitness and mobility, monitoring and measuring their progress. Yet the current ringfence round transferred public health cash should stay, he says. "Nationally, there is great variability about how councils are using the money and about their understanding of public health. There's a small group of councils that really don't want to know."

Middleton has found the move from the NHS into local government a revelation in some key respects: the quality of procurement and legal expertise far exceeds that in the NHS, he believes. He has also found it a good discipline to have to justify spending priorities to senior colleagues and elected members.

"I have to acknowledge that some things I thought were working well turned out, when they were scrutinised, not to be doing what I expected," he admits. "Where we had invested in community health networks supporting minorities, for example, we found that the Indian community had moved on and were aspirational and in many ways self-reliant, but eastern Europeans were not being covered at all."

Partly because of this, TB rates in Sandwell are 50% above what they were in the late 1980s. Rates of HIV infection are also causing concern. But Middleton can point with satisfaction to highly positive trends on headline indicators such as teenage pregnancy – on which he thinks the Sure Start effect may already be starting to show – and heart disease: a programme of identification of at-risk individuals has led to the saving of some 70 lives a year since 2007.

Perhaps his single biggest contribution to wellbeing, though, was ensuring that every GP practice in the borough had access to welfare rights advice, enabling poor families to claim their full entitlement and bringing millions of pounds into the local economy.

Leaving will be a wrench. But one thing he won't miss is working with NHS England. He describes the body created in last year's health service shake-up to run the NHS across England as "a dysfunctional organisation that really isn't fit for any purpose I can see".

He has more time for the equally new Public Health England, where he thinks chief executive Duncan Selbie is doing a good job in difficult circumstances, but, like many working in the health field, Middleton is generally exhausted by structural change. "I've had 13 reorganisations in 26 years, seven of them national, so I've never been able to get to the point where I think I am doing the same thing twice," he half-jokes.

Sandwell has changed beyond recognition since he joined the health authority in the late 1980s. Then, there were still 18 foundries and 88 scrapyards supplying them. Today there are just three foundries and, although manufacturing and metal recycling survives, the borough's newfound growth is based on its dormitory housing for Birmingham commuters, its resilient minority-ethnic communities and its surprising green spaces.

Middleton will host a final local health summit on his last working day this Friday, at the vast Balaji Hindu temple in Tividale, before heading off for retirement, when he hopes to have more time for his other great passion, the blues harmonica. Following a long and rich tradition of "blues doctors", Middleton is an accomplished frontman with Dr Harp's Medicine Band. Blues doctors, he says, "show people a good time, making ordinary lives richer and healthier". He hopes he has done that.

Career 2012-present: director of public health, Sandwell council; 2002-12: director of public health, Sandwell primary care trust; 1988-2002: director of public health, Sandwell health authority; 1987-88; consultant in public health, Sandwell; 1983-86: registrar in public health, West Midlands; 1979-83: various junior doctor posts, principally in hospitals in Liverpool and Salisbury.