Councils taking on the burden of responsibility for public health will face a major cultural barrier when making the transition to town hall control, local government leaders have heard.

Addressing council managers in London this week, Oswin Baker, director of Rockpool Research Associates, warned that as councils had not managed public health since the 1970s, they would have to get used to managing unfamiliarity.

They would also inherit cultural problems when managing clinicians transferred to work in local government. "The NHS views the public as there to be treated, and local authorities view the public as there to be served," he explained. "As you're taking on that part of the health system, you also take on a lot of those historic battles for power within that system."

Another problem facing local government was the fact that "everybody loves their doctors", he added. A study of attitudes found that 88% said they trusted their doctors, but just 47% felt the same way about civil servants.

Council officers working on public health may also face a stumbling block: elected members found it difficult to canvass on public health strategies among voters who "like to have a drink and a smoke".

David Buck, senior fellow in public health and health inequalities at the King's Fund, said local government risked the "paralysis of analysis" in taking over responsibility. It should prioritise, he said, and accept what it could – and could not achieve – in each area.

Dr Jessica Allen, deputy director of the Marmot Review team and an academic at University College London, said research revealed huge health inequalities across the UK. In some boroughs, the poorest residents were not only living shorter lives but were more incapacitated, some living up to 25 years with a chronic condition or disability. Taking public health into local government control represented a real opportunity to tackle inequalities, she said.

David Kidney, head of policy at the Chartered Institute of Environmental Health, said local government had the statutory power to really make a difference on public health matters. But he warned that councils would have to learn to work together as major public health crises, such as a flu pandemic, could not be confined to council boundaries.