Last summer, a community arts project supported by Hackney council set up an installation in a London shopping street to welcome passers-by. It comprised a large banner with the slogan, "Hackney is friendly" and two attendants dressed up as if they were attending a 1970s folk festival. For some, it was an alienating sight. "What's this about?" asked one man, shaking his head. "They don't want people like me here any more."

It's one example of the difficult question facing communities and public services: when an area is gentrified, and the demographic and local culture changes, what happens to those who were there before and feel left behind? What happens when the local boozer becomes an expensive gastropub and the corner shop turns into a coffee shop serving flat whites for £3.80 a cup?

Hackney is just one of the case studies identified in an IPPR report, Love thy neighbourhood, which addresses this problem. The study focuses on the example of the borough's Herbert Butler estate, which found itself cut off from the economic and social benefits of regeneration.

"The influence of higher-income households into private housing facilitated a degree of gentrification and service improvement. However, on the Herbert Butler estate residents have become isolated and crowded out of job opportunities," the report concluded. "The sense of isolation and of being in a poverty trap makes the neighbourhood effect appear very real to those who live there – a fact exacerbated by the apparently low levels of community activity and aspiration."

The report's author, Ed Cox, director of the IPPR, says there is no simple or easy answer to this sense of isolation but he says it's up to public services to find new ways to work together to tackle the problem, from pooling budgets to putting more trust in local people.

Cox raises questions about the people working in the public services today. "We need frontline workers to be able to relate not only to the needs of particular communities but also to recognise the assets and qualities that those individuals and community organisations actually bring," he says. "There are some people who really get it; there are others that don't."

Broadway Market in Hackney, east London, has seen a huge change in demographics and has been the target of anti-gentrification protests. Photograph: Dosfotos/PYMCA/REX

In Hackney, councillors are spending money on projects to try to put the problem right. Jonathan McShane, cabinet member for health, social care and culture, says he is looking to learn from the experience of London's schools, which are improving faster than anywhere else. "A big part of that is white, middle-class Oxbridge people mentoring young disadvantaged kids," he says. "A core factor in that is relationships are building that people wouldn't normally encounter."

McShane admits that councils have tended to run local projects for specific cultural groups and it can be tough to bring people from different walks of life together. "It's something we think about all the time. I think it's our job [to do that] because so many good things flow from people mixing better and people understanding each other. It's worth putting resources in to try and help people mix better."

London is the obvious example of a divided community, but the problems of social isolation and inequality are not solely caused by urban regeneration. Winchester is one of the wealthiest cities in England overall, yet it has boasts two wards classed among the most deprived. Figures suggest that 1,800 children in the city are living in poverty.

Gavin Smart, deputy chief executive of the Chartered Institute of Housing, is a local resident. "I have always been acutely aware of what the income distribution in Winchester was likely to be like. It's important to think about patterns of economic and social division within areas that are regarded as being pretty affluent," he says.

"Winchester comes up in the list of wealthiest but, in terms of relative disadvantage, there's an interesting story there. If you start to look at the council estates … the incomes of those people in comparison with elsewhere in Winchester are going to be very low because they're going to be on zero-hours contracts or minimum wage, or benefit dependent."

Winchester's affluent appearance hides the fact that 1,800 children in the city live in poverty. Photograph: Alamy

Winchester council has this month established a working group on poverty and inequality. Ian Tait, a councillor, has called for the city to become more "inclusive", and the group will come up with new policies to help bring residents closer together in the coming weeks.

Head of policy Antonia Perkins explains: "The group's remit is to look at what the particular issues are for inequality in the district and, working with partners, look at what we are doing already to address them and what further needs to be done. Winchester district is relatively affluent and prosperous and so any deprivation will be very marked against that context – so we are very keen to do the background work to pinpoint the areas affected and direct resources there."

It might take its lead from Oxford, which faces a similar income disparities. The city has been named the least affordable, with house prices costing 11 times the average local salary. The growth of the private rented sector in cheaper areas of the city has also led to a changing population profile. Geographer Danny Dorling, who is based at Oxford University, predicts that plans for a new rail station linking Oxford to London Marylebone will only exacerbate the divisions between rich and poor.

House prices in Oxford have increased to 11 times the average local salary. Photograph: travelbild.com/Alamy

Oxford city council identified and concentrate resources on the five most deprived neighbourhoods of the city. It has invested in schemes to bring people from within estates and between areas together. New community centres, a visiting health bus and community events have all played a part.

"People were living parallel lives," says council chief executive Peter Sloman. "There can be a tendency for people to not know their neighbors and other divisions start to creep into their lives … All of that work has led to quite a strong community relationship in that area."

The council has also worked with the university to get older people and school children into the city's museums. "The university is now employing people specifically to go out into our communities and run things in schools which bring people in," Sloman adds. "Community cohesion is one of the things that local authorities need to concern themselves about. If you go back over the last 25 years, every time that there have been any major urban riots there has been a report which says more needs to be done to glue communities together and get them to understand each other."

Brixton in south London has been extensively regenerated but the consequence, a councillor admits, is an increase in house prices. Photograph: Dan Kitwood/Getty Images

Back in London, Lambeth council has also faced criticism for the rapid gentrification of Brixton. Councillor Ed Davie understands the concerns. "We obviously want to regenerate a run-down area but we don't want to drive out local people who may have been there a long time and are a part of the integral character of the area. If you improve an area you drive up house prices: it's a strange and unfortunate situation," he admits.

But he is also angry at uninformed criticism of public services and their efforts. "This idea that it's social cleansing," he says, "it's bollocks actually."