‘Retracing Betjeman’s journey, as I have been doing for BBC radio, reveals a very different Metroland.’ Illustration by Matt Kenyon

Simon Woolley vividly remembers his introduction to the English suburbs. He was detained and handcuffed outside his own home by police officers who refused to believe he lived there. But for the intervention of his white English neighbours, who hurried to vouch for him, the black Briton was on an ignominious route to the local station. These things stay with you.

Two decades later, Woolley – a national figure as director of the pressure group Operation Black Vote – is now a veteran suburbanite. What is remarkable in our national story is the extent to which thousands of other black and Asian Brits have followed that trajectory. Demographers make much of white flight – the movement of white Britons from the inner cities to the suburbs and beyond – interpreting its meaning and consequences. Less talked about is the growing movement of visible minorities into the heartlands of Englishness.

In post-referendum Britain we have seen sudden waves of intolerance. Eastern Europeans told to go home. A BBC reporter branded a “Paki” in the English town where she was born. An African-American on a bus told by fellow passengers to go back to Africa. This is distressing. But underneath, there is something more encouraging; the quiet riptide of communities adapting to significant, structural change.

John Betjeman would have spotted it immediately, for it is via the former poet laureate and his celebrated 1973 BBC documentary Metroland that the traditional perception of the English suburbs has been formed. On a gentle jaunt, in his gentle way, he described communities stretching from Neasden in north-west London all the way to the Chiltern Hills, along the path of the Metropolitan underground line. Betjeman’s suburbia spoke of cricket pitches, golf clubs, women’s institutes and verdant farmland: a new life for indigenous Britons at arm’s length from the bustle and smoke of London.

Many migrants came from rural Caribbean, Africa and India. There is no particular cultural affinity for concrete

But that journey retraced, as I have been doing for a BBC radio documentary, reveals a very different Metroland. “Funereal from Harrow draws the train,” wrote Betjeman. “On, on northwestwards, London far away.” Harrow then, according to the 1971 census, had an Asian population of about 8,000 from a total population of 200,000. A further 1,500 settled there after Idi Amin’s mass expulsions from Uganda in 1972. Today, 31% of the residents there describe themselves as white-British; 69% are ethnic minorities. Betjeman’s Pinner, where he saw “sepia views of leafy lanes”, is much changed too. In 1971 there were fewer than 1,000 visible minorities. Pinner’s population of 19,000 is now 38% ethnic minority.

Were Betjeman able to take other routes from other cities into other suburbs, he would encounter the same phenomenon. Analysis by thinktank Demos suggests that just 800 from 8,850 council wards in England and Wales now have a 98% white population.

Geographers at Leeds University led by Prof Philip Rees tracked the “internal migration” of people from one local authority to another, using census data to show that minorities were leaving core city neighbourhoods for lower-density suburban areas. In London, all minority groups are drifting out of the inner city. The trend is also noticeable around most other big English cities, including Leeds, Birmingham, Nottingham and Leicester.

And it will persist, not least because the absolute numbers of minorities will increase too. White Britons are expected to account for 70% of the UK’s population by 2061, with the ethnic minority population 30%. And those who have been here the longest are most likely to strike out for suburbia.

This tells us various things about our recent history. It means that, over time, the suburbs have become less forbidding for minorities. People seem more willing to move away from traditional inner cities, particularly when others have already done so, mirroring a phenomenon demographers have noted in parts of the US. That process isn’t always tension-free. The racist British National party sought to make political capital in Barking, east London, with claims that black Africans were being lured into the area with public funding. In fact, those were largely families buying into an area with some of the cheapest house prices in the country. The rumours persist though – surfacing again as residents vented to reporters during the Brexit referendum campaign. Compared to the scale of change, however, there has been relatively little conflict. It tells us that minorities themselves feel psychologically able to move to new areas.

In making the documentary, I did encounter two highly successful professionals who buck that trend. One said he would be reluctant as a matter of principle to move from inner city to suburb because being away from his community would be undesirable. He actively wanted to stay among folk who were like him. The other felt that the departure of figures like him would – in time – degrade the social capital of his area.

But for the most part, the “internal migrants” I saw equated their move outwards with practical and social advancement. And all this was aided by Britain’s improving infrastructure. Businessman Wasim Rehman grew up in Aston in Birmingham, then moved his family to Chalfont St Giles in Buckinghamshire. It’s 100 miles away, but a clear M40 allows him to meet the challenges of being away from family and community. He can be back for cultural events in less than two hours and does so, he says, “even if we only stay for an hour”.

In addition, culturally specific purchases merely require a 40-minute trip to Southall, 15 miles away. “Being part of your community no longer has to be about geography,” he tells me.

And, of course, as sufficient numbers move into a suburb, eventually they create a local network of shops, community facilities and places of worship, further speeding up the process. The mainly black African King’s Family church in Chadwell St Mary, Essex, caters for a new community but also acts as a conduit. People visit the church, befriend the congregation. Many then move in.

Inevitably, this alters the look and character of the suburbs, and the nature of the local economy. Food, hair, clothes and jewellery shops, temples, mosques, gurdwaras. Traditional pursuits are adapted to new realities. What would Betjeman have made of Bollywood night at Pinner cricket club?

A defining question arises that goes beyond the fabric to the people: will the new surburbanites change or be changed by their new surroundings? And this has a particular relevance to politicians. Labour had hoped the phenomenon, reflecting traditional support from ethnic minorities, would help in the capture of suburban Tory marginal seats, but it didn’t work out that way in the 2015 election. In places like Metroland too many of the upwardly mobile Asian voters plumped for the Tories. Hendon in north London stayed Tory – with an increased majority. Croydon Central, a recipient of aspirational minority south Londoners, could have fallen but didn’t. Neither did Harrow East. Quite the conundrum for politicians as they seek to recalibrate how votes are sought and won from an increasingly diverse electorate.

One can’t be Pollyannaish about this, as alongside black flight, demographers detect an extension of white flight. Minorities move out; many white Britons move out even further. But perhaps there is a natural filtering system at play, and perhaps that’s for the best. Those who can be comfortable with a changing Britain embrace it or make the best of it. Those who can’t just pack up and leave.

There is an inevitability about diversity in the inner cities. Poorer Britons of all origins live there. Migrants looking for jobs and cheap housing gravitate there. But it is not surprising that minorities should want a piece of Metroland. Many came from rural areas of the Caribbean, Africa, India and Pakistan. There is no particular cultural affinity for urban sprawl and concrete.

What the geographers detect is the sum of decisions – integration by design, not accident. Overall, we can be quietly hopeful about how it’s playing out.

• Black Flight and the New Suburbia is on BBC Radio 4 on Sunday at 1.30pm