The Manchester Modernist Society leads tours around the city centre. Photograph: Christopher Thomond

In Piccadilly Plaza, Manchester, a bunch of brutalists are hanging on to Eddy Rhead’s every word. Concrete to the left of us, concrete to the right, Eddy is in modernist heaven. He introduces us to the concrete wall built by the Japanese architect Tadao Ando to separate the bustle of buses from the serenity of the gardens. Though it is pushing it a little to say Ando built the wall – he never actually came to Manchester, but he did give his name and blessing to it.

When I was growing up in Manchester, I came to this part of town to do soup runs for the homeless. Back then, before the Modernists made merry, the gardens were old-fashioned flowery affairs. But even then there was not much serenity to be had. This was where the smack heads and the sex workers, the dipsos and the dossers, hung out. There was something moving about the community – and it was a real community. If one of the homeless had moved on or had an accident, we could rely on the prostitutes to keep us informed.

Today, it’s rather soulless in its concrete and steel garb. “It’s just starchitecture,” says Dale Lately dismissively. Starchitecture? “Yeah, a building designed by a star architect who comes in from the outside. This isn’t really true brutalism.”

Eddy, the co-founder of the Manchester Modernist Society, tells the crowd of 30 trendies, councillors, hippies, architects and anarchists gathered for his walk, that this is brutalism light – a Disneyfication of concrete culture. But we’re only at the start. He is going to take us to the centre of the city’s concrete jungle, and that, he says, is where we’ll recover our soul.

By the way, he says, brutalism doesn’t come from the word “brutal” as we think we know it, however appropriate it sounds. This brutalism is from Béton brut, which is French for raw concrete. We turn to the back of Piccadilly Gardens. Nothing unfamiliar or new here. Just the nondescript hotels that lead into Chinatown.

But as Eddy talks, it morphs into a concrete wonderland. He’s right – the Piccadilly Plaza does have a touch of Thunderbirds about it. He points out a beautifully curved car-park ramp. Eddy is known for his half-arsed topographical anecdotes, and it’s time for another one.

“In the early 1970s, the architect Louis Kahn, who was a god to me, came to Manchester to give a lecture at the university. He was in his late 80s and he was picked up by a lecturer, who took him to the hotel. He was driving up the circular ramp and the car conked out with Louis Kahn in the passenger seat on the ramp. He put the handbrake on, it was late at night, he was panicking, didn’t know what to do. Suddenly headlights came up in his rearview mirror. He looked out, there was a transit van behind, out jump five or six Afro’d men. It was the Commodores! They’d been doing a gig in town. They jumped out and pushed Louis Kahn up the ramp to the plaza. So the Commodores saved the day. It’s a beautiful story. On that note, we’ll move on.”

Close to the plaza is the City Tower hotel. At first glance, all you notice is its height. Close inspection reveals beautiful concrete artworks on the gables. “The story goes, this is to reflect the circuit boards because as you know Manchester invented the computer, no matter what anybody in America says. So that is a proud nod towards Baby, the first computer.”

The original UMIST campus, now integrated into the University of Manchester. Photograph: Christopher Thomond for The Guardian./Christopher Thomond

Here is the concrete complex at Umist, Manchester’s science and technology university, built in the 1960s and once the gleaming symbol of Harold Wilson’s white-hot technological revolution.

Eddy explains that this was inspired by the great Brazilian architect Oscar Niemeyer, whose buildings were designed to reflect the light and shadows of Brasilia. Even Eddy doesn’t know why so many were built in Manchester. There is something heartbreakingly ugly about these buildings. But there is also something strangely beautiful – the simplicity, the vision, the ambition (for a better world), the hope (for a sunny world).

Parallel to Canal Street, plonk in the middle of a regular road, is the most surreal example of Mancunian brutalism: a concrete building with red concrete windows and 1911 stamped into its roof. Nobody seems to knows why it was built, what it was used for and why it’s still there. It’s every bit as ghostly as Rachel Whiteread’s House – only this is for real. Eddy is bewitched by it, and so am I.

“It’s BEAUTIFUL,” I scream.

“Beauty is in the eye of the beholder,” says Joan Davies, slightly disapprovingly. As a councillor, she’s more used to showing off Manchester’s traditional hotspots – the town hall, the Royal Exchange, St Peter’s Square.

Every so often, one of the party passes on a tip to me. A man called David taps me on the arm. “Do you know there’s a website called Fuck Yeah Brutalism,” he whispers. Is that a fan site? “Of course,” he say. What made him a brutalist? “Insipid modern buildings,” he says. “I like the unloved.”

Simon admires the Mancunian Way flyover, winner of a concrete society award. Photograph: Christopher Thomond for The Guardian/Christopher Thomond

So we head off to the crowning glory of concrete Manchester – the Mancunian Way, a two-mile mini-motorway. Eddy points out the monument to the Mancunian Way, made in two complementary concretes. It reads: “1968. The Concrete Society Award was awarded to the Mancunian Way.”

Actually, it feels as if the Mancunian Way is held together by graffiti. “Future anarchy”; “Take LSD”; “Acid makes everything better”; “There’s no time to sit on the fence, fight.” Pure grime, pure gorgeousness. Welcome to Manchester.

• This article was amended on 9 February 2015. An earlier version misspelled the name of the architect Louis Kahn.