Just as many Londoners have never set foot in the superb Museum of London, many New Yorkers I meet have never been to Ellis Island. The former gateway to America is now a museum that profoundly shapes an understanding of the city and the country.

Along with the battered suitcases and ghost-filled registry hall, I've never forgotten the testimony of an Italian immigrant who spoke for many when he said: "I came to America because I heard the streets were paved with gold. When I got here, I found out three things: first, the streets weren't paved with gold; second, they weren't paved at all; and third, I was expected to pave them."

There are Africans who regard Johannesburg, a city born in a gold rush, as the New York of this continent. They speak with wonder of its high-rise buildings, shopping malls and can-do dynamism. The tourist board describes it as "a city of commerce and opportunity" where people come to "realise their dreams". Like New York, Johannesburg has been a magnet for immigrants trying to escape penury and to better themselves. Like that disappointed Italian, many have discovered they were seduced by a myth.

When gold was discovered in the area known as the Witwatersrand in 1886, prospectors arrived and a mining town exploded into existence. Men came from Britain, North America, Australia and China to dig for gold, but most were from 12 African countries, drawn from 55 tribes. Thousands of bars and brothels sprung up in Johannesburg, which was described as "hedonistic, wild and outrageous".

Today the headgear of Shaft 14 of Crown Mines – once the biggest, deepest, richest gold mine in the world – is surrounded by a Wimpy, ice-cream parlour, cinema, farm, ferris wheel, coconut shy and fairground rides such as Thunder Mountain and Tower of Terror. This is Gold Reef City, a Disney-style simulacrum of a mining town next door to the highly esteemed Apartheid Museum.

Gold Reef City has its own railway, main street, hotel and chapel. Congratulations to Grant Sandon and Mitzi Botha, who got married there at 3pm last Saturday. Around the corner, visitors formed a circle around a silver-painted street artist dancing to a bouncy disco beat with some young would-be Michael Jacksons.

But all this disappeared from view as I plummeted 226 metres below ground in a clangy miners' lift. I was among a group of tourists, mainly Japanese, wearing helmets and clutching torches on the Jozi's Story of Gold tour. We got out at level 5, since going all the way to level 57, some 3,200 metres deep, would have taken the lift two hours.

A guide led us through the two-metre high tunnels of the mine that employed 30,000 men, producing 1.4m kilograms of gold, before closing in 1977. He told how miners created a common language, Fanagalo, predominantly Zulu but mixed with English, Afrikaans and Xhosa. It took only three weeks to learn as it consisted of 2,000 words – a quarter of them swear words.

Back above ground, we see a demonstration of liquid gold being poured into a brick mould at 1,062C. Nearby, the small Miners' Museum bears the legend "Yena Lo Kalo Madoda" (Place of the Strong Men). It doesn't attempt to sugarcoat the past. An estimated 80,000 miners were killed on the job in the last century, it says, and that does not include deaths from mine-related diseases. A further million suffered disabling injuries.

These hapless men were crammed into hostels, sometimes sleeping on bare concrete floors, and worked 12-hour shifts. In 1913, the minister of native affairs remarked: "If the mines continue to employ these people, it would be little less than murder."

A month ago, 86 miners, more than half of them from Lesotho, were killed in an inferno in a disused section of a gold mine in the Free State. Last year 62 people died and about 100,000 were displaced in a wave of xenophobic attacks that shocked South Africa and saw the word "pogrom" back in discourse. Last month Somali shopkeepers in some townships were handed letters telling them to get out or face the consequences.

But it is Zimbabweans fleeing President Robert Mugabe's destructive apparatus who, perhaps more than anyone, find that the land of opportunity is another dead end. Stories are legion of Zimbabweans kidnapped, raped or murdered while attempting to cross the border into South Africa.

A 17-year-old who was among seven Zimbabwean girls trying to enter by bus told Médecins Sans Frontières that they were taken hostage in a forest. "There were 13 men watching us all the time and they raped us every day," she said. "Eventually they started to let us go to urinate by ourselves and that is how we escaped. Now I am four months pregnant."

Some Zimbabweans earn a pittance working illegally in mines, on farms or as prostitutes. Others have made it to Johannesburg's Central Methodist Church, a cruel, stinking parody of Ellis Island that cannot contain them all. Last week hundreds of the destitute refugees, forced to sleep on the street in the winter cold, were rounded up and arrested for vagrancy and loitering.

On another occasion, they were sprayed with putrid sewage by assailants who sped off, presumably thinking it was all a lark. The migrants were soaked, homeless and humiliated. They came to South Africa because they heard the streets were paved with gold.