I'm standing at the edge of the men's swimming pond at Hampstead Heath. And I can completely understand why I'm virtually alone. Just as you wouldn't find 4'33'' by John Cage on the karaoke machine, and in the same way that Marcel Duchamp's Fountain never featured among Athena's range of iconic posters, most people don't see the point of spending a drizzly autumn weekend lowering themselves into ponds.

Like a piece of music with no notes, or an upturned urinal displayed as art, there's something about the practice of swimming in cold water that touches a nerve among right-thinking people. Instinctively, they feel that it's dangerous or somehow corrupting. And in many ways, they are correct.

The cold-shock response, an involuntary instant reaction to cold-water immersion involving hyperventilation and a racing heart, can cause a person to swallow water and drown, or even suffer a heart attack. Hypothermia or non-freezing cold injury potentially await survivors – but before that, the functioning of the limbs will increasingly weaken as blood flows to the core body to maintain warmth.

One thing that encourages people to endure the pain is the accompanying cocktail of endorphins that arises in the brain, resulting in a lasting sense of euphoria and calm. "It sets you up for the day," says one swimmer I encounter in the changing area. He's one of 20 or 30 who swim here throughout the year, even in the depths of winter, when the temperature is close to zero.

Various health benefits are also claimed (though the evidence is inconclusive) for regular cold-water swimming, including boosts to the circulation, immune system and libido. In theory, it sounds good. But theory counts for nothing when faced with the brute reality of these chilly, murky waters. It's a congenial environment for moorhens – but human beings?

And yet cold-water swimming is undeniably growing in popularity. The number of participants in the biennial Winter Swimming World Championships has risen from 500 in Helsinki back in 2000, to 1129 for the latest event in Latvia last year. Local events have sprung up too, in places ranging from Slovenia to Sweden. Meanwhile, South African Ram Barkai is attempting to formalise ice swimming as a sport – with his International Ice Swimming Association defining an "ice mile" as one mile in water below five degrees – and dreaming of inclusion in the Winter Olympics.

One of the world's most famous cold-water swimmers is Lewis Pugh, who has swum across a glacial lake on Everest and completed 1km across the North Pole in water of -1.7 degrees (four months later, he recovered the feeling in his hands). Pugh's trainer, Professor Tim Noakes of the University of Cape Town, says that Pugh can mentally raise his body temperature by 1.4 degrees before entering cold water in a process called "anticipatory thermogenesis". But how? One possibility Noakes suggests involves brown fat – the tissue that allows newborn babies and hibernating mammals to keep warm, but was until recently thought not to exist in adult humans.

If it sounds like science fiction, that only illustrates how little we actually know. The bizarre physical and mental experiences cold-water swimmers often report underline that Pugh and Barkai are in effect the Duchamp and Cage of swimming – relentlessly pushing boundaries, forcing us to redefine what we consider possible. No matter that their work is misunderstood by mainstream society. This only strengthens these pioneers' camaraderie, their commitment to discovering new ways of seeing. They are, as Christopher McDougall said of ultrarunners, "body artists, playing with the palette of human endurance".

And I'll be joining them. Having previously dipped my toe into this world in the name of research, I've now decided to go native, and try to acclimatise to the cold throughout the winter. Because what we do know is that repeated exposure reduces the negative symptoms of cold immersion, while retaining the potential benefits. Perhaps gaining the respect of this pursuit's practitioners will help me figure out what makes them tick. But attaining cold-water credibility is a complex process. It might be instructive to draw parallels to another area: music.

Wetsuits, for example, are an obvious faux pas – the equivalent of putting Mumford & Sons on the pub jukebox. Right now, the water is 16 degrees, and I can stay in for a good 15 minutes without too much trouble. That's respectable though unremarkable, much like confessing an admiration for David Bowie. But I'm planning to keep going as the temperatures plummet, and then enter the next Winter Swimming World Championships in Lapland in March – where I'll see how fast I can swim 50m.

The lifeguard here says he gets increasing numbers of inquiries about cold-water swimming. But if you are tempted, then be safe. It's not for those with heart problems, and it's recommended you consult a doctor beforehand. It's important to swim regularly (2-3 times per week) and the ideal time to start is during the summer, when the water is warmest. Don't jump straight in, and don't stay in for too long. Four minutes 33 seconds is probably too much.

And as for urinals, it's best not to visit one straight after your swim. If you're male, that's one area where cold water is not good for the ego.