Having a celebrity parent is rarely easy. When that parent is famous for answering an ad from a strange man to spend a year as his "wife" on an uninhabited desert island and for the film that portrayed the stormy relationship – in all its nakedness – on the big screen, it is perhaps more than usually tricky. If that parent is also your only parent and has never lost her enthusiasm for islands, isolation, adventure and self-revelation, there is bound to be an impact. Robin Irvine knows – he is the son of "Castaway" Lucy Irvine.

Lucy, you may remember, hit the headlines in the 1980s with her bestselling memoir and the film Castaway starring Amanda Donohoe as Lucy and Oliver Reed as Gerald Kingsland, the man with whom she spent the infamous year on the tiny tropical atoll of Tuin, just a mile long, in the Torres strait between New Guinea and northern Australia. She and Gerald turned out to be wildly incompatible and the island no paradise. They might well have died without the timely intervention of neighbouring islanders. Lucy, however, never regretted her year on the island – and it has informed her, and her children's, lives ever since.

Robin, now 22, has read the book, he says, and seen the film – at school in an end-of-term chemistry lesson. "It was a bit odd, me and my classmates watching 'my mum' naked. Oliver Reed appalled me, but I enjoyed it as a film … There were bits in the book that could have disturbed me, but they never did."

After her tropical sojourn, Lucy bought an isolated cottage in Scotland called Inchreoch, meaning Heather Island, advertised as "generous for one, no neighbours". Quite quickly, though, she found herself the single mother of three boys: the oldest (now 24 and a professional diver) the result of a one-night stand on a book publicity tour to South Africa, the younger two, of whom Robin is the older, of a longer, but ultimately unsuccessful, relationship in Scotland.

Between the birth of her first child and Robin's arrival, Lucy wrote Runaway, a prequel to Castaway in which she set out to explain the life that led to her accepting the desert island challenge. It begins with her aged 12 running away from her nice London school and middle-class parents: her magnetically attractive, self-centred father and cheated-on, unhappy mother, as well as her fading, anorexic older sister. Lucy is headstrong, which her father admires, and refuses any formal education from the age of 13. By 15, despite all the usual warnings, she is drifting and hitching alone around Europe. Rape at knife-point on a rural side-road outside Athens leads circuitously to a return to England, mental breakdowns, summers working in her father's Scottish hotel, a criminal conviction, a night as a prostitute and a job with the Inland Revenue before "that ad".

Has Robin read Runaway? "I know what my mother went through in those years, so I probably have … but reading it is one thing, connecting it to my mum – to the person bringing me up – is another." So what was his upbringing like? Isolated, certainly, at least to start with. Lucy wrote in an outhouse while the boys played together – there were no other children for miles. "We were a single-parent family with a Lucy Irvine twist," says Robin. "She is eccentric. She occasionally came to school events, but they stressed her out. I've told her myself: 'You're socially inept, Mum.' She knows. She's comfortable with that.

"You might think that a woman like her would be cavalier about our upbringing, but she wasn't. She thought a lot about it and was very keen on education." There was no television in the house, instead Lucy plied her sons with books, gave them general knowledge quizzes and saw that they listened to musical classics. Lucy later sold the house to pay school fees (and moved from one rented place to another, which she hated), although the money ran out when Robin was 14 and the two younger boys were pulled from their boarding schools ("I didn't mind much. My brother minded more"). There was also a road-trip through West Africa and the Scottish wilderness.

The boys roamed for miles, as well as working hard around the home. "Mum had a regime," says Robin, who by age nine was in charge of the logs for the stove, cleaning out the fires and taking tea to his mother at appointed times while she wrote. They all carried water from the stream when the house water supply gave out as it frequently did. When Lucy seriously damaged her back mending the wood-burning stove and had open-spine surgery, the boys took on even more, cycling a 14km round-trip to the nearest shop.

Before she had fully recovered, though, Lucy was planning the next adventure. "That's Mum," says Robin. "She took us for a little camping trip not far from where we lived. I think she was testing us a bit. It was quite hard. Walking a Highland path with the moor on one side and the forest on the other, she popped this image into our heads of tropical fish and beaches and palm trees – desert island stuff – and it was quite appealing." So began Robin and his brothers' own castaway year on Pigeon Island, a remote five-acre atoll in the Solomon Islands.

Diana Hepworth, a long-time British resident of Pigeon Island, had read Castaway and written to Lucy suggesting she come and spend a year on Pigeon Island with her boys and write the Hepworth family story. Diana had brought up her own children in the tiny community, and her son had married an islander. Lucy put the idea to the boys – and they were keen to go.

The final leg of the six-day journey might have put them off islands for life. A storm brewed up and waves were breaking hard over their open boat. Lucy later wrote (in her memoir Faraway) of 10-year-old Robin (called Joe in the book) "sodden … dead white … bailing automatically" while his younger brother "screamed and screamed". Actually, says Robin, he was more fatalistic than frightened: "I was planning exactly what to do if the boat capsized. I'd worked out which bit to hang on to. Mum still had a bad back. I didn't look to her for protection."

The boys quickly became island kids. Robin kayaked 30-40 minutes alone across the sea to trade for fruit on a neighbouring island, and they would canoe-surf five to a kayak, losing the passengers one by one into the waves. "When I think about it now, some of what we did was ridiculously dangerous," says Robin thoughtfully. Where was their mother? "Oh, nowhere to be seen," he says casually, "but she knew we were looked after. The islanders were like extended family."

"Things did go horribly wrong at times," he admits. "My little brother nearly took his thumb off – it was literally hanging – chopping open a coconut with a machete nearly as long as him … He learned from it though." Robin pauses before adding, "That sounds really terrible from a 21st-century mothering point of view, doesn't it?" But he doesn't think it was wrong: "Definitely not. Things go wrong very infrequently. If you take away the risk, you also take away a huge learning environment. If things go wrong 0.5 per cent of the time, 99.5 per cent of the time people are learning.

"God knows how I'd bring up my own kids," he says, "but I will probably never have any. I don't see myself with children." He didn't really know his own father until two years ago. "I remember the night he left – I was perhaps five – but I don't remember it being traumatic."

He doesn't know what happened between his parents – this is one of the few things he does not discuss with his mother. Two years ago, however, Robin got a birthday card with "love from Dad" written inside. "There was a little piece of paper stuck on the back. I prized it off and there was a phone number." Robin rang it. At the time he had been offered a place at Cambridge to study Chinese but was having second thoughts. "I thought I wanted a father who would give me direction. In fact, he just listened, and I sighed with relief." Robin now visits his father regularly, which also gives him a home in Scotland since his mother moved.

A couple of years ago, on the day her youngest son finished school, Lucy upped sticks and moved to a remote crumbling mud-brick house in Bulgaria. "It is bucolic," says Robin, "great for reading. No distractions." When he visits, he and his mother sit and talk, he says, their relationship now not so much mother and son as close friends.

Robin turned down the Cambridge place but three years later reapplied to study archaeology and anthropology and is now in his first year. In between he worked, travelled extensively – including riding through the desert in Mongolia – and created new a "family".

Most of his work was for Cumbrian Heavy Horses, Britain's only specialist heavy-horse-riding establishment. Robin has always loved animals ("animals are my children"), and he has become an integral part of the organisation and of the family that runs it. He now regards the owners, Annie Rose and Tim Ancrum, as his "pseudo-family". Tim is the father figure he lacked as a child, he says: "He came on the scene before my dad reappeared and sort of picked up where my dad left off. Tim's a practical man's man. Almost my only memory of my father is of him teaching me to wield an axe. Tim took it from there … and Annie really looks after me.

"I also have my Turkish mum," he says. He stays with her in Istanbul on his way to Bulgaria, "and I'm working on one in Cambridge." What they all have in common, he says, is that they are "open-house people". "Mum is not an open-house person. She likes the idea, but she needs her privacy. The one thing she didn't teach me was how to talk to people."

Robin is, in fact, highly articulate and has (like his mother) a disarming honesty about the workings of his own mind and emotions. His unusually varied and independent life has left him with extraordinary confidence in some areas – he is happy to take off alone to any part of the globe for instance – but perhaps also uncertain about how he fits into the social world. He has been partially deaf from birth and has a hearing aid in each ear. "I always felt I had to prove it didn't matter – prove myself."

This summer he is off to the Gobi desert to ride and join some anthropological field work, to Korea to dive and to Astana to visit a Kazakh he met in China … Oh, and to Istanbul, the Lake District and Bulgaria to see his three mothers.

What about after graduation? He grins: "Mum has given me some very bad ideas – I see she's made a life out of doing stupid things and writing about it. So why can't I?" Something tells me that – if his luck holds – we may be hearing more from Robin Irvine.