Survivors of notorious cult The Family are waking up for the first time knowing the world is rid of their chief tormentor, kidnapper and abuser, Anne Hamilton-Byrne, who died on Thursday at a Melbourne nursing home aged 97.

But Hamilton-Byrne, described by ex-detective Lex De Man as one of Australia's most evil criminals, lives on in other ways.

For victim Adam Lancaster she's in his ruined dreams of being a navy officer; the memories of failed suicide attempts; the years lost to the cult's deranged cause, or to drugs and the streets.

Adam Lancaster (centre) with Anne Hamilton-Byrne and her husband Bill many years ago.

Mr Lancaster, 48, says he is happy these days. Social skills stunted by decades of isolation have somewhat recovered and he has good friends, support and a peaceful life. He goes on holidays.

Advertisement

But there is not a day he doesn't "struggle in some way or another" from his time in The Family, and he suspects it's the same for others who grew up within the cult (at its peak, there were 28 children).

Mr Lancaster feels relief at Hamilton-Byrne's death. There had been no "release for people with her still on the planet", he said.

Loading

He also feels grief. After all, "Aunty" Anne helped raise him. She was like a mother, the head of the only family he knew until he finally united with his biological family five years ago.

"I'd describe her as a crystal bullet," Mr Lancaster told The Age. "She was open and loving but if you crossed her or did something wrong she was deadly.

"I was joking with a friend before that I'm guessing she's not on the lift going upstairs - it might be the other way around. I don't know if there's an afterlife but, if there is, she's going to pay the price. Good luck to her."

Adam Lancaster with Anne Hamilton-Byrne on his second last visit.

Hamilton-Byrne was never charged with anything more serious than fraud, which led to $5000 in fines.

She had friends in high places and there were legal technicalities in the case against her. There was also government incompetency, including a representative of the Education Department who decided the cult was suitable to provide home schooling.

Mr Lancaster believes he and the other children of The Family should be compensated for their trauma, preferably out of her considerable estate.

The Family, formed in the 1960s at a property in the Dandenongs by Hamilton-Byrne and Melbourne University academic Raynor Johnson, went from teaching yoga to a cult that at one point had 500 followers.

It centred around Hamilton-Byrne, who set about raising a "master race" while preaching she was Jesus incarnate.

The kids supposed to inherit Hamilton-Byrne's new world were kept at a property at Lake Eildon, where they were starved, abused and drugged.

Mr Lancaster arrived just days old, after being 'adopted' from hospital. He grew up believing cult member Elizabeth Whitaker was his mother.

Adam with a friend on a holiday in Phuket.

On August 14, 1987, when Mr De Man and other police rescued the children, he not only found out the adoption story was a lie, but that his age was not 16 as he'd been told, but 17.

He had to decide that day whether to stay or go.

"Lex (De Man) offered for me to become a ward of the state," Mr Lancaster said. "But for my own safety it was a case of 'better the devil you know', and in the end I feel I helped tame the devil, that being Anne."

He made it out for a time in the early 1990s, but life replaced The Family with drugs and homelessness. After offers of money from Hamilton-Byrne, he returned.

It wasn't until "mum" Elizabeth Whitaker died that he left the cult for good, and not on good terms.

"It's been an amazing journey in the last eight years," he said. "I knew all through my adult life I'm not going to succeed until mum's gone, and when she died I went 'boom' and never looked back.

"I live a very quiet life and a relatively peaceful life. I'm very much a homebody. But life's good. I'm still alive."

Two years ago, he visited Hamilton-Byrne at her nursing home. She was ravaged by dementia and could not remember him.