One summer, when I was 18 years old, I was walking along the seafront in Brighton when a skinny man in an ill-fitting suit approached me thumbing a copy of Dianetics. I hadn’t heard of the book or its author, L Ron Hubbard, but my new friend invited me to accompany him to a nearby office and fill out a questionnaire. When I answered his 200 questions, he came back with a graph whose line dipped in the middle to a precipitous fall. He’d once been a drug addict, he confided, and he felt sure that some similar weakness was holding me back. Did I have any idea what it could be?

Thanks to Lawrence Wright’s new book, I now know that what my adviser was up to was textbook Scientology: whetting my spiritual appetite by seeking out the particular “engrams” that were preying on my psyche and preventing me from achieving my real potential. Scientology, as Wright makes abundantly clear, is not religion as we know it; its scriptures are written not as weighty morals and metaphors but in the language of self-help and engineering, allied to a cosmology that is revealed in a glittering staircase of levels as the supplicant progresses through Hubbard’s oeuvre.

If I’d got as far as level three, I’d have known this: 75 million years ago, in a planetary system called the Galactic Confederacy, a cruel warlord called Xenu summoned an entire population of otherworldly beings called thetans and killed off the whole lot. The frozen bodies were immediately packed off to Teegeeack, which was then the name for earth, where they were hurled into volcanoes and nuked with hydrogen bombs. It’s these troubled, vacuum-packed spirits that are today the cause of most of our psychic ills. The goal of Scientology is to put believers on a path where they can eventually “go clear” of spiritual contaminants and find total freedom. If I’d progressed further up Scientology’s ladder of spiritual accomplishment, I might even have become a fully “operating thetan”, bouncing around in a jacuzzi with Tom Cruise.

Written out like this, it’s easy to make other people’s beliefs look ridiculous. The achievement of Wright’s book is to make us realise that Scientology’s wacky eschatology was never the most interesting part. Built on the foundations of a long article published two years ago in the New Yorker, Wright tells the story of the Church of Scientology through the career its late founder, Hubbard, and the testimony of a prominent dissident from the movement, Paul Haggis, who granted him a series of interviews.

Like most religious ideas, Scientology was a product of the cultural upheavals of its time. It arrived amid a flood of exotic new religions in the shifty, paranoid years immediately following the Second World War, at a time when many of the old ideologies had been shattered and there was a thirst for something new. Dianeticswas an immediate bestseller, one of the first of a new genre of self-help writing, but Hubbard wanted more. His approach started out as a kind of alternative psychotherapy but Hubbard bathed it in the rhetoric of science and engineering and when that seemed to pay dividends he tarted it up into a religion, too.

Eschewing the shouty provocations of some journalists who’ve tried to take on Scientology, Wright’s approach to the material is judicious and even-handed. What’s clear, according to Wright, is that from the beginning Hubbard was endowed with “an impressive capacity to summon others to join him on what was clearly a shaky enterprise”. It helped that he started out as a prolific writer of pulpy sci-fi novels, from which he seems to have learned a good deal about myth-making and self-mythology. Ingeniously, Hubbard developed his self-help potboiler into a series of veiled revelations, each of which promised greater abilities and increased spiritual power. “To keep a person on the Scientology path,” he once told one of his associates, “feed him a mystery sandwich.”

Wright takes refuge in deadpan. One disillusioned disciple “had long since come to the conclusion that Hubbard was not an operating thetan”. For an organisation that sets such store on its ability to help addicts, its own origins seem eerily bound up with hallucinogenic experiences. Discussing an early experiment with religious ritual, Wright tells us about a ceremony that, “likely aided by narcotics and hallucinogens, required Hubbard to channel the female deity of Babylon as Parsons performed the ‘invocation of wand with material basis on talisman’ – in other words, masturbating on a piece of parchment. He typically invoked twice a night.” The secrets of human existence appear to have been disclosed to Hubbard while he was at the dentist and under the influence of laughing gas.

The best criticism of old-time religion was that it offered empty solace to the impoverished masses but Scientology preys not on the poor but on the vain and the needy – actors and media types, for the most part. Scientologists are certainly impressively well organised when it comes to protecting their interests and the church is well known to be litigious when confronted by journalists. (Going Clear didn’t get past our plaintifffriendly libel laws: perhaps cowed by the threat of expensive libel suits, Wright’s UK publishers pulled out.) Yet after a long career in the church, Haggis’s very public falling out with Scientology for its mild homophobia – compared to most religions, Scientology is rather forward-thinking – seems overcooked and a little too righteous. As Wright cleverly suggests, the charge of “brainwashing” levelled against Scientology comes from the same paranoid B-movie thinking that gave us McCarthyism and the Red Menace.

As mainstream institutions crumble, it’s easy to have a go at the cultish organisations that are thriving on the margins but it’s more productive to ask why the rest of us feel so threatened by tightly knit groups with strong beliefs. Scientologists informally claim a membership of millions but, according to Wright, one former spokesperson for the organisation put membership of the Inter - national Association of Scientologists at only 30,000. The inflated network and numbers, as well as our grisly fascination, may only add to the mystique.

Was L Ron Hubbard God? Just in case, in each of the main Scientology offices around the world, they still keep a room ready for his second coming. Much of the intrigue in this enthralling book comes from Wright’s refusal to declare him either a con artist or a nut. If anything, he was simply a masterful storyteller. Hubbard was surely right, too, that any modern religion needs a powerful dose of narcissism and a series of secrets that are only available to the chosen few. His mistake, if there was one, was to let anyone ever find out what was inside the box.

James Harkin’s latest book is “Niche: the Missing Middle and Why Business Needs to Specialise to Survive” (Abacus, £9.99)