Tim Minchin: the man who waltzed Matilda

His score for the stage version of Roald Dahl's Matilda has transformed the one-time struggling standup into a leftfield Andrew Lloyd Webber. Next month, the production relocates to Broadway. But how will New York take to the anarchic Australian?

But for an encore, Tim Minchin, 37, might have missed the opportunity of his life. Back in 2008. the Australian was performing for a packed house at London's Bloomsbury theatre. Wearing eye makeup, his red hair straightened into chaotic spikes, he sang the droll and thoughtful piano ballads that had been charming comedy fans around the country. Unknown to the performer, he was being scouted that night. Matthew Warchus, the director who'd put on West End adaptations of Ghost and The Lord of the Rings, had been hired by the RSC to direct a stage version of Roald Dahl's Matilda. (The production opens on Broadway in April, transferring to New York after monstrously successful runs in Stratford and the West End.) Warchus was on the hunt for someone to write the songs.

"A fantastic musician and an outstanding lyricist" was Warchus's verdict of Minchin's set, with its anarchic, pointed songs on subjects as diverse as religious complacency and dancing bears, superstitiousness and inflatable sex dolls. Only come final bows, Warchus didn't feel he'd heard the subtlety of emotion that would be necessary to capture the essence of Dahl's knotted and unsentimental children's book. Then Minchin returned to the stage for an encore, in which he sang a relatively straight song called White Wine in the Sun. It explored the strange, elemental pull of having parents and siblings who were 10,000 miles away, back in Oz. Warchus recalled: "The whole audience cried."

"Tell me what I must do to convince you I'm your man," Minchin said, when Warchus made his approach soon afterwards. In the end Minchin wrote a couple of dozen songs for Matilda, in collaboration with an arranger, 16 of which ended up in the production that opened at the Courtyard theatre in Stratford in December 2010. "[It's] not avant garde, cutting edge," Minchin later said of his work on the production, alongside musical supervisor Christopher Nightingale and playwright Dennis Kelly, "but emotionally, it hits the mark."

Reviews were uniform – "smash", "gem", "triumph" – and Minchin's songs were agreed a highlight. Within a year, Matilda had moved to the West End and then, shortly before it took a record haul of seven Olivier awards, transfer to Broadway was confirmed. The New York Times has already called Minchin's score "one of the most original in years".

During this period of colossal success, Minchin might have wallowed; papered his walls with photocopies of the Telegraph review, say, that called him the most exciting thing to happen to musical theatre since Lloyd Webber, or otherwise spent his days calculating and recalculating the millions that can be made once a musical starts to look globally tourable. While Matilda prospered, though, he continued to ply the musical comedy circuit.

The night after Matilda opened, Minchin was playing a gig in Brighton, where he launched a UK tour. Performing songs he described as "Wagnerian in their epic stupidity", and accompanied by a 55-piece orchestra, Minchin went on to do a show for 20,000 at London's O2. The opening verse of his opening number that night was typical: a rational if somewhat harsh truth made palatable by seductive piano. "Nothing ruins comedy like arenas," he sang, "but your enjoyment is less important than my self-esteem is." As Minchin once put it: "That's the power of musical comedy: you can say horrible things while sugaring the pill."

He had begun to put on weight, and was starting to look like a chubby, bedraggled Damian Lewis; a dramatic slimming, in 2012, was surely the outward sign of a career suddenly moving in new directions. In spring that year, he signed up to the David Duchovny drama Californication (Minchin's episodes recently aired in the US) and joined the cast of Jesus Christ Superstar. He also rebuffed an approach from Pepsi, which wanted to see about a drink endorsement. "Six years ago," Minchin told the New Yorker, which ran a flattering five-page profile on him last week, "I couldn't fucking get a gig."

Six years was an exaggeration – call it eight – but certainly by profile-in-the-New-Yorker standards, Minchin was an absolute nobody until he was 30. The Australian was actually born in the UK, while his father, a surgeon, was on secondment at a hospital in Northampton. Within a year, the family had moved back to Perth, where Minchin's upbringing sounds ideally Australian. Swimming on the Rainbow Coast. A sheep-shearing shed in the garden.

He first took up the piano because his older brother played the guitar and demanded "someone to jam with". Efforts at formal lessons were abandoned when Minchin was about 12; the family had a decrepit pianola on which Minchin taught himself to play. He met his future wife, Sarah, in his teens at the University of Western Australia. She went on to become a social worker and would have watched as Minchin, in his 20s, pursued less stable careers.

"Sometimes composing for theatre," he later recalled. "A hell of a lot of cover bands." He tried and failed to get a record deal, and acted in two productions of Jesus Christ Superstar in Perth, understudying Judas but never getting to play the role. For a while, at about 23, Minchin was seized by an idea to turn Matilda into a musical. The project didn't take off. "It became one of those lost ideas."

It sounds perverse but, as Minchin tells it, he moved into piano-based comedy as an act of exorcism. He wanted to stop messing around, to focus properly on serious music, and so resolved to dispel a penchant for comic ditties by gathering together all those he had written and singing them out of his system at a cabaret club. The songs went over well, though, and he ended up gigging at the Melbourne comedy festival. Serious music faded away.

Minchin understood well the confusions of hopeless, wannabe rock stars, being one himself, and decided to perform his cabaret act as a recognisable monster: Keith Richards eyeliner, shirt open to the nipples, delusional from the barefooted toes up. His audiences were very, very small, and he thought the disparity enhanced the comedy – "a clash between the act and the room". Minchin has kept the look ever since. (In that recent New Yorker article, he was described as "a kind of rationalist Russell Brand", which must have stung; the Australian once floated the idea that Brand had ripped off his haircut, back in the mid-00s.)

Debuting at the Edinburgh festival in 2005, Minchin overcame a damning first review – one-star, later immortalised in song – to win best newcomer. The prize allowed him to get an agent, a DVD deal and, best, a few years' relevance with British audiences. When he returned to the UK for the next Edinburgh, he brought a pregnant Sarah, too. They relocated to London.

"Someone pulled away the set from my life and replaced it with another," Minchin has said of that time. He started touring in America, where the prickly, sceptical songs (there was a long one about the egotism of prayer) infuriated many but made him an absolute hero to a few. Non-Christian Kentuckians wept with affinity on his shoulder.

In Texas, a local piano firm refused to lease their instruments to a "God-hater". Meanwhile, in Seattle, he was offered a threesome: declined.

"I'm basically un-seduceable," Minchin once said. "The sacrifice of not being able to shag other women is a full-on sacrifice… But there is no way I would risk not being able to live with my kids." In interviews he has tended to acknowledge his wife's great sacrifice, too. Sarah Minchin gave up her career in social work to raise their family. On her Twitter account her bio reads: "Having the time of someone else's life."

The Minchins currently live in north London, with six-year-old daughter Violet, newly arrived when Minchin started work on Matilda, and four-year-old Caspar. "It opened that door back [to childhood]," Minchin said of composing Matilda when he'd recently had a child. "It reminded me just how innocent innocence was." Clearly he captured something: on opening night, Dahl's widow, Liccy, gave the composer a hug and told him he had the spirit of her husband's work precisely.

Back in the 90s, when Minchin first wrote to the Dahl estate to ask about a Matilda musical, he was told they'd consider it if he showed them a script. At that point Minchin gave up; he didn't want to do the work without getting paid.

It took a while, but his payment came – and will continue to come, should the Broadway production be the success everyone's expecting. Asked to give a quote to promote the New York show, the Australian was cheeky. "Kids and adults [will feel] energised and naughty," he said. "And hopefully, they'll go away with a copy of the album and a couple of T-shirts." It was very Minchin: anarchy advocated, plus a dash of cold truth.

THE MINCHIN FILE

Born 7 October 1975, to parents Dave, a general surgeon, and Ros. His father was on a fellowship at the Royal College of General Practitioners in Northampton when Minchin was born; within a year the family (Minchin, an older brother, two younger sisters) were back in their home town of Perth, western Australia.

Best of times After months wowing audiences and critics, Matilda wins seven prizes at the 2012 Laurence Olivier awards - the most won by a single production in the ceremony's 36-year history. Collecting the prize for best new musical, Minchin praised "the little twerps" in the cast.

Worst of timesA Guardian review of Minchin's Edinburgh festival show in 2005, by Phil Daoust, wondered if Minchin should be tarred and feathered for the low quality of his act. Discussing the incident on Desert Island Discs seven years later, Minchin still sounded aggrieved.

What he says Success is "beyond anything I could have conceived, if only because I am largely a self-taught musician".

What they say "Brilliant, incredibly intelligent comedy songs," said David Tennant, choosing Minchin's White Wine in the Sun on Desert Island Discs in 2009.

Sign up for the Guardian Today

Our editors' picks for the day's top news and commentary delivered to your inbox each morning.