Lynton Crosby, the campaign director for the Conservative party. Photograph: Stuart Clarke/Rex Features

The unveiling of Lynton Crosby as mastermind of the next Conservative election campaign was conveyed to the public in contrasting ways. In the Sunday Times, a relatively sober front-page report: Cameron nabs Boris election guru. And in the Mail on Sunday, an Exocet: PM's new fixer in racist rant at Muslims. Crosby, said the paper, urged subordinates not to get hung up on seeking support from "fucking Muslims".

Had he not been the target, the strategist might have found much to admire in the Mail on Sunday's pre-emptive act of aggression. It was proactive. It aimed to kill. It could have come from the Crosby playbook.

David Cameron must have known his hiring of the man often described as an "evil genius" would draw him into the ambit of bare-knuckle politics. It's what Crosby brings to election campaigns. Four election victories for the rightwinger John Howard, making him the second longest-serving Australian prime minister. Two victories in the London mayoral election for Boris Johnson, confounding the notion that his charge is a buffoon and London a Labour fiefdom.

The attraction is obvious, but that doesn't diminish the significance of Crosby's appointment. In the desperate quest for a majority, the Cameroonian project to detoxify the Tory party has been paused, perhaps abandoned. Had Steve Hilton, symbol of that makeover, expired rather than just moved on from his job as director of strategy, he would be spinning in his grave.

Crosby, bluff as a barrack room sergeant with a vocabulary to match, would shed few tears about that. "He really hates Steve Hilton," said a source who observed him. "He thinks all that repositioning stuff and the Big Society doesn't work. He likes things to be simple. Pick very simple themes and go for your core vote. Anything else he sees as a waste of time."

Understand this and one begins to understand the Mail on Sunday's Exocet about Crosby and Muslims. The alleged comment has not been fully denied. Crosby and Johnson merely say that neither can recall it. Does Crosby have a problem with Muslims? There is no evidence of that.

One insider says: "He was dismissive of a lot of people. He was ruthless about who in the electorate was and wasn't worth bothering with.

"I remember him saying that if you put a headscarf around a bus, perhaps Muslims might vote for us. He was a lot keener on some minorities than others because they were more likely to go for Boris."

The idea that a credible mayor should have broadest possible appeal never seemed to cloud the thinking of a man described as "a cross between Alastair Campbell and Crocodile Dundee" by one pundit. "The thing with him," said the observer, "is he will win at all costs."

This brought him into some conflict with Guto Harri, who served Johnson as director of communications and later chief of staff. Harri, now director of communications at News International, wanted to win but couldn't share Crosby's narrow blueprint of how this was to be done. He saw Johnson as a figure to be embraced by cosmopolitan London.

By contrast, Crosby preferred to restrict Johnson's appearances and to once again concentrate on the Conservative core in the outer boroughs. Johnson won by 63,000 votes, but critics insist that with inner-London votes, the majority should and could have been bigger. Still, a win is a win.

Crosby grew up in Kadina, South Australia. The son of a cereal farmer, he read economics at the University of Adelaide, before working for a petrol company. He had a shortlived and unenjoyed attempt to get elected in Queensland, but with that sorry experience behind him, became a Liberal party official there, rising to state secretary. By 1996, he was playing a key role in the victory of John Howard.

He repeated the trick two years later when Howard won with marginal seats targeted by Crosby. The strategist guided his charge to another win in 2001 but it was achieved at high cost. Howard claimed that Afghan boat people were throwing their children into the sea – using moral blackmail to enter Australia. The myth was disproved and claims that the tactic had dragged Australian politics to a new low continue to haunt the retired prime minister – and Crosby.

A year later the strategist struck out, creating with Mark Textor their consultancy, Crosby Textor, the entity that will seek a Cameron majority. As a consultant, Crosby achieved John Howard's fourth election win in 2004.

Crosby and Textor are big wheels in Australia and beyond. They have, they say, worked "on every continent", veterans of 250 commercial campaigns in 60 countries and 750 research projects. The elections attract headlines, but sources say the act of running the campaigns doesn't make money. "It's a bit of a loss leader," said one expert. The most profitable element may be the polling activity. Crosby is expected to rely on his own number crunchers rather than those who have served the Tories to date. On the back of successful election campaigns come more commercial contracts. Textor acknowledges that the combined business is worth "in the order of magnitude" of about A$20m (£13m) globally.

"He is a very, very sharply honed campaign operative and manager," says Pamela Williams, a journalist and author who covered the 1996 Liberal campaign – Crosby's first at a national level. "They are very good at mapping out a 35-day campaign strategy in which there is a new announcement every day."

He is a master at negative advertising, she says, and brilliant at protecting his candidate. "If there's bad news, you take it full on, straight in the face, deal with it as fast as you can and then move on to try to stop it bogging you down. You make yourself as small a target as possible on the negatives. They turned that into an art form in campaign after campaign."

Justin Di Lollo, strategist for the "enemy" Labor party, bore the brunt. "The hallmarks of the Crosby campaign are negative campaigning, often around race or immigration," Di Lollo says. "It's the type of campaigning that involves really tearing at the fabric of society for shorter term political gains.

"A political party doesn't take that road unless it feels like it's in quite a lot of trouble … In politics, it is entirely possible to excite negative attitudes in the community and turn them into votes, but that can come at a terrible price. It can undermine community harmony and attitudes towards tolerance. He has probably one of the world's best capacities to utilise this sort of campaigning."

It doesn't always work. It didn't work here. There were no Afghan boat people in the then Tory leader Michael Howard's Crosby-directed campaign in 2005, but the niceties were stripped away, leaving the essentials: crime and immigration. Initially this seemed likely to pay dividends. But once the strategy was unleashed via the now infamous ads, which read: "It's not racist to impose limits on immigration … Are you thinking what we're thinking?" Crosby's approach was mocked as hamfisted. Labour won a 66-seat majority. Perhaps, it was said, the Crosby brand of politics was not for Britain.

The Australian brought to Johnson's 2008 mayoral campaign the smack of firm management, an ideal arrangement for Johnson – for whom politics is essentially performance. Johnson, initially the long-shot candidate, beat Ken Livingstone by almost 6%. It was classic Crosby in several respects. He assumed total control. "It's his way or the highway," recalls a source close to the campaign.

Sonia Purnell, author of the biography Just Boris, tells how Crosby and the grandee Lord Marland took Johnson to dinner at the London restaurant Quirinale. Her account, never denied, has them issuing the warning: "Don't let us down or we'll cut your fucking knees off."

Allegations suggesting waste and chaos at Livingstone's London Development Agency were filtered out by the Tories to the BBC and the Evening Standard, a Johnson flag-waver. The Standard used them to destructive effect. Labour was overwhelmed.

Crosby repeated the trick in 2012. Again, the iron hand on the tiller, concentration on the outer ring, and careful curbs on his candidate, protecting him from territory that might prove hostile as well as journalists who might cause problems. Tony Travers of the London School of Economics says it showed: "You could see the strain on Boris trying to be serious."

Once again, there was the attack story – contested allegations that Livingstone had avoided taxes hit the former mayor from out of nowhere. That plan had been long in gestation. "They were looking at Ken's tax about a year before the election," says a strategist. "They thought it would be helpful. They didn't know the impact it would have." Labour tried to shine similar light on Johnson, but it didn't work. "Boris knew he had to have his tax in order because he knew they were going to go for Ken."

This was Crosby in his prime. "He would have a crack of dawn meeting every day," says a volunteer. "He would get the grid out, work out what we were doing and what the anti Ken bus – NotKenAgain – was doing. He spent as much time thinking about the NotKenAgain bus as our own.

"He was brilliant at motivating the troops and would make great passionate speeches." Star campaigners would be identified – "bloody fantastic job". Slackers got a rollicking.

Crosby even found time for opponents who posed no threat, counselling the Green party mayoral candidate, Jenny Jones, that she was doing well but could profitably tack left – perhaps a further ruse to destabilise Livingstone. She ignored the advice. Still she was intrigued. "I said to him, 'you're good; perhaps you should work for the Green party.' He said: 'You couldn't afford me.'"

His professional capabilities are loved and loathed, but relatively little is known about the man. We know from past interviews that he considers his grandchildren and his "normal family life" his biggest achievement. But who he is isn't usually open for discussion. Textor, his friend and business partner, probably knows him best.

"We're great mates," he says, speaking from Australia. "He and I share the same value system which says that it is OK to have robust conversations [and] you let the electorate decide. You do not decide for them. Be civil to all those you meet personally. It's a very Australian conservative values system. Australians are very egalitarian. You treat people fairly, maybe not always with kid gloves, but fairly. You have a mindset without prejudice: very firm but fair."

Crosby is in his mid 50s and married to Dawn, a Liberal party official in the 1990s. He has two adult daughters and, according to Textor, loves the theatre, a good steak and a glass of pinot. "He was quite addicted to a restaurant in Canberra called the Ottoman which served great Turkish food. He was addicted to that food. I think the lemon veal was his favourite … I couldn't get him out of the place so I'm glad he left because he could have died in there."

David Cameron's decision to hire Crosby, initially on a part-time basis but full time as the election draws closer, has triggered many questions. Will he have a free hand steering a fractious, habitually undisciplined Tory party? Will he be deflected by noises off from those he has supplanted, such as the chancellor, George Osborne, who ran the 2010 campaign, or Lord Ashcroft, who has been desperate to broaden the Tory base to attract minorities, northerners, women? His "Are you thinking what we're thinking" campaign in the 2005 election was one reason the Tories felt obliged to detoxify in the first place.

It is worth considering Crosby's experience with the Libertas campaign, which contested elections to the European parliament in 2009. The party was formed by the Ireland-based businessman Declan Ganley, who spearheaded the no vote in Ireland's referendum on the EU's Lisbon treaty in 2008. With Crosby's help, it fielded more than 500 candidates a year later. All but one failed.

On reflection, Ganley is generous about the new Tory strategist. "I think the guy is a star, really capable, very smart." But there were issues over strategy. Crosby urged Libertas to strike a more strident Eurosceptic note pre-election, but Libertas sought a United States of Europe with an elected presidency. Ganley held his principled line, lost his project but retained his political integrity. Would Cameron do the same?

Potted profile

Name Lynton Crosby

Born Kadina, South Australia

Career Political strategist

High point Four election victories in Australia for John Howard; two in London for Boris Johnson

Low point Derided 2005 Tory campaign for Michael Howard

What he says "Winner"

What they say "Dog-whistler"

Additional reporting: Alison Rourke in Sydney, Hélène Mulholland in London and Henry McDonald in Dublin