Hazel Blears spoke about the need to tackle non-violent extremism in Britain's Muslim community in December. She was correct to do so. The government and British Muslim communities need to work together to tackle not only those who directly advocate violence but also those who spread the intolerant ideologies that make such terrorism possible. However, the right must also confront its own non-violent extremists. Just as many Islamists see non-Muslims as an immoral and brutish mob, so many leading figures on the right have routinely demonised Muslims collectively, portrayed the most reactionary interpretations of Islam as being typical and depicted Muslims as a faceless, monolithic bloc whose very existence threatens the foundations of western society.

The most prominent of the right's "non-violent extremists" is Mark Steyn, formerly a regular writer for the Telegraph and the Spectator. In his book America Alone, he compared Muslim immigration with an invasion, writing that "a fearless Muslim advance has penetrated far deeper into Europe than Abd al-Rahman" and dispensing entirely with an Islam/Islamism distinction to write that "the religion [of Islam] itself is a political project – and in fact an imperial project". For Steyn, the consequences of this are clear: a European Union that "will be well on its way to majority Muslim by 2035". In summer 2008, Steyn spoke at a conference on "libel tourism", organised by Douglas Murray (of whom more later), where he launched a fantastical diatribe against Muslims in which he compared himself with the heroes of the B-movie Tremors who are pursued by giant carnivorous worms. In the audience were journalists, professional lobbyists, politicians and other key opinion formers. Alarmingly, many were laughing at his "jokes".

Another of the right's prominent rabble-rousers is Melanie Phillips, a writer for the Daily Mail and the Spectator. Although Phillips generally manages to differentiate Muslims from Islamists, this is usually obscured by her increasingly crazed rhetoric. For example, in one article she wrote of "the steadily rising number of Muslims coming to settle in Britain who, refusing to assimilate, are steadily changing its demographic, cultural, and political identities". Elsewhere in the article, she warned of the "steady Islamisation of British public space", telling her readers that a "war of Islamic conquest is being waged against the west". Her paranoia knows few bounds. On one occasion, she predicted that Scotland's 40,000 Muslims (less than 1% of the population) could create a "Caledonian Caliphate" that would amount to an "Islamised country on England's border". More recently, she has begun labouring under the delusion that any Muslims who do not unequivocally support Israel are closet anti-western Islamists.

Other icons of the right have similarly attacked Muslims collectively, dangerously blurring the lines between Muslims, Islam and Islamism. Rod Liddle, for example, wrote that "Islam is largely to blame for the viciousness which is periodically unleashed upon us all in the form of bombings – that it is the credo, rather than the individual, which is principally to blame". On another occasion, he said that "Islam is masochistic, homophobic and a totalitarian regime. It is a fascistic, bigoted and medieval religion." If he is right, where does that leave those who believe that Islam can be a liberal, tolerant faith? Do they have any chance of succeeding? Or are they doomed to failure?

Finally, there is, of course, Douglas Murray, "Britain's only neoconservative", who has often failed to distinguish Islam from Islamism. In just one speech, for example, Murray referred to the "violence, intimidation and thuggery of Islam" and "the problem of Islam". Like Steyn, Murray has also represented Muslims as a collective threat, referring ominously to the "demographic time-bomb which will soon see a number of our largest cities fall to Muslim majorities". He concluded that "conditions for Muslims in Europe must be made harder across the board" – a phrase that could easily be interpreted as a call for the collective punishment of Muslims.

At this point I must add that until recently I worked with Murray at his Centre for Social Cohesion, which I joined because, in mid-2007, few other thinktanks were willing to seriously address the problem of Islamism at all. My time there was a constant struggle to "de-radicalise" Murray and to ensure that the centre's output targeted only Islamists – and not Muslims as a whole. This October, however, I had finally had enough of this constant battle and resigned. To his credit, Murray has privately retracted many of his more noxious comments – but he apparently lacks the courage to do so publicly.

There is more than political correctness at stake here. Failure to distinguish adequately between Islam and Islamism, and between Islamists and ordinary Muslims, has important consequences. It plays into the hands of Islamists by accepting their own narrative that their politicised understanding of Islam represents the "true" Islam. It can also lead non-Muslims to assume that all Muslims harbour – perhaps secretly – the totalitarian aspirations of Islamism. Even more troubling are the implications of Steyn's argument that all Muslims – by mere virtue of their existing and giving birth to other Muslims – pose an existential threat to western civilisation. This risks encouraging other Britons to see all Muslims as the enemy – regardless of their individual qualities. In the past, such blanket demonisation of entire peoples has ended in genocide.

The need for the right to rein in its extremists is growing urgent. There are increasing signs that such hate-preachers are close to inadvertently producing terrorists of their own – just as Islamists have done for years. In recent months, alarming numbers of white British nationalists have been jailed for terrorism or put on trial for planning a bombing campaign against a mixed race couple. In December, a Grimsby man was found guilty of planning a violent campaign against a local Muslim man and his wife. In 2007, Robert Cottage was jailed for stashing explosives in preparation for the racial war that he believed was imminent. Is it unreasonable to believe that such people have been influenced by the relentless paranoia offered by Phillips, Steyn and others? Just as resolving the present Arab-Israeli crisis means recognising wrongs committed by both sides, we can only build a successful multi-faith society on a basis of true equality and equal citizenship. And that means that the right must tackle its own extremists – just as British Muslims are now standing up to the extremists in their communities.