Mainstream political parties must tackle far-right groups through doorstep "hearts and minds" campaigns that tackle anti-Muslim sentiments at local level, according to two reports on challenging extremists.

The rise of career politicians, and the fall of grassroots activists, has left a vacuum across Europe for populist anti-establishment groups, warns the paper Right Response, published by the international affairs thinktank Chatham House. These groups now tapped into voters' feeling of being "disenfranchised" by out-of-touch politicians.

Muslims are increasingly the focus of anti-immigration and anti-minority group activity, says the report, and it means growing public hostility to settled Muslim communities.

It says the government's Prevent strategy should no longer be seen as offering alternatives to those who might be tempted into terrorism by al-Qaida and like-minded groups, but that the strategy should also tackle rightwing extremism.

Matthew Goodwin, the author of Right Response, said mainstream parties had become increasingly professional and managerial, concentrating on marketing techniques and relying on computer-generated canvass returns, tightly-scripted phone banks, focus groups and opinion polls, rather than on personal contact, except at election time. Extreme parties often had more innovative websites.

"Politics is about winning the hearts and minds of voters, not seeking to win arguments on intellectual grounds," said Goodwin, an associate fellow of Chatham House and lecturer in politics and international relations at Nottingham University. "To do this, mainstream parties should be part of the community, have an active and visible presence, and forge stronger links to local groups and forums. In practical terms, this means standing full slates of candidates at the local level, engaging with voters face-to-face and redirecting some resources to revitalising grassroots campaigns."

The rise of extreme parties was not only linked to anxiety over threats to jobs, social housing and the welfare state posed by immigrants, said Goodwin. Mainstream parties needed to challenge more forcefully claims of national cultures being under attack; that meant going beyond the economic case for immigration and arguing for cultural diversity.

Politicians also needed to be more honest. "Existing responses … typically focus on plans to reduce the number of immigrants, or curtail overall levels of immigration. Yet at the same time, international treaties have greatly reduced the capacity of governments to deliver demonstrable outcomes in this policy area." The outcome was "a disconnect" that could further fuel public dissatisfaction.

The EDL report highlights the group's use of central websites carrying its line, and sites and blogs focusing on single issues such as "no more mosques". This, combined with the strategy of "march and grow", has given the EDL a sustained culture of grassroots activism, the report says.

It adds: "Given [the] licence to violent extremism, tackling the EDL and other new far-right groups, needs to become a core focus of the Prevent strategy. Generally speaking, the nation's wider economic success impacts on the fortunes of far-right movements. Yet this needs to be understood in relation to specific localities, not merely nationally.

"Without resolving underlying economic and social tensions within areas identified with EDL and new far-right support, it is likely the movement will continue to find fertile conditions in more deprived pockets across the country … to combat this, a more relevant and empowering politics is crucial to tackling support for extreme nationalisms."

Michael Ellis, the Tory MP for Northampton North, says in a foreword to the report that he has got "every confidence" the coalition's planned revamp of the Labour-devised Prevent strategy would help combat "the rise of the new far-right, and potential for lone wolf terrorism". He adds: "One must only look at the terrible atrocity this summer in Norway at the hands of a murderous terrorist [gunman Anders Breivik] – in the name of a crazed war against Islam – to see the relevancy and currency of this report."