Ukip leader Nigel Farage has accepted an invitation to address a rally of the Thatcherite Bruges Group at the Tory party conference in Manchester in September, as he steps up attempts to poach Conservative MPs and members who want to leave the EU.

In a move that will alarm a Tory high command already worried about desertions, Farage has been booked to speak at one of the conference's biggest and grandest venues, the Great Hall in Manchester town hall, on "Conservatives and Ukip: Allies or Enemies?"

The event, certain to be a key attraction of conference week, will have greater poignancy as it will be the first conference gathering of the Bruges Group since the death of its former president, the late Baroness Thatcher. The group was set up in 1989 to promote the idea of a less-centralised European community and to commemorate Thatcher's Bruges speech of 1988, in which she warned of a new "European super-state exercising a new dominance from Brussels".

Farage has been "showered with invites" from Tories running events on the conference fringe. "I suppose my presence will be a bit controversial, but whatever happens I am sure most people in the audiences will agree with my policy positions," he said. He will also speak at events run by the Freedom Association and the Freedom Organisation for the Right to Enjoy Smoking Tobacco.

Lord Tebbit, who succeeded Thatcher as president of the Bruges Group, said he was relaxed about inviting the leader of a rival party. "It is up to the Bruges Group to invite anyone to speak who is of a Eurosceptic persuasion," Tebbit said.

It has been claimed that Ukip has been in talks with about eight Tory MPs about them defecting to the party, which advocates immediate withdrawal from the EU. A series of clandestine lunches has been held by Stuart Wheeler, a former Tory donor who is now trying to persuade MPs to jump ship.

Ukip has been enjoying a strong run in the polls since coming second to the Liberal Democrats in the Eastleigh byelection in February and performing well in the May local elections. Conservatives now fear that it could emerge as the largest party in next May's European elections.

In an Opinium/Observer poll, Farage's party is down 3% but is still polling strongly on 16%, double the level of support for the Lib Dems, who are up 2% on a fortnight ago, at 8%. Labour, up 1% at 39%, has an 11% lead over the Tories, who are up 1% on 28%.

Robert Oulds, director of the Bruges Group, said: "We have to start to have a debate and to understand each other and to find out if there is some way we can co-operate."