The killers of Drummer Lee Rigby are believed to have worshipped at a mosque whose imam was once recorded apparently urging British university students to wage jihad in Palestine.

Whitehall sources have told the Telegraph that Michael Adebolajo and Michael Adebowale had links with the Lewisham Islamic Centre in south-east London, less than six miles from the Woolwich street where they murdered the soldier.

The mosque’s imam, Shakeel Begg, was caught on tape by the Sunday Times in 2006 apparently telling Kingston University students to “take some money and go to Palestine and fight, fight the terrorists, fight the Zionists”.

A senior Whitehall source insisted the centre had a large number of worshippers that were of no concern to the authorities.

But the source added: “It does attract a radical crowd and radical speakers and has its fair share of converts. From that perspective it is significant.

“Adebolajo and Adebowale did go there, and anywhere that attracts extremists is of interest.”

The Lewisham Islamic Centre said it was “feasible” that Adebolajo and Adebowale may have attended on occasion because it was a public place of worship open to everyone.

However, a spokesman for the mosque said the killers did not take part in any meetings or classes there, adding: "Firstly it should be noted that we do not allow any closed meetings/classes to take place at the Centre. For the avoidance of doubt, we can confirm that these individuals did not attend any meetings or classes.

"Secondly, the Centre is a public place open to all like any other mosque or place of worship and it does not exclude any member of the public from entering therein.

"It is therefore feasible that the individuals may have attended the Centre as they were also likely to have done with many mosques in and around London. Likewise, it is equally feasible that the individuals also attended churches at some point in their lives before they had any association with Islam."

The Lewisham Islamic Centre claimed Mr Begg had been misquoted and suggested that he had in fact been talking about the “illegitimate nature” of terrorist activities.

The mosque, which was set up in the late 1970s, is a registered charity with its own primary school and after-school academy for young Muslims.

One of its former trustees, Dr Ijaz Mian, was filmed in an undercover Channel 4 Dispatches programme broadcast in 2007 apparently telling worshippers at Sparkbrook Mosque in Birmingham: “You cannot accept the rule of the kuffir [non-Muslims]. We have to rule ourselves and we have to rule the others.”

Adebolajo and Adebowale are believed to have moved on to the centre after years of close links with the now-banned al-Muhajiroun group, founded by Omar Bakri Mohammed and later run by Anjem Choudary.

Bakri claimed to have converted Adebolajo to Islam and Choudary said he first met him in 2005, but claimed not to have seen him in the last two years.

Choudary said he could not remember Adebowale but Bakri disputed that.

One in five terrorists convicted in Britain over more than a decade were either members of or had previous links to al-Muhajiroun before it was banned in 2010, according to the Henry Jackson Society, a counter-extremism think tank.

Others linked to the group included Omar Khyam, who was jailed life in 2007 for leading the “fertiliser bomb” plot, which was to target London nightclubs and other public buildings.

Whitehall sources said people would join the group “slightly radicalised” and leave it “very radicalised”.

Bakri helped organise a seminar after the September 11 attacks in favour of the "Magnificent 19" and went on to call the July 7 bombers the "Fantastic Four".

Speaking from Beirut, where he now lives in exile, Bakri said of Adebolajo: "Because he is a convert I can still remember him. At that time there were a lot of conflicts around the world, and in Iraq and in Afghanistan especially. We talked to him about these and he sympathised with the Muslim people, it seemed. He was a quiet boy who didn't ask many questions. He used to come to our open talks and speeches."

Choudhry said: "He was interested in Islam, in memorising the Koran."

Adebolajo became more extreme and took part of numerous Islamist protests around London.

In 2006 he was arrested outside the Old Bailey during a violent demonstration surrounding the trial of fellow fanatics for soliciting murder and inciting racial hatred during a protest against the publication of a cartoon of the prophet Mohammed by a Danish newspaper.

In 2007 he was filmed by the BBC protesting outside Paddington Green police station following the arrest of another fanatic. He is seen holding a placard which complains of a "Crusade Against Muslims".

And in 2009, a video shows him ranting on a platform in front of young Muslims outside a mosque in north London.

He is heard saying: "Do not be scared of the filthy kuffar. They are pigs."

Adebolajo and Adebowale are both also said to have had links with Usman Ali, an alleged extremist and former member of Al – Muhajiroun, and Khalid Fikry, an extremist cleric and supporter of convicted terrorists.

Adebolajo and Adebowale attended a prayer group at the Glyndon community centre in Plumstead, south – east London, organised by Usman Ali. He was banned for life from the Greenwich Islamic Centre in 2007 after the mosque's trustees won an injunction against him.

After Mr Rigby’s murder, Mr Ali said in a statement that he had not been involved in any group since leaving al-Muhajiroun in 2004.

He added: “I would not and have never encouraged Muslims to commit such abhorrent acts, and I utterly denounce these acts.”

One close associate of Adebolajo was jailed in 2008 for inciting terrorism abroad, alongside Abu Izzadeen, another Muslim convert who was radicalised by Abu Hamza, the hook – handed cleric awaiting trial for terrorism offences in America.

Others in their circle were said to include Richard Dart, who was jailed earlier this year over a plot to target soldiers' funeral cortèges at Royal Wootton Bassett.

Adebolajo is also understood to have attended talks by preacher Mohammed Hamid, who called himself Osama bin London, and visited a bookstall run by Hamid in Marble Arch, central London.

Hamid, a 55-year-old former crack addict, was jailed on 2008 for "grooming" young Muslim men for jihad.