“Let's be straight about this,” Mr Bailey said. “The Conservatives haven’t been brilliant around race. I’m not telling you it’s a whole big lie and they’re spot on. No, they’ve had questions to ask and many of the questions they haven’t answered.”

The Telegraph in May disclosed that Mr Bailey lost his job as a Downing Street special adviser before being moved to a part-time role in the Cabinet Office. Mr Bailey at the time told friends that he had been excluded from Mr Cameron’s inner circle because he was “different”.

He now works as the Government’s “youth and engagement champion” and is based in the Cabinet Office.

Related Articles

Mr Bailey yesterday said that Conservatives need to in future be “more welcoming”.

“The real thing is people right down to the constituency level, to the activist level, to the people delivering leaflets, [need] to be more welcoming.

“We need to speak about race. Our weakness is we don’t talk about it. If we don’t talk about it, the other side get to tell everyone what you believe about it. We need not to be afraid about it.”

Mr Bailey still has hopes of becoming an MP and plans to stand in the 2015 election.

He added: “If someone said to me why I am so interested in politics and being an MP and all that, I want to show you can be of my race and my class and be a front bench member, a back bench member, a fully paid up member of the Conservative party. I think for our social progress it’s very important.”

Mr Bailey said that there is “a lot” of racism in the UK.

“You are in a fairly racist country,” he said. “It’s a fact. It’s not as racist as America, but there’s a lot of racism in this country. That is a fact.”

Mr Bailey's appointment after the 2010 general election was lauded as a sign of the inclusive nature of Mr Cameron’s office, a view which took on particular importance after the summer riots of 2011.

However, in January he was moved quietly to the Cabinet Office. His appointment to the new position was not publicly announced.

A Tory source said: "The Conservative party's policies reflect the values of hard work, strong families, opportunity, aspiration and responsibility - the values that run through BME communities."