The pope's visit to the UK is mired in controversy after one of Benedict XVI's senior advisers dropped out after comparing an arrival in multicultural Britain to landing "in a third-world country".

Cardinal Walter Kasper – the Vatican's leading expert on relations with the Church of England – also accused the UK of discriminating against Christians.

The cardinal's remarks, made hours before the papal party was due to land in Edinburgh this morning, came in an interview with the German news magazine Focus, in which he noted that Britain was a "secular, pluralistic" country.

The cardinal's comments on "aggressive" atheism drew an angry response from secular campaigners who said the UK did not need a "lecture" on religious freedom and belief from the Vatican – but were welcomed by some Christians.

They also came after an article today in the pope's own newspaper by Tony Blair, in which the former prime minister appealed to the Roman Catholic leadership to listen to the views of ordinary priests and churchgoers on doctrinal issues and show greater interest in ideas dismissed as secularist.

The pope's spokesman, Father Federico Lombardi, said tonight that the cardinal "had no negative intention, nor [a] lesser appreciation for the United Kingdom", but had been referring to Britain's multi-ethnic composition. He said the pope's former adviser recognised "the great values of British culture".

In a statement, Lombardi said Kasper "had meant to refer to the fact that from the moment of arrival in London airport – as happens in many big metropolises of the world today, but in London particularly because the unique role played over time by the UK's capital – you realise from the outset that you are in a country in which many human realities of the most diverse provenances and conditions meet and mingle; a crucible of today's humanity, with its diversity and problems".

Lombardi added that, in speaking about atheism, the cardinal "was obviously referring to the positions of certain well-known authors who put themselves forward particularly aggressively and dress themselves up in scientific and cultural arguments, but who do not in fact have the value they show off".

Last night the Catholic church in England and Wales distanced itself from the cardinal's comments, which a spokeswoman said "do not represent the views of the Vatican, nor those of bishops in this country". The spokeswoman added: "Clearly they are personal views … Catholics play a full part in this country's life and welcome the rich diversity of thought, culture and people so evident here. This visit marks a further development of the good relationship between the United Kingdom and the Holy See. We are confident that it will be a huge success."

A government spokesman said the cardinal was expressing his own views: "As a church spokesperson has said, his comments do not represent the views of the Vatican, nor of bishops in this country."

Despite the timing of the comments, the Vatican insisted that Kasper had withdrawn from the trip "for health reasons".

Lombardi told the Guardian that the 77-year-old prelate's absence from the four-day visit "had absolutely nothing to do with anything else". The cardinal's reference to British Airways has also revived questions about why, in a departure from the norm, the pope will not return to Rome aboard a plane from the country he has visited – normally that country's national flag carrier.

At a briefing last week, Lombardi said the decision to fly there and back with the Italian airline Alitalia had been taken for reasons of "simplicity".

He added: "In any case, British Airways is no longer state-owned." Alitalia was also privatised two years ago.

BA issued a statement today saying that Kasper had been "seriously misinformed. It is completely untrue that we discriminate against Christians or members of any faith. Allegations to the contrary made by one individual have been repeatedly rejected by the courts."

Commenting on the cardinal's remarks, Andrew Copson, chief executive of the British Humanist Association, which has opposed the papal visit, said Britain had little to learn from the Vatican.

"The UK is fully signed up to treaties like the European convention on human rights," he said. "It is not hostility to religion that characterises the negative reaction of many to the pope's state visit, but hostility to the Holy See's bigoted position on so many issues. We should be proud that – unlike the Holy See – the UK is a place of liberalism and tolerance."

Clifford Longley, from the Catholic newspaper the Tablet said Kasper was "obviously talking nonsense". Longley told the BBC: "I don't think he believes Britain is in the grip of secular atheism, and he shouldn't have said so."

Simon Woolley, a founder of Operation Black Vote, said Kasper's remarks were "shocking and ignorant".

"If he's complainting about Britain being multicultural, it's ignorant because of Britain's slave and colonial past. In reality, most black people are here because Britain was there," said Woolley. "It's shocking because it's so disparaging. The remarks are really unhelpful."

The group had voiced no particular opinion on on the Pope's visist until now.

Darcus Howe, the broadcaster and writer, said: "What's he talking about, it's crap."Others, however, praised Kasper for speaking up. "We do have to become aware of the fact that Christians are finding it increasingly difficult to live out and express faith in the public sphere," said Andrea Williams of the Christian Legal Centre, which works to protect and promote freedoms of Christians in the UK. "This aggressive secularism amounts to privatisation of faith."

The country, she said, would do well to remember the good that its Christian heritage had done "not just for Christians, but for the whole of society".

The last-minute withdrawal of the cardinal led to a flurry of activity in the Vatican. He was to have played a central role in the ecumenical aspects of the pope's visit.

Until July, Kasper was the head of the department that deals with relations with other Christian denominations, where he had worked since 1999.His successor, a Swiss archbishop, Kurt Koch, speaks English, but he is not as fluent as Kasper. He also has far less experience of dealing with what Kasper in his Focus interview called the "difficult dialogue" with the Church of England.

This is not the first time Kasper has invited controversy. Last year, after the pope lifted the excommunication of an ultra-traditional British bishop who had questioned the extent of the Holocaust, Kasper raised eyebrows in the Vatican with an interview in which he criticised a lack of consultation and said there had been "misunderstandings and management errors" in the papal bureaucracy.

Tony Blair's front-page article in L'Osservatore Romano, the Vatican's semi-official daily, is also likely to focus attention on the Catholic hierarchy's attitude to secularism. Although Blair specifically excluded the pope from the scope of accusations, his piece marked a rare instance of the church leadership being upbraided on its own turf.

Blair praised the writings of Cardinal John Henry Newman, an earlier notable convert whom Benedict will beatify at a service in Birmingham on Sunday.

In particular, he singled out Newman's idea that church doctrine was constantly developing and acknowledged that church teaching was about defining what constituted "true" development.

But, he added: "Newman defined the consent of the entire 'body of the faithful' on doctrinal questions as '[the] voice of the infallible church'. I ask myself whether this voice is still taken seriously enough, or if we have fully understood the implications of these ideas."

The former prime minister, whose Faith Foundation aims to promote understanding between religions, added: "The tendency of some religious leaders to put a great number of different ideas into a single envelope with the label 'secularism' and then consider it as something sinister creates divisions in pluralistic societies. This rules out for the Church the possibility of new developments in its thinking."

Taken in isolation, his comment might be interpreted as a rebuke to Benedict, who has frequently inveighed against the secularisation of modern Europe. But Blair said: "The pope's dialogues with important secular thinkers [represent] … a very different example."

It was announced in May that Benedict had ordered the setting up of a foundation to reach out to atheists and agnostics. The Vatican hopes to stage a series of debates in Paris next year. But a senior official said the Catholic leadership was only interested in engaging with "noble atheism or agnosticism, not the polemical kind".

Benedict's visit, the first state trip to the UK by a pope, has attracted considerable criticism. Today, more than 50 public figures signed a letter to the Guardian arguing that the pope should not be given the "honour" of a state visit. The signatories, including Stephen Fry, Terry Pratchett, Philip Pullman and Richard Dawkins call for "Pope Ratzinger" to be stripped of the right because of the Vatican's record on gay rights, abortion and birth control.