Salman Rushdie has called on Indians to wake up and fight for free speech because "voices are being silenced" and "a combination of religious fanaticism, political opportunism and public apathy is damaging that freedom upon which all other freedoms depend: the freedom of expression".

Rushdie was speaking last week at a conference in Delhi. The author had replaced Pakistani cricketer turned politician Imran Khan as lead speaker at the India Today Conclave, after Khan pulled out, citing the "immeasurable hurt" which Rushdie's novel The Satanic Verses had caused to Muslims, and saying he would not appear on a programme which included Rushdie.

After a thorough demolition of Khan for his comments – "In the real world, I would say 'immeasurable hurt' is caused to the way in which Muslims are seen by the terrorists based in Pakistan, who act in the name of Islam, including those who attack this country from Pakistan, backed by the Lashkar-e-Taiba, with whom Imran Khan now wants India to sit down and talk" – Rushdie challenged the politician to a debate over the literary merits of The Satanic Verses. "[It's] a book which I would be willing to place a substantial bet that Imran Khan has not read," said Rushdie. "Back in the day when he was a playboy in London, the most common nickname for him in the London circles was 'Im the dim'. The force of intellect which earned him that nickname is now placed at the service of his people, and its enemy, it seems, is my book. If Imran really wants to argue about the literary merits of The Satanic Verses, I am happy to meet him in a debate on that subject anywhere and any time. Well, maybe not anywhere."

He went on to make an impassioned plea for the importance of free speech to his audience. "Who would be against freedom? It's a word everyone would automatically be 'for', one would think. A free society is one in which a thousand flowers bloom, in which a thousand and one voices speak. And what a simple and grand idea that seems. It's like that copper goddess standing in the harbour, enlightening the world. But in our time, many essential freedoms are in danger of defeat, and not only in totalitarian or authoritarian states," said Rushdie in his speech, now posted online in full by the conference's organisers.

Earlier this year, the author was prevented from attending – or even speaking via video link – at a literary festival in Jaipur over fears of violent protests. "In the debacle of Jaipur, it was suggested that for me to turn up at all was wrong," he said in Delhi. "This is a case of the world turned upside down. What's happening here, tonight, is what I would call 'normal'. A writer of Indian birth, who loves this country, who has spent much of his life writing about it, shows up to talk to an Indian audience about India. I would call that normal. What is abnormal is for that to be prevented. And we seem to be in danger of getting this upside down."

Following the cancellation of the Rushdie event in Jaipur, a group of authors read extracts from The Satanic Verses aloud to an audience, and could now be faced with legal action. "I was extremely shocked at Jaipur when writers who stood up for my work were not defended by the literary festival, and remain, at this point, still in danger of prosecution for what that they did. This in spite of the fact that people have read from The Satanic Verses in India many times since the book came out in 1988, without any question of prosecuting them. And despite the fact that the only prohibition for The Satanic Verses is a customs ban; a typically Indian piece of sleight of hand. Don't actually ban the book; just stop it coming into the country. In theory, somebody could print the novel in this country. There is no law against it. Any publishers interested? See me later," said Rushdie in Delhi. "You can download the book. This ban is an absurdity in the electronic age. And yet it exists and there are four writers who may be in danger of prosecution for having read aloud from the novel, while the men who threatened violence go free."

Voices, said the author "are being silenced. Publishers are more frightened to publish. Galleries are more afraid to display certain kind of art; certain kind of films would not be made that might have been made 15-20 years ago. The chilling effect of violence is very real and it is growing in this country."

He called on the Indian public to rouse itself from apathy over attacks on free speech."People here are asleep, I think. Very largely asleep to what's going on and you need to wake up," Rushdie said. "There is a line in my novel Shalimar the Clown in which one character says to another, 'Freedom is not a tea party, India. Freedom is a war.' You keep the freedoms that you fight for; you lose the freedoms that you neglect. Freedom is something that somebody's always trying to take away from you. And if you don't defend it, you will lose it."

India, concluded Rushdie, "deserves to be led better than it's being led", to be returned "to the non-sectarian, non-communal land which the nation's founders envisaged". And this can be achieved only if all of us "have the ability to speak our minds. To speak freely without fears of religious or governmental reprisals".

The human being, said the Booker prize-winning novelist, "is essentially a language animal. We are a creature which has always used language to express our most profound feelings and we are nothing without our language. The attempt to silence our tongue is not only censorship. It's also an existential crime about the kind of species that we are. We are a species which requires to speak, and we must not be silenced. Language itself is a liberty and please, do not let the battle for this liberty be lost."