The distaste of eminent writers such as Peter Carey at PEN American Center’s decision to honour Charlie Hebdo at its gala next month, highlights once again the fundamental inconsistency that underpins attitudes towards free speech. Within days of the "Je Suis Charlie" outpouring of solidarity, French police were arresting citizens for glorifying terrorism, while David Cameron’s government was busy pushing a counter terrorism bill through Parliament that would severely curtail universities’ liberty as a forum for freedom of expression. We may be used to the double standards of politicians, but what about writers?

Becoming a member of PEN (one of the oldest human rights organisations in the world, and the largest international community of writers) means pledging "to oppose any form of suppression of freedom of expression in the country and community to which [writers] belong, as well as throughout the world wherever this is possible". That’s a sentiment one would hope any writer might be happy to support and PEN depends, on a regular basis, on its members speaking out and standing up for fellow writers at risk. That’s the source of its influence and moral authority.

One of the writers, Francine Prose, a former PEN President, who decided to withdraw from the gala in protest, was reported as saying that giving an award signified "admiration and respect" for the winner’s work. "I couldn’t imagine being in the audience when they have a standing ovation for Charlie Hebdo." But Charlie Hebdo is in fact being recognised for its courage: the courage to publish in the face of threats and intimidation, and the courage to continue publishing after the shocking murders in January.

We are more used to seeing that courage at a greater distance – in Mexico, Russia, Bangladesh or Egypt – and feel safe celebrating writers and journalists who may be prosecuted for outraging public morals in their own culture. On our own doorstep, when faced with a satirical publication that provokes and offends, there is an underlying view implicit in the protest of Peter Carey and fellow writers that this kind of speech is not worth defending. Carey questioned whether it even was a freedom of expression issue; the writer Deborah Eisenberg voiced concerns (as have many others) about Charlie Hebdo’s "denigrating portrayals of Muslims". Yet one of the most important, if uncomfortable, responsibilities for any free speech advocate is to defend the right to express speech which may be shocking, disturbing or offensive. Without that broad defence, the limits of everyone’s speech, as well as writers and publishers, are at risk of being restricted to suit the political agenda or prevailing morality, at a cost to artistic licence as well as individual freedom.

Most of the great free speech battles in history have been fought over issues that were not deemed deserving of defence. The subjects of the famous obscenity prosecutions of the 70s in the UK (the Oz trial or Linda Lovelace’s memoir) were seen as publications of no merit. But what was at stake, as in the case of Charlie Hebdo, was the principle: the freedom to publish and the freedom to write. A freedom on which all writers depend. Victory in court (in the face of moral outrage) led to greater freedom for publishers and writers. In one of his last interviews, the writer and barrister John Mortimer, who famously defended both the Oz and the Lovelace trials, spoke of the retreat from "the abiding principle ... that you lived in a country where you could read anything you like". The growth of the idea that we should at all costs avoid causing offence (and that this may be even more important than defending the right to free speech) continues to undermine that principled protection for freedom of expression.

Salman Rushdie, a notable supporter of English PEN and the PEN America Center, who has excoriated the withdrawal of Carey and others from the gala, was similarly criticised 26 years ago for causing gratuitous offence (by fellow writers) after the fatwa. Roald Dahl even called him a dangerous opportunist. There may be "good faith differences of opinion within our community" as PEN American Center generously acknowledged on Sunday but it’s in the interests of all writers to stand up for the principle.