One of the things that became increasingly clear to me while I was coordinating the English PEN Free Expression is No Offence campaign - against the government's incitement to religious hatred legislation, which would have protected religion in the way race already is - was that free speech, the very basis of our argumentative parliamentary democracy, is hard to bear.

The free expression most people want is the kind that permits them to speak and the kind they agree with. Otherwise, like a father proud of his child until she embarrasses him with the sudden revelation of what he thinks are family secrets, the tendency is to raise a threatening hand and bark out a "Keep quiet!".

The family analogy is a useful one. Many of the coordinated attempts to silence public expression have come from faith or immigrant groups and have been directed against their own. It was young Muslims who, back in 1989, burned Salman Rushdie's The Satanic Verses. It was Sikh rioters and a critical community who managed to close Gurpreet Bhatti's play Behzti, because the rape she depicted of a girl by an elder was considered shaming. The filming of Monica Ali's Brick Lane had to move to a secret location after protests from a small group of local Bangladeshis.

Groups - who may in some way feel vulnerable - confound dissent with disloyalty. They are tolerant of neither and would prefer them prohibited or punished.

Sadly - and perhaps oddly, given a tradition which is based on constant disputation - the same spirit of intolerance, the same attempt at silencing dissenting views has infected dominant Jewish groups. Those who have openly voiced criticism of Israel's occupation of the Palestinian territories or her actions have found themselves subject to the kind of orchestrated hate-mail barrage that the BBC received from evangelical Christians over its screening of Jerry Springer - The Opera. Worse, this "hate mail" breached the privacy, let alone the civility, barrier, which public letter-writing still conforms to.

The highly respected historian, Tony Judt, author of Post War, and himself Jewish, found a lecture he was to give at the Polish Consulate in New York cancelled because of pressure exerted by Jewish groups.

He withdrew from a second lecture at Manhattan College after threats that a protest by holocaust survivors would take place, branding him as a "state of Israel denier". This slippage of meanings is one many Holocaust survivors might themselves find offensive. Denying historical facts based on overwhelming evidence is hardly tantamount to criticising a government's policy or actions, something most do on a daily basis in Europe or America. Israelis themselves have the good fortune of being able to do so vociferously in their own press.

At the risk of drawing venom, might I venture that over-sensitivity to slights, the labelling, by certain Jewish groupings, of all criticism under the catch-all of anti-semitism, may signal unvoiced recognition that something is indeed amiss. To confound Jewishness with Israel's politics is to mirror exactly what the most rabid jihadist websites do.

To demand that all Jews, whether secular or more religious in their orientation, speak with a single voice, the voice of the so-called "community" - as we saw in recent attacks on the head of the Institute for Jewish Policy Research, Antony Lerman - is a form of authoritarianism that is dangerous whatever its cloak of "faith".

That said, all sides find free expression which stands counter to their own position hard to bear. The academic and cultural boycotts of Israel, which would stop the free flow of intellectual and artistic exchange, are also moves to silence speech, to punish and to censor expression and alternate visions of the world, to block out what one doesn't want to hear. They are a mirror image of their opposite number. They forget that talk and negotiation is at the basis of all peace plans.

The importance of this new initiative is that at its heart is the recognition that a robust society needs diverse independent voices and their free expression. When the government and media, in their recently found way, have a self-elected "community" leader speak, supposedly on behalf of a group of co-religionists, they might do well to remember that there are any number of other voices from the same designated "identity" group who would prefer a secular public sphere which guarantees both the freedom of diverse religions and that freedom, fundamental to our way of life, of contesting speech.

Click here for a full list of articles in the Independent Jewish Voices debate.