The media storm created by Ken Livingstone's remarks to the London Evening Standard reporter Oliver Finegold finally seems to be abating. I empathise with those who feel that it is time to "move on". However for me, as the elected leader of the British Jewish community, moving on is not that simple. I wish it were.

The subsequent column inches have obscured the nature of the original remarks. It is worth remembering exactly what happened. After a reporter clearly identified himself as Jewish, the mayor responded with a remark deliberately calibrated to cause maximum pain and offence.

Ken has stubbornly refused to apologise for the hurt he caused, despite being called upon to do so by just about everybody in public life. This has highlighted the almost total lack of accountability of the elected mayor to the London assembly. If we are to move on, then Ken must also move on, beyond what seems to be a blind spot in his dealings with the Jewish community.

Ken is sincere when he states that he regards the Holocaust as the worst crime of the last century. Using the Holocaust as a moral reference point can be a force for good, but the mayor's record in this regard is lamentable. His Holocaust comparisons are premised upon his own political prejudices, rather than any objective study of human suffering and deliberate industrialised mass murder.

He has used them as reference points not just for Daily Mail group journalists, but also for international capitalism, Britain's record in Ireland and, in 1987, Camden council's housing policy for homosexuals. On this occasion he went one step further. By employing Holocaust imagery to address an insult to somebody who identified himself as a Jew, the mayor overstepped the bounds of acceptable behaviour for an elected official. That is the core of our complaint.

The hurt was compounded by his subsequent refusals to apologise for the offence and furore that inevitably followed. What is the option for the Jewish community - to remain silent when the Holocaust is trivialised before the last survivors are even dead?

We are repeatedly told that Ken is a steadfast opponent of anti-semitism. He certainly hates the swastika and jackboots variety, because that fits his world-view. But the sad truth is that there are other, more modern variants that he simply ignores.

His recent mayoral submission to the House of Commons home affairs select committee inquiry into terrorism and community relations mentions many forms of prejudice but makes no reference to anti-semitism - despite two previous Jewish communal appeals to him regarding the wave of anti-semitic violence and terrorism that Jews worldwide have endured since 2000. Similarly, the mayor has made no mention of this month's statistics that showed that the British Jewish community suffered a 41% escalation in anti-semitic incidents during 2004. Is this another example of the mayoral blind spot?

Ken has quoted polls to claim the support of Londoners, as if we should take a perverse comfort that debating Holocaust metaphors has become this week's media hype. In any case, many polls have told the opposite story. But the issue runs beyond public opinion. As mayor of London, Ken is obliged to represent all faith and minority communities in the capital. As the CRE and others have suggested, this incident makes it very difficult for him to do just that.

There is deep anger in the Jewish community, not just at the remarks themselves, but at the manner in which the mayor has dragged us through this controversy, continually ascribing our legitimate concerns to media campaigns and attempts to secure political gain.

The mayor's carefully crafted statements and rebuttals throughout this episode have had another effect. They have made the Jewish community feel tense, isolated, humiliated and a target for insults. Members of our community have received hate mail and been accused of hijacking the media as a result of expressing our concerns. Most shocking of all, the Guardian reported an an allegation made in City Hall about a Jewish primary school pupil being called a Nazi by another child. When challenged by his teacher the child claimed to be following the example set by the mayor.

This is the real danger of simply moving on. Sixty years after the liberation of Auschwitz, what is it we are being asked to move on from - an offensive remark by a leading public figure, or our apparently futile attempts to preserve the memory of the Holocaust and the safety of future generations of Jews and all mankind?

Without appropriate closure this saga will leave a lingering stain on Ken Livingstone's tenure as mayor. He has let us down. He has let his office down. Oliver Finegold deserves an apology, Holocaust survivors deserve an apology, the Jewish community deserves an apology and most of all, London deserves an apology. Then we can all move on.

· Henry Grunwald QC is president of the Board of Deputies of British Jews