William Hague had little choice. The hand of the Iranian regime was pretty visible in the ransacking of both the British embassy compound and the northern Tehran UK staff residences. Safety required the pulling out of British diplomats, and diplomatic pride required that he demand the expulsion of their Iranian counterparts in London.

But while right may seem to have been done, these tit-for-tat expulsions rarely end well for either side. Broader objectives get put aside as time-consuming negotiations to restore full diplomatic ties displace more serious business. Without embassies the basic function of diplomacy – keeping some kind of dialogue going even when views are diametrically opposed – is essentially suspended.

Until now Britain has been a ringleader of efforts to squeeze Iran into compliance with international restrictions on its nuclear programme. Without an embassy it actually becomes a bystander, as it will quite quickly know less about what is going on in Iran than others at the table. Both as a minister and before that at the UN, I always noticed that Iran was an issue on which the US and others tended to defer to British views because, unlike Washington, we had our embassy, our own listening post. We were better informed.

But that privileged position also carried the seeds of its own undoing as it fed a real suspicion of Britain in Iran. We were seen as the Americans' proxy, doing their business – and in the eyes of many Iranians, particularly in the regime, no doubt spying for them. These suspicions were particularly toxic because of a long history of perceived British perfidy. Iranians have long nursed the conspiracy theory, for example, that tunnels snake out from the British embassy deep into Tehran, along which British spies ply their trade.

More troublingly, you do not have to be an Iranian with a persecution complex to concede that Britain has not used its privileged knowledge of Iran to particularly good effect. It has been a loyal camp follower of a narrow American diplomacy that, other than briefly at the beginning of the Obama administration, has been consumed by the nuclear issue at the expense of a broader view of Iran's agenda – or rather, its anxieties.

Iran's preoccupation with its own security and relations with what it sees as the threats of the US, Israel and Saudi Arabia have always offered the prospect of a wider canvas on which to provide guarantees against outside interference, in return for curbing Iran's nuclear and conventional armament programme. The real value of a more imaginative diplomacy of this kind would have been to remove the prop that has kept this unpopular regime going: the threat of foreign intervention.

Such a diplomatic overture might have come from a country such as the UK, with its historic links and understanding of Iran combined with its influence in Washington. It would have needed all Britain's diplomatic skill and patience because as negotiating partners Iran's leaders are as unreliable as any.

But long before we closed our embassy in Tehran, we had anyway lost the opportunity of any such role, as we were seen as being in the Americans' lap. And so the decision to step up banking sanctions, which provoked the attacks on Tuesday, represented the latest step in a deepening confrontation, so perhaps the embassies were redundant anyway. This was already a showdown.

The good news is sanctions are working and putting great pressure on the regime. They are also, typically, biting at the less well off, thereby causing resentment that was reflected in the protests in Tehran. But the bigger problem, which cannot be allowed to fester, is that Iran's fresh progress towards nuclear enrichment is provoking talk of war in Israel – at least among government hawks, if not more prudent generals.

In the meantime, for all its other difficulties, Iran enjoys a better ally in a US- and UK-liberated Iraq than it ever did in Saddam Hussein's years. These are developments that Saudi Arabia and other Arab neighbours cannot let go indefinitely unchecked. So the region lurches towards instability and possible conflict as the west desperately ups the ante on sanctions, hoping this can break the regime and avert conflict.

It will be no comfort for British diplomats that they will now not be granted front-row seats in Tehran for what follows. And given the real dangers, that is all our loss.