Make no mistake, the latest rift in Australia-Indonesia diplomatic ties over allegations of spying on Indonesia’s president and his close circle is serious. But it’s one moment in a relationship that has had many ups and downs.

A cursory glance at our recent history underscores this point. After Australia’s 1999 involvement in Timor-Leste, a 1995 security agreement between Australia and Indonesia was torn up and ministerial meetings cancelled. Diplomatic relations were frayed but eventually restored.

The Bali bombing in 2002 further complicated Australian public perceptions of Indonesia, but it opened the door for co-operation on counter-terrorism between law enforcement agencies of the two countries. Enhanced military ties followed with the recommencement of special forces co-operation – suspended after Timor – in 2003.

If you needed further evidence of the relationship’s sometime fragility, but ultimate durability, look no further than developments in 2006. Earlier that year, Indonesia recalled its ambassador to Australia in light of the Howard government’s decision to grant 42 West Papuan asylum seekers temporary protection visas. Despite these sensitivities, by late 2006 both governments signed an agreement that outlined a framework for security co-operation known as the “Lombok treaty”. Since then, the treaty has been often cited as a sign of the strength of the Australia-Indonesia relationship and the basis for expanding future co-operation.

Unfortunately, some of the military and security co-operation fostered under that treaty has been wound back this week with the suspension of military co-operation and intelligence sharing related to people smuggling. This is worrying because people smuggling is an important issue to both countries. And there are flow-on effects as well: the suspension of military co-operation impacts on the ability to respond quickly to emergencies of all kinds, including natural disasters and terrorism.

Some instances of co-operation have been remarkable and should not be forgotten. Based on the intelligence and information exchange systems developed after the Bali bombing of 2002, it now takes only a matter of hours for Indonesian and Australian authorities to respond to each other’s needs in times of crisis. This serves the interests of both countries.

That said, Australia and Indonesia are not always natural partners, and in some instances their security interests diverge. These spying revelations, as uncomfortable as they may be, have flushed these divergent positions out into the public eye, and might provide an opportunity for Australia and Indonesia to be frank with one another. The two countries need to decide what kind of relationship they want in future and how deep a level of co-operation they are prepared to commit to. This might be the right moment for our leaders to have such a conversation.

Intelligence might even be the right place to start. It’s reported that Indonesia is now looking to boost its intelligence capabilities. If that happens, Australia might consider how to help shape those processes. As my Australian Strategic Policy Institute colleague Benjamin Schreer has argued, it's in Australia's interest to have a more capable Indonesia. Without overstating the level of influence we have on Indonesia’s processes, there’s an opportunity to explore what role Australia can play in working together with Indonesia in intelligence.

It's not going to be easy to make progress in the short term. The latest row is over a substantive issue and Indonesia is understandably disinclined to do Australia any favours. But the Lombok treaty is still in force, and reflects the fundamental fact that the two countries recognise that they have to work together. In time this might be seen as a turning point for the good.