I can't remember which relationships counsellor it was – it could have been the second we went to, or it might have been the third – who told us we were setting too much store by sex.

"Many couples lead very satisfying lives together without that physical dimension," she said. I remember thinking it sounded like something you might be told if you'd just found out you were paralysed. "Many people lead rewarding lives without the use of their legs." I'm sure it's true, I remember thinking, but why, if you had the choice, wouldn't you have it all?

That's what came to mind when I read Helen Mirren's quotes in Woman & Home magazine last week. "People get together for reasons other than sex and, although it's important in the beginning for most couples, it's not what makes marriages last," said the star of the forthcoming film, Hitchcock, which depicts the long and reputedly sexless marriage between the Hollywood director and his wife, Alma Reville.

"But I think the power of partnership in marriage is under-recognised in our society. That's what makes marriages work, not sex."

On one level, what the Great Dame says is entirely sensible. Of course, sex in itself – that tragicomical arrangement of bodily parts – isn't the be-all and end-all of a relationship. That would be preposterous when there are far more important things, such as children and finances and illnesses and Scandinavian boxed sets, to bind a couple together.

But sex is so rarely "sex in itself". Sex is loaded with other stuff. Other stuff oozes and spills from every orifice: resentment, history, anger, baggage, fear, punishment, resentment (oh, did I mention that already?).

When the sex went out of my marriage like a faulty bulb, my husband and I never imagined we would remain in the dark for three years (and counting). We fully believed we'd get around to replacing it eventually. As soon as we'd ironed out the problems that led to this point, we thought, we'd be up and running again, all guns blazing, good to go.

But – and this is where I think Helen Mirren's argument falls down – unless you've both made a proactive choice to do without sex, like those couples who give up drinking during the week, or eating meat, the absence of sex can become as loaded as the sex itself.

We stopped having sex because of resentment (on my part) and apathy (on his). There were long-standing problems in our relationship that we'd never quite managed to address, and from these came niggles that threaded themselves like fissures through the foundations of our marriage.

Instead of forcing my emotion-phobic husband to confront those deeper-seated problems (which, if you're interested, revolved around the predictable triumvirate of money, motivation and responsibility), I took what seemed at the time to be a cunning shortcut. I stopped having sex with him. To show him I meant business.

I never imagined that when faced with a choice of not having sex or having the difficult emotional conversations that needed to be had, he'd choose the former. I never imagined I'd let him.

But something weird happens in a relationship when sex is allowed to go from a shared comfort to an individual bargaining tool. Something toxic enters in that spreads to every part, filling the gaps sex left behind.

Of course it didn't happen that brutally, the no-sex thing I mean. At first there were frequent lapses, tacit forays in the dark. But what had once felt so natural now felt freighted with meaning. Sex was a transaction in which things were given and things taken away, and both of us were left silently calculating what had been won and lost.

Eventually, these occasions proved too painful and petered out altogether, leaving us as one of the estimated 15 to 20pc of couples living in so-called sexless relationships.

(It's a measure of the state of our relationship that, when I discovered "sexless" is defined as having sex fewer than 10 times a year, I laughed a hollow laugh. Ten times? Why, that's practically a sexathon!).

At first we tried to pretend it wasn't happening, that it was normal for a couple to go without sex for six months, a year, two years. I found statistics to make myself feel better. I learnt about couples who had regular sex breaks before resuming normal service. I didn't realise you could find statistics to fit every argument. I didn't realise statistics were a poor substitute for physical closeness.

A year ago, we took ourselves off to marital counselling. My husband would have preferred to eat his own arm than tell a stranger he no longer slept with his wife. But he came along. Which just shows how bad things had become.

The first counsellor was a married man who went cruising for women at the gym. We only found out because a good friend was one of those women. I'm not one to judge, but when you're taking marital advice from someone you expect them to at least be better at marriage than you are.

The second counsellor, or it could have been the third, said that thing about us setting too much store by sex. "Don't confuse it with intimacy," she urged us.

Which, I think, might be what Dame Helen was getting at.

While I know it's perfectly possible to have sex without intimacy and intimacy without sex, once you've had them both it's hard to go back to just one or the other. Like a ghost limb following an amputation – the one you keep wanting to scratch.

And, let's face it, intimacy is key to a relationship. Intimacy is what makes a marriage special, what sets it apart from the relationship you might have with a friend or a sister. Intimacy is the marital holy grail, the ambrosia that oils the wheels of coupledom. And, in most cases, sex is the quickest and easiest, and most damn pleasurable, way of achieving intimacy.

That's not to say you can't have intimacy without sex, but you have to work harder to get it. And it's almost impossible if you're having to wade through a treacle of built-up resentment while you're at it.

Don't get me wrong. I'm sure there are many couples who have perfectly satisfying, intimate relationships without sex. But they're the ones who have mutually agreed it.

They don't have it foisted on them by someone else, or fall into it by mistake while they're looking the other way. They don't have to go through the agony of being rebuffed by someone who once couldn't get enough of them, or learn the sad repertoire of deflective gestures choreographing the message "No".

Helen Mirren says that it's partnership rather than sex that makes a marriage last. She's right, of course. But partnership means being broadly in agreement about the important things in life – goals, values and what's that third one? Oh yes, sex. And the having of it or not.

We're on our third counsellor now. Everyone tells us it's a question of finding a good fit. I suspect the search for a perfect counsellor has become a displacement activity. It's what we do instead of having sex. The thing is that none of the counsellors is telling us what we want to hear – that sex will come back eventually.

Instead, they set us homework to try to gradually reintroduce physical intimacy into our lives – a kiss here, a cuddle there. And they try to get us to talk about the resentments that led to the lack of sex in the first place – except that I can no longer remember exactly what they were.

All I know is that sex has become so tied up with pain that it's easier and safer to ignore it altogether.

A long-term relationship is like a car engine – it requires lots of different parts to keep running.

Helen Mirren is right to suggest that we put too much emphasis on sex at the expense of other more enduring components. But if you take the sex out altogether, you might just find the engine packs up completely.

Maybe I'll bring that analogy up in counselling. Beats doing the homework.