When Ken Livingstone made his infamous comment about the Tory party being "riddled" with gay men back in February, he ignited a row over his choice of words. But funnily enough, I don't remember anyone taking him to task on the basic fact. They couldn't really, as it is a fact: the Conservative party does have a large number of gay members and supporters.

Guy Black, the first openly gay Tory in the House of Lords, put it most eloquently. He's quoted in Tory Pride And Prejudice, a recent history of the Conservatives and homosexuality: "It was one of those strange phenomena that when the Conservative party appeared nationally to be at is most homophobic, at the very heart of the organisation were all these influential gay men."

This phenomenon is seen as one of the enduring mysteries of minority politics: why do so many gay men support a party they may think doesn't (or once didn't) support them? (Another mystery, incidentally, is why a Liberal Democrat party keenly supportive of gay rights should have suffered so many high-profile embarrassments over closeted MPs, but that is for another day.)

The "mystery" was driven home to me when I was recently invited to speak at ParliOut, an LGBT group for people working in the houses of parliament. I happily turned up for a meeting in front of a friendly, substantially male audience of maybe 40 staff. With the formalities over, we retreated to the bar where, to my surprise, it became obvious that this cross-party gay group was about 80% Tory.

I spoke to Ben Furnival, chairman of the group at the time, as to why this might be, and he seemed no clearer about the answer than me. And it got me thinking: what exactly is going on here?

In trying to answer that question, it's worth saying at the outset that one has to be ready for complexities, contradictions and surprises. For example, I remember spending a day at the home of Lord Tebbit to record a feature for the Today programme. We chatted about many issues, including social liberalism, homosexuality and Alan Duncan (the first serving Tory MP to make a decision to come out). Lord Tebbit has famously robust views on sodomy, and once suggested that gays shouldn't be allowed to hold top government jobs such as home secretary. Would he be beastly to me? Not a bit of it. He couldn't have been more civil or full of respect for gay individuals, including Alan Duncan.

So the Conservative party, while sometimes negative about homosexuality in the abstract, is perhaps generally well-mannered at the level of the individual. It certainly did not get where it is today by excluding people. And maybe that kind of personal open-mindedness has made it easier than you would think to be a gay Tory.

Matthew Sephton, who runs LGBTory, the Conservative LGBT group, told me he has never experienced hostility as an out gay man at a local party level in his 10 years as a member. In the same vein, Furnival speculates that the Tory party's acceptance of eccentricity might have made it appealing to many a gay man. It is, after all, a party that believes in individuals making decisions for themselves. Or does it?

As I say, complexities and contradictions abound. I quickly find that the more people I talk to about the gay Tories, the less clear I get. Is it the polite party or the rude one? The tolerant or intolerant? You find bits of everything in different accounts.

Duncan – who does indeed enjoy a friendship with Tebbit – thinks local parties did have some unpleasantness about them: it was impossible to be an openly gay candidate in the early 90s. "The people like the area agents felt it was their duty to flush out those risks," he says. He came out only once he'd established sufficient seniority to be known not only for his sexuality.

So, why then do gay men join the Tories?

The least satisfactory account is what I would call the cognitive dissonance theory. Some people harbour an awkward clash of feelings – homosexual attraction on the one hand and shame or embarrassment about that attraction on the other. It is well known that the mind struggles to sustain conflicting views. So the theory goes that, in this case, the tendency is either to drop the shame or to bury the homosexual feelings.

The shameless will naturally tend to gravitate to the left, because that is a political choice that validates their self-perception and reduces the mental discomfort of the internal conflict. And for the self-loathing, the choice goes the other way – a neat way to be less homosexual is to join the party that doesn't like homosexuals.

It sounds plausible, and maybe this does explain why certain homosexual men drift towards a career in the mid-ranks of the Catholic church; but I find myself unconvinced by it in regard to the gay members of the Conservative party, who are gay with a capital G, not "men with homosexual tendencies". They are not, by and large (or are no longer), the kind who get married to women and occasionally succumb to the temptation of a furtive encounter in a public toilet. Those members of ParliOut, for example, looked as comfortably gay as anyone I know on the left. So bang goes that account.

A second interesting theory puts it down to the aesthetic charm of the Conservative party; or, to put it more bluntly, the fact that the party is sometimes a bit camp. Certainly it shares a taste for theatricality with many in the gay community. And you can detect a hint of gay sensibility in other ways the Tories project themselves, too.

For example, gay culture holds a special place for those who put on an extravagantly brave front (think Hyacinth Bucket); or for those who maintain strength against the odds, or who face rejection or adversity (or their own demons) with style. Take the classic gay icons – Judy Garland, Liza Minelli and Barbra Streisand. I wouldn't put Margaret Thatcher into the same family as them, but her struggle against the conventions of her time perhaps makes her a distant cousin.

As I write this, I can almost persuade myself there is something in it, but unfortunately I suspect it's a theory for the wrong kind of gays. Those programmed with the relevant emotional response are not, by and large, the ones you encounter in Conservative circles. The Tory ones fail to live up to the camp stereotype, they don't know the words to great songs from the musicals and they aren't particularly artistic. They're just ordinary political folk with ordinary political interests.

Which brings us to theory number three: class. All socio-political phenomena in the UK come laden with the baggage of a class-based theory or two attached to them. In the case of gay Tories, there is one particularly silly variant of the category, which asserts that gayness is bred in public schools and thus fits with Conservatism like hand in glove.

Alas, there is strong evidence that gayness is not bred in any kind of schooling at all, but in the mother's womb. And anyway, the Conservatives can't realistically be portrayed as the party of public schools given that it consistently attracts a 30-40% share of the vote.

While that version fails, there is a more general class-based hypothesis that gayness, like the Conservative party, is a middle to upper-class tendency. This, in fact, can't be dismissed out of hand. A study published in 2002 of incomes of gay couples and comparable straight ones found that the gay men were, on average, slightly better paid (9%) and somewhat better educated. In fact, the authors' research suggested that, given their skill levels, the gay men should have been earning 15% more. A sign perhaps that we are a cut above average?

I'd love to believe so. But whether that pay differential is 9% or 15%, it is hardly the difference between a middle-class professional and a working-class labourer; it's a small difference in a wide distribution. So, for me, the class-based theory doesn't get far. The best that can be said of it was summarised by Furnival. In comparing the "gentleman's club and working men's club, one is arguably more approachable for a gay man". That's about as far as I would stretch it.

All in all, it doesn't sound as though I'm doing much to explain the strange phenomenon of the gay Tories at all. So what message might we draw? After a few days' thought, several conversations, a bit of reading, some writing and rewriting, the conclusion turns out to be remarkably simple. Both gay men and Conservatives come in many different varieties, you can make only the loosest of generalisations about either, and it is thus not surprising that there is some intersection between the two. Because gay men are not all self-loathing, camp or middle class, or any of the above, you don't need a theory that draws on any specific characteristics at all. And the Tories? As Alan Duncan reminded me, there is more than one Tory party. There is an economically liberal one and a socially conservative one. You can join one enthusiastically and simply tolerate the other, just as people joining Old Labour have to tolerate New. This isn't a strange phenomenon; it is politics as everybody has always played it.

In short, when you stop reducing all the players in this political drama to stereotypes, the strangeness of the phenomenon of the gay Tories evaporates. Gay men have bills to pay, aspirations to fulfil and beliefs about the shape of society that drive their party affiliation. The results vary. Even the gay Tories themselves are not all one and the same. They, too, come with different motives and beliefs. Er, that's it.

It is still interesting to reflect on the mindset of those who stuck with the Conservatives in the 1980s; but for most of them tribally loyal to the party at that stage, I suspect its negativity towards homosexuality was mostly seen as an aberration. Gay supporters just had to recognise it as such, and fight against it as best they could while upholding support for the rest of the Conservative platform.

The judgment that Clause 28 was a temporary fault and that normal service would soon be resumed did eventually come to be vindicated with the rise of a new generation and a relatively socially liberal leader in the form of David Cameron.

If there is a lesson, it simply that the Conservative party and the gay community are perhaps more diverse coalitions than most political and social caricature allows. The same is undoubtedly true of any group, be it the Labour party, the trade unions or company executives. I suppose we must just get over it.

• This article was amended on 21 April 2012. In the original, Ben Furnival was misnamed. This has been corrected.

• The following correction was published on 26 April 2012:In an article about the numbers of gay members of the Conservative Party, Evan Davis, the author, referred to remarks by Ben Fenton, the FT journalist. This was an error; Davis had intended to quote Ben Furnival, the former chairman of the Lesbian, Bisexual, Gay and transgender group ParliOut. We are happy to point out that Fenton made no contribution to this article and that Davis would have had no reason to include him as Fenton is neither gay nor has he ever given any public indication of his political leanings. The Guardian regrets the error (Glad to be Tory, 21 April, pages 38 and 41, Weekend).

There are two ways to get your placard noticed at a gay rights protest: be cute or be funny. Ahead of the Guardian Weekend magazine gay issue, out Saturday 21 April, we pay tribute to the best gay rights protesting placards