Andy Warhol had it nearly right. In the future everyone will be ordinary for 15 seconds and every half-baked columnist will cause a tweet uproar for 10. I don't want to go over last week's storm in a double-D cup, and feel I should really address a much more innocuous subject.

But you know what? This whole shebang raised a few issues for me that won't go away. The wrath of the transgender community has been insane. They say I haven't apologised enough and I probably haven't. No one has apologised to me for saying that I should be decapitated and I support the English Defence League. The sexual and political confusion is nasty and, while I accept some of it is my fault, is it all my responsibility?

I don't really care what people do with their bodies. Honestly. Martin Bright said in his sweet defence of me in the Spectator that I am a freak. I am talking freak in a Sugababes/Rick James kind of way, though he could have meant something else.

Indeed, I feel increasingly freakish because I believe in freedom, which is easier to say than to achieve and makes me wonder if I am even of "the left" any more. I am a freak because I question certain words: cis, date rape, Islamofascist, for instance. I am a freak because I believe in sexual liberation, which is not the same as equality.

And I am serious about freedom of speech. If Lynne Featherstone can call for a journalist and an editor to be sacked, this does not bode well for having politicians and lawyers running the press, does it? Do you actually want to be governed by humourless, authoritarian morons? Don't answer that, I may be offended. You don't commission someone like Julie Burchill to launch an Exocet missile and then say: "Oh dear, we only really wanted a sparkler."You cannot unpublish something any more because of the internet, something that Lord Justice Leveson failed to get his considerable head round.

How has the left ceded the word "freedom" to the right? It maddens me. We can argue about sexuality and gender till the sacred cows come home. Obviously my politics come out of feminism and did I need to say that I have never personally condoned the murder of a single woman, Brazilian, trans or otherwise? Nor did I make up stuff; I merely reported what is actually happening in Brazil. According to an Associated Press report: "The trans-models have a proverbial leg up on their female colleagues. Unlike even the thinnest of women, without cellulite and stretch marks ... once they've lasered away facial and body hair, they can look more feminine than models who were born female." This description has as much to do with the average transgender person as I do with Naomi Campbell, but the artificiality of femininity is something I often write about.

Is this hatred to say so? Isn't the struggle for people to identify themselves genderwise part of a much bigger battle?

More and more I realise I am on the side of liberation, which seems so often to be on the opposite one to equality. That makes me sad. As Labour became increasingly authoritarian and took us to war in the name of "freedom", many turned to the Lib Dems to protect our civil liberties. They turned out to be as liberal as Ann Widdecombe on Advocaat.

No party represents freedom now. What we have is a few rightwingers who took some E in a field once and so claim to be libertarians, but are in fact Thatcherite misogynists. We have the double-think of "free schools", which exclude those who most need them. We have "freedom" for the very rich to take from the very poor while lecturing them on their moral poverty. We have women and gay people pushed into the conformity of lifelong monogamy, even though it clearly does not work for so many.

None of this is new; it is an age-old political argument. If you believe, as Emma Goldman did, that justice requires the redistribution of wealth, then you are always going to be at war with those who say coercive redistribution curtails the individual liberty of the tax avoiders. How to be free in a world in which we look at Mali or Gaza, or remember the beautiful women doctors in Kabul in the 70s. Yes, I certainly check my privilege all the time. People died for my right to offend you. I live for a left that is about freedom, a sexual politics that is about choice.

That, to me, is liberty and my heroes who write so well about anger and pain nearly always write in the end about love. For only those with intelligence can take real liberties. As the great James Baldwin says: "Love takes off the masks we fear we cannot live without and know we cannot live within." William Hazlitt, too, talks of loving liberty as being about loving others, loving power only being about loving ourselves.

So I regret not making it clearer that we need both love and anger to be free. And you may continue to hate me, put me on lists, cast me out of the left. Free-thinking is always problematic. But if you take away my freedom to love, be intemperate, silly, angry, human, ask yourself who really wins? Who?

Sign up for the Guardian Today

Our editors' picks for the day's top news and commentary delivered to your inbox each morning.