By now it seems fairly clear that the comic actor Billy Crystal did not intend to single out gay sex scenes in his reply to a question this weekend about how uncomfortably graphic some television has become. He was speaking broadly about a lot of what’s on TV now of all sexual varieties, but a reporter clipped the remark for maximum impact. The scolds of the Internet, always on alert, did the rest of the work. But set aside the insane idea that Crystal — the first straight actor to play an openly gay TV role on “Soap” back when it was career-lethal and truly cutting edge — might be some sort of sex-negative homophobe. The preposterousness of that is baffling enough; if anyone in the history of the cathode ray deserves the benefit of the doubt, it would seem, it would be him.

And yet — what if Crystal’s unremarkable remarks hadn’t been sensationalized? What if, in fact, he was just a 66-year-old grandfather who finds the increasing visibility of same-sex intimacy, especially on network TV, off-putting or startling to his sensibilities? What if seeing the bobbing head of a woman as she implicitly performed some sex acts on another woman’s lower half, say, makes some viewers uncomfortable — and they have the gall to admit it when asked?

I ask because those people exist. There are, in fact, a lot of them. The smug folks in their L.A. and New York bubbles might think they’re backwards, irrelevant, and scarce, but they’d be wrong. Their points of view are easy to understand, really. And it does not make them “haters” or any of the other epithets thrown at Crystal this week.

In fact, gay activists are as responsible as anyone for the fact that a large swath of Americans who thought they were OK with gays are finding themselves surprised by their own reactions to what they’re starting to see. This is, after all, a civil rights movement that aggressively worked for many years to downplay the mechanics of gay sexual behavior.

The gay-rights push may have started in the 1960s and 1970s with its pursuit of the fundamental liberty to have consensual sex with whatever other adults one pleased and to break free of traditional gender roles, but it morphed in the 1980s into one that preached, rightly, that being gay was about much more than mere sex. When gay male intercourse in particular became equated in the minds of millions with the transmission of AIDS, the best option was to de-emphasize it and instead make the case that our relationships are the same in every meaningful way as straight ones. Also, it did nobody any good to have our parents, much less our grandparents or our teachers or our bosses, visualizing us naked in sexual positions.

This is how the fights over the integration of the military and legalized same-sex marriage were won. Every time anti-gay forces tried to gross out the nation by referencing the gritty details of, say, anal sex, gay advocates would reply by accusing them of being secretly titillated by and obsessed with it. Whenever some crusty old military hack would grouse about gays being naked in showers or barracks with straight soldiers, gay activists did everything they could to insist gay people are supernaturally capable of stifling every fleeting sexual thought even when something attractive is before them.

Related Stories

Gays won, essentially, by persuading everyone to view us as sexless because we knew that how we express love and lust was at best unfamiliar — and at worst repulsive — to many people. Looking back now, it was probably the right strategy, and it was definitely a successful one. It enabled enough fair-minded people to look at couples like Edie Windsor and Thea Spyer, together for more than 40 years before Thea died and the government tried to disinherit her octogenarian widow Edie, and imagine how they’d feel if the law did that to them after a lifetime of companionship. When the Supreme Court fixed that for Windsor in a landmark 2013 decision that forced the federal government to recognize their marriage, most Americans felt a grievous wrong had been set right.

But now, after decades of telling the folks that what we do in our bedrooms is not of their concern and should not matter to their opinions of us, we demand that they be happy to look upon depictions of gay sex with a big shrug or, even, a round of applause. If anyone breathes even the slightest hint of discomfort or dismay, they must be berated and humiliated. Where once the anti-gay legions used shame to scare and silence us, now it’s the pro-gay forces who think it’s a weapon.

So here’s a dirty little secret: I’m gay and I hate watching straight sex scenes in TV shows and movies. I always have. It subconsciously conjures up the apprehension and anxiety I felt when I was still closeted and tried to force myself to want to experience that. I know what straight people do, how they do it and what all of their parts look like. But I could do without bouncing breasts in every other scene of “Boardwalk Empire” and “Ray Donovan.” It’s the opposite of what I want to do or feel or see, so I wait it out and enjoy the rest of the show. It’s part of the bargain of consuming art sometimes and occasionally it actually even has a purpose.

Does all that make me a heterophobe? I hope not. Some of my best friends are straight. It’s just not my preference. And yes, in this case, we’re talking about preferences and not orientations. As Crystal said, albeit in a broader context, it’s about one’s personal tastes, nothing more.

Hollywood and its audiences are in a transitional period regarding how to cope with the modern era of out gays and the specifics of how to show and talk about it. Some straight people, especially older and more traditional ones, will be uncomfortable. Others won’t. Most will, as we gays have regarding explicit heterosexual sex scenes, come to accept it even if they never love it.

And all of that should be fine. Gays have enough actual enemies: people who really don’t want us to live happy, productive lives as our true selves. We certainly don’t need to turn our straight allies — be they grandparents or groundbreaking actors — into bogeymen, too.

Thank you!

For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

TIME Ideas hosts the world's leading voices, providing commentary on events in news, society, and culture. We welcome outside contributions. Opinions expressed do not necessarily reflect the views of TIME editors.