“No, no, no. I was just into my magazines and the drawings," he said. "I had a very strict upbringing, almost puritanical. I lived there all the way through college. I was in my grandmother’s house, and I respected that!”

Talley told Grigoriadis he rejects the gay “label," but admitted to having had some “very gay experiences." Designers not included. "[Y]es, I swear on my grandmother’s grave that I never slept with a single designer in my life. Never, ever desired, never was asked, never was approached, never, ever bought, in my entire career. Never. Not one. Skinny or fat. Never.”

He also said he has never been in love with a man, but has been in love with two women, including an unnamed socialite whom he fell in love with after an evening of dancing in Manhattan.

Talley -- named No. 45 on Out magazine's "Power 50" list of gay cultural icons in 2007 -- is not the only one in the fashion world to rebuff being pigeonholed. Designer Tom Ford also rejects the notion.

"I hate to use the words 'gay' and 'straight,' and I think we're all on some sort of sliding scale," the fashion mogul said during an interview with British Vogue last year. "Of course, I'm at the Richard Buckley end of the scale, but I hate that gay/straight thing."

Talley may not choose to publicly discuss his sexuality, but he is willing to dish on what he finds sexy. Earlier this month, he talked about a particularly erotic moment in high school when the homecoming queen asked him to powder her back in between dances, The New York Times noted. It was then that he had "a first frisson of the spine as an erotic zone."

Hollywood juggernauts and best friends, Matt Damon and Ben Affleck, are <a href="http://www.nydailynews.com/entertainment/gossip/matt-damon-talks-gay-rumors-playboy-article-1.1220585#ixzz2Kz1uVL4Z">no strangers when it comes to gay rumors</a>. Their seemingly more than a bromance, which stems from their Boston upbringing, had never been denied, causing people to further believe the gossip until Damon decided to speak up about it in December 2012.
Damon proclaimed: "I never denied those rumors because I was offended and didn’t want to offend my friends who were gay. As if being gay were some kind of f--king disease. It put me in a weird position in that sense."