The fashion industry elite knows Sean Patterson best as the president of Wilhelmina Models, one of the largest, most successful model and talent management agencies in the world. Reality television fans may know him better from VH1’s The Agency, TV Land’s She’s Got the Look, and guest appearances on America’s Next Top Model.The Advocate readers also know him as the lucky guy that Fergie recently fingered as her “gay BFF.” Joining a panel of judges that includes Candy Spelling, poker pro Vanessa Rousso, and Melody Thornton from the Pussycat Dolls, Patterson now gives away cash to the most deserving dreamers on Bank of Hollywood, a new E! game show from executive producer Ryan Seacrest. Patterson speaks exclusively to Advocate.com about his high-profile pals, his Andy Cohen-ish self-promotion, and his recent decisions to deny funds to LGBT hopefuls.

Advocate.com: What makes you qualified to be on the Bank of Hollywood judging panel?Sean Patterson: When Ryan Seacrest and the other producers offered me a spot on the panel, they were looking for someone to be a grim dose of reality that only an agent can deliver. They wanted me to be the guy who says, “Look, it’s all nice and flowery what you’re saying right now, but this is the way life really works.” As an agent, people are constantly pitching me things, and we’re also pitching our talent to clients, so I can smell when something’s fishy. Basically, I was there to make sure nobody ripped us off.

In the first episode you chose not to give a drag queen, Latrice Royale, the money to compete in the 2010 Miss Gay USA at Large pageant. Even so, I was happy that there was at least one gay person up there weighing in on the decision. Before I worked at Wilhelmina, when I was at NYU, I was a club promoter. And now I work in a very gay industry, so I have lots of friends who have had the same uphill battles and who have gone through the same sort of discrimination that Latrice has gone through. So that was a very complex decision for me, because as much as I wanted to make a statement by helping somebody who was an amazing gay performer, there was some baggage that came along with Latrice — she’d gone to jail and had some other things in her past that I thought were a little questionable. We had the ability to do so much good with the money, but it wasn’t infinite, so we had to be careful to only give it away to the most deserving people.

You also refused Maya Jafer, the Indian trans woman who wanted money for facial feminization surgery. Well, that concerned me because it’s a very painful, very dangerous operation, so I thought we needed to give serious thought to giving somebody the money to go do something they may regret or that could kill them. Because money won’t solve some of these people’s issues; for some it’s more of a psychological issue than it is a material issue. But I still think I was a strong voice for gay and transgender people, and you’ll see that in an upcoming episode when I make a dramatic statement about the discrimination that gays go through. I’m not going to give it away, but I actually stopped the show at one point because I felt like we were potentially heading down a road I was not comfortable with.

Does the money you give away really come out of your own pockets? It does because of the fact that our fees for appearing on the show are what we give away. So if we don’t give the money away to the hopefuls, it’s ours to keep.

Couldn’t you just take it all out of Candy Spelling’s check? She certainly doesn’t need the money. I know, right? The last night of shooting we actually had an impromptu wrap party at Candy’s place, which you know is the most expensive house in America, and it was the most surreal experience. I never believed that I would ever be drunk at Candy Spelling’s house. A lot of the crew and producers on the show were gay as well, so it was almost like a tea dance at Candy’s.

Did you see the doll collection and the gift-wrapping room?Are you kidding? Of course I did. The doll room was one of the first parts of the tour. We also bowled in the bowling alley, and I was not sober when we were bowling. It was, like, midnight on a Wednesday and we were still getting tanked at Candy’s. That was a reality show right there.

Did Tori show up with the kids?No, but I had spoken to Candy briefly about all that while we were filming, so I was very happy to hear that she and Tori had reunited afterward. I didn’t know what to expect going into the show, and I was probably fearful that she’d be villainous because of the way she’s sometimes portrayed with her daughter, but I have to tell you that Candy is one of the most elegant, amazing, down-to-earth women that I’ve ever met. We talked about everything under the sun, and she can get off-color too, which I love. She told me one story about a dinner party when Tori was a kid that’s so off-color I can’t even repeat it. Candy’s also got one of the most unbelievable art collections you’re ever going to see. You know that famous painting of dogs playing poker? She has the real one, right in between her bowling alley and her doll room.