Russell Swan: Being on Survivor Brought on Depression

On day 15 of Survivor: Samoa, a dehydrated Swan collapsed during a challenge, nearly scaring the khakis off host Jeff Probst. Three years later, he returned to the game only to face one of the worst losing streaks in Survivor history and being voted out fourth.

Swan, a 45-year-old attorney from Pennsylvania, tells PEOPLE about his discouragement and depression – and what he really thinks about Jeff Probst.

So, it didn’t go well this time out. What was the problem?I’m not going to throw [my tribesmates] under the bus, but this is a hard game.

I respect anybody who will take it on, but they have to be engaged in the process. My tribe wasn’t engaged; the other tribes were hungrier.

You had some strong players on your tribe, though.I don’t know that I agree with you. The other tribes were stronger than us.

How did you prepare to return to Survivor?I trained hard for this. I rode my bike several miles from home to the office; I went to my wife’s swimming class; I went to the gym three times a week. I was doing something six days a week to get ready to play this game. But I didn’t train to dive down 20 feet to get puzzle pieces. Name one city guy who can do that!

But the fact remains that your tribe lost four consecutive challenges. What does that do to you mentally?It’s hell. You know those theater faces with the happy and sad faces? It’s like that. A tragedy. The failures were mine. I’m the first returning player that is out, so it feels like a failure. No one likes to fail, and I hate to lose, so I am still processing it day by day. But yeah, you could say I have a lot of regrets. … I’m pissed off.

When I came back from the Philippines, I went into denial. I couldn’t sleep; I was irritable. I thought, “I’ll pretend like it never happened.” I hid everything that had to do with the experience. And it was working, until the show started airing. When they announced that I had returned, I thought, “Oh, crap. It really did happen. I want this to go away; I need this to go away.” But every week the show would play, and it’s like I relived it again: the failures, the discouragement.

I have never really experienced depression, but this is activity-induced depression. I can’t be like this. I have a daughter, and she won’t understand if I’m up in my own head all the time. So I have to compartmentalize being on Survivor and move on.

Coming into the game, what were you expecting? I was there to get the check; I wanted to have an “up from the ashes” narrative. And that’s not what happened, and I have to deal with that.

Let’s talk about your gameplay. The hidden immunity idol was on the rice bin, right in the camp.It was under my nose the whole time. If you want to completely destroy somebody, this is the way to do it: put something under their nose, and then have them not able to find it.

I tried to look for it, but my other teammates were always hanging around, so I couldn’t. They did a good job of limiting my ability to look for the idol – which does not excuse my ineptitude.

My sense of self-worth and capability: physical, intellectual, emotional, all of it has been thrown into question. Basically, to find out that you suck at every aspect of the game, it’s really tough.

But you’re an accomplished guy; you just lost a reality show. I wasn’t playing a character; I was just being me. So all the things that people say to me on Twitter, they can hurt. The things Jeff [Probst] has said about me, the way we interacted, that has been tough, too.

But you know, I could care less what Jeff Probst thinks of my performance out there. I did what I could. And don’t worry; you won’t read about me jumping off a cliff because of this. I’ll be fine.