When FX’s Justified returns tonight with its season 3 premiere (catch up here), Timothy Olyphant’s Deputy U.S. Marshal Raylan Givens will be uncharacteristically a bit off his game. ”It can only continue for so long, because I have to continue to live,” Olyphant says. “I think it was more or less the idea that we’re starting with a guy who’s just got a lot of things up against him: He’s recovering from a gunshot wound, he’s got a pregnant girlfriend who he used to be married to. He’s got a lot on his mind.” In addition to Raylan missing a few things that he wouldn’t normally let slip past him (enjoy the fun elevator scene in tonight’s episode, which should make Elmore Leonard proud), there is another upside: Raylan’s bandage needs to be changed. Shirtless scenes! ”That was my idea,” Olyphant jokes. “I thought if we’re gonna go shirtless, let’s do it in the first episode, and I’ll really prepare for it. And then after that, I can just let myself go, and we’ll just wear lots of coats. I would just want to say also, for the record, that Natalie Zea has her shirt on in the sex scene. That was not my idea, and I don’t want to hear any complaints coming my way. I don’t think she has a problem with it, I just think someone, for some reason being polite, didn’t mention it. I know certain people are very upset about it.” Here’s more to look forward to in the season premiere and beyond (besides, as Olyphant puts it, “a lot of bullets flying right off the bat and bodies dropping”):

• The introduction of Robert Quarles, the Detroit mobster played by Neal McDonough: He arrives in the premiere to get the Dixie Mafia back on track – and sees a business opportunity that will keep him in town. “Those things are blue, aren’t they?” Olyphant says of McDonough’s eyes. “And that hair’s blond. He stands out.” That’s the point, really: “[Justified exec producer Graham Yost] and I have talked about this. It’s the idea of this Northerner coming in, and as the season goes, that becomes a little bit more rich. It’s not just a bad guy coming into town, it’s a guy who represents something that goes back hundreds of years – the guy from the North coming in, underestimating these people, trying to take advantage of these people. That kind of thing gets me excited. We’ve done that in the past very successfully. Last year, with the character Carol [Rebecca Creskoff] working for the coal company, it wasn’t just this woman from the coal company, but it was who she represented, and the history going back generations – what it was like for characters like Boyd and Raylan who grew up working for the coal company, and the coal company being the source of employment for these people and at the same time, taking them to their graves and raping the land. That stuff is what makes our show work at its best – when it feels very specific to 2012, but then feels rich with the history of who these people are, and where they came from, and the things they believe in. So it’s not just about good guys and bad guys.”

• The appearance of Ellstin Limehouse, the hill-country badman played by Mykelti Williamson, in episode 2: He rules Noble’s Holler. ”I don’t know much about that world, but as soon as I heard about it – this holler up in Kentucky that was set aside for emancipated slaves – there’s a rich history there. The idea that people are basically banking with this guy, because no one dare go up there and try to get it, I think that’s fantastic,” Olyphant says. “Mykelti’s great. He’s one of those guys that I don’t know that much about. I’ve seen a few things that he’s done. But as soon as someone said his name, I was like, ‘That guy’s fantastic. I can’t wait to work with that guy.’”

• More Boyd and Raylan: We’ll see them get into it before the opening credits in the premiere, but with Quarles and Limehouse in the picture, Olyphant says our favorite duo will be on a bit of a slow-burn at the start of the season. Will Boyd and Raylan end up being on the same side by the finale? “We could,” Olyphant says. “Or, we could go the other way. Or, we could have them on the same side and still bumpin’ heads. You want to keep it simple, but at the same time, if there’s an alliance, you want it to be an uneasy one. You don’t want to lose sight of who these guys are and that there’s just no way it’s not gonna go a little south. If there’s no conflict, there’s no scene. The key though is to find that within the tone and rhythm and cadence of Elmore Leonard. That’s what allows us to be a little more funnier than usual, a little more folksie than usual. People are threatening one another’s lives, but it sounds like they’re talking about meeting later for drinks. That is the fun of it all, taking scenes that in another world would seem heated and very clear on the surface and twisting them, giving them that little twinkle, violence and comedy going hand-in-hand.”

• A season that hopefully builds to as satisfying a finale as seasons 1 and 2 did: If you think about topping previous seasons, you’ll start to make decisions for the wrong reasons, says Olyphant, who seems to earn his producer credit by bringing ideas to scripts that up the Elmore Leonard quotient. ”I can only think, ‘Here’s our story. What’s the best way to tell it? Where’s the conflict? Where’s the fun? How many ways can you look at it until something comes up where you’re like, ‘God, that makes me laugh. I can’t wait to shoot that.’ You just hope that other people feel the same,” he says. It seems to us that for shows with a prolonged adrenaline rush and an escalating body count, likeSons of Anarchy last season, finales can be particularly tricky, we tell him. “The challenge in any kind of storytelling, as far as I know it is, you’re looking for the moment that is completely unexpected, but at the same time, you think to yourself, of course, that’s the only way it could have gone. It has to feel like it’s the truth of it. It’s easier for me think about movies like the Coen Brothers’. You think, I didn’t see it coming, but of course. I remember on Deadwood, [David] Milch having that same kind of balancing act at the end of the third season with Hearst [Gerald McRaney]. It was just building, and building, and building, and at the end of the show, Hearst just drove out-of-town. Nothing happened,” Olyphant laughs. “But it felt like of course, that’s the way it was gonna go. Where else could it go? First of all, I’ve read my history books, and we’re not doing Tarantino. We’re not gonna kill Hitler. But it is tricky how to pull that off and still satisfy people’s appetites for what you promised. At the end of the day, the job is to tell the story that you promised to tell, and do it in the most entertaining and perhaps surprising way you can think of. With these television shows, it’s not easy.”

Below, Olyphant talks more about how he collaborates with EP Graham Yost and the show’s writers. “Every episode, one of those guys is on the set, and no one’s having to call into the office every time we change some dialogue. Everybody has a voice in it,” Olyphant says, including his fellow castmates. “I think that what’s happened over the course of the last two years is there’s been a trust that’s been built. Graham really feels like, You guys are good. You got it. You know what you’re doing.”