It’s no secret Iron Man 3 marks the end of Robert Downey Jr.‘s contract with Marvel. The actor has been vocal about it, the president of the studio Kevin Feige has talked about it, but everyone agreed concentrating on Iron Man 3 was the most important thing. That film is now done and hitting theaters in mere days. So talk once again turns to Downey’s future with Marvel. The actor’s charismatic turn as billionaire playboy Tony Stark is credited for igniting the massive success of the Marvel Cinematic Universe, and while he feels like an integral part of that universe Downey recently revealed he has an end in mind for his Marvel run.

In a new interview with GQ magazine, the actor said he’s had a ton of fun playing Iron Man and hasn’t decided anything for sure, but suggests about three more years is as far as he can go with the role, at least physically. After that, he’d like to direct and star in a Halloween-set thriller. Read the quotes below.

Downey knows that no ball can bounce forever like his has in recent years. “This period of time, this shall pass,” he says. “Fortunately I’ve been around the block enough—I’m not ill-prepared. And I love change. I love it when a lightning bolt hits the genny and you’re down for two hours on the set. Now, this isn’t a lightning strike, this is like the warning of a gathering storm.”

When I ask him what the gathering storm is in this case, he answers by telling me about the injury he sustained on the Iron Man 3 set in North Carolina that shut down production for several weeks. He was doing the kind of wire jump he’s done hundreds of times over the past few years, and he blew his ankle out.

“It got me thinking about how big the message from your cosmic sponsor needs to be before you pick it up. How many genre movies can I do? How many follow-ups to a successful follow-up are actually fun? Because, as quiet as it’s kept, I come from a family of very innovative writers and directors and actors and artists, and the circle of friends they were in were the people I heard having pun-offs playing poker at two in the morning, and it was just the most comforting aspect of my childhood. So there’s this kind of legacy of souls from what I consider to be a very particular time in entertainment, and I’m sensing a return to that—it’s what me and the missus are doing next. It’s not unlike: I heard Brady signed on for three more years with New England, and then he’s done being a QB, because he’ll be 40. I’m 47, and I’ll be 50.”

So that’s the line in your head at the moment?

“Yeah, roughly.”

So what stops at 50? Superheroes?

“I don’t know. I don’t know. Right now I don’t have a contract to do anything, and I did for the last five years.”

In the article, Downey also admits he was paid around $50 million for his work in The Avengers. You’d imagine that would be a difficult thing to move away from. When we spoke to Downey a few months back, he admitted money was a consideration. “Me, the agents and the lawyers are having a bit of a ball right now,” he said. However, with this new three year window in play, that’s doesn’t sound like the issue. He’s more concerned about his health and doing new, exciting things professionally.

Downey seems particularly excited about freeing himself of obligations to do smaller films. One he mentioned specifically, which he’d direct and star in, would feature a “Village Voice reporter on the run” on or around Halloween. “Nobody has cornered Halloween as a market since Halloween,” he said. That film would be much smaller than other movies:

The funny thing is even the stuff that I would want to do as an artistic expression of my own I would like to do in a genre, I would like to do for a price, and I would like to be pretty much sure that it was going to be a hit before I started.

Back to Marvel, three years would be 2016, and that would suggest he’ll appear in The Avengers 2 and, maybe, another Iron Man movie. Actually, though he doesn’t have a contract for The Avengers 2 yet, if Iron Man 3 ties into that film as expected, his participation seemingly confirms he’ll at least be back for that one.

But what if Downey really does hang up the armor after that film? Does Marvel recast the role as once suggested? Or do they do the unthinkable — something hinted at at the end of The Avengers? Will Joss Whedon kill Tony Stark?

Lots to think about but, one thing is for certain. Enjoy Robert Downey Jr. as Tony Stark while you still can.