“I was very confident at that time,” Jones, 23, says in the current issue of PEOPLE of his decision to leave Men in 2013 after telling Seventh-Day Adventist filmmaker Christopher Hudson, “You cannot be a true God-fearing person and be on a television show like that.”

Walking away from Hollywood, Jones enrolled at the University of Colorado at Boulder as an environmental studies major. But at the end of his freshman year, he switched majors to Jewish studies, a sign of his evolving relationship with organized religion.

“Over the last three years I’ve been involved with various faith-based organizations,” says Jones. “Right now, I’m stepping away from the organizational business-model programs. I’m interested in seeing where I go without an organization putting a stamp of approval on if I’m good or bad or whatever.”

Matt Hoyle/CBSMatt Hoyle/CBS

When he’d departed from Men after season 10, there were reports Jones would guest star on season 11 of the sitcom. But he did not return to the show until the series finale in 2015.

“Getting to be back on the set with everyone again kind of showed me how much I did like it and how much a part of me acting,” he says.

For more from Angus T. Jones — including what his life is like now — pick up the latest issue of PEOPLE, on newsstands Friday

“The door is definitely still open for me to do that, but I’m taking things slowly,” says Jones, who recently starred in his former college roommate’s short film and still has an agent. “But I’m kind of liking the ability to travel and to move around at a moment’s notice and not have to be in one spot for years at a time.”

Regardless of his future, Jones refuses to second-guess his decision to step away from the spotlight.

“In terms of regrets, I try to avoid those completely,” he says. “There’s no changing anything. There’s only moving forward.”