As more details emerged and the circle of those involved widened, there has so far been no evidence of a criminal violation. The fact remains that Petraeus spent 37 years in the military — at one time as the top U.S. commander in Afghanistan — and is considered by many to be one of the top military intellectuals in the U.S.

Regardless of professional accolades, it's often a long road to recovery from such a public downfall. Just four-and-a-half years ago, former New York Gov. Eliot Spitzer resigned after being embroiled in a prostitution scandal.

Spitzer, who now hosts Current TV's Viewpoint, tells weekends on All Things Considered host Guy Raz that the public is entitled to judge public officials by the entirety of their existence.

"When one stands for elected office or appointed office, the entirety of one's life is subject to scrutiny," he says. "[But] I do think people can and perhaps should weigh different aspects of those lives in different ways."

That's the question, Spitzer says, that people should ask themselves in regard to Petraeus: Should his personal affair be considered a disqualifying factor for his public position?

"It's too bad it has been viewed that way," he says.

Spitzer says he wishes President Obama would have let Petraeus remain as CIA director, while recognizing the personal violation and the public fallout from the scandal.

"Petraeus is a uniquely talented individual when it comes to the military ... and the public should get the benefit of his talents in one domain even if there have been flaws or failures elsewhere," he says.

Spitzer, who was a major figure in cracking down on Wall Street corruption while serving as attorney general of New York, transitioned to a television career with hosting jobs on CNN and Current TV to continue that fight. He says he hopes that once all is said and done, Petraeus can somehow continue to work on matters of national security.

"Maybe not as the director of the CIA," he says, "but in some advisory capacity in the White House where his knowledge could be tapped."

As far as moving on goes, Spitzer says he has tried to focus on the issues and individuals he cares about most and just "lead life."