To most of America, the downfall of decorated CIA chief David Petraeus was a shock. Aides and associates have long described the retired four-star general as a talented military tactician and sound leader, with accomplishments “comparable to what World War II generals achieved,” says Jack Keane, a one-time Army vice chief of staff and a key architect of the Iraq surge Petraeus led. Among civiilians, the general’s image was even shinier, burnished by hagiographic media accounts like the one penned by biographer Paula Broadwell, the woman with whom he had an affair.

Beneath this glowing reputation, however, Petraeus also had his critics. These detractors, including some from within the military his career helped shape, paint a picture of Petraeus that is sharply at odds with his public persona. They believe the general’s ambitions outpaced his accomplishments. They privately dubbed him “King David” for his high self-regard and chumminess with reporters. And they describe him as a man who found power by cozying up to senior officers and maintained it by working the press.

“Petraeus is a remarkable piece of fiction created and promoted by neocons in government, the media and academia,” argues Douglas Macgregor, a retired Army colonel and author known for his book Breaking the Phalanx, which takes the Army to task for the way it organizes and uses its ground forces.

Macgregor describes Petraeus as a “man who for 35 years shamelessly reinforced whatever dumb idea his superior advanced regardless of its impact on soldiers, let alone the nation, a man who served repeatedly as a sycophantic aide-de-camp.” Petraeus “was always a useful fool in the Leninist sense for his political superiors — Wolfowitz, Rumsfeld, and Gates,” he says. “And that is precisely how history will judge him.”

As a young cadet at the U.S. Military Academy at West Point, Petraeus found his first important Army mentor in the form of Lieut. General William Knowlton, the father of his future wife Holly and West Point’s superintendent. A top graduate, Petraeus did stints in anonymity – but close to power — carrying bags for Army generals like NATO commander John Galvin, Army chief of staff Carl Vuono, and Henry Shelton, who would go on to become chairman of the Joint Chiefs.

Some colleagues criticized Petraeus for spending too much time with the brass, and not enough with the troops. In person, he seemed to be a coiled spring perpetually under tension. His ambition burned brightly, and kept him from becoming close friends with many comrades. Instead, he relied on a coterie of junior officers, many of whom had served multiple tours with him in Iraq and Afghanistan. Petraeus never fit the mold of a traditional back-slapping, hale-fellow-well-met Army officer. He had the aura of a cerebral general, a whip-smart but somewhat aloof commander who knew – or at least you suspected he believed – that he was the smartest guy in the war room.

As he rose through the ranks, Petraeus carefully nurtured this image. He raised eyebrows when he invited outsiders into his war rooms to question strategy and sent them out in military aircraft as professional second-guessers. “The man has always been controversial, and therefore his legacy was going to be controversial,” says Stephen Biddle, a military expert at George Washington University who says the lack of any formal review of the U.S. military’s successes and failures in the post-9/11 wars will make Petraeus’ fall from grace cast a long shadow over the conflicts.

Biddle served on three Petraeus advisory teams – one for Afghanistan, one for Iraq, and one when the general took over Central Command – and says he was stunned by the general’s undoing. “Petraeus is one of the most disciplined humans who ever walked the face of the Earth,” he says. “I find the analogy to John Edwards kind of interesting, personally. Here are two very accomplished, very ambitious, very self-aware people, who both fell for their biographer. A certain degree of narcissism looked like it had something to do with both men’s falls.”

Though the affair was out of character, Petraeus has overstepped boundaries before. In July 2009, when he was the chief of the U.S. Central Command, he gave a speech to an annual Marine Corps Association Foundation dinner that made the Air Force the butt of a joke about pilots’ ponytails. Air Force partisans got wind of the comments and described them as “beyond outrageous,” arguing they “belittled the contributions of the Air Force to the joint force.”

The following year, when Army General Stan McChrystal was forced out for insubordination, Petraeus effectively took a demotion to replace him as commander of the Afghan campaign. Obama and his advisers had been somewhat leery of Petraeus; there was a whiff of desperation in the air when they tapped him to succeed McChrystal. Obama was determined to keep Petraeus at arms-length, several links away in the chain of command. Pentagon officials exulted at the change; there would be far less direct communication between the commander-in-chief and his most famous commander.

While the majority of the military is still singing Petraeus’s praises, others have greeted his downfall with an unsparing appraisal. “When a man becomes more reputation than substance, his reputation had better be invulnerable,” says Ralph Peters, a retired Army officer and author. “Every successful man has encountered at least one Paula Broadwell. The smart ones don’t take her calls.”