The Pariah

Where did it all go wrong? How did one man on the Supreme Court—David Souter—become the symbol of everything the right wing hates about government?

all summer long, they called him names. He was an activist, a liberal. He was a turncoat. He was socially, perhaps even sexually, perverse. Most of all, he was to be avoided—the Republican Party's most embarrassing mistake—because they'd been fooled by assurances, fifteen years ago, that he was a true conservative and thus sent him on to a lifetime appointment on the Supreme Court.

After which he betrayed them.

“We're all aware of the tragic consequences that have resulted from the David Souter appointment by President George Herbert Walker Bush,” said James Dobson, founder of Focus on the Family, in a radio address this July. “He has turned out to be a radical liberal who has done great damage to this country.” With an opening on the Court, the Republican Party could not afford another Souter. John Roberts? Well, he seemed solid enough, saying and doing the right things. But then again, in the beginning, so had Souter.

Back in September 1990, David Souter was sitting, gaunt and bleary-eyed, before the Senate Judiciary Committee, interviewing to become the 105th justice of the U.S. Supreme Court. The committee chairman, Joe Biden, started the hearings by asking Souter to provide “a little glimpse into your heart.” It might have seemed a silly thing to ask of a Supreme Court nominee, but in truth, people had wondered about Souter. He was a 50-year-old bachelor. Had very few friends. Lived alone on a dirt road in New Hampshire. Was said to enjoy books more than people. Seemed, in fact, not to require much in the way of human contact at all.

Souter nodded at Biden and began to speak. “I spent most of my boyhood in a small town in New Hampshire,” he said. “The physical space, the open space between people, however, was not matched by the inner space between them, because as everybody knows who has lived in a small town, there is a closeness of people in a small town which is unattainable anywhere else.…”

For three days, he sat at a table with a pencil and a blank piece of paper and a roll of spearmint Life Savers and fielded the senators' questions. When asked, he declared that the Fourteenth Amendment “does recognize and does protect an unenumerated right of privacy”—prompting Biden to respond, “I like what you are saying.” When the (very) senior senator from South Carolina, Strom Thurmond, interrupted an answer with “That's sufficient,” Souter joked, “You are going to turn me into a laconic Yankee if you keep doing that, Senator.” And when the committee's harshest interrogator, Ted Kennedy, prodded Souter to reveal whether he thought abortion was immoral, the judge ducked: “What I do believe, Senator, is that for me in this forum to start…[with] an expression of my views of the morality on that subject would be taken by a substantial number of people as the beginning of a commitment on my part to go in one direction or another.” He was forthright, he was funny, and he was crafty.

But still, nobody really knew the man. He'd served on the U.S. First Circuit for less than two months when he was nominated to the highest court in the country. He'd penned nothing controversial. President George H. W. Bush's chief of staff, former New Hampshire governor John Sununu, had been telling conservatives like Paul Weyrich and Gary Bauer that Souter would be a “home run,” but even Sununu didn't really know him. Souter's biggest supporter, New Hampshire senator Warren Rudman, was himself confoundingly moderate. Was there a hidden truth?

If so, no one could dislodge it. Over the course of those three days, Souter revealed himself as a quiet, genial sort who would probably tilt the Court's balance to the right. One of the deans of the conservative movement, columnist James Kilpatrick, rejoiced at the time, “With Souter aboard, it's the conservatives' turn.”

That was back when we thought we knew David Souter.

We know differently now.

“i think david's view is that if someone wants to kill him,” one of Souter's friends told me, “they probably will.” Recently, a man incensed by a Souter opinion granting local governments the right to seize private property threatened to erect a hotel on Souter's land. This is what happens when a professed conservative upholds a woman's right to an abortion, bans the Ten Commandments from courthouses, and now allows a state to seize someone's land.

Back in Washington, Souter haters show a little more restraint. “As a conservative, we aren't very happy with him,” deadpans Gary Bauer. Grover Norquist agrees: “He's been a consistent vote on the left for some time.” Bauer says he was assured by Sununu back in 1990 that this would not be so and therefore kept his doubts to himself during the confirmation hearing. (“I think I'll take a pass on this,” Sununu says today.)

The right sees it like this: They played ball with Bush 41…and got hoodwinked. They've made it clear to Bush 43 that it's not going to happen again. Alberto Gonzales? That's Spanish for Souter, they said. Try again.

So George W. Bush went with John Roberts. Much like Souter—whom Bush 41 introduced to the nation as a judge who would “interpret the Constitution and not legislate from the federal bench”—Roberts was a brainy, unassuming, nearly track-recordless conservative who Bush 43 vowed would “strictly apply the Constitution in laws, not legislate from the bench.” A queasy sense of déjà vu wafted over Washington. “It remains a mystery how Roberts will handle something like Roe v. Wade,” frets Bauer. The right consoles itself with Roberts's “stable” lifestyle. “People say he may be like Souter—but no, he's got a family and friends here,” says Norquist, who agrees with Bauer's theory that Souter “fell into the influence of the prevailing liberal ethos of Washington.”

But the notion that David Souter fell captive to Beltway elitism and morphed into the Supreme Court's liberal counterweight to Antonin Scalia has only one flaw:

Souter has nothing whatsoever to do with Washington.

In his fifteen years there, he has forged few, if any, relationships. “He said, ‘When you have a job like I have,’ ” one of his associates from New Hampshire told me, “ ‘the friends you have are the ones you had before you got the job.’ ”

“Does he like to go to embassy parties and big dinners and follow the social scene in Washington? Clearly no,” says former senator Rudman, one of Souter's closest friends. But it's not only the frivolous stuff that Souter avoids: You won't see him at many Supreme Court Historical Society events, either. “Saying Justice Souter had some difficulty settling into Washington makes it sound like he has settled into Washington,” says Meir Feder, one of Souter's clerks during his first term. “He sees himself as temporarily there.”

Even by the monastic standards of the Supreme Court, Souter's withdrawn nature is conspicuous. “I don't remember any justice as private as Souter,” says journalist Lyle Denniston, who has covered the Court for the past forty-seven years.

The thing that most infuriates conservatives who feel betrayed by Souter is the dawning reality that he will never answer for his sins. He has not spoken on the record to any member of the press since his appointment. He has written no books. His former clerks abide by an implicit vow of silence. Not one of them cooperated with Souter's biographer, Tinsley Yarbrough, despite Yarbrough's pronounced admiration of Souter and despite the fact that the publisher, Oord University Press, is the imprint of one of Souter's alma maters. (Of the twenty former clerks contacted for this story, only three agreed to be interviewed.) And his close friends speak of him with extreme guardedness, reminding inquirers that

David is very, very private.

Here, according to those who know him best, is David Souter's life:

Every morning before nine, he departs the same tiny apartment in southwest D.C. that he has lived in since arriving in a U-Haul truck in 1990. He wears a suit, unless it's the weekend, in which case he wears a blazer and tie. In his chambers on the Court's first floor, he reviews the day's business with his clerks, who uniformly adore him. “One of the common experiences among Supreme Court clerks,” says one of them, “is that you end your clerkship thinking much less of the justice than when you started. I don't think that ever happens to a Souter clerk. He's impressive in his intellectual power, his integrity, and his humanity. He treats everyone as an equal. Not all the justices are the same in that respect.”

At his desk—behind which hangs a portrait of Chief Justice (and fellow New Hampshirite) Harlan Fiske Stone—Souter writes his opinions in a script that is both elegant and indecipherable. (A computer sits off to the side, gathering dust; Souter has no idea how to use it.) He reads by natural light his clerks' bench memos, which unlike those for other justices are a mere two pages long, since Souter doesn't delegate the reading of briefs to his subordinates. He has lunch in his chambers: plain yogurt and an apple. (He eats the whole apple, seeds and all.)

Now and then he will drop in on the other justices, but only rarely. It has been said that the Supreme Court is like nine separate law firms, and as Souter once told a gathering of legal professionals, there is “great truth to that metaphor.” Rehnquist clocks out by four, and one by one the other justices follow suit—leaving the building to Souter, who often stays until eight or nine. After which he drives home, changes clothes, jogs alone around the track at Fort McNair, buys some groceries at the local Safeway, returns to his apartment (“I would say the word is spartan,” says Rudman of Souter's style in general), prepares his own dinner, and reads himself to sleep (he is a big fan of British history).

He does not go to movies, doesn't watch television. Occasionally, he will visit a museum or take a woman out to dinner—though Barbara Bush failed in her efforts to fix up Souter with one of her cousins. (It has long been rumored that Souter is gay. Nothing bothered him more than this insinuation during the confirmation process, which compelled a couple of his es to come forward and out him as a…heterosexual. “That really ripped him up,” Rudman recalls. “He thought it was a totally intrusive thing.”) He celebrates his birthday in New Hampshire, says his friend Bill Glahn, “by running up a really steep hill in Dunbarton, just to prove to himself that he can still do it.” He is, as another friend says, “a creature of the seventeenth century.”

on june 29, 1992, twenty months after Souter had been confirmed, Warren Rudman stepped out of a train and saw Senate Judiciary Committee chairman Joe Biden running toward him at a full gallop. The Democrat from Delaware threw his arms around his Republican colleague, exclaiming, “Warren, you were so right!”

What Rudman had been “so right” about was Souter, of course, and the source of Biden's ecstasy was that day's Supreme Court ruling in Planned Parenthood v. Casey, the biggest test of a woman's right to have an abortion since Roe v. Wade sanctioned the practice in 1973. Since that time, liberal justices William Douglas, Thurgood Marshall, and William Brennan had vacated their seats. Pro-lifers expected O'Connor, Kennedy, and Souter to deliver the goods. Their hopes were dashed. All three sided with the Roe flank. From the bench, Souter said of the 1973 abortion case that to overrule it “would subvert the court's legitimacy beyond any reasonable question.” Nine days later, in The New York Times, Robert Bork responded. “The three who controlled the outcome—Justices Sandra Day O'Connor, Anthony M. Kennedy and David H. Souter—are called by the media ‘centrists’ and ‘moderates,’ ” he wrote. “They may be that on the political spectrum, but on a constitutional spectrum their joint opinion is more properly termed ‘radical.’ ”

Not every conservative went ballistic. Senator Orrin Hatch would later recall that during the confirmation hearings, “I argued that Judge Souter would be fair and would follow precedent.” Roe v. Wade happened to be the precedent in this case, and Hatch behaved like a grown-up about it. And today another former Judiciary Committee member, Alan Simpson, says, “I haven't changed my opinion of him at all. He's got a great, facile mind—a splendid man, writes his opinions based on legal scholarship. I've heard that crap about every one of the seven I put on the Supreme Court. My view is, if they were good lawyers and used sound reasoning, I didn't give a damn where they stood on abortion. I don't even think male legislators should vote on the issue.”

Still, from Casey onward, the right would regard Souter with growing outrage. The language from that majority opinion—“At the heart of liberty is the right to define one's own concept of existence”—horrified social conservatives like Gary Bauer, who saw in him a man bent on “social engineering.” Top conservative legal minds have also expressed reservations. “My own view is that Justice Souter tends to be more result-oriented than some other members of the Court,” says William Bradford Reynolds, an assistant attorney general under Ronald Reagan, “and thus increasingly appears to allow his desired result to persuade him to stray from the letter of the law and rely upon extraneous considerations to shape his opinions.… It appears Justice Souter's years on the Supreme Court have made him more, rather than less, comfortable with this approach—an approach with which, at the time of his confirmation, he expressed considerable—dare I say ‘complete’—discomfort.”

In the coming years, the frustration would only deepen. Souter ruled consistently against the use of state funds for religious activities as being “categorically forbidden under the Establishment Clause” of the First Amendment. He often voted to review death-penalty cases. In one of his many dissents against the Scalia-led charge to grant states immunity from being sued by individuals, Souter likened this judicial meddling to the Court's early-twentieth-century “experiment in laissez-faire, the one being as unrealistic as the other, as indefensible, and probably as fleeting.”

In an earlier world—say, 1973, when decidedly unliberal justices Warren Burger, Potter Stewart, and Lewis Powell all ratified Roe—such positions might have been appropriated into the Republican Party platform. But by the mid-'90s, both the party and the Court had left Souter behind. Scalia's dissents began to call him out by name, taunting his opinions as being “linguistically and conceptually absurd.” A clerk who had helped draft one such target went pale as he read Scalia's stiletto-like dissection.

Souter smiled and told him, “That's just Nino being Nino.”

Being Scalia also meant championing Christianity's traditional place in American life. Spiritually, Scalia, a devout Roman Catholic, had nothing on Souter, who has attended the same Episcopal church in Hopkinton, New Hampshire, for decades. And so when Scalia proclaimed, in a case involving whether a Kentucky courthouse could post the Ten Commandments on its wall, that government could favor religion “over irreligion,” Souter reacted with characteristic caution. “This is no time,” he wrote, “to deny the prudence of understanding the Establishment Clause to require the Government to stay neutral on religious belief, which is reserved for the conscience of the individual.”

Souter prevailed, 5–4, in that case. But the specter of John Roberts—and the possibility of his joining the Scalia faction—suggests that victories such as these will be increasingly hard for Souter to come by. His role on the Court has come to echo that of John Marshall Harlan II and the Great Dissenter, Oliver Wendell Holmes Jr. His explorations into when precedents must finally give way to workable new constructs would someday, as one Court expert puts it, “amount to a far more important jurisprudential deposit than Scalia's glib and clever writing.” And as for his “activism,” Emory University law professor David Garrow believes the opposite is true: “Activist is one of the last words I'd use for Souter. If anything, he views himself as playing a defensive role against other, activist judges.”

At no time was that more apparent than on December 12, 2000. It was one thing to be on the losing end of Bush v. Gore. But after the Supreme Court's 5–4 ruling halted the Florida vote recount and effectively awarded the presidency, Souter worried privately what others spoke aloud—namely, that the decision did not bear the aroma of judicial impartiality. After years of championing “state sovereign immunity,” his colleagues on the right had seen fit to butt in to Florida's affairs. And the Rehnquist-led majority had done so with uncustomary timidity bordering on sheepishness: The ruling, they said, would be applicable only to this one case. It only added to the irony that the singular result was to elevate to the White House a candidate who decried judicial activism.

Souter ended his dissent with the customary “I respectfully dissent,” unlike Ruth Bader Ginsburg, and he didn't join the openly hostile dissent written by John Paul Stevens. He refused all interviews, as always, and declined all speaking invitations. And when he vented to his closest friends, he made it clear that his sentiments must never be aired, though one of them would later recall, “It was a very hard winter for him.”

Yet winter did pass. In the spring, Souter held an annual reunion for his former law clerks—as always, an unpublicized event, with even the clerks' spouses forbidden to attend. A few clerks performed a skit lampooning Bush v. Gore, and so it wasn't a surprise that when it came Souter's turn to speak, the dominant topic was the landmark case. “He viewed it with a lot of sadness,” recalls a former clerk. “He said that he had come to terms with it, though it had been difficult. He had to figure out a way to understand it so that he could go forward and continue working with these people without thinking that they were ideologically driven and unprincipled.”

He revealed that “way to understand” later in the month, to a private gathering of judges and lawyers. “His point in the speech,” recalls one who was in attendance, “was ‘We don't know if we've gotten it right, and the only way we'll know is through history.’ And he was willing to consider that he might have gotten it wrong.”

while beltway noisemakers were trashing him all summer long, Souter was back home in New Hampshire—“decompressing,” as he puts it. He was having dinner with an old Harvard classmate, attending a neighbor's funeral, submitting to well-wishers at the Shaw's market in Concord, but mostly he was passing his days reading cert petitions in his Concord office and his evenings in the old family farmhouse at the end of an unmarked dirt road.

Is there another man in Washington who wields such influence over American life while being so far removed from that city's culture?

Aloofness has long agitated those who disagree with the Court. White racists pilloried Earl Warren for filling their schools with colored children. Nixon supporters were dismayed when Burger, Rehnquist, and Powell—the very men he'd sent to the Court—brushed off his claims of ecutive privilege and effectively forced him out of office. Today it's Souter who offends, even as he appears to lead a life of quiet decency that his detractors can only admire. Souter “is intensely scrupulous,” says David Garrow. “He in no way wants to be used or, for that matter, to appear to be used.”

There has been creeping judicial activism in recent years, yes, but Souter has been its enemy, not its exponent. Fundamental to his cautious rulings, says his former clerk Meir Feder, is that “people order their lives around what the law is. One reason not to overrule something is that it has become part of the social fabric.”

That fabric is currently being rewoven. Activists from the right are pushing for more profuse religious expression. Death-penalty advocates seek an unclogged ecution process. The Bush administration wants greater autonomy in prosecuting the war on terror. As Souter sees it, these things are not always constitutionally grounded. And as a result, a fundamentally conservative soul is now viewed as a betrayer of conservative values. “The Souter story,” says longtime Court reporter Linda Greenhouse of The New York Times, “is really a story of what's become of the Republican Party in our time. Souter is an old-line Republican. His great-grandfather was New Hampshire's representative at Lincoln's funeral. He would certainly say that he hasn't changed.”

Already there's been speculation that a newly confirmed John Roberts might align himself with the pragmatic Breyer and the ambitious Kennedy in a center-right coalition that would seek to steal votes away from the Scalia-Thomas-Rehnquist troika. That would leave Souter in the marginalized company of Stevens and Ginsburg. He would be stranded out on the left pole, where no one in 1990 ever would have pictured him.

Use of this site constitutes acceptance of our User Agreement (effective 1/4/2014) and Privacy Policy (effective 1/4/2014). GQ may earn a portion of sales from products that are purchased through our site as part of our Affiliate Partnerships with retailers. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with prior written permission of Condé Nast.