Most of the furnishings in the dining room of Sam Wyly’s Aspen, Colorado, condominium do not reflect his own taste. His wife, Cheryl, did the decorating. A pair of portraits of grim-faced children flank the dining room table. They were bought in Prague. The table itself, a hulking 12-seater carved from a reclaimed barn door, came from Mexico. A curving staircase leads from the front door to the dining space, and it is lit by a massive spotlight that hangs from the ceiling. It was found on a Hollywood salvage lot and dates to the 1930s, the same decade in which Sam was born.

“Ah, but this is more my style,” he says, as he leads me away from the dining table, through a corridor, past a study, and to a landing at the top of yet another stairwell. There he points to a framed tattered flag that bears a resemblance to the Stars and Stripes. Sam lets out a high-pitched giggle, the one that often punctuates his speech, and he asks mischievously, “Would you have ever guessed that this is a Confederate flag?”

It’s a battle flag that was flown by a Texas infantry regiment in the Civil War. Sam’s maternal great-great-grandfather, Edward Sparrow, was a Confederate senator who fought with Louisiana regiments. He also built a plantation called Arlington, which the family still owns. It’s where Sam married Cheryl, his third wife, in 1994.

On the wall of Sam’s office, which lies just beyond the stairwell with the Confederate battle flag, there hangs a photograph of the couple posing in front of Arlington. And in the couple’s home on Beverly Drive in Highland Park, there’s a painting they commissioned called Moonlight and Magnolias, in which they are seen dancing at Arlington during a Civil War-era ball. They are wearing Gone With the Wind garb—she in a hoop skirt, he in the gray uniform of a Confederate general.

“And there’s our buddy George,” Sam says, giggling again and pointing at a small photo of himself with George H.W. Bush. The 41st president was one of 192 Republican candidates and causes that Sam and his older brother, Charles, have backed with their money, contributing about $10 million, by their own estimates. That total dates from 1968, when Sam directed Richard Nixon’s election effort in the state of Texas. He stopped working in politics shortly thereafter but kept writing checks, including those that supported the elder Bush in his 1988 win and his 1992 loss. “I was friends with George, and I was very good friends with Ross Perot,” Sam says. “I had a heck of a decision. Two friends running for president. But we stuck with George.” With both Georges, actually. The Wylys were among the bigger donors to George W. Bush during his presidency. Indeed, Wyly money was pivotal to the younger Bush’s securing the GOP presidential nomination in 2000.

But during George W.’s final term, the Wylys were forced to stop giving money directly to candidates. They discovered that their cash was no longer welcome in Washington, D.C., thanks to a public inquiry into their finances conducted by the U.S. Senate Permanent Subcommittee on Investigations, which came on the heels of a less public Securities and Exchange Commission investigation.

The latter investigation led to an SEC lawsuit filed against the Wylys in the summer of 2010. It alleges that the brothers knowingly sought to bypass securities disclosure laws through a complex system of offshore trusts and shell corporations. It also claims the Wylys engaged in an insider trade, buying up shares of Sterling Software, a company they had spent a decade building into a giant, right before the firm was put up for sale. The SEC says that trade netted the brothers more than $31 million in profits that it wants returned. Add in the other allegedly improperly disclosed trades made offshore, and the SEC seeks a total of $550 million. To put that in perspective, that’s the amount the SEC got from Goldman Sachs in 2010, the largest penalty ever imposed on a Wall Street firm. It’s also half of what Forbes estimated was Sam Wyly’s total net worth in 2011, when the magazine ranked him the 393rd richest man in the world.

The SEC’s case against the Wylys is made all the more unusual because it seeks money from a dead man. Sam and Charles had been inseparable, from their childhood growing up in a Depression-era farmhouse in rural Louisiana; to their playing days on a high school football team that won the state championship; to their five decades in business together, building two billion-dollar companies, Sterling Software and Michaels Stores—even to their joint choice of Aspen as a summer vacation spot. That’s where Charles died in August 2011, when an SUV T-boned his Porsche 911 Targa as he tried to cross a highway. Five months later, the SEC won an unprecedented court decision that allows it to seek payback of any illicit gains from Charles’ estate.

“I miss him,” Sam says. “I miss him every day. I still feel the pain when I drive down where the car wreck was. It’s hard to drive down that way.”

You know what’s not hard for Sam Wyly? Being locked out of Washington, facing a $550 million lawsuit by himself, and trying to clear his brother’s name. If you thought this fight would consume the billionaire, maybe cost him sleep, cause him to pace around his Aspen condo in the middle of the night while trying not to run into that huge wooden table or get freaked out by those creepy kid paintings, then you thought wrong. Not only is Sam Wyly sleeping well, he’s hardly thinking about the lawsuit at all.

When I met Sam Wyly in late fall, it was the first sit-down interview he’d done since his brother’s death (and he’d talked about the allegations of offshore misdealings only two other times, with the Dallas Morning News in 2006 and with the New York Times in 2010). I had already talked with Charles. We met for what would turn out to be his last interview, in his Crescent Court office, three months before his death. The accident happened only a few miles from Sam’s seven-bedroom, 7,000-square-foot, $7.6 million downtown Aspen condominium. Well, technically Sam doesn’t own the condo. His wife, Cheryl, owns it. Or, rather, the Cheryl R. Wyly Marital Trust owns it.

According to that 2006 report from the U.S. Senate Permanent Subcommittee on Investigations, the Wylys’ condos were part of more than $80 million in U.S. real estate purchased using funds Sam and Charles had transferred to a network of offshore trusts that stretched from the Cayman Islands, in the Caribbean, to the Isle of Man, in the Irish Sea. The Senate’s Democratic investigators say $1 billion in Wyly money was shipped offshore between 1992 and 2005. Much of that was in the form of stock and stock options and warrants the Wylys received as compensation for their roles as directors of Michaels Stores and Sterling Software. The trusts took possession of the stock. In exchange, the Wylys got annuities that paid them a set amount of money over a long term. That setup bought the brothers a massive tax deferment. The Senate subcommittee’s chairman, Carl Levin, a Democrat from Michigan, says the Wylys’ offshore system helped them avoid paying $300 million in federal taxes over 13 years.

Levin’s investigators also implied that the Wylys intended to undermine and circumvent U.S. tax laws with their offshore network of 58 trusts and dummy corporations. But despite pouring through more than 1.5 million pages of Wyly documents and issuing subpoenas to most of the Wyly associates who oversaw the offshore network, the subcommittee’s investigators failed to deliver anything that could convince the IRS to take action against the Wylys. In fact, the Wylys’ attorneys say the IRS never notified them that it was even looking into the brothers’ offshore network.

“We got invited to the hearing,” Sam recalls, now sitting at his desk, framed by a window offering a spectacular view of mountains splashed with bright yellow Quaking Aspen trees. “And you know who qualified for it? The Democrats picked me and the Johnson Band-Aid guy to be up there, and the Republicans picked the biggest Democratic fundraiser at the time, whose name I’ve forgotten. But why would we go out to Washington? So we could get beat up on all day long? Instead, I watched it from here.”

He ends the sentence with that giggle. It’s a reflex, the giggle, a quirky habit like the one he has of repeatedly brushing back his longish brown-gray hair when he’s telling a long story. The New York Times, in its 2010 story on the SEC case, said the high-pitched giggle makes Sam seem “a little bit manic.” I respectfully disagree. The giggle, the hair flip, the way he shouts—squeals, practically—down the curving staircase when his daughter Lisa and her son enter the door, “Oh! It’s my genius daughter and my genius grandson!” it all just makes Sam Wyly seem completely at ease with himself. And happy. Crazy happy. Which is about the very last thing you’d expect from a man whose name, and that of his dearly departed brother, has been dragged through the mud for the past six years and whose fortune—which, no one would disagree, he took extraordinary measures to protect—is now at risk. A Dallas-based business colleague of Sam’s told me before I arrived in Aspen, “Sam’s a weird duck.” That’s one way to put it.

Sam’s religious beliefs also help explain his generally sunny disposition. Sam’s mother, Flora, a onetime New York dancer, was an early devotee of Christian Science, which was founded in 1879 by Mary Baker Eddy. Every Sunday morning when the brothers were growing up, Flora would take Sam and Charles to the nearest Christian Science church, some 40 miles from their house in Lake Providence, Louisiana.

In accordance with the teachings of “Mrs. Eddy,” as he calls her, Sam knows that Charles isn’t gone. Though Sam’s eyes well with tears when he speaks of his brother, Christian Science teaches him that Charles’ spirit lives on. This is not in the Pearly Gates sense that other Christians think of, but in a sort of Zen, oneness-with-the-universe sense. Christian Science teaches that life, the physical manifestation of it, is an illusion. The only reality is spiritual reality. That’s why some Christian Scientists believe you can and should pray away pain rather than take medicine or submit to surgery. To them, the feeling of pain is simply the presence of evil, or the “error” of the universe, which has fooled us into believing in our physical existence when we really exist only on a spiritual plane. In more secular terms, think of it like the Matrix.

Sam references Christian Science often during our conversation and, without being asked, gives me an overview of why the religion is important to him. In his autobiography, he wrote: “What I got from my spiritual training is much more than whether or not to take aspirin. I learned to focus on spirituality, on mental power as opposed to material power, and I developed a philosophy of individual empowerment. ... Mrs. Eddy taught me always to be positive. She wrote, ‘Stand porter at the door of thought,’ not to accept negative ideas from others.”

A small writing desk sits in the corner of Sam’s office. Given his frequent references to Mrs. Eddy, its provenance is a curious one. It belonged to Mark Twain, who, three years before Mrs. Eddy’s death in 1910, wrote “Christian Science,” a series of scathingly satirical essays on the religion. But Sam has an affinity for Twain because he traced his family’s lineage to the master American wordsmith (and also to Andrew Jackson, Daniel Boone, Davy Crockett, Sam Houston, Ulysses S. Grant, and Teddy Roosevelt, which would also make him related to Teddy’s fifth cousin, Franklin Delano Roosevelt, though no one talks about that last connection). To accompany the desk, Sam commissioned a portrait of Twain. On the day of my visit, the valuable antique isn’t displayed as a museum piece. It’s being used as, well, a desk. It’s covered with scraps of paper that have hand-written calculations on them, plus a few printed pages that look like invoices, a Hewlett Packard Reverse Polish Notation Calculator, and several stacks of books—but only two different titles.

One is Texas Got It Right!, which was penned by Sam and his son Andrew. Released to stores last year, the book is a treatise on the Lone Star State’s entrepreneurial spirit and the prowess of its limited government. The other book is The Wyly Family: Coming to America, Vol. 1, a gold-leaf, leather-bound tome that’s part American history textbook, part compendium of personal anecdotes from Sam, and part Wyly family yearbook. Every year, Sam assembles his large family—six kids, 12 grandchildren, two great-grandchildren—and takes them on a trip. They are pictured in Aspen, Dallas, Paris, Malibu, Milan, on ski slopes and sandy beaches and at backyard barbecues. Sam wrote The Wyly Family and is already at work on the next two volumes. “I don’t know if my grandkids are interested in it,” he says. “But I hope they are.”

This is how he spends his days now: sorting through boxes of family photographs. What he is most definitely not spending his days on is the case that could cost him and his kids and their kids a quarter of the fortune that it took the Wylys a lifetime to build. “I almost never think about that,” Sam says of the SEC case. “We have people to handle it. And I’ve always known we had nothing to fear. But it is a burden. It is costly. It has cost money for a long time.”

The meter started running on November 16, 2004. That Tuesday, two weeks after George W. Bush thumped John Kerry in the presidential election, the Manhattan District Attorney’s Office opened its investigation into the Wylys’ offshore network. The SEC investigation opened around the same time. But, a year later, the Manhattan DA’s investigation was dropped after the Wylys filed securities disclosures restating some of their holdings in Michaels Stores. In those filings, the brothers declared that they were the beneficiaries of offshore trusts that controlled a significant percentage of Michaels’ stock.

The restatement didn’t end the SEC investigation. And six years later, after Barack Obama was elected and installed his own SEC commissioner, the agency filed suit against the Wylys. There was discussion of a settlement, but it was brief, according to Sam’s longtime attorney, William Brewer of Dallas-based Bickel & Brewer. “My clients declined to resolve the case on any basis that in any way would damage the reputations they had established over the course of a lifetime,” Brewer says.

The Wylys have the money to protect that reputation. “Some SEC settlements happen when people simply don’t have the resources to fight,” says Wayne State University Law School securities law expert Peter Henning, one of the few academics who has closely followed the SEC’s arcane case against the Wylys. “But the Wylys do. They can match the SEC dollar for dollar, lawyer for lawyer.”

Maybe that’s why Sam is so unruffled about the whole mess. Maybe the mere fact that he can afford to fight the case for as long as it takes explains why he rarely addresses it in the family meetings he has with his kids every 60 days. “We talk about the case maybe only once every four meetings,” Sam says.That works out to once every eight months. “There’s nothing to say, really,” he continues. “We trust the firm to handle it. And I’ve always known we had nothing to fear. The government is going to make a lot of noise, but they are going to lose.”

How can he be so sure? I agreed not to ask either Sam or Charles about the specifics of the case. But Brewer says the brothers have been confident of victory because they are certain of their innocence. Of course, that’s exactly what you’d expect someone to say if you’re paying him $1,000 an hour to say it. What you might not expect Brewer to say is that he basically agrees with the SEC. Everything the SEC claims the Wylys did? Brewer says they did it. With one big exception: the SEC leads off its lawsuit by alleging that the Wylys engaged in a “fraudulent scheme to hold and trade tens of millions of securities of public companies while they were members of the boards of directors of those companies, without disclosing their ownership and their trading of those securities.” Swap “fraudulent scheme” for “plan,” and add “trusts” after the word “their.” Yes, that’s exactly what happened, according to Brewer.

“The battleground here is over intent,” Brewer says. “What happened is not in dispute.”

The way the Wylys see it, they did set up a system of offshore trusts that was intended to defer their tax burden and spread out their assets so that creditors, litigators, the government, or whoever couldn’t take away everything they’d spent a lifetime earning. And they see nothing wrong with that

“I pay a lot of taxes,” Sam says. He claims he’s paying 39.6 percent of his annual income to the federal government every year and another $86,000 to the Highland Park School District. The school taxes he doesn’t mind so much, but the federal taxes rankle him. He says if he could reduce that amount, he’d be able to give more to charity. A source in Dallas I spoke with who has worked with both Sam and son Evan Wyly on various business ventures, including as a competitor, puts it more bluntly. “Sam Wyly is the most tax-averse bastard you’ve ever met,” the source says.

Tax aversion is one reason the Wyly brothers began moving their money offshore. This was in 1992, back when few individuals were using offshore trusts for tax deference. But the Wylys, according to the Senate subcommittee’s 2006 report, enlisted the advice of attorney David Tedder. He told them they could establish offshore trusts that would exchange the Wylys’ stock market holdings for annuity payments, small payments made over a long period of time. This setup would allow the Wylys to avoid capital gains taxes on their stock shares because, in a legal sense, the Wylys would no longer own them. The catch: the Wylys could not exercise direct control over the trusts—which is what the SEC alleges they did. But “control” is a legally ambiguous concept.