In contrast, shortly after he confessed, Madoff declared his household net worth to be between $823 and $826 million, according to court documents. While the Madoffs clearly lived opulently, no evidence has emerged that their combined assets and expenditures approached the amount the Picower family is alleged to have withdrawn from the scheme.

In an era when billions of dollars are being tossed about in financial collapses and government bailouts, remarkably little attention has been paid to Jeffry Picower's extraordinary success with Bernie Madoff. If Picower has penetrated the popular consciousness at all, it is as a Madoff victim. The victim narrative is buoyed by testimonials from the nonprofits who received funding from his charitable foundation – which quickly closed on the heels of the swindler's confession. For this reason, ProPublica decided to take a closer look at both Jeffry Picower and the complaint filed against him by Madoff trustee Irving Picard.

Fortunately for the trustee and the federal investigators presently swarming over the case, Madoff apparently kept detailed notes of communications between his office and his clients. But despite this documentary evidence, which is cited but not provided in court documents, Picard's complaint raises more questions than it answers. Above all, what was the exact relationship between the two men? The complaint is larded with the legal catch-all phrase, "knew or should have known," to describe Picower's cognizance of Madoff's fraud, but the intricacies of the relationship are left to the imagination.

The complaint states that the Picowers were beneficiaries of the Ponzi scheme for more than 20 years. The withdrawals listed between 1995 and 2008 reveal a pattern of large quarterly disbursements, transferred to Picower-controlled accounts by check or sometimes wire, that peak in 2003. Three years later something happens that causes the amount to drop precipitously. It recovers slightly the following year, but the highest-flying days are over for good.

One question is the role that Picower's charitable giving played in all of this. The amount Picower withdrew for his foundation is separate from the quarterly withdrawals for his personal accounts. During the 1995-2008 time span, Picower took out about $291 million from Madoff for the foundation account. During the same period, the foundation doled out a little under $207 million in donations, according to tax forms.

Perhaps the most pertinent question: If Picower withdrew $5.1 billion in "profit" from Madoff, where did all the money go? The Picowers own a home in Palm Beach that is appraised at a little over $28 million. They also have a 28.4-acre compound in Connecticut valued at $4.5 million. A search of numerous online sources, both aggregate databases and county property records for the couple, their daughter, and the companies named in the complaint, reveals few other major assets. If someone needed the skills to hide billions of dollars, few would be better equipped than Picower, an attorney and accountant who has been described as a "tax shelter expert." Even so, it's curious our search did not even uncover a boat or plane under the Picower name.

Messages left for Picower and his wife Barbara requesting comment for this story were not returned. Their lawyer, William Zabel, declined to comment to ProPublica on the Madoff matter. Earlier, Zabel told The Wall Street Journal that the couple "were in no way complicit in" Madoff's scheme.

Emailed questions to David Sheehan, an attorney at Baker & Hostetler who is working alongside Picard on the case, went unanswered.

Picower, 67, began his career as an accountant and lawyer in New York but seems to have made much of his fortune as an investor in the medical industry. He has avoided media interviews and, with a few notable exceptions, succeeded in keeping a low profile. If the Picowers were recognized at all, prior to their Madoff notoriety, it was through praise for their philanthropy. Yet even here, their ties to Madoff loomed large. The growth of their largest foundation was attributed to their Madoff investments. Madoff himself served as a trustee on another Picower foundation.

The court-appointed trustee makes a powerful, albeit still largely circumstantial, case in court filings that Picower knew Madoff's fund was illegitimate. Although Madoff ostensibly produced eerily consistent 10-12 percent annual returns for his clients, the returns he provided Picower were other worldly:

In 14 instances between 1996 and 2007, a group of Picower trading accounts experienced annual returns of more than 100 percent. On 25 occasions, the annual return exceeded 50 percent. During this same period, the biggest annual gain in either the Dow Jones Industrial Average or the S&P 500 was 31 percent, for the S&P in 1997. The S&P 500's annual average for that period was slightly under 9 percent.

The annual rate of return for two of Picower's regular trading accounts in the four years between 1996 and 1999 ranged from about 120 percent to more than 550 percent annually.

In 1999, one account earned 950 percent.

Picower belonged to a select group of Madoff investors who received souped-up returns. A Wall Street Journal story published in May cited unnamed sources saying that prosecutors were looking into eight investors who appear to have received special treatment from Madoff. Among the eight named, Picower seems to have withdrawn the most money, with the bulk of it coming from an account called "Decisions, Inc." According to the Madoff trustee's court filings, "the account reflected little trading activity and relatively few holdings," yet Picower took hundreds of millions out of it. At the time of Madoff's arrest, the account had a reported negative net cash balance of more than $6 billion.

At the beginning of each quarter, the Picowers received sums that grew from an annual total of $330 million in 1996 to $1 billion in 2003. These withdrawals were divided into odd numbers spread over various accounts. Added together, they usually equaled large even sums. For example, on January 2, 2003, Picower withdrew $1,378,852 from his account Jln Partnership. Yet when withdrawals across all accounts were totaled for that day, they amounted to precisely $250 million.

Picower's quarterly withdrawals reached their zenith in 2003 and then decreased by half the next year, eventually dropping to their lowest point in 2006. For some reason, the quarterly withdrawals totaled an uneven $16,975,422 in 2006, only to rebound to exactly $40 million in 2007.

Picower's extraordinary gains do not appear to have been achieved at random. The trustee's complaint details how Picower, often acting through a subordinate, ordered up "returns" which Madoff's office then delivered. In some cases, Picower is alleged to have requested backdated returns for trades or sales of securities.

On April 18, 2006, Picower wired $125 million to Madoff to open a new account. Madoff's office began "purchasing" securities in the account, but "it backdated the vast majority of these purported transactions to January 2006" when the stock market was at its lowest for the period, according to the complaint. Twelve days later, the net equity value of the account was $164 million, a gain of $39 million – or more than 30 percent – in less than two weeks.

The complaint details conversations between Picower associate April Freilich and Madoff's office beginning around May 14, 2007, when Freilich stated that the Picower Foundation needed gains during January and February of 2006, a year earlier.

On May 18, Freilich specified that the foundation needed "$20 mil in gains" and "want[ed] 18% for year[] 08 appreciation" for January and February of that year. Five days later, Freilich changed the amount to $12.3 million. Subsequent statements reflected gains of $12.6 million.

On December 22, 2005, Picower or Freilich allegedly faxed a letter to Madoff dated December 1, directing him to sell specific holdings. The statement for that month reflects that the sales were finalized on December 2, a process that typically takes three days. The clues that the letter was back-dated: A fax with the tell-tale December 22 date and an attached portfolio appraisal dated December 16 that included the positions that were supposedly "sold."

On or around December 29, 2005, Freilich allegedly faxed a letter to Madoff asking for $50 million in gains across certain accounts. Subsequent statements generated by Madoff for the accounts show stock sales, presumably to satisfy the request, that were supposedly recorded around December 8 and 9, 2005, approximately three weeks prior to Freilich's letter.

Who is Jeffry Picower?

The Picowers' generosity to deserving charities, particularly in New York, Florida, and Massachusetts, has earned them admiration and respect. However, the image belies a more complex reality.

The Picowers gave to a host of worthy causes from the Children's Aid Society to the New York City Ballet, but Jeffry Picower's passion centered on health issues, particularly funding for medical research. On December 1, 2005, the couple made a rare public appearance at a ceremony at the Massachusetts Institute of Technology to dedicate a new center to study the brain, the Picower Institute for Learning and Memory. The Picowers' gift of $50 million, spread over five years, was the single largest in the school's history. In a video of the ceremony, Nobel Prize winning Japanese scientist Susumu Tonegawa told the crowd that without the Picowers there would be no institute. The Picowers stood by silently as Tonegawa unveiled a portrait of the couple to conclude the ceremony.

The gift to MIT was the largest single donation to an outside entity the Picower Foundation had ever made, according to tax forms the foundation filed. Prior to late 1995, when Madoff trustee Picard's records start tracking Picower's Madoff activity, the Picower Foundation was relatively small in size and scope. At year's end in 1994, it had assets of just under $75 million and had donated $375,754. By 2007, it was reporting about $958 million in assets and about $23.4 million in donations for the year.

Picower's attention to cutting-edge medicine was the sole focus of a second foundation, The Picower Institute for Medical Research, created to find cures for human diseases. He launched the Institute in 1991 with a $10 million donation from the Picower Foundation. Madoff served as a trustee of the Institute.

In 2001, the St. Petersburg Times revealed that Picower used both his foundations and a private corporation called PharmaSciences, of which he was the majority shareholder, to gain control of a potentially lucrative medical discovery. In 1999, Picower merged PharmaSciences with a for-profit spinoff of his institute called Cytokine Networks, essentially negotiating with himself. The merged company called Cytokine PharmaSciences had the rights to develop a new drug that could help minimize such illnesses as arthritis and multiple sclerosis. The newspaper raised the question of whether Picower had shortchanged his nonprofit in the deal.

An IRS audit concluded that the Picower Foundation had not jeopardized its tax status or incurred extra liability during the period in question. The Foundation's lawyer William Zabel provided ProPublica with a letter from the IRS dated September 2006 that he said "cleared the Foundation." Addressed to Barbara Picower, the letter is from the IRS' Office of Exempt Organizations and formally accepts the Foundation's tax returns. Zabel also said that shares Picower received from the merger were given to other charitable organizations.

The publicity-shy Picower is no stranger to lawsuits or regulators, a trip through several decades of legal and regulatory filings reveals. In 1984, the SEC cited him for a late disclosure over how much he owned in a company called Bradford National Corporation. The SEC filing alleged that Picower was part of a scheme to take over the company.

A year later Picower had to pay out a $21-million settlement when shareholders sued over the collapse of Physicians Computer Network. Picower controlled 45 percent of the stock and chaired the company before it went belly up. In 1989, Picower paid an undisclosed settlement over a questionable tax shelter he helped set up years earlier for a client. In 1990, it was Picower's turn to recover money – from a settlement involving infamous insider trader Ivan Boesky. Picower had been one of his investors.

Perhaps the most revealing case against Picower was a lawsuit filed for failure to pay for renovations on his New York office. Picower alleged that general contractor McHugh, DiVincent Alessi had done such a bad job on his office that the toilets didn't even flush properly, according to Jeremy Morley, the firm's lawyer at the time. Rather than take Picower at his word, the judge decided the jury members should see for themselves. Arriving by city bus from the courtroom, the judge, with jury in tow, made a surprise visit to Picower's office.

"The jury wandered around, looking at the office, testing the toilet," Morley said. "They quickly realized that the case was full of crap in more ways than one."

The judge awarded Morley's client what Picower owed them and some of their attorney fees, according to the lawyer.

Picower's legal and regulatory history was outlined in an article by Forbes Magazine. The 2002 article, which didn't mention Picower's activities with Madoff, said Picower was "worth at least $300 million." That same year, the trustee reports, Picower's quarterly withdrawals from Madoff totaled $895 million.

ProPublica Research Director Lisa Schwartz and Kitty Bennett contributed to this report.