Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

The Bernie Madoff scandal has rocked the genteel world of Palm Beach, where sprawling mansions are hidden behind tall hedges and security gates and high society events and grand parties are nothing out of the ordinary. Madoff and his wife, Ruth, bought a home in the exclusive community 15 years ago.

ABC

Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

The Bernie Madoff scandal has rocked the genteel world of Palm Beach, where sprawling mansions are hidden behind tall hedges and security gates and high society events and grand parties are nothing out of the ordinary. Madoff and his wife, Ruth, bought a home in the exclusive community 15 years ago.

ABC

Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

Madoff lived the high life in Palm Beach. Here, David Neff, of the ritzy Trillion clothing boutique, shows off Madoff's last purchase at the store he frequented often: a $2000 pair of worsted spun cashmere charcoal slacks.

ABC

Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

The exclusive Palm Beach Country Club was the setting for many introductions of potential investors to Madoff. Formed in the 1950s by people who were rejected from other prominent Palm Beach clubs because they were Jewish, the club is now notorious for a different reason: many of its elite members are now victims of Madoff.

ABC

Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

Many of Madoff's victims are struggling to stay afloat. Palm Beach jeweler Patti Esbia said she is overwhelmed at all of the Madoff victims who are hawking their prized jewels. Her boutique has bought several hundreds of thousands of dollars of jewelry, at distressed prices, since Madoff's arrest.

ABC

Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

There are more than dollar signs on victims' minds. Donald Trump, one of Palm Beach's most prominent social fixtures, tells ABC News that at an exclusive party in the ballroom of his Palm Beach club Mar a Lago, pictured here, just one day after Madoff was exposed, a fight broke out. Who was involved? One of Madoff's "feeders," those people who found investors for him, and one of his alleged victims. "People went a little bit wild," The Donald told Brian Ross.

ABC

Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

The man who was accosted at Donald Trump's club is Robert Jaffe, seen here with his wife. Many victims in Palm Beach blame Jaffe for introducing them to Madoff, and now authorities tell ABC News that he is emerging as a significant figure in the investigation.

ABC

Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

Jaffe lives three doors down from Madoff in Palm Beach in this waterfront mansion. He has been identified as "a piece of this puzzle" by Massachusetts Secretary of State Robert Galvin, who said, "He obviously could have provided us with a lot more information about how the Madoff empire worked. He declined to do so." When questioned by Galvin, Jaffe took the Fifth Amendment.

ABC

Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

Despite the ongoing investigation, Jaffe is still out on the town. Last week, he dined at Bice, a top Italian restaurant in Palm Beach. He declined repeated requests for comment from ABC News. His attorney said Jaffe is a victim and was misled like everyone else.

ABC

Inside Palm Beach: Brian Ross Investigates the Secret World of Madoff

Far away from Palm Beach misery and the town's swaying palm trees and manicured golf greens, an ABC News stake out found Madoff surfing the internet in his designer-decorated den inside his multi-million Manhattan penthouse. While many of his victims are facing foreclosure, financial ruin and even suicide, Bernie Madoff is still living a life of luxury.