Fallout

Madoff in Manhattan

Bernard Madoff’s scam was global, but his center of gravity was Manhattan’s Upper East Side. Speaking to longtime residents of the tony enclave, including many who lost millions with Madoff, the author explores the thorny issues of class and religion that the scandal has brought to the surface.

Reporters surround Bernard Madoff as he arrives at his New York City apartment after being placed under house arrest, December 17, 2008. By Justin Lane/EPA/Corbis.

Of all the incidents that would unspool after the exposure of Bernard Madoff’s $50 billion Ponzi scheme last December, the scene that stays with me took place at a coffee shop on East 62nd Street two days after Madoff’s arrest. The platoon of reporters had yet to camp out in front of Madoff’s building, which was just two blocks away, but you could feel the tension at that local Burger Heaven, in a neighborhood of people who have known one another for much of their lives and who come to their favorite coffee shop to read the Financial Times in their sweats after working out at the gym. “My father warned me about this guy,” said a woman having lunch with a friend in a booth. She wore a sheared mink coat and spoke softly, still in a state of shock. She referred to the Madoffs as Bernie and Ruth. “Ruth and I played with her grandchildren. She and Bernie did everything together. My father said, ‘Be careful of them. This is a big Ponzi scheme.’ And that was 15 years ago.” Her father was a real-estate titan, and his daughter now ran the company, but she was hardly preening about her foresight. Her husband had lost most of his foundation to the Madoff scam. I happened to be there that day with a friend who had spent years on the co-op board of Madoff’s building. The board would often meet in the Madoff apartment, with its wraparound terraces and black-and-white marble floors. “It was strange,” he said. “I could never understand a word that Bernard Madoff said.” The remark was Upper East Side code. It meant that, for all Madoff’s success, the hungry kid from Far Rockaway, in Queens, had never learned the implicit language of New York’s upper-middle class. My friend might have encountered a young Bernie Madoff in his university days, but he probably would have looked through him, not necessarily with reflexive snobbery, but with what V. S. Naipaul has described as a lack of “larger comprehension.” In the days following Madoff’s arrest, the shibboleths of New York from the time of Gentleman’s Agreement, when buildings were restricted up and down Park Avenue, were heard again. There were those who argued that the rejection Madoff had experienced may have acted as the trigger in his behavior: I’ll show them.

Read more on the financial crisis in our archive “Charting the Road to Ruin.” Illustration by Edward Sorel.

Meanwhile, an angry swarm of bloggers went into overdrive on the subject, with a mix of titillation and Schadenfreude, and the implication was chastening for the city: O.K., you spoiled rich people, now you see how the rest of America has always lived. Madoff’s victims were not only on the Upper East Side, however; they were in Palm Beach and Los Angeles and Houston. They were country-club members in Minneapolis and Detroit. New Jersey senator Frank Lautenberg lost most of his foundation, which had supported cancer research. There would soon be reports of scores of victims in Switzerland, France, Spain, and Italy, including, some said, Rothschilds and their clients and even some Russian oligarchs whom Madoff had lured into his funds through a Viennese banker. Madoff had used the Studio 54 approach, the con of exclusivity. He understood the swamp of insecurities that lay underneath many of his clients’ designer clothes, their face-lifts, their need to be part of an insiders’ club. A former governor of the New York Stock Exchange had once gone to see Madoff on behalf of a client who wanted to invest. “I don’t need your money,” Madoff told him, “and I don’t discuss my investments.” Then he ushered him out the door.

Great con men always understand the vulnerabilities of their victims. Madoff’s clients trusted the fact that he invested money not only for such important Jewish institutions as Hadassah but also for his closest friends. “If you had asked me, ‘Do you want to invest with the guy who makes the money for Yeshiva University and Steven Spielberg’s foundation,’ would I have signed up in a heartbeat?” a friend of mine said. “You bet I would.” Madoff’s history as a scrappy, rags-to-riches success struck so deeply into the psyche of many of his Jewish victims that they put aside their common sense for him. No computer access to an account? No chance to ask a question about due diligence? Oh, fine. Someone close to one hedge-fund manager who lost a billion dollars believed that Madoff had reminded his friend of his father. “Shirtsleeves to shirtsleeves in three generations,” many of his investors had been told as children. The essence of Madoff’s genius was his ability to invoke the romance of grandparents and great-grandparents coming to America and making good. Madoff tapped into his clients’ innocence and their grandiosity. A not inconsiderable part of his victim pool came from a group who thought of money with a complex tangle of shame and attraction, as if they believed that understanding money would drag them back into the stereotyping of their immigrant roots.

New York is used to scammers who blaze their way into the tabloid headlines, and here was a beauty, a nebbish hustler who looked like a kindly uncle. He defied all attempts to place him. In spite of the many comments and stories written about this century’s financial Jeffrey Dahmer, the mystery at the center of his personality remained as elusive as his crime. We ask questions when a great con man surfaces, as if a label would reduce our fear. New York had prospered in a golden era, and now that period of luxury and comfort was gone. “How can I know that any stock is actually in my account?” someone asked. “What if everything is a fraud?” The questions circulated and fell back on themselves, a Möbius strip without any answers to explain the deviant workings of a singular criminal mind. “What is important to understand is that criminals come from every walk of life,” said Stanton E. Samenow, the author of Inside the Criminal Mind. “They may be highly educated, they may have family and friends and to all outward appearance not only seem normal, but superlative. They commit a crime not because they need the money or they hate humanity or they had a terrible childhood or a smothering mother, or because they are Jewish or Catholic or Christian. What connects them is basic: Their self-esteem rises or falls at the expense of another. Bernard Madoff shares personality traits with embezzlers and serial killers.”

“It’s the rule of New York,” one money manager remarked. “Take any 10 bankers in New York: 5 will be mediocre, 3 truly gifted, and 2 should be in jail.” You heard a lot of pithy sound bites like this as the city of over-knowing experts attempted to label a con man of almost incomprehensible evil. “Was Madoff a crook who covered his bad trades like Ken Lay?” someone asked, referring to the late criminal C.E.O. who brought down Enron. “What kind of Jew victimizes Elie Wiesel?” another wanted to know. Madoff masked the predator in the cloak of the familiar. His clothes said London—Barbour jackets, cashmere sweaters—but there was something in his gaze that troubled many who encountered him. After his disclosure, he was compared to Augustus Melmotte in Trollope’s The Way We Live Now, who sells a fake railroad to Mexico, and to the oily and treacherous Uriah Heep, who takes over his employer’s fortunes in Dickens’s David Copperfield. “We don’t know enough about his early childhood,” one psychologist said. Still, significance was heaped on the inconclusive outline of his life story. He was a nobody poor boy from Far Rockaway who went to Hofstra and scrimped and saved $5,000 as a lifeguard and made his wife his bookkeeper. He worked his way through Roslyn, Long Island, until he caught the shiniest of brass rings and eventually rose to become the chairman of nasdaq. A few deplorable comments made the rounds about Jews and their history of money lending. All of this was predictable, as were the fears many Jews had that the arrest of Madoff would set off a wave of anti-Semitism. At a time when America had just elected a black president, it quickly became apparent that hoary labels were no longer fashionable. It seemed unnecessary to state the obvious: Nobel Prizes had been won and New York City’s grandeur established by vast numbers of other poor boys with backgrounds not too different from Bernie Madoff’s.

Within days the Madoff affair had morphed into a global catastrophe, involving feeder funds run by such well-regarded financiers as Walter Noel and J. Ezra Merkin, which had funneled billions of dollars from banks and investors in France, Switzerland, and South America. Merkin, a Harvard Law School graduate, was one of six children raised in a family of Orthodox Jewish philanthropists who had given tens of millions of dollars to Yeshiva University and built a concert hall in Lincoln Center. Debate raged as to whether he was a victim or a victimizer. Merkin had charged his investors a 1.5 percent fee and taken a commission from Yeshiva University for shoveling their money to Madoff. Merkin quickly resigned from the institutions his family had helped to build. On the Upper East Side his sister the writer Daphne Merkin, who knew little about her family’s secretive business dealings, avoided calls from reporters and maintained a fiercely loyal sibling silence. She had heard about her brother’s troubles in a call from a New York Times editor. I heard from a cousin who swore me to secrecy about an aristocratic French banker who had lost $1.4 billion. Three days later the banker, René-Thierry Magon de la Villehuchet, a man of honor, slit his wrists with a box cutter in his office.

Bernard Madoff lives on a quiet block on East 64th Street, not far from the restaurant Gino, where 40 years ago members of the Lehman banking family often ate dinner in the dining room with zebra-patterned wallpaper. Gino is still there, but the social order of that era is long gone, as are Lehman Brothers and Kuhn, Loeb & Co., the most distinguished banks of the Jewish upper class. Much of New York’s real wealth is found in these side streets of the East 60s, for many years part of the zip code 10021, which extended to the East 80s and is known as the silk-stocking district. As soon as Madoff could afford it, he moved from Roslyn, Long Island, to 133 East 64th Street. The address gave him an insider’s cachet, signaling that he was a player in a world of old-money rules. His neighbors were a Galbraith and a Loeb. Madoff lived with the new-money details he might have pulled from Quest, the society monthly—a triplex in the right neighborhood, a house in Palm Beach on the waterway, another in Montauk. He camped out at the Hôtel du Cap, in the South of France. In fact, his lifestyle was no more elaborate than that of other Midases profiled in magazines and on television. The disguise Bernard Madoff put on was that of the very class it seems he set out to maim.

“There were two kinds of people. There were ‘people we visit’ and ‘people we wouldn’t visit,’” Stephen Birmingham wrote 42 years ago in Our Crowd, a history of the German Jews of New York. The term “our crowd” immediately became a code of the culture and was used commonly in conversation. “They are very our crowd,” one might say. Or, “They aspire to be our crowd.” Birmingham eloquently depicted a world of what he called “perpetual reassurance,” the upper-class world of Loebs and Lehmans, Guggenheims and Sachses. Our Crowd became, for many, a guidebook for the emerging Jewish middle class’s cultural aspirations. The book was an immediate popular success, but it turned into something more profound, a template for achievement that was discussed in many Jewish homes all across America. Our mothers underlined significant stories, such as the one about August Belmont’s name change and the one about a member of the Lehman family who took a psychology test and was startled to learn that she was an anti-Semite. Our crowd did not wear flashy clothes or elaborate jewelry. “No stones” was a mantra. Birmingham identified the core values of the new society, a group that prided itself on a lack of showiness.

The expression “our crowd” passed out of fashion in the 1980s, with the arrival of Ronald Reagan to the White House, deregulation, pouf skirts, the leveraged buyouts of Michael Milken, and the vulgarities of Donald Trump. It took Bernard Madoff to bring the term back again. You heard it frequently in the days after he was indicted. He wanted to be our crowd. He was never showy. He gave so much to charity. Madoff knew the psychic buttons of his victims.

Madoff was 29 when Our Crowd was published. He was on the generational cusp, out of Hofstra University at a time when Wall Street was opening up to men of ambition and intelligence who had not been born into the world of our crowd but comported themselves as if they had. Wall Street was still reasonably small, divided into the Wasp houses of Brown Brothers Harriman and Morgan Stanley and the elite Jewish firms—Lehman Brothers, Goldman Sachs, and Kuhn, Loeb. “In 1967 there were a lot of guys like Bernie Madoff trying to make their way,” former Lehman Brothers vice chairman Peter Solomon said. “Everyone was still categorizing then. [New York mayor] Ed Koch said to me one day, ‘You German Jews!’ and I said, ‘Ed, I am Russian.’ And he said, ‘I don’t care if you are poor and Polish, to me you are always rich and German.’” Madoff was the kind of kid who worked his way up through a commercial-paper house, who was kept in the back room with the brain trust so that he would not embarrass the clients. In those years, an ambitious young man would have set his cap on a membership to the Harmonie Club, the bastion of the Jewish upper class, or the Century, the Westchester club where the children of the great Jewish families met and socialized. Growing up, Madoff probably understood implicitly that this world excluded him. “If you were Bernie Madoff in 1965, the entry to a major firm was through the trading floor,” Peter Solomon said.

Madoff made his name as a “market maker,” a term that few understood. He offered trades and markets in securities and outpaced his competitors by developing an ingenious online technology that became nasdaq. Madoff ultimately became nasdaq’s chairman and served on its board of governors. His company was a family affair, a closed corporation that included his two sons, Mark and Andrew, who worked on the trading side, and his younger brother, Peter, who was the chief compliance officer. He even brought in his niece, whose husband had once been a lawyer on the Securities and Exchange Commission.

By the 1980s, Madoff was making 5 percent of the total trades on the New York Stock Exchange. He paid for orders—an unusual practice at the time—which he likened in an interview to a commission you pay a store to display your brand of stockings on a special rack. Madoff’s employees called his investment-management division “the hedge fund,” and knew that the 17th floor, where it was run by a staff of fewer than 24, was off-limits. It is still not known if these facts about Madoff’s apparently legitimate trading operation troubled his employees or his sons. In 2001, Barron’s questioned the legitimacy of Madoff’s astonishing returns. Did his sons never ask their father about the Barron’s allegations? Madoff kept his profits at a reasonable 12 percent, camouflaging the fact that there were perhaps no profits, and for that matter perhaps few investments, in a dizzying blur of printouts consisting of fake trades that he sent monthly to his clients. Several intriguing questions about this case remain: Who composed those fictions? Did they appear in the reports prepared by Madoff’s 80-year-old accountant, who worked in a strip mall next to a pediatrician’s office? Did none of his prestigious clients, such as J. Ezra Merkin or Walter Noel, ever ask him, “Why aren’t you audited by Ernst & Young?” Hindsight is always dangerous, but surely some degree of greed put blinders on Madoff’s legion of true believers. At a time in the city when hedge-fund managers boasted of their 30 percent profits in Asia and India, Madoff’s consistent returns seemed to be the stuff of T-bills. “Jewish T-bills,” they called Madoff investments at the Palm Beach Country Club.

Every decade of excess in conjunction with a failure of regulation has delivered a marquee con man once the bubble broke. In the 1960s, the freewheeling bald playboy Bernie Cornfield’s mutual-funds con game had a famous slogan: “Do you sincerely want to be rich?” The collapse of this $3 billion scam ushered in the bankrupt 1970s. (Cornfield was indicted for embezzlement but later acquitted.) In 1989 the junk-bond crime spree of Michael Milken resulted in a 98-count indictment and the collapse of Drexel Burnham. Like Madoff, Milken was famously secretive, surrounding himself with yes-men, believing that he could make his own rules. In the end he pled guilty to six counts of securities fraud. After serving less than two years in prison, he launched a massive public-relations campaign and set up a foundation to find a cure for prostate cancer, which reunited him with the very man who had once prosecuted him, Rudy Giuliani.

During the 19 years that Alan Greenspan was chairman of the Federal Reserve, irrational exuberance drove the markets. Morality became a subjective matter. Madoff was the perfect crook for the era of subprime-mortgage madness and incomprehensible derivatives. From the day of his arrest, he became the symbol of our recent mass delusions—here was an icon of evil, a face to glue on the shadowy money markets, unregulated hedge funds, and government mortgage Ponzi schemes. We rode a decade in which people conferred special status on those who lived in $30 million apartments, leased NetJets, and filled their closets with $5,000 purses. Did Bernard Madoff see his opportunities and seize them, or did the casino atmosphere in New York spawn the ordinary man in the Barbour coat who would become the greatest con artist of all time? Madoff provided his victims with a mirage of an impossible dream house—with investments that paid 12 percent in good times and bad—and their tragedy and ours was that they believed him.