NEW YORK (CNNMoney.com) -- Bernard Madoff, who stole billions from investment clients, was ordered jailed Thursday after pleading guilty to all 11 criminal counts in one of Wall Street's biggest swindles ever.

"I operated a Ponzi scheme," said Madoff to the packed courtroom, after U.S. District Judge Denny Chin told him to explain his crimes.

"I thought it would end quickly, but it proved impossible," said Madoff, who stole from thousands of victims through his investment firm. "I am ashamed for these criminal acts. I always knew this day would come."

U.S. District Court Judge Denny Chin remanded the 70-year-old to jail following his confession. He was taken to Manhattan Correctional Center, near the courthouse in lower Manhattan.

Madoff admitted that he never invested his clients' money, and that he deposited the funds into a "Chase Manhattan" bank.

"When money was requested, I paid it out from the Chase account," he said.

As Madoff wrung his hands and made other nervous gestures, the judge suggested that he pour himself a glass of water.

Madoff had successfully avoided detention after his December arrest, posting $10 million bail and cloistering himself with his wife in their $7 million Manhattan apartment.

He was able to remain in his residence, despite accusations from federal prosecutors that he tried to hide his assets from seizure by mailing diamond-studded jewelry to relatives. Only a fraction of the missing money has been recovered.

The investors

Madoff created a decades-long scheme in which new investments were used to fund payoffs to earlier investors, to falsely create the appearance of legitimate returns.

Richard Friedman, an accountant who said he lost $3.1 million to Madoff, told CNN that he hopes Madoff is sentenced to the full 150 years and that he "lives a very long life" in prison.

"The crime is really unimaginable," said Friedman. "It's not just a typical Ponzi scheme. It affects society as a whole. You don't just have to be a Madoff investor to be affected by this."

Friedman also blamed the Securities and Exchange Commission for allowing the scheme to go undetected for so long, and said that Madoff's family members are probably complicit and should go to prison.

"There's no way he could have acted alone," said Friedman. "There had to be other people involved. The whole family, as far as I'm concerned, once proof becomes available, should be thrown in jail."

In Thursday's court proceeding, Madoff faced many of his alleged victims -- with 50 courtroom seats reserved for them. Some of the investors entrusted all their savings to his firm, Bernard L. Madoff Investment Securities LLC.

But not all is lost for these burned investors. They could be eligible for $500,000 from the Securities Investor Protection Corp., a government entity that provides funds to victims of failed brokerage firms, according to Irving Picard, the court-appointed trustee for the liquidation of Madoff assets.

Fuming at the SEC

At a Feb. 20 meeting at the U.S. Bankruptcy Court in Manhattan with several hundred Madoff investors, Picard said that $650 million had been recovered thus far. He said the assets would be divided among the creditors, commensurate to how much they put in.

Many of the investors seem to be angrier with the SEC than with the man who stole their money.

At the Bankruptcy Court hearing, 77-year-old investor Raymond Spungin, who claimed he lost $700,000 to Madoff, yelled, "We're not just the victims of Madoff; we're the victims of the incompetence and irresponsibility of the SEC!"

He was answered with loud applause from the crowd of ripped-off creditors.

The SEC had been alerted to Madoff's fraudulent behavior by whistleblower Harry Markopolos, a Boston accountant who repeatedly contacted the commission but was ignored for his efforts.

"I gift-wrapped and delivered the largest Ponzi scheme in history to them and somehow they couldn't be bothered to conduct a thorough and proper investigation," he said, in testimony to the House Subcommittee on Capital Markets on Feb. 4.

But the times are changing. In January, Mary Schapiro assumed the role of SEC chairwoman, after the former chair, Chris Cox, resigned under severe criticism.

Schapiro on Wednesday requested more funds for the commission to ramp up enforcement and examination staff. Also, the SEC is taking steps to centralize whistleblower tips to better follow leads, rather than have them funneled through separate departments.

Markopolos, in testimony, said he's already alerted to the SEC to two new fraud cases, but provided no further details.

Allan Chernoff CNN senior correspondent and Julian Cummings, Laurie Segall and Deborah Brunswick of the CNN New York bureau contributed to this report.