It was nearly 10 p.m. Wednesday, and Perrelli was still working the phones inside his large office on the fifth floor of the Robert F. Kennedy Justice Department building, trying to nail down the final details of a 16-month negotiation with federal officials, bank executives and attorneys general across the county to provide financial relief to distressed homeowners.

At the end of a long, harried day, Perrelli, the lead government negotiator, was about to tell reporters on a conference call that a historic deal had finally been reached that would force the mortgage firms to pay $25 billion.

But at the last minute, he had to pull the plug on the call. The banks were pushing back. It was the sixth time that day they had planned to announce the settlement but had to pull back.

The moment was typical of a precarious process that produced the landmark settlement, an agreement with the nation’s five largest mortgage servicers to end foreclosure abuses and help homeowners who were hit hard by the housing bust. At several junctures, the negotiations, which involved nearly 90 people, seemed to fall apart.

Thursday morning, the key players stood side by side on a Justice Department stage in a room packed with reporters, praising their colleagues for pulling off the landmark deal.

“Today’s settlement . . . is a remarkable example of cooperative law enforcement,” said Attorney General Eric H. Holder Jr. Added Iowa Attorney General Tom Miller, one of the leaders of the group: “This agreement came about as a result of strong, healthy bipartisan relationships.”

Behind the scenes, the negotiation was a complicated and emotional process of fits and starts. There were 25 Democratic and 25 GOP attorneys general, most of whom were skeptical of the plan. There were colorful and sometimes difficult personalities. There was an endless stream of conference calls, sometimes involving 40 people, trying to work out intricate details. There was a constant push and pull between the federal and state agencies and the banks.

“We went down a lot of rabbit holes,” said a government official involved in the negotiations.

So many state and federal officials, bank lawyers and executives were involved that 50 people or more regularly crowded into negotiating sessions, some lasting more than eight hours, in hotel conference rooms in and around Washington. The group had a running joke that the negotiations resembled the Paris peace talks during the Vietnam War.

On Monday, more than 40 states were on board. Major states such as New York and California joined late in the process, after holding out for months.

By late Wednesday, every state had signed on but Oklahoma, whose attorney general said that he thought the deal “overreached the authority of state attorneys general” and unfairly rewarded homeowners who had stopped paying their mortgages.

At 10 p.m., there was a final sticking point. According to government sources, federal and state officials wanted to be able to pursue other claims against the banks after the civil settlement was in place — and the banks wanted more protection from those claims.

“We wanted to be able to pursue the conduct that brought this about,” said one government official involved in the negotiations.

It took more than another two hours to craft language that satisfied all parties, though the banks got only limited legal protections. At nearly 1 a.m., word came. “By that time, I was too tired to jump up and down,” said one Justice Department official close to the negotiations.

A few miles away, the attorneys general from Iowa, Connecticut and North Carolina had gathered inside the offices of the National Association of Attorneys General on M Street near Dupont Circle. They also had worked the phones all day and sent last-minute emails about the deal.

The endless back-and-forth of recent days had staff member quoting Tom Petty lyrics: “The waiting is the hardest part . . . ”

After word came that the deal was done, Patrick Madigan, the assistant Iowa attorney general who had played an integral role in crafting the settlement, waited around with other state officials until 2:45 a.m. to make sure there were no last-minute speed bumps. Finally, he walked the two blocks back to his hotel to try to grab a few hours’ sleep, hopeful but still uncertain that the deal he and so many others had worked on for more than a year would become reality.

Only when he was in a cab headed for the Justice Department in the morning did he let himself think, “This is really happening.”

Sari Horwitz covers the Justice Department and criminal justice issues nationwide for The Washington Post, where she has been a reporter for 30 years.

Brady Dennis is a national reporter for The Washington Post, focusing on food and drug issues.

Comments our editors find particularly useful or relevant are displayed in Top Comments, as are comments by users with these badges: . Replies to those posts appear here, as well as posts by staff writers.

To pause and restart automatic updates, click "Live" or "Paused". If paused, you'll be notified of the number of additional comments that have come in.

Comments our editors find particularly useful or relevant are displayed in Top Comments, as are comments by users with these badges: . Replies to those posts appear here, as well as posts by staff writers.