WASHINGTON—J.P. Morgan Chase & Co. has reached a tentative deal with the Justice Department to pay a record $13 billion to settle a number of outstanding probes of its residential mortgage-backed securities business, according to people familiar with the decision.

The deal, struck Friday night, doesn’t resolve a continuing criminal probe of the bank’s conduct, which could result in charges against individuals or the bank itself and possibly increase the penalty tab. The two sides continued to disagree over an admission of wrongdoing that would end the criminal probe and decided instead to resolve the civil allegations related to the mortgage securities.

The deal includes $4 billion to settle claims by the Federal Housing Finance Agency that J.P. Morgan misled Fannie Mae and Freddie Mac about the quality of loans it sold them in the run-up to the 2008 financial crisis, another $4 billion in consumer relief, and $5 billion in penalties paid by the bank, according to people familiar with the decision. How the consumer relief and penalties get dispersed and distributed is largely up to the government, and those details are still unclear, these people said.

The agreement hasn’t been completed and some particulars are still being discussed, such as the final wording, these people said. If completed, the deal would represent the largest settlement the U.S. government has reached with a single company. The deal would also resolve a separate suit brought by New York state’s attorney general, Eric Schneiderman, the people said.

J.P. Morgan disclosed last week that it set aside $9 billion additional legal reserves in the third quarter, giving it a total of $23 billion to absorb future settlement and lawsuits.

The general terms of a deal were struck Friday night in a phone conversation between Attorney General Eric Holder, his lieutenant Tony West, J.P. Morgan CEO James Dimon and the bank’s general counsel, Stephen Cutler, a person familiar with the matter said. However, another person close to the talks said the two sides had decided earlier in the week that they would probably not reach an agreement on the criminal prosecution.

Once the two sides agreed to general terms—and decided they weren’t going to agree on how to resolve the criminal investigation of the bank—Mr. Holder and Mr. Dimon got off the phone, and Mr. West and Mr. Cutler continued to negotiate specific language of settlement documents. Those discussions will continue, and it wasn’t clear how long it would take to get a final deal in place.

Mr. Dimon cleared the proposed deal with the bank’s board before the Friday night phone call, according to one of the people close to the talks.

The bank ultimately decided to proceed without a pledge to drop the criminal probe in part because it does not think the government has a strong case, according to a person familiar with the negotiations.

The tentative settlement comes as J.P. Morgan tries to put as many legal woes behind it as possible.

Earlier this week, J.P. Morgan agreed to pay $100 million and acknowledge wrongdoing to settle allegations by the Commodity Futures Trading Commission related to its botched “London whale” trades. Last month, the bank agreed to pay $920 million to settle similar charges with U.S. and U.K. regulators related to that 2012 trade.

In the past month, J.P. Morgan also agreed to pay more than $1 billion to end an array of investigations into a 2012 trading debacle that cost the bank more than $6 billion and raised questions about governance and oversight.

Now the bank is trying to end a large number of cases that revolve around residential mortgage-backed securities that J.P. Morgan, Bear Stearns Cos. and Washington Mutual Inc. issued between 2005 and 2007. J.P. Morgan purchased Bear Stearns and the banking operations of Washington Mutual during the 2008 financial crisis.

Those purchases—encouraged by U.S. regulators five years ago—are now costing J.P. Morgan as investigators examine whether mortgage bond investors were misled about the quality of loans made during the housing boom. In 2011 FHFA sued J.P. Morgan and 17 other banks, taking issue with the quality of nearly $200 billion in mortgage investments sold to Fannie and Freddie.

The agencies bought the securities when they offered rich returns. But the investments produced large losses once the housing downturn hit. Fannie Mae and Freddie Mac had to write down those investments sharply in 2008 when they were taken over by the federal government.

J.P. Morgan had the second-largest exposure of the 18 financial institutions sued by the FHFA, with $33 billion of the $200 billion at issue. Bank of America Corp had the largest, with $57.5 billion. Not counting J.P. Morgan, three other institutions have reached settlements with the FHFA thus far. The regulator announced pacts with Citigroup Inc in May and with General Electric Co. in January but didn’t disclose the amounts. In July it reached a $885 million settlement with UBS AG.

The settlement with the Justice Department is remarkable not just for the record-setting dollar amount, but for the fact that it still leaves a major potentially costly issue still unresolved—the criminal probe. That probe grew out of a broad investigation launched last year by an Obama administration task force charged with looking into banks’ handling of residential mortgage-backed securities. When the value of such securities dropped with housing prices in 2007 and 2008, it sparked a crisis in the financial system.

The task force issued a series of subpoenas to various financial companies, seeking internal documents. Those documents held a number of promising leads, one of which was assigned to federal prosecutors in Sacramento.

Investigators in that case discovered an email by a bank employee, warning her higher-ups that the bank was vastly overstating the value of the mortgages being securitized, according to people familiar with the probe. That employee, who has since left the company, has been cooperating with federal prosecutors, who expect to call her as a witness if the case ever goes to trial, according to people familiar with the case.

While the Justice Department considers the evidence in that case to be strong, officials at the bank strongly disagree, according to people familiar with the negotiations.

In August, Mr. Holder signaled in an interview with The Wall Street Journal that decisions would be made in a matter of weeks on major cases involving conduct in the financial community that contributed to the 2008 financial crisis. While he wouldn’t specifically address the J.P. Morgan cases, the attorney general emphasized he would be taking action soon.

In late September, as the Justice Department neared its own deadline to file a civil lawsuit in the case, the bank offered $3 billion to settle the case. Attorney General Eric Holder rejected that offer, and government lawyers prepared to file the suit. The bank then offered billions more, if the government was willing to throw into the settlement separate cases, raising the total price and resolving more of the bank’s legal headaches.

As the negotiations intensified in September, Mr. Dimon sought a face-to-face meeting with Mr. Holder to try to resolve the remaining sticking points. The two met Sept. 26 at the Justice Department, but the meeting failed to settle the outstanding issues. As talks continued over the remaining weeks, the size of the deal swelled, but the two sides continued to disagree over an admission of wrongdoing that would end the criminal probe.

On Friday night, Mr. Dimon and Mr. Holder decided they were just not going to come to terms on the criminal issue—and take the deal on the terms where they did agree.