The family that controls The New York Times empire has lost more than 86 percent of its fortune and may have sell their controlling stake to get out of debt.

The Ochs-Sulzberger family, which has run the venerable paper since 1896, may also face unusual pressure from about two dozen descendants to cash out and restore their comfortable lifestyles snatched away suddenly by hard times.

Until this year, the family had been living on wealth valued as high as $425 million. But today the family is down in their Times’ annual income to a paltry $4.5 million, which could shrink even more in the recession.

Soaring losses amid a devastating media slump have drained much of the corporate cash, pushed the company deeper into debt of $1.3 billion and beckoned a stock vulture to its door — Mexican billionaire Carlos Slim, who this year bailed out the Times with a high-interest $250 million loan that also threatens family control.

Slim, already the second-largest owner of Times common stock, is poised to become the biggest Times shareholder of common stock because he’s allowing his loan to be repaid in six years with stock — either from the family’s main trust or a weakened corporate treasury.

If the first year’s payment of $47.5 million, due in nine months, is made in stock in lieu of scarce cash, it would soak up a whopping 7.9 million shares — based on a current price of around $6. At that depressed stock price, it would take about 48 million shares to repay Slim’s IOUs.

However, the family trust holds just 8.9 million common and super-voting shares. At that pace of using stock to eliminate debt, most of the shares of both classes available to the family and the Times could be depleted soon, especially since the Times halted its stock buyback program more than a year ago.

Since 2003, it has bought back 12 million shares for its treasury, some of which were used to pay employee bonuses and options.

Just four months ago, the family lost its biggest single source of steady cash from the company — a 92-cent annual dividend paying nearly $8 million on shares the family trust holds. The family’s stock fortune also crashed this year from a high of $411.5 million four years earlier to a low in February of just $30.8 million, rendering it virtually useless to borrow against or pledge in other money-making ventures.

Eight family members continue to work at the company or serve on its board, bringing in total annual paychecks of $4.5 million — currently the family’s only reported source of company income — which itself is down by nearly half in the last two years.

The biggest breadwinner, Chairman Arthur “Pinch” Sulzberger, 57, saw his $100,500 dividend checks disappear when dividends got canceled earlier this year. His other compensation for running the company got slashed to $2.4 million in 2008, down from $4.4 million in 2006.

The only sliver of good news for the family came last week following deep cost-cutting throughout the company to strip fat for possible sales. The moves sent shares up 4 percent Friday to $6.37, shoring up the trust’s stock fortune to $56.6 million from its $30.8 million low three months earlier.

But that’s hardly enough to bankroll and restore the approximate $20 million a year the family had been spending on its lifestyle and personal charities — even if they cash out voting-control shares at a premium.

The family trust holds voting control through about 746,000 super-voting Class B shares spread among 30 holders. The non-traded Class B shares allow holders to pick 70 percent of the company’s 13 directors, but they lose those rights when they are sold on the market.

Selling to outsiders is prohibited by the family’s trust unless the trust’s eight family trustees unanimously approve such sales.

Just 14,202 shares of Class B have been sold or retired in the past four years — half during the recent year of financial stress, slipping from family fingers at an average of just $10.13 a share.

While the Class B stock is a fortress around the family business, filings say a contract loophole exists that allows six trustees to vote out two of the other trustees in order to break the covenant, if the future of the paper’s existence depends on cashing out and tearing down that wall.