Homeowners who stay put—living in the same home for 15, 20, 30 years—can pocket millions of dollars when they finally make a move.

In 1976, Abba rocked the radio, Gerald Ford was president, and Leonard Ross purchased one of the most expensive homes in the country.

Mr. Ross, an attorney and real-estate investor, bought the roughly 30,000-square-foot Beverly Hills, Calif., estate of newspaper magnate William Randolph Hearst for about $1.7 million at a time when the median home price was $38,000.

“People asked, ‘Why are you buying a house that big?’” said Mr. Ross, who was single at the time and living with his German shepherd, Lark. “I said, ‘I’m family oriented. I’ll grow into it.’ ” Mr. Ross, who grew up poor on New York’s Lower East Side in the 1950s, said he bought the mansion in part because it was a trophy property.

Forty years later, Mr. Ross, 72, is listing the 5-acre compound, including the now 50,000-square-foot mansion, for $195 million—which would be a record sale price in Los Angeles, where the Playboy mansion recently sold for $100 million. Adjusted for inflation, that’s a compound annual growth rate of 8.7%. The S&P 500 stock-market index averaged an inflation-adjusted annualized return of just 4.1%. That doesn’t factor in carrying costs or improvements, but neither does it account for Mr. Ross occasionally renting the space for upward of $600,000 a month.

In luxury real estate, good things often come to those who wait. The number of years that home buyers remain in their home is at a record high, in part because of lingering uncertainty in the housing market. For those who can afford to play the long game—15, 20, 30 years—staying put can mean a windfall when they finally make a move. But buyer beware: Time isn’t always on your side.

For Yvonne Dunleavy, the quiet village of Sagaponack, N.Y., in the Hamptons seemed like a smart place to buy property in 1978, when she purchased a modest home there for $140,000. Now she and her husband, architect Joost Schiereck, are listing the 1,800-square-foot modern home for $5.75 million, for a compound annual growth rate of 6.61%.

“I don’t see it as a luxury house—a luxury location, certainly,” said Ms. Dunleavy, co-author of “The Happy Hooker,” about an Upper East Side madam. Over the years, she watched the sleepy vacation community become a hot spot for the jet set. In the 1990s, she rented the home to John F. Kennedy Jr. and some of his family.

Prices are still climbing in the area. A neighbor who bought a similar property recently sold for about $5.5 million this summer. The new owner promptly demolished that home to make way for an as-yet-unbuilt modernist spec house, which is already listed for $16.995 million.

All told, Ms. Dunleavy estimates spending $350,000 on renovations over the past 40 years, including the addition of a new wing, some new sliding glass doors, adding a pool, and replacing the roof and kitchen. The couple is selling the home because they’d like to travel more. “It was sheer luck,” she said about buying when she did.

A lot can change over the course of homeownership. In an analysis of four metro areas—New York, Los Angeles, Atlanta and Cleveland—Realtor.com tracked the sale prices of about 11,600 homes that were sold in 2000 and again in 2016.

The most expensive market, Los Angeles, saw a median price gain of 141%, or $320,500 in the 16-year period. Adjusted for inflation, that translates to a compound annual growth rate of 3.5%. New York saw an annual growth rate of 1.2%; Atlanta shrank 0.6%; and Cleveland fell 2.3%.

“It’s a function of strong local economies,” said Javier Vivas, an economic researcher for Realtor.com. He also notes that speculators have pushed prices higher in surging markets. While Cleveland saw a decline in that period, he said the luxury tier fared best, in part because affluent buyers are less prone to market fluctuations.

Wait long enough, though, and the peaks and valleys seem less important. In San Francisco, Ed McGuirk, 65, and his family are selling an eight-bedroom Victorian home for $6.95 million; his grandfather bought it for $30,000 in 1926.

“That was a ton of money back then,” said Mr. McGuirk, a retired schoolteacher. “But now it’s skyrocketed in price.” He, along with his siblings and a cousin, are selling the roughly 5,100-square-foot home because they all live elsewhere; his aunt, the last owner, died in 2015.

Unlike most homes in the area, the Queen Anne-style home still has most of its original layout, with 12-foot ceilings and grand parlors. In the basement, his grandfather’s Prohibition-era bar is still intact. Behind a mirror is a passageway that led into the front yard—presumably to make a quick escape from police, he said. He used to play in the tunnel as a child. (It has since been sealed to elude more likely raiders—neighborhood raccoons.)

The home was listed in September for just under $7.5 million and reduced to $6.95 million in October.

Homeowners are staying in their homes longer—a median of 10 years, the longest tenure since at least 1985, according to the National Association of Realtors. Chief economist Lawrence Yun said the shift is partly due to an aging society and tight inventory in many markets. “Home sellers know they can find a buyer, but they would have difficulty finding a new home once they sell,” he said. He says luxury homeowners are likely staying put for even longer, because their homes take longer to sell.

In Shaker Heights, an affluent suburb of Cleveland, Barry Minoff is realistic about the roughly 9,000-square-foot home he purchased in 2000 for $1.6 million. “I have more into it than I’m asking,” he said of the Georgian Revival mansion, now listed for $4.995 million—a compound annual growth rate of 5.18%.

The average luxury sales price in the area is around $1.5 million, said listing agent Adam Kaufman of the firm Howard Hanna.

Mr. Minoff, chairman of Kichler Lighting, says he is selling because a lot in his life has changed. His main home is now in Fort Lauderdale, Fla., and he has recently branched into acting; he starred opposite Corey Feldman in the 2013 film “Exposure.”

“I’m at the stage of life where you can always get more money, but you can’t get more time,” he said about his decision to sell.