There’s been a lot of speculation about why for-sale inventory levels remain so stubbornly low.

With home prices still well off peak levels from 2005-2006 and mortgages still near historically low interest rates, many industry observers were expecting to see home sales increase much more rapidly than they have.

One of the main reasons that sales have lagged expectations is that there simply aren’t many homes available to purchase — particularly at the low end of the market, where first-time buyers shop.

An excellent report published by Zillow’s Svenja Gudell documents the large number of U.S. homeowners who are still underwater, owing more on their mortgages than their homes are worth. While down from the peak of 31.4 percent in 2012, 18.8 percent of homeowners are in a negative equity position.

Equally problematic, as the report points out, is that many homeowners who have escaped from being underwater are barely treading water: 36.9 percent of homeowners with a mortgage are “effectively underwater,” according to Zillow, since they simply don’t have enough equity in their homes to consider selling the property and moving up to a new, larger house.

Short sales, the only way to sell a home with an underwater mortgage, have been declining rapidly. According to Hope Now, there were over 83,000 short sales in the first quarter of 2013 and just over 33,000 in the first quarter of this year (there are a number of reasons for this decline, but that’s a subject for another article).

To put all of this in its simplest terms, between 10 million and 18.5 million homeowners are effectively off the market, decimating inventory levels of existing homes for sale.

New-home construction hasn’t recovered

New-home inventory has improved marginally since bottoming out in 2012, but is still nearly at the lowest levels in the 40 years that this information has been reported, according to NAHB data compiled by Calculated Risk.

Current inventory levels are roughly half of what there would be normally, and about a third of what they were at the peak in early 2007. Builders overbuilt as the housing boom passed its peak, and stopped building almost entirely when faced with the daunting task of trying to compete with their own recently built inventory that had been foreclosed on and was now available to buyers at a significant discount. Builders shifted their strategy towards building fewer new homes, but larger and more expensive ones.

While there is limited inventory at the low end of the market for existing homes, there is virtually nothing available in that segment of the market for new-home buyers. And although housing starts have been inching up, the headline numbers are a little misleading: Most of the new construction is on multifamily units, as builders are betting on continuing declines in homeownership rates, and strong rental demand.

Foreclosure activity has slowed

Distressed homes made up a significant part of the home sales inventory, peaking with the sale of over 1 million lender-owned properties in 2010. But foreclosure activity has been slowing down for the past few years.

A recent report from RealtyTrac notes that foreclosure starts have declined for 22 consecutive months. Foreclosure activity, while still between two to three times higher than normal, is at its lowest levels since 2006, and running at about 50 percent of where it was at the peak.

While this is good for the housing market — and good for the economy — it has dramatically reduced the number of distressed properties available for purchase. Short sales and foreclosure sales dipped to 14.5 percent of home sales in the first quarter of 2014, down from 18.5 percent the prior year.

From an inventory standpoint, this all adds up to something of a “perfect storm.” All three categories of homes — existing homes, new homes and distressed homes — fell at the same time. Which is why it feels like there aren’t enough homes on the market.

The big question, of course, is what it will take to resolve this situation. The decidedly unpopular answer is: more time. But given some time, it’s very likely that the situation will resolve itself.

Tight inventory leads to price appreciation. Price appreciation leads to fewer underwater borrowers, and more existing homes on the market. More time to pay down principal balance and build equity also adds to this equation. Higher prices also make for a more attractive new-home market, which should encourage builders to start breaking ground at least a little bit more often. And as the remaining distressed assets in the “shadow inventory” gradually make their way through the foreclosure process, we should see somewhere in the neighborhood of 2 million of those homes enter the market.

The NAR report on April existing-home sales showed a significant jump in home inventory for the first time in months — from a 5.1-month supply in March to a 5.9-month supply in April. Whether this is a harbinger of things to come or a pent-up burst of activity after a nearly dormant first quarter remains to be seen.

There are no quick fixes for the inventory problems we’re facing, but at least the market is moving in the right direction.

Rick Sharga is the executive vice president of Auction.com, a real estate site that brings buyers and sellers together in collaboration with buyer’s and seller’s agents, using a transparent platform to realize the true value of any real estate asset.