If you're looking for a reason the housing market is so tight, perhaps the Baby Boomers are to blame.

For generations, after the kids grew up and moved out, many parents put the big family house up for sale. According to one market watcher, that doesn't appear to be happening nearly as often nowadays.

"The traditional 'empty nest' storyline where parents would be downsizing is not happening," Svenja Gudell, Zillow's chief economist told CNBC's "On the Money" in an interview. Research from the real estate data company found the inventory of homes for sale is the lowest in four years.

Gudell said a contributing factor to fewer homes on the market is more baby boomers are staying put in their "home sweet home," rather than downsizing.

"Baby Boomers have to live somewhere, "Gudell said, "but they are having a hard time downsizing and finding that available inventory. And so it's tough for people to move into their homes and that is causing that disruption in the normal 'move-up' market."

As a result, Boomers aren't able to find a home that's smaller and cheaper than the one they're in "that would entice them to move," Gudell explained, "so for many it does not make sense to sell."

"What it comes down to is we have two very large generations kind of butting heads," Gudell told CNBC. "…You have millennials that are just entering the market, that are even larger of a generation than the baby boomers, and you really just need more homes."

"We're currently dealing with '1994-level' inventory, despite having 63 million more people in the country," she said. "So at this point we have to build some more homes in order to accommodate all those people."

If the demand for affordable starter homes is so high, why aren't those homes being constructed?

"I think a lot of builders will tell you that they're having a hard time meeting that price point right now given how expensive labor, lumber, and simply regulation costs are," Gudell stated.

"So when it comes down to making the profits work on their end, they're going to build a home that's slightly too expensive for the entry-level market," Gudell added.

Recent data show growth in higher end properties being built, and sold. Last week, the National Association of Realtors reported sales of homes costing less than $250,000 have fallen because of declining supply.

Zillow's Gudell said the tight housing market is getting increasingly tighter by keeping potential sellers on the sidelines.

She explained owners who may want to downsize are reluctant to list their current home and "have to become a buyer" and "join the fight for that small inventory." The two forces are competing against one another.

"The musical chairs is great for sellers, but you don't want to be a buyer now," she said.

"On the Money" airs on CNBC Saturday at 5:30 am ET, or check listings for air times in local markets.