But this doesn't necessarily have to be the case, according to Gluskin Sheff's David Rosenberg.

An improving job market and rising incomes matter more to real estate than mortgage rates, he argues:

"If mortgage rates were going up absent a rise in housing and loan demand, that would be a problem indeed. Say, for example the backup in yields was due principally to rising inflationary expectations. That doesn't do anybody much good, that is for sure (unless you are long TIPS, gold and collectibles). But the current rise in market rates reflects the better tone to the real economy so it makes sense that the two would increase in tandem.

"And our in-house regression work shows that incomes and jobs actually matter more to the real estate market than mortgage rates — not that rates don't matter. They do. But it's all relative. We went back to those years when mortgage rates rose but income growth accelerated and found that, in those episodes, on average, there was no perceptible impact of the rising yield environment on sales or starts."

So, for those arguing that higher mortgage rates could blow the housing recovery, Rosenberg says, "give income a chance."