The wave of positive housing data has buoyed markets, homebuilders and bank stocks. Housing prices have risen for seven consecutive months. Housing starts rose 3.6% in October to a four-year high of 894,000, the Commerce Department said Tuesday.

But, while the developments are encouraging and residential investment will likely be a source of growth and jobs, ", a number of factors continue to prevent the sort of powerful housing recovery that has typically occurred in the past," the Chairman said.

One reason was because of tight credit conditions even for borrowers with good credit, a point Bernanke recently noted. The Chairman said that while some of the tightening of credit was appropriate given earlier excesses, the "the pendulum appears to have swung too far, restraining the pace of recovery in the housing sector."

Other factors slowing the recovery include over 20% of borrowers being underwater who are unable to refinance or sell their homes and substantial overhang of vacant homes which holds down prices and reduces the need for new construction.

"While these headwinds on both the supply and demand sides of the housing market have clearly started to abate, the recovery in the housing sector is likely to remain moderate by historical standards," Bernanke said.

In the Q&A following the speech, Bernanke said that the Fed was working to improve credit conditions. He said the Fed's policy of keeping mortgage rates at record lows would play a very important role in boosting housing and affordability.

He also said that banks will likely loosen their purse strings a little as they continue to see good data on housing prices, starts and home sales, as part of the reason banks have not lent money has been fear that housing markets will remain depressed.

That should set off a "virtuous cycle" of data improving, leading to better credit conditions, leading to stronger data.

The Chairman also believes that as the profitability of mortgage lending improves, banks will increase capacity to originate more mortgages.

On the regulatory side, the central bank is trying to finalize rules on capital weights on mortgages, rules for securitization and so on. He gave no indication on when the regulators will complete writing the rules.

On the supervisory side, the Fed was trying to steer banks in the direction of making more loans while remaining prudent, offering foreclosure alternatives and so on.