Janet Yellen's big concern: Housing slowdown

Federal Reserve Chair Janet Yellen is upbeat on the U.S. economy for the most part, but there's one sector that causes her some concern: Housing.

"One cautionary note, though, is that readings on housing activity -- a sector that has been recovering since 2011 -- have remained disappointing so far this year and will bear watching," she told the Joint Economic Committee in prepared remarks Wednesday.

The housing sector was a key part of the economic recovery last year but has since fallen short of economists' expectations.

Building permits -- a gauge of future home construction -- fell 2.4% in March, and existing home sales were flat. Economists had expected both these indicators would improve more, especially after blizzards and a cold winter put the housing sector largely on hold.

Yellen cautioned that the warmer weather may not be enough to turn things around.

"The recent flattening out in housing activity could prove more protracted than currently expected rather than resuming its earlier pace of recovery," she said.

The Federal Reserve has kept its key short-term interest rate near zero since 2008 in an effort to stimulate more spending in the U.S. economy.

Low interest rates make it cheaper for businesses and consumers to take out loans, including mortgages. One of the most potent ways the Fed impacts the economy is through the housing market, where record-low mortgage rates had been spurring a recovery for the last two years.

While mortgage rates are still near historic lows, they have been rising since last summer as the Fed slows its stimulative policies by buying fewer bonds each month.

Yellen staysvague on timing of future rate increases

Despite several probing questions from Rep. Kevin Brady, a Republican from Texas, Yellen refrained from laying out a timeline for when the Fed may eventually raise short-term rates.

Instead, she opted for vague language, saying it will take "some time" and a "considerable time" after the Fed's bond-buying program ends later this year.

Yellen had previously said in a March press conference that "considerable time" could be defined as about six months, and stocks rallied on that news. Since then, Fed officials have backed off from that language.

Fed sees economy picking up

As for the broader economy, Yellen isn't too worried. The U.S. economy stalled in the first quarter, growing at merely a 0.1% annual rate, but she blames the weakness largely on the harsh winter.

"I see that pause as mostly reflecting transitory factors, including the effects of the unusually cold and snowy winter weather," she said.

"With the harsh winter behind us, many recent indicators suggest that a rebound in spending and production is already under way, putting the overall economy on track for solid growth in the current quarter," she added.

Overall, Yellen expects the economy to grow faster this year than it did last year and the unemployment rate to continue declining gradually. As of March, the unemployment rate was at 6.3%, still well above its pre-recession range between 4% and 5%.

"While conditions in the labor market have improved appreciably, they are still far from satisfactory," Yellen said, expressing concerns about the long-term unemployed, stagnant wages and part-time workers who would prefer to work full time.

U.S. stocks started the day higher, turned negative when Yellen began speaking and then rebounded during the question and answer period. The Dow was up about 100 points when Yellen concluded her remarks.