Long-term bond yields are still relatively low. But rates have edged sharply higher in the past few months as recovery hopes (and some inflation fears) have taken hold. That could lead to higher mortgage rates.

NEW YORK (CNNMoney.com) -- The economy seems to be getting better. That's what Wall Street is telling us. Stocks soared Wednesday thanks to a slew of decent earnings reports.

That's also what C Street -- home of the Federal Reserve -- is telling us. In the minutes from the Fed's last meeting, released Wednesday afternoon, policymakers indicated that the recession could be over "before long."

But on Main Street, people seem to have a decidedly different view. It's hard to find things to be happy about when the unemployment rate is at a more than quarter-century high of 9.5% and the housing market remains in shambles.

So even though investors, CEOs and Fed chair Ben Bernanke may be seeing green shoots everywhere they look, many average Americans have to squint very hard in order to find them. Just try telling a Zillow.com-obsessed home owner who's watched the value of his home continue to plummet in recent months that the economy is getting better.

"The big unknown variable in the economy still is housing," said Haag Sherman, managing director with Salient Partners, an investment firm based in Houston. "The worst may be behind us with subprime loans, but I don't think housing has found a bottom. We could have a recovery in Corporate America that's much narrower than the recovery in the broader economy."

Talkback: Are you thinking of buying or selling a house now or do you think prices will fall much further? Leave your comments at the bottom of this story.

Sherman pointed out that banks are starting to experience waves of delinquencies and defaults with higher-quality mortgages such as alt-A loans and prime loans.

The bank said that net charge-offs, a figure that measures the amount of debt written off as bad, were $1.3 billion from home equity loans, double the $511 million of a year earlier. And net charge-offs related to prime mortgages quadrupled to $481 million from $104 million last year.

JPMorgan Chase is widely considered one of the healthier banks around. So if it's experiencing more difficulties in its home loan business, it's likely that two of the more troubled big banks, Citigroup (C, Fortune 500) and Bank of America (BAC, Fortune 500), will also disclose more bad news from their mortgage units when they each report their second-quarter numbers Friday morning.

Rising delinquencies could lead to more foreclosures. If so, it's harder to envision a scenario where prices will rebound since foreclosed homes just add to the inventory of unsold real estate.

Steven Kyle, professor of applied economics at Cornell University, said the fear that prices will fall further is discouraging some homeowners from even putting their homes on the market because they're worried about the existing glut of homes.

Kyle added that with unemployment nearing 10% and likely to exceed that level before long, he worries many consumers won't be willing to buy homes -- even if prices stay relatively affordable and mortgage rates remain fairly low.

"Housing is not going to go rocketing off anytime in the near future. Interest rates are already low, so you won't get a boost from that. And unemployed people aren't buying houses," Kyle said.

Sherman said that, ironically enough, more talk of an economic recovery could actually hurt chances of a housing rebound, since it could lead to higher rates in the future.

That's because some fixed-income investors may start to fear inflation and start selling long-term bonds, which would drive up their yields. Bond rates and prices move in opposite directions.

In fact, it's already happened to some extent. The yield on the U.S. 10-year Treasury has surged from a low of about 2% in December to about 3.5% currently. Many fixed-rate mortgages are closely pegged to longer-term Treasury rates, so a further dumping of Treasurys won't help prospective home buyers.

"You can have housing stabilize for a bit and then take another leg down. Continued affordability is what we want to see, but if bonds sell off, that weighs on affordability," Sherman said.

Nonetheless, there are some pieces of good news about the housing market. The National Association of Home Builders reported Thursday that builder confidence in July was it its highest level since last September.

In addition, finance professors Paola Sapienza of the Kellogg School of Management at Northwestern University and Luigi Zingales of the University of Chicago's Booth School of Business, said consumers are much more confident about the housing market now than they were just a few months ago. The professors run a quarterly survey of consumers that looks at their trust in the financial system.

Sapienza said that a majority of consumers polled in June thought that prices in their market would be stable over the next 12 months and that only 26% thought prices would decline. By way of comparison, nearly half of the consumers surveyed in December were predicting price drops, Sapienza said.

We'll find out even more about the state of housing Friday morning when the Census Bureau reports its latest figures on housing starts and building permits. Both numbers are key measures of potential demand for new homes.

Economists surveyed by Briefing.com are forecasting that the number of permits rose slightly in June to an annualized rate of 524,000 from 518,000 in May and that starts dipped a tad last month -- from a rate of 532,000 in May to 530,000 in June.

The consensus estimates for both permits and starts are considerably higher than the record lows set in April. So as long as the June numbers are close to forecasts, one could make the case that the housing market is stabilizing and that the worst may be over.

But as I pointed out in Wednesday's column about earnings, stabilization is not the same thing as a recovery. And even though Wall Street still seems willing to subsist on a diet of green shoots salad, consumers are hungering for more substantial evidence of a recovery.

"The end of the world didn't in fact happen so people got more optimistic earlier this year. But to say the end of the world was avoided doesn't mean we're now swimming in rose petals," Kyle said. "The real estate market is still looking rocky. The danger is we just bump along in this stagnating saggy state for awhile."

And as long as that's the case, consumers are unlikely to feel much better about the economy.

"People are still nervous. We haven't seen an actual turn in housing yet, just signs of bottoming," said Brad Sorensen, director of market and sector research with Charles Schwab. "This isn't rocket science but an improvement in the housing market will make consumers feel more confident and more willing to spend and that will have a trickle down effect on the overall economy."

Talkback: Are you thinking of buying or selling a house now or do you think prices will fall much further?