NEW YORK (CNNMoney.com) -- The sputtering U.S. economy has gotten everyone from the financial markets to the Federal Reserve to Congress in a panic.

But here's a disheartening message for those already worried about economic growth -- it could get much worse.

Most economists who believe a recession is already here or at least near are looking for a relatively short and mild downturn, perhaps lasting only two or three quarters.

But many of those same economists say they also can envision a worst-case scenario where spending by consumers and businesses falls off sharply, unemployment heads higher than normal during a typical recession and housing and credit market problems worsen.

"I can easily imagine [the economy] going into a free fall," said Dean Baker, the chief economist for the Center for Economic and Policy Research. "The danger is that housing prices continue to tumble and accelerate, people's ability to pull out equity will evaporate, and you'll see a serious downturn in consumption."

We talked to three more leading economists to find out their biggest economic fears. Here's what they had to say.

Greenback blues David Wyss, chief economist with Standard & Poor's, said that among his biggest concerns is that overseas investors could pull back on investing in the dollar and other U.S. assets.

That could cause an even greater sense of fear among U.S. consumers and businesses, as stock prices fall and bond yields rise, which in turn would lift mortgage rates and be a bigger drag on the already battered housing market.

"Americans could just get scared by a barrage of bad news," Wyss said. "The stock market could continue going down because of foreigners pulling money out, and between that and home values going through the floor, it could lead to a real pullback of spending, particularly by Baby Boomers who are getting close to retirement."

Wyss said he's also concerned that oil prices could shoot higher, even if a recession cuts into global demand. He said supply disruptions in the Middle East could send oil prices up to $150 a barrel and help deepen any recession.

Wyss said that in his worst case scenario, the unemployment rate would climb to 7.5 percent by early 2009, up from its current level of 5 percent.

He also believes gross domestic product, the broad measure of the nation's economic activity, could wind up as much as 2 percent lower at the end of 2008 than it was at the end of 2007. That would be the biggest downturn since 1982. Many of those forecasting a recession this year are expecting GDP to show a slight gain by the end of the year.

House of pain. Edward McKelvey, senior economist at Goldman Sachs, agreed with Wyss that, in a worst case scenario, GDP could fall 2 percent this year..

His biggest fear is that home prices could fall much further in the coming months. In fact, Goldman and economists at Merrill Lynch have both predicted that home values could fall another 15 percent, on top of the 10 percent drop from earlier peaks that has already taken place.

McKelvey said further declines could cause much deeper problems for consumers and credit markets.

"One of the most likely candidates would be credit markets acting more violently than we thought, a tightening of the supply of credit to businesses and households," he said when asked what could bring about his worst case outlook.

"You could also see a more substantial response by businesses to the downturn through layoffs, cuts in their spending and business plans," he added.

Bank woes just beginning. Paul Kasriel, chief economist at Northern Trust, said he thinks there's a good chance that the economic pullback will be much steeper than now widely assumed. This weak forecast is based on his belief that the billions in dollars of writedowns already reported by Merrill Lynch (MER, Fortune 500), Citigroup (C, Fortune 500), JP Morgan Chase (JPM, Fortune 500), Bank of America (BAC, Fortune 500) and other big banks are just the beginning of the problem in the financial sector.

Kasriel said that if banks have to report more losses due to bad bets on subprime mortgages, they will be unwilling, or unable, to make large loans to businesses and consumers.

So even if the Fed keeps cutting interest rates, the impact of the cuts may be "less potent" than rate cuts in previous recessions since consumers and businesses may not be able to borrow enough to keep spending. That could make this recession more like the one in 1991-92 than the relatively short and mild recession of 2001.

"Historically, and not surprisingly, recessions accompanied by declines in consumer spending tend to be more severe. And people are going to be constrained from spending by the declines in housing," Kasriel said.

He added that state and local governments might have to cut back spending as a result of declining tax revenue. And that would be another sizable blow to the overall economy.

"People forget about state and local government spending, but it represents 11 percent of GDP," Kasriel said.