In October, Cuker got the second job that brings in an extra $5,000 a year. It came just in time. In November, Belgium-based InBev completed its takeover of Anheuser-Busch. By Christmas, Jennifer's job was gone.

The family's income dropped from about $93,000 to $47,000 a year. After paying their $1,700 mortgage and some of their roughly $50,000 in credit card debt each month, Cuker says, there's barely $200 left.

So they tap their savings, and weigh every expenditure — from an ice cream cone to a haircut.

"Sometimes you can't afford to get food," he says. "We just scrape whatever's in (the pantry)."

Jennifer remains unemployed, but is searching for a job.

In February, Joe tried to get food stamps and make payment arrangements with his credit card companies. But he was turned down for both — because he earned too much money to qualify for food aid, and he already had the lowest interest rates offered on his cards.

"What's most stressful is trying to be proactive and responsible," he says, "and that doesn't get you anywhere."

Joe Cuker is part of the age group that worried about money the most. About 43% of people 30-49 who were surveyed throughout last year said they had worried about money the day before, according to the poll.

"They're planning for their families' futures," says Frey. "They're going to be the mainstay of people older and younger than them. And they're just scared."

Latinos feel the strain

Among ethnic groups, Latinos were the most likely to be concerned, with 42% saying they worried about money the day before. Blacks were close behind, at 41%, while 35% of whites and 35% of Asians said the same.

"Latinos are probably the most aspirational of all the groups in the U.S.," Frey says.

During the last decade, "They got a foothold into the suburbs. They got a foothold into the housing market, and now it's all crashing down around them."