The report, released for the first time on Thursday, found 34% of households said they were “somewhat worse” or “much worse” financially in 2013 compared to 2008. Only 30% reported being better off to some degree.

Getty Images

Considering that respondents were asked to compare their incomes to 2008, during the depths of the recession, “the fact that over two-thirds of respondents reported being the same or worse off financially highlights the uneven nature of the recovery,” the Fed report said.

That data comes from the Fed’s Survey of Household Economics and Decision Making. The survey was conducted for the first time last year as part of the central bank’s effort to monitor America’s recovery from the recession and identify any risks to households’ financial stability. The new report offers a view of how households assess their own situations that is not found in other U.S. data sources.

Despite relatively few Americans feeling their finances are improving, most appear to be faring relatively well. The survey found 23% reported “living comfortably” and 37% said they were “doing OK.” Less than 40% said they were “just getting by” or otherwise struggling.

The survey also drilled down into several specific topics:

Housing

The financial crisis rattled the housing market, but Americans now feel pretty confident about their real estate. The survey found 40% of homeowners expect the value of their property to increase in the next year, while less than 10% anticipate values to decline in the coming 12 months.

A majority of renters, meanwhile, say they’d like to buy a home but are financially blocked from doing so. In the survey, 45% said they rent because they can’t afford a home and 29% said they do so because they can’t qualify for a mortgage.

Retirement

Almost half of Americans have largely failed to plan for retirement, with 24% saying they’ve given “a little thought” to the subject and 25% saying they’ve done no planning at all.

Of those surveyed, 31% said they have no retirement savings or pension—including 19% of those ages 55 to 64.

The long-term shift from defined-benefit plans, such as pensions, to defined-contribution plans, such as 401(k) accounts, “places significant responsibilities on individuals to plan for their own retirement, (but) only about one-fourth appear to be actively doing so,” the Fed report said.

Medical Expenses

A vast majority of Americans had some form of health insurance last year—before plans offered under the Affordable Care Act took effect—but many still struggled with the high cost of medical care.

Despite 84% of Americans having coverage, only 21% of those surveyed said it was likely that they could afford to cover a major out-of-pocket expense. As a consequence of that inability to pay, a quarter of respondents said they had avoided seeking medical treatment in the past year.

Five years after the start of the recession, “a small, but significant, core of respondents continues to experience economic hardship on multiple dimensions,” the Fed report said. “A sizable fraction of respondents appear to be financially vulnerable to unexpected events such as a serious illness, unexpected expense or job loss.”

About Real Time Economics

Real Time Economics offers exclusive news, analysis and commentary on the U.S. and global economy, central bank policy and economics. Send news items, comments and questions to the editors and reporters below or email realtimeeconomics@wsj.com.