Why Even Rich People Are Terrified of Retirement

Just because they have a big pile of cash to cushion their fall doesn't mean wealthy workers lack concerns about retirement.

As the folks at UBS Wealth Management discovered in a recent survey, nearly half of wealthy pre-retirees (45%) have a retirement savings target between $1 million and $3 million. Roughly 91% believe they have the financial tools and knowledge necessary for a comfortable retirement, with 89% saying they are confident they will have enough money saved. In addition, 74% believe they know how long their savings will last once they retire.

But money isn't the only thing on their minds as retirement closes in. When asked what they'd miss most, 64% of wealthy workers (those with more than $1 million in investable assets) said they'd miss the comforting stability of the work schedule more than their salary. Others fear the adjustment to retired life (59%), leaving work colleagues behind (57%), experiencing an initial shock (39%) losing a sense of purpose (36%) and filling the hours of free time (34%). For those engrossed in their work and excelling at their jobs, retirement can seem more like a looming inevitability rather than a welcome respite.

"Baby Boomers have been known for 'living to work,' having been focused on their careers for so many years," says Paula Polito, client strategy officer of UBS Wealth Management Americas. "Now, as many of them look toward retirement, they need to start 'working on living,' - figuring out how they will fill their time and find their purpose once they leave the workforce."

Though only one in five wealthy retirees worry about outliving their assets, their health remains an issue. Of those surveyed, 73% say getting sick is their top retirement concern. Nearly half (47%) are also worried they will not have anyone to take care of them.

In that small way, they share the fears of the average U.S. worker. Voya Financial points out that 59% of working Americans are very or extremely concerned about outliving their retirement savings -- with 74% having never calculated their monthly retirement income needs. Both Voya and U.S. Bank note that retirees will need 70% to 80% of their current annual income to continue their current lifestyle in retirement.

That's easier said than done. The Voya study found that 61% of workers were significantly concerned about their inability to pay for healthcare expenses in retirement. Meanwhile, 58% were also significantly concerned that they would end up with fewer Social Security benefits than expected. That's not great news, when 45% of retirees plan to rely on Social Security as a major source of their income in retirement and 66% of workers planned to start taking Social Security at age 66 or younger — well short of when full benefit payouts begin at age 70.

Though GOBankingRates notes that 13% of workers have $300,000 or more saved for retirement, about 30% of those age 55 and over have no retirement savings and 26% have less than $50,000. In fact, only 26% of Baby Boomers nearing retirement age have $200,000 or more, while 31% of Boomers over 65 can say the same.

That still beats the 52% of Generation X (ages 35-54) who still have less than $10,000 in retirement savings after the recession wiped out 45% of their net wealth on average, according to Pew Research Center. Roughly 31% of Gen Xers have no retirement account at all, though 40% of Gen Xers over 40 have more than $50,000 in retirement accounts. Among that group 7% have between $200,000 and $300,000 socked away, while 15% have $300,000 or more. Meanwhile, though 60% of Millennials (18 to 34) have started a retirement fund, 72% have saved less than $10,000 or nothing at all. A full 42% have no retirement savings, though that percentage shrinks to just 36% of those older than 25.

That's all well short of the recommended thresholds. GOBankingRates and J.P. Morgan have determined that, if you're age 40 and making the $66,693 median salary, you should have more than $100,000 saved for retirement. Unfortunately, only 20% of people at that benchmark do. For the 50-year-olds making a median of $70,832 a year -- the peak of their earnings -- there should be close to $212,496 socked away. Only 22% have hit that mark. As for 60-year-olds coasting into retirement at $60,580 a year, only 26% have the recommended $260,500.

Even for a normal retirement, T. Rowe Price recommends saving 15% of your earnings -- including employer contributions. If you start at age 30, you can earn you upwards of $1.7 million by the time you retire. Now, that assumes 7% annual returns on your investments, a $50,000 starting salary at age 30, a 3% annual salary increase, a 4% annual withdrawal rate beginning at age 65 and 3% annual inflation. It also assumes that you wouldn't forfeit about $570,000 by saving just 10% a year... and won't retire early. However, knock 15 years off of that total and you're down to $457,000 based on those same variables -- or less money to cover a longer retirement.

Prudential Investments found that although 80% of people they surveyed before the end of the year considered retirement their to priority, the average grade they give themselves for the retirement preparation is a "C." A very honest 12% give themselves a failing grade.

"Understanding the hurdles keeping people from a secure financial future is critical to helping them meet their goals," says Stuart Parker, president of Prudential Investments. "This research reinforces the need for people to seek advice and the need for the investment community to give advisors the best tools and solutions available."

A whopping 74% of you think you should be doing more to prepare for retirement, while 40% don't know what to do to prepare. Though 24% of workers think they'll need $1 million or more to retire, 54% have less than $150,000 saved in employer-sponsored plans. It doesn't help that 20% of workers don't believe they'll ever be able to retire, while 35% say they'll never be able to save enough, so it doesn't matter when they start saving. That kind of retirement nihilism is leading to all sorts of bad decisions, with 57% of Americans saying they would use savings to cover a financial emergency.

That's not helpful when 51% of all retirees retired earlier than expected, with 50% retiring five or more years earlier than expected. If you're among the 2% of that group who retired early because they were tired of working, congratulations. However, if you're among the 52% who retired because of your health problems or those of a loved one or the 30% who were either laid off or bought out, it's can be scary out there. Among those of you who haven't retired he, 57% say healthcare costs could bite into retirement savings, another 57% say changes to Social Security might alter their plans and 45% say the potential for dealing with an illness or disability has them worried about their savings.

None of that has scared you into saving, though. According to a recent study of 1,000 U.S. workers by financial services firm Edward Jones, 45% of non-retired U.S. workers aren't saving for retirement at all. Of that group, only 36% plan to do so in the future and almost 10% say they aren't planning to save for retirement at all. While 58% of respondents 18 to 34 years old have not yet started saving, 90% say they have or plan to start saving for retirement before they turn or turned 30. However, as a testament to the power of procrastination, 26% of 35- to 44-year-olds say they plan to start saving in their 40s.

"When it comes to retirement savings, there's a big difference between planning to save and actually doing so," said Scott Thoma, principal and investment strategist for Edward Jones. "While intentions to save for retirement are legitimate, individuals tend to satisfy more immediate, short-term spending goals and push off their long-term saving goals. This behavior can be incredibly detrimental for individual investors, particularly as they enter the critical savings periods of their 30s and 40s when they have (and unfortunately waste) a tremendously valuable asset -- time."

That said, even the wealthy don't always feel so secure in their financial status. In a survey of 2,215 U.S. investors with more than $1 million, UBS found that 58% of all millionaires report feeling increased expectations for their standard of living over the last ten years. Among working millionaires with children at home, 52% feel like they are stuck on a treadmill, unable to get off without sacrificing their family's lifestyle.

Millionaires' wealth hasn't done much to assuage their fears either. Half (50%) of those with $1 million to $5 million are afraid that one major setback (job loss, market crash) would have a significant impact on their lifestyle. Even 34% of those with $5 million or more feel the same. Give those same millionaires kids, and suddenly 63% feel that one major setback would have a significant impact on their lifestyle.

Maintaining that lifestyle comes at a cost, as 64% of millionaires report that they have had to give up family time to achieve their dreams. Most millionaires (68%) admit to having regrets, most commonly around making mistakes in a relationship with their spouse or family and not spending more time with family. They worry that their children will grow up without the right values, with two in three (67%) already feel that their children take things for granted and more than half (53%) are at least somewhat worried that their children act entitled. Millionaire parents expressed concern that their children do not understand the value of money (65%), lack motivation (54%), have unrealistic expectations (54%) and fear that they will embark on an unstable career path (50%).

However, retirement tends to assuage those fears quickly. Half of wealthy retirees took no time at all to adjust to retired life. Another third took less than a year. A full 84% of wealthy retirees say they are happier than at any point in their lives. If wealthy retirees had the chance to do it over again, only 19% would have delayed their retirement.

Still healthy and financially stable, 90% of wealthy investors are very satisfied with life in their 60s and 70s. That's higher than investors in any other age group, including those in their 30s (68% satisfied) and 40s (83% satisfied). One wealthy retiree told UBS retirement was like being young again, "but with money and no curfew."Yes, wealthy workers still have nightmares about retirement, but actual retirement wakes them out of it pretty quickly.

"Based on the experience of current retirees, wealthy pre-retirees can lay their fears to rest," says Sameer Aurora, head of client insights for UBS Wealth Management. "Most investors are happier in retirement than they have ever been."

NFL Hall of Famer Mike Haynes is regarded as one of the best corner-backs of all-time. TheStreet talks with Haynes about his career, financial lessons learned along the way and his outlook for the NFL.