The wealth gap between the richest Americans and the typical family more than doubled over the past 50 years.

That trend is happening for two reasons: Not only are the rich getting richer, but the middle class is also getting poorer.

Most Americans below the upper echelon have suffered a decline in wealth in recent decades. The median household saw its net worth drop to $57,000 in 2010, down from $73,000 in 1983. It would have been $119,000 had wealth grown equally across households.

The top 1%, on the other hand, saw their average wealth grow to $16.4 million, up from $9.6 million in 1983. This is due in large part to the growing income inequality divide, as well as the sharp rise in value of stocks over the period.

"The distribution of wealth is way more unequal than the super-unequal distribution of income and wages," said Heidi Shierholz, an economist with the institute.

The biannual report, entitled The State of Working America, looks at the changes in income, jobs, mobility, poverty, wealth and other areas in recent decades, as well as during the Great Recession.

The effects of the Great Recession

While the wealth and income gaps have been expanding for decades, the report shows that the trend was accelerated during the Great Recession.Median family income was 6% lower in 2010 than a decade earlier.

As for wealth, while the housing bust and the spike in unemployment hurt people at all levels of the spectrum, it affected middle-class and lower-income Americans to a greater degree.

The average wealth of the top 1% dropped just 15.6% between 2007 and 2010, while the median net worth of American households sank 47.1% That large decline in median wealth is largely responsible for driving the gap to such heights.

Homeowners at the bottom of the wealth distribution were, on average, underwater, meaning they had no equity in their homes because their mortgages were more than the property's value.

The Great Recession hit black and Latino households particularly hard. It wiped out half the wealth of a typical black household, leaving them with a median net worth of $4,900. And the median wealth of Latino families plummeted 86.3% to $1,300.

This compares to $97,000 for white households.

Super rich take advantage of tax havens Tax Havens & The Super Rich

Part of the reason for the eye-popping statistics is because blacks and Latinos had a relatively small amount of net worth so the drop is larger in percentage terms. Also, their homeownership rates grew faster than whites' during the housing boom, but fell further when it collapsed.

The typical black and Latino households don't own any stocks, and the typical black family has no home equity.

Overall, the widening of the wealth gap in recent decades is due to two things, Shierholz said. The increase in income inequality means the wealthy have more to save and invest every year. Furthermore, the growth of Wall Street means that the rich, who are much more likely to own stocks, accumulated wealth even faster.

"The growth in wealth we did see got funneled to the top and it didn't spur faster growth in the middle," Shierholz said.