They are barely a mile apart, separated by a few gritty streets and a thin muddy stretch of water known as the Harlem river. They are in the same city and have experienced the same recession.

But New Yorkers living in the city's 14th and 16th congressional districts – electoral districts with populations of around 600,000 each – often occupy completely different worlds. Their lives provide a shocking example of growing inequality in America, where the rich are leaving a growing mass of the poor completely behind.

The numbers are stark enough. Last week a census report revealed that 46 million Americans live in poverty, the highest number ever recorded. At the same time, the richest 20% of Americans control 84% of the country's wealth. Indeed, just 400 families have the same net worth as the total of the bottom 50%. America's Gini coefficient – which measures inequality of income distribution – now nears that of Rwanda.

The Gini figure is just a number – but to walk the streets of the 14th and 16th districts is to see that story of growing inequality in terms of people living almost next to each other but separated by education, job prospects, health, race and class.

The 14th occupies a chunk of Manhattan and Queens. Not all of it is wealthy, but at its heart lies the Upper East Side, by Central Park, a neighbourhood that is home to New York's moneyed classes. It is here that the titans of finance, whose recklessness brought on the near collapse of the American economy, live and play. They raise their families in gigantic apartments, send their children to the best private schools and patronise the pricey bistros that dot the street corners. Old money New York has long considered the Upper East Side its natural home, viewing Central Park as its backyard and Manhattan as a private playground.

The same cannot be said of the 16th. That district spans the South Bronx. It has been occupied by waves of immigrants, now mainly Hispanics from Mexico, the Dominican Republic and Puerto Rico, plus black Americans heading out of the south or fleeing higher rents in gentrifying Harlem. It is rife with gangs, drugs and crime. Well-paid jobs are scarce.

To travel between the two districts is to go from a world of unimaginable luxury to one of fear and poverty. It takes about 10 minutes on the subway.

Felix Santiago, 51, has certainly felt the impact of the great recession on the 16th district. He arrived from Puerto Rico 30 years ago and made a home when the neighbourhood was scarred by the drug epidemic and racial troubles of the 1970s and 1980s. Now he sees it going downhill again. "If you live in this neighbourhood, you are poor. If you try to be middle class, you just can't do it," he said.

Santiago has tried. He works as a handyman in a local church; his daughter joined the US Marines. But he struggles as rents and food prices go up. He shook his head at the idea that America's economy has recovered since the financial crisis. In the South Bronx, he said, it is still getting worse. "I think this year has been the real critical one. There is no work. Prices are going up. It is getting ridiculous."

Life is definitely hard in the 16th compared with the 14th. Life expectancy is three years lower in the South Bronx than on the Upper East Side. The median household income of $23,000 (£14,500) in the 16th is barely above the official poverty level. In the South Bronx, nearly 40% of people live below the poverty line; in the 14th, the figure is less than 10%.

The murder rate is four times higher in the 16th and the number of robberies more than twice as high. There are 90 Starbucks coffee shops in the 14th. There is just one in the 16th.

Just a few blocks away from where Santiago works are some of the toughest streets in the Bronx. On one corner, even though it is barely 10am, police have a lookout post on top of a mobile crane. It looks eerily like a medieval watch tower. Nor does it stop drug dealers from plying their trade nearby, though Santiago warned that times are tough for them too. "There is no money for people to spend on drugs," he said, blaming that for a spate of muggings as frustrated dealers turn to robbery. Local shops sell cheap clothes and corner stores accept food stamps. It would be suicide not to. The use of food stamps in the US has risen by 75% in the past four years and now covers 15% of the population.

But if you hop on the number 6 subway line and travel a few stops south to the Upper East Side, food stamps are not an issue. The streets are crowded, luxury shops sell the latest fashions and French restaurants are doing a roaring trade. There is anger at the recession here, too. Certainly Sam Durant is furious. He runs a high-end jewellery store on Madison Avenue and his trade is down as Wall Street bankers are now often paid bonuses in stocks not cash.

Durant knows where the blame lies. "People are not spending," he said "That asshole in the White House has taken away their bonuses. He doesn't want them to have what he doesn't have," he said. His disapproval of Barack Obama is fierce. "He's the most hated president in history, did you know that?" he said.

Politics inspires worry in the South Bronx, too. But in a different way. In St Ann's church, the Rev Martha Overall watched last week's Republican debate in dismay, especially the attacks on government. She fears the impact that enormous government cuts are already having, let alone the sort that any Republican president might bring in. "It's social Darwinism. It's people being pitted against people. I also believe it is un-American. I don't believe this country was founded on a sense of every man for himself. It was founded on community," Overall said.

Rather than fretting over Wall Street bonuses, Overall is scared about reports that the local post office, a major employer in the area, might shut "It is also kind of a social centre here," she said.

"Don't walk across the park," warned Santiago, referring to St Mary's Park in the South Bronx. Such sentiments are rarely voiced on the Upper East Side about Central Park. Last week it was its usual haven, packed with young mothers and their children and, more frequently, with dark-skinned nannies wheeling white children around.

One young mother, who gave her name as Ally, said the recession had hit the neighbourhood. Her husband worked in finance and, while he had kept his job, things had been nerve-racking. "It's not been easy. You don't feel you have security and so you watch your spending," she said.

But in general the finance industry is roaring again. Last year the ratio of wage bills to company earnings rose at 10 out of 16 major US and European banks. This year again will see bonuses worth hundreds of thousands of dollars for many and millions of dollars for the lucky few. Such payouts fuel the Upper East Side property market.

A glance at an estate agent's display on Madison Avenue revealed a townhouse going for $26m. Prefer to rent? Another could be had for $58,000 a month. Even a modest-looking two-bedroom apartment cost $1.9m. On 75th Street, one opulent mansion has just been sold by financier Christopher Flowers for $36.5m (he had left the property empty for the past couple of years).

It is a different picture a few miles away. There are vacant properties in the Bronx, but they tend to be derelict. Yet there are links between the two neighbourhoods. Overall is one herself. She is from the Upper East Side but works with church groups in what she calls "the other congressional district".

St Ann's welcomes volunteers from the Upper East Side and schools operate exchange programmes. But on the whole, the people of the South Bronx pay little attention to the people of the Upper East Side and vice versa. Both are too busy with their own concerns.

Mike Manigault, 21, came here from South Carolina. He had just got a precious job at the Fat Boy Barber Shop. He was glad his year-long quest to find work had ended, even if he was just sweeping up hair. "It's tough, but you just do what you got to do. You either quit or you keep trying," he said.

And as for life as it's lived on the Upper East Side? He shrugged. He has never been there.