More than 1.6 million people between the ages of 20 and 40 are living with their parents because they cannot afford to rent or buy their own home.

The figures are revealed in a report by YouGov on behalf of the housing and homelessness charity Shelter, which said they showed a "vivid picture of arrested development" for young people.

Nearly half of the parents surveyed said they did not believe their children would ever be able to get on the housing ladder. A similar number (44%) were concerned that living at home was holding their children back from leading an independent adult life.

Of those living at home, just over a third said they felt embarrassed about admitting to their housing situation, and others believed that their relationship with their parents had deteriorated. Nearly two-thirds thought that it was harder to form relationships.

Campbell Robb, Shelter's chief executive, said: "These figures paint a vivid picture of 20 and 30-somethings in arrested development, with our housing crisis putting the brakes on their aspirations for the future. Our chronic lack of homes that young people can genuinely afford to rent or buy is at the root of the problem.

"There's no doubt that young people are grateful to be able to live with mum and dad to save money. But we have to question whether it's acceptable that this is becoming the norm for people to live at home into their mid-30s, when we know that they are desperate to be independent and make their own way in the world."

He added: "As rents soar and deposits become even further out of reach, the government needs to look seriously at how it can meet these young people halfway, and make housing more affordable so that this generation and the next can get on in life."

The issue of housing and younger people has become more conspicuous as the government has moved closer to removing housing benefits from people aged under 25 as part of a further package of welfare cuts aimed at saving £10bn. In June, David Cameron said many of those claiming benefit were able to move back home, and were only on benefits so they could be independent.

Campaigners for single people, who fare worst when it comes to allocations of social housing, have become more prominent. Pennie Quinton, chair of Phoenix Housing Co-operative, which was established in 1980 by a group of single people – some former squatters – to create safe accommodation, told the Guardian's housing network: "It is time for a new debate over statutory rights for housing for all single people.

"Social housing should exist as a resource for all who find themselves unable to participate in the private housing market. There is no rational reason for targeting single people or for regarding us as having less need of a roof over our heads than anyone else."