Documentary photographer Lauren Greenfield was trying to form trusting relationships with members of a Mayan tribe in Mexico in the early 1990s when she picked up a discarded copy of Bret Easton Ellis’s Less Than Zero. Before she’d finished the cult novel – which charts the parties, drug taking and sex lives of rich college kids in Los Angeles – Greenfield had decided to swap photography subjects from the Maya of Chiapas to the rich kids of her home town.

A quarter of a century later, Greenfield has amassed 500,000 images of the often absurd lives of the wealthy. The highlights – including a picture of go-go dancers hired for a 13-year-old’s bar mitzvah – are published by Phaidon in a £60 2.5kg tome called Generation Wealth. An accompanying behind-the-scenes documentary film is released in the UK next week.

The value system changed completely. It wasn’t about who you are, but about what you are worth

Greenfield introduces us to characters all motivated by the accumulation of wealth. “No matter how much people had, they still wanted more,” Greenfield says of her subjects. We meet Florian Homm, a hedge fund manager living in self-imposed exile in Germany to avoid extradition to the US where he has been sentenced to 225 years in jail. Smoking cigars and dripping in gold, Homm, who became known as “the antichrist of finance” for ripping off his investors for hundreds of millions of dollars, tells Greenfield that morality changed in the 80s. “The value system changed completely. It wasn’t about who you are, but about what you are worth… Morals are completely non-productive in that value system.”

Money to burn: Florian Homm, sentenced to 225 years in jail in the US. Photograph: Lauren Greenfield/INSTITUTE

As his hedge fund was imploding during the financial crisis of 2008, Homm, now 58, fled the €5m Majorcan villa he shared with a 27-year-old Russian lingerie model. With $500,000 stashed in his underwear and a humidor in hand, Homm boarded a plane to Colombia and disappeared for five years. We learn that he used his fortune to buy his son, then 15, the services of a Dutch prostitute. Homm was later arrested at the Uffizi Gallery in Florence.

One of the photos that best sums up Greenfield’s work is of the DJ, rapper and producer Lil Jon flashing a smile showing $50,000 worth of diamond and platinum dental work. There’s also the former porn star Kacey Jordan, who received $30,000 to attend a sex party at Charlie Sheen’s home. Jordan tells Greenfield of seeking greater wealth through extreme sex, which may have made her temporarily wealthier but didn’t make her happy. She made numerous suicide attempts, and ends the film in the same dead-end job she had before starting porn.

Then there’s Eden Wood, six, a beauty pageant princess and star of reality TV show Toddlers & Tiaras, who tells Greenfield “My favourite princess is me” and says beauty means “that I get money, and I’ll be a superstar.”

Greenfield says the intimacy she shares with her subjects at their most vulnerable led her to consider the impact her obsession with their lives has had on her own family, and turns the camera on her sons, mother and husband. Her 16-year-old son Noah tells her: “I got used to growing up without you around. The damage has been done.”

The stories Greenfield tells about rich families detached from the world around them, living in bubbles separated from reality by armies of nannies and household staff, chime with my own experiences covering the super-rich as wealth correspondent for the Observer and the Guardian. From Knightsbridge to Monaco, the Upper West Side and the Hamptons, the wealthiest people in society are richer than they’ve ever been before. More of them have five, six, seven or even a dozen homes in the world’s most desirable locations and it is not unusual for them to fly their nannies, tutors and security details between them on private jets. Their houses may cost as much as the £135m a Ukrainian billionaire paid for a penthouse in One Hyde Park, but that doesn’t mean they’ll spend much time there.

You have to have £20m to live on The Boltons, no real people live here. You have to be an Arab prince, or an oligarch

The Boltons, a street of grand Victorian mansions in South Kensington, has been an address that only the very well-heeled can afford since the 19th century. But the influx of wealthy Russian and Middle Eastern families to London means prices have gone beyond the reach of all but the very richest of Brits. The average house price is a fraction under £20m – 88 times the UK average – making it the third most expensive street in the country (after London’s Kensington Palace Gardens and Grosvenor Crescent). Over cups of tea and strawberries and cream at the annual church fête held in a private garden square overlooked by the mansions this summer, locals tell me of a disconnected community. “We must be standing on some of the most expensive real estate in the world,” says David, a long-standing member of the St Mary The Boltons congregation, who lives a few streets away. “But you won’t find any of the owners at our fête. You probably won’t find any of them at home from one month to the next.”

Portrait of the artist: Lauren Greenfield (right) at work on her massive project. Photograph: Lauren Greenfield/INSTITUTE

A growing number of academics warn that the widening gulf between the richest 1% and everyone else could lead to a backlash. The richest 0.1% of the world’s population has increased their combined wealth by as much as the poorest 50% – or 3.8 billion people – since 1980, according to the World Inequality Report. The report, by the French economist Thomas Piketty and 100 other researchers, also found that the richest 1% of the global population “captured” 27% of the world’s wealth growth between 1980 and 2016. Piketty warns that inequality has ballooned to “extreme levels” in many countries, and will only get worse unless governments take co-ordinated action to increase taxes and prevent tax avoidance.

A four- storey, nine-bedroom house complete with cinema, indoor swimming pool and Japanese garden sold for £51m in 2015 to an offshore company. The stamp duty alone set back the unknown buyer £7.6m. The owner is paying £220,000 a year in extra tax in order to keep their identity a secret.

Greenfield says the true absurdity of extreme wealth hit her when she was documenting the lives of the Siegel family, who were attempting to build the biggest private home in America, for her film The Queen of Versailles. When “timeshare king” David Siegel loses billions in the 2008 financial crisis, the family are forced to travel by commercial jet and one of the children turns and asks, “Mommy, what are all these people doing on our plane?” Greenfield met the Siegels after striking up a friendship with Donatella Versace, whom she met via her work documenting the lives of rich kids in LA. She went back to Crossroads, the $38,000-a-year private school in Santa Monica she’d attended alongside Hollywood’s rich and famous offspring. Greenfield says her position as an “insider and outsider” – she went to the school herself, but her parents (both academics) couldn’t afford to kit her out with the designer bling the other kids had – gave her “exceptional access to the world of the wealthy in LA”.

Bags full of it: at a private opening at a Versace store in Beverly Hills. Photograph: Lauren Greenfield/INSTITUTE

Greenfield never wanted to become an expert on the rich or pass judgement on their lives or spending. “I wanted to be a guide, to take the viewer on a journey and to help them understand the shift in society.” She admits weeping a few times while putting the book and film together. .

We meet for lunch at London’s five-star May Fair Hotel, surrounded by fellow diners who could easily be her subjects. “The photos are evidence of a sea change in our values over the last 25 years,” she says. “I felt like we had gone from the American dream of opportunity for all to a desire for ever more wealth, in the currency of money, fame, beauty or youth.” She says it’s not just an issue that effects the wealthy, but everyone in society. “We’re all complicit in generation wealth.”

Greenfield interviewed several experts for the film, but just one survived the cut. Former New York Times journalist and leftwing activist Chris Hedges is quoted as saying “Wealth is whatever gives us value” and warns that “Societies accrue their greatest wealth at the moment they face death.” This last remark might trouble Americans in particular: last year Professor Philip Alston, United Nations special rapporteur on extreme poverty, made a statement accusing Donald Trump and the Republican party of consciously distorting the shape of American society in a “bid to become the most unequal society in the world”.

Alston, who acts as a watchdog on extreme poverty, said Trump’s administration had passed tax laws that “overwhelmingly benefitted the wealthy and worsened inequality”. He said Trump’s policies “seem deliberately designed to remove basic protections from the poorest, punish those who are not in employment and make even basic health care into a privilege”.

But Greenfield says the chasm between rich and poor was widening well before the reality TV star’s 2016 election. “The American dream – that everyone has equal opportunity – became a fiction long before Trump,” she says. “Americans don’t hate the rich, as they imagine they could become the rich. They don’t want high taxes as they think they could become rich and won’t want to pay them. But what they dream of is an increasingly unbelievable fantasy.”

She says that while examining her photos it became clear to her that “We have left behind the American dream of my dad’s generation where there was the possibility of social mobility and the belief that anyone could make it. The things that were valued then – discipline, hard work and frugality – are not so important now. We have a culture that prizes celebrity, bling and narcissism.” Trump, she says, “is the apotheosis of generation wealth. With Trump you have wealth and celebrity achieving the ultimate goal. Trump is the natural evolution of the values of our culture.”

When Greenfield started the photography series – which includes portraits of a 12-year-old Kim Kardashian – getting permission from parents to document their kids’ lifestyles sometimes proved difficult. “The parents wanted to be a little discreet – they indulged in the lifestyle but didn’t necessarily want to be seen doing it, but that has changed,” she says. “I went back again in 2007 and the parents’ attitudes had completely changed. A lot of times parents would be present at the interviews and they would be proud, saying: ‘Our kids are just like Paris Hilton.’”

“‘Keeping up with the Joneses’ has become ‘keeping up with the Kardashians’,” she says as we tuck into seared tuna with caponata and gambas al pil pil. “Instead of wanting the slightly better house down the street, you want the mansion that Kim has. Reality TV and social media has made the lives of the wealthy more accessible, and it has made people hungrier for it.” The children Greenfield met were sometimes more aware of the potential damage created by free flowing cash than their parents. She recalls a 13-year-old called Adam who told her: “Money ruins kids, money has ruined me”.

Greenfield went to Adam’s bar mitzvah and was “blown away” by the scale of the production, including the go-go dancer. “Kids in his circle had to be seen to spend £50,000 on their bar mitzvah or they’re shit out of luck,” she says. Hanging out with Adam affected her more than most of her subjects, because even at 13 he was perceptive enough to know that something had gone wrong. She describes herself as friends with many of her original subjects, including Adam. But he wouldn’t allow her and the cameras back for her latest film.

Neither would David Siegel, 83, the real estate mogul she shadowed in The Queen of Versailles. “He hates the way the film ended when he loses the houses,” Greenfield says. “I don’t think David will ever let me back.” But Siegel’s third wife, Jackie, 52, is still friends with Greenfield and attended the premiere of The Queen of Versailles. “She thinks it’s the best thing she ever did,” Greenfield says. Early on, Jackie, a former Mrs Florida beauty pageant winner, told Greenfield: “You can never be too rich or have big enough boobs.” Jackie’s breasts, which have been augmented four times, fill the frame of one photo.

There has been a sea change. We’re all complicit in generation wealth

At the end of the film, Greenfield says the Siegels appeared to have learnt that wealth is not as important as health and happiness. But, having recovered from the financial crisis, the Siegels bought back the house and extended their ambitious plans. The house built on a man-made hill in a gated community in Orlando will have 14 bedrooms, 32 bathrooms, a 30-car garage, bowling alley, five swimming pools (three indoor, two outdoor), a two-storey cinema and a ballroom with capacity for 500 guests. The much delayed house is set to be finally habitable next year.

“They had this realisation that it was family that mattered,” Greenfield says. “Yet they are back at the house building bigger and better. I wonder, did we really learn anything from the financial crash, or did we just go back to the way it had always been? In the US and the UK we fixed a broken system and now the real estate market is even more on fire, the stock market is up and Trump is the president and tax laws are making things even more unequal.”

Numerous official reports detail the widening gap between rich and poor. UBS, the Swiss bank which prides itself on advising more uber-wealthy families than any other bank, published research showing that the super-rich hold the greatest concentration of wealth since the time of the Carnegies, Rockefellers and Vanderbilts at the turn of the 20th century. There are now 1,542 billionaires across the world, more than ever before. The richest 500 people alive increased their wealth by 23% last year, taking their combined fortunes to $5.3tn – more than twice the gross domestic product of the UK.

Josef Stadler, the lead author of UBS’s report, says his billionaire clients are aware of the widening gulf between rich and poor and fear that hard-pressed people might rise up and take direct action. “We’re at an inflection point,” he says. “Wealth concentration is as high as in 1905, this is something billionaires are concerned about. The question is, to what extent is that sustainable and at what point will society intervene and strike back?”

He says the “$1bn question” is how society will react to the concentration of so much money in the hands of so few. Anger at so-called robber baron families who built up vast fortunes from monopolies in US rail, oil, steel and banking in the late 19th century, an era of rapid industrialisation and growing inequality in America that became known as the Gilded Age, led to President Roosevelt breaking up companies and trusts and increasing taxes on the wealthy in the early 1900s. “Will there be similarities in the way society reacts to this gilded age?” Stadler asked.

Looking back on her work, Greenfield recalls being startled by ostentatious displays of wealth but says what might have been shocking then is day-to-day reality now. In her first few days visiting Crossroads, three boys asked her what she was doing by the school gates every day. When she told them she was working on a documentary about growing up in LA, they told her it was “all about money” and pulled cash out of their pockets, posing for the camera.

“It wasn’t until I developed the film that I saw that these 13-year-olds were waving $100 bills.”