Art is a luxury, the ultimate luxury. Imagine the glory of having an original work of art by a great artist on your wall. It beats the best car, the best helicopter. Art is money and if you want people to know your wealth, you must buy art.

Sorry if this imaginary blurb for the art market seems offensive, but that is kind of the idea. The market in modern art is truly offensive. It is becoming more sickening by the day. This week saw businesses go bust and an entire nation on the edge of the economic abyss. In Britain, famous high street names such as Thorntons and Habitat hit the buffers. In Greece, riot police held back protesters as punitive austerity measures were imposed by parliament.

Meanwhile, a sale of modern art at Sotheby's on Wednesday night made £108.8m, a London record according to the auctioneers. A Bacon went for £8.3m, a Warhol portrait of Deborah Harry for £3.7m. Spectacular sums were also paid for works by German contemporary artists, while a Damien Hirst spot painting topped a million quid, suggesting he is still attractive to the people he needs to be attractive to.

But who are they, these people? I would genuinely like to know. The popular assumption seems to be that today's art collectors are "Russian oligarchs". Certainly the spectacle of Roman Abramovich's yacht drew attention to the oligarchic presence at this year's Venice Biennale. One thing is certain – the big-time buyers of art are people in the financial sector who are weathering our troubled times a lot better than high street businesses, nations picked on by Standard & Poor's, or public sector workers.

And yet, for the last couple of decades, contemporary art has flourished through an alliance of the rich and the not-so-rich. It is the same educated, probably public-sector-employed middle class (many of whom marched this week) that enthusiastically visit galleries and art fairs. It is these fans of modern art who have helped, by their acclaim, to generate the charisma that makes it apparently worth so many millions.

In the 1990s, a credit-fuelled sense of affluence made the excesses of the art market seem fine, even entertaining. Besides, contemporary art has a dual nature. On the one hand it is – like all fine art down the ages – a plaything of the rich. But that is not the whole story. It is also a public art. Spectacular installations, accessible videos such as The Clock, and free display spaces like the Tate Turbine Hall, make the art of today a common property, capable of communicating in exciting ways across nations and generations. It has a utopian aspect.

So spare us the conservative attacks on modern art. I do not think the prices paid for Warhol or Bacon reflect on the artists themselves – as it happens, a lot of good art changed hands at the Sotheby's sale. And for all the fuss over the Abramovich yacht, the reality is that people from all walks of life are visiting the Venice Biennale this summer and finding it, as I did, a stimulating overview of the best new art on the planet.

But how long can this go on? How will the growing, grotesque disparity between our belief that we "own" modern art and the glaring reality that it is bought and sold by the super-rich, survive these times? In 2009, Athens was being touted as a rising contemporary art centre, with collectors, fairs, new galleries. Art is fully globalised, and seems to be operating as a separate world system while all around it crashes. I am not prophesying disaster for it. If people go on believing in it, art may even be a clue to the survival and recovery of world capitalism. On a more local level, if British people keep on loving new art even as the rich carry it home, it probably also means the coalition is destined to a decade or so of power and the left is toast. Or if the times here and elsewhere prove harder to stabilise, if the rocks in the road get bigger – well, the art system will probably still go on. But will we be looking?