Behind the storm-in-a-wineglass feuds that surround the Man Booker Prize, a true and even tragic sub-plot may be starting to unfold. To be mass-market blunt rather than literary-novel elliptical: is the British audience for ambitious fiction dying off, losing faith, or just drifting away? As usual, I feel a yearly spurt of outrage, bewilderment or gratification – this time, respectively, at the exclusion of James Kelman from the long-list, of Michelle de Kretser from the shortlist, and at the judges' recognition of the far-from-shouty merits of Linda Grant.

It does seem, however, as if a dwindling band of domestic readers shares this annual passion. In the five weeks after the long-list announcement on 29 July, the 13 titles of the "Booker dozen" sold fewer than 14,000 UK copies; on average, barely 1,000 each. This is, frankly, pathetic. Writers and retailers will pray the shortlist delivers a bigger boost. Before the early-1980s Booker battles caught the public imagination, the prize tiptoed politely from year to year as a small-time coterie event. Now, a quarter-century of starlight has begun to fade.

Both marketplace and media offer ever-shrinking space for "literary fiction". The prize ceremony, which once secured its own BBC programme, now has to make do with a scrappy insert in the news. The Orange and Costa contests loom far larger as heavyweight rivals. Rather sadly, award director Ion Trewin has been reduced to branding the Booker as "Richard and Judy for grown-ups". Adland lore: insult the competition, and you look rattled.

More disturbingly, a run of eccentric choices since the truly bizarre DBC Pierre victory in 2003 may have damaged the prestige of the prize among its natural constituency. This sounds like a harsh verdict, given that the judges pride themselves on opting for the book that they adore rather than bowing to record or reputation. And both Kiran Desai and Anne Enright – the most recent victors – prevailed with spirited, invigorating works. All the same, one may ask if an alternative list of laureates since 2003 – selected from the actual shortlists – would have left the prize in better shape to flourish in tough times. It might, after all, have read: Monica Ali, Brick Lane (2003); David Mitchell, Cloud Atlas (2004); Zadie Smith, On Beauty (2005); Sarah Waters, The Night Watch (2006); Ian McEwan, On Chesil Beach (2007). Wrong for those years? Possibly. Right for an embattled prize? Certainly.

Meanwhile, in another part of the literary jungle, Little, Brown's imprint Sphere has promised to turn Sharon Osbourne into a bestselling fiction "brand" who will publish ("write" is not le mot juste) one glittery blockbuster per year. The reality-show diva is a friend of Jackie Collins, whose sequinned schlock ruled the charts through the penny-pinching years of the Seventies and early Eighties. As recession bites, the in-store reality that Booker-level fiction will confront may well mean – fantasy. I hope I haven't seen the future, but if I have: it sparkles.