On Thursday at the 2,800-seater Hammerstein venue in New York, Sony is expected to kickstart the next generation of video game consoles. The company has not officially confirmed anything, but pundits are certain this is where the world will get its first glimpse of the long-awaited PlayStation 4.

Meanwhile, Microsoft is working on the sequel to its Xbox 360 console, expected to be unveiled at this summer's E3 exhibition in Los Angeles.

New games platforms are usually welcomed with excitement by tech fans, but this time things are a little different. Since PlayStation 2 dominated the world of gaming in the early 2000s, the industry has been transformed by the arrival of smartphones and tablets.

These devices offer cheap, convenient gaming experiences to a new demographic of players who do not necessarily want to pay £60 for a game and then sit in front of their TVs for hours to play it.

"Why would I want another noisy black box sitting under my TV when the phone or tablet I have can do almost everything such a console can?" says the games analyst Oscar Clark.

"In a couple of years that console will be out of date compared with my other devices. At the same time, my relationship with my TV is changing. It is no longer the first screen, it's just the shared screen where I consume content with the other members of my household. My primary screen is my tablet or mobile, so where does the console hardware fit?"

Retail sales of games are falling around the world. In the US, according to the market analysts NPD Group, sales of physical games fell 22% in 2012, to $13.26bn; in the UK the market was down 17% to £1.59bn. Demand for hardware is also falling: console sales were down 37% in 2012, NPD said, and newcomers such as Nintendo's Wii U and the handheld PlayStation Vita have so far fared poorly.

Of course, decline is natural at the end of a console cycle, but this time the market lethargy could cost Sony, Nintendo and Microsoft dearly because there are new challengers on the horizon. The US games developer Valve plans to launch its own console, Steam Box, a cheap, tiny PC that plugs into the TV and offers hundreds of downloadable games. This year will also see the launch of at least two new sub-$100 consoles based on Google's Android technology. Ouya and GameStick are both open platforms, which means anyone can make games for them without authorisation from the manufacturer, an appealing prospect to developers.

Hovering in the background are Apple and Google, both of which have online app stores and TV services. Apple is rumoured to be working on a new version of its Apple TV set-top box, or even a dedicated Apple television set, that will give access to thousands of cheap games via the App Store, without the need for a dedicated console.

But Sony and Microsoft may have a few new weapons to bring to the battle. Both are creating cross-platform ecosystems: PS4 is likely to integrate with Sony's Xperia mobile phones and the Vita handheld console; the next Xbox will doubtless be compatible with Windows 8 PCs and Windows mobiles.

Last year Sony spent $380m on the California-based company Gaikai, which has developed a cloud-based technology that lets users play as games are streamed over the internet. Microsoft has hinted that the new Xbox will support cloud-based gaming.

It is likely, then, that both next-gen consoles will offer something akin to Spotify for games, allowing players to stream or download new titles in exchange for a monthly subscription.

In the US, Microsoft has toyed with a smartphone-style subsidised price model where buyers get the Xbox 360 for a lower price if they sign up for a two-year online gaming subscription. This could well be used to keep the cost of the next-gen machines down.

All this is not great news for retailers who depend on selling boxed games, but it fits in with new consumer purchasing habits. "I am convinced that smart, scalable cloud technology is going to be critically important," says Clark. "But we mustn't forget that it depends on the reliability of the connection, bandwidth and latency. The challenge [is] how much to rely on the device and how much on the server."

It is possible that a new device will come in to disrupt the market – and that few will see it coming. Games were an afterthought on the iPhone and on Facebook, but these platforms spawned huge new markets with hits such as Angry Birds and Farmville. This could also be the point at which the industry breaks away from its reliance on genres such as sports sims and military shooters.

"The living room is wide open right now," says the games designer and author Will Luton. "Some device will come in with the purpose of doing something else – likely movies, TV or music – but will be games-capable.

"People won't have to go to specialist shops for games about musclebound men smashing something's skull in. There will be new content, it will be for everybody and, unlike mobile or social games, it will be long-form and narrative-driven.

"This will be the biggest change games will ever see. Finally we'll be able to compete creatively and as widely as every other media art form."