Microsoft want to use your GPU’s AI skills to make your games prettier… and more deadly

In one of the first – of likely many – big announcements coming from the Game Developer’s Conference in San Fran this week, Microsoft are adding further utility to their AI hardware acceleration API, WinML. The goal is to create more intelligent, more beautiful, and even more difficult videogames.

Microsoft’s WinML API has already been implemented in various applications across the Windows platform, from security all the way to video creation. But now, Microsoft – along with support from AI big cheese, Nvidia – are planning to put their machine-learning centric API to good use in gaming.

Alongside the joint efforts of these two companies to implement real-time raytracing in games, Microsoft are also hoping to make the most of Nvidia’s Volta Tensor Cores for what they were born to do: infer. Not that machine learning is Tensor exclusive, of course, and any DX12 GPU can try their hand at a little inference, too.

Microsoft’s WinML API allows neural networks to carry out inference tasks straight on a PC’s hardware, and makes it even easier for developer’s to implement various deep neural networks (DNN) models within their games and have them function correctly on users’ machines.

Utilising a DNN, Microsoft hope to engage players on a personal level never before seen. This involves creating a game environment, difficulty, or appearance that is tailor-made for the player. It can do this through varying models of neural networks that change and adapt in real-time to the stimuli received from the player. Whether that’s the players play style, personal preferences, or skill level… basically, whatever a dev can tap into with the tools at hand.

I’m sure most of us have experienced at least a passing thought of what AI could hold for gamers. Take for example a Dark Souls game that actively moves enemies depending on the path most taken in your previous runs? Warranty returns on smashed keyboards would be through the roof, and I can only dream of the speed run upset potential.

It’s not all behind the scenes for AI, either. Nvidia have already showed that machine learning-powered supersampling can be used to remove jaggies and pixelation in an upscaled image. The machine learning algorithm picks the best colour required and applies what it infers into a much more visually-clear image than one created via traditional sampling techniques. Think of the same process being used to upsample game textures the closer you get to them.

The potential implementations for machine learning aren’t just with gamers, either. Game development also lends itself to inference-based applications with potential uses developing facial animation, texture creation, character design, and much more on the cards (literally). Unity are already rolling out their ML-Agents framework, showing that it can be baked into a game engine too. Game design comes as just another form of entertainment under the belt for AI, what’s next, our jobs? Yeah, probably.

Assumedly, the Tensor Cores will offset the performance hit of ML supersampling or AI gameplay, when present. However, despite the machine learning hardware acceleration tech (DirectML) being built upon Direct3D – and with aspects of DirectX – it could still be incredibly demanding for DX12 GPUs without these cores – ya know, almost every other GPU under the sun.

With only Nvidia’s Titan V and super high-end Tesla cards currently featuring their Tensor Cores – 640 of them, in fact – this may offer some indication to dedicated AI circuitry making its way into even Nvidia’s gamer-focused GeForce cards to deal with the impact of these features, and with real-time raytracing, in the near-future.

As for AMD, their GPUs have proven quite capable for compute, as made evidently clear by the hordes of them gobbled up by miners. But, we’ve also seen how well tech championed by Nvidia has gone in the past for team red… it wasn’t pretty.

Ignoring the mass cultural, economic, social, technological, and existential upsets lying in wait with AI, more importantly, it could be be massively important to the future of our videogames – from a games production standpoint all the way down the ladder to the nuances between how we, as individuals, play them.