A normal user's look into the world of 3D Stereo Technologies

Today Nvidia has officially released the 1.0 version of two powerful VR software development kits (SDKs) – the Nvidia GameWorks VR and Nvidia DesignWorks VR that are targeted at headset, game, and application VR developers in order for this relatively new category of display devices to offer better performance and user experience. Delivering good VR games and experiences is a complex challenge, especially since immersive VR can require multiple times the graphics processing power compared to traditional 3D apps and games you not only need a good GPU in terms of performance, but also one that is optimized for VR. With these SDKs developers on Nvidia hardware should now have the tools to create amazing VR experiences, increase performance, reduce latency, improve hardware compatibility and accelerate 360-degree video broadcasts. Both SDKs deliver a comprehensive set of APIs and libraries for headset and app developers, including the new Multi-Res Shading Technology. Available publicly for the first time, Multi-Res Shading is an innovative rendering technique that increases performance by as much as 50 percent while maintaining image quality. Also the 1.0 SDK releases also add support for the new Windows 10 operating system.

GameWorks VR
For game and application developers, the GameWorks VR SDK includes:
– Multi-Res Shading — an innovative rendering technique for VR in which each part of an image is rendered at a resolution that best matches the pixel density of the warped image required by the headset. It uses the NVIDIA Maxwell chip architecture’s multi-projection capability to render multiple-scaled viewports in a single pass, delivering substantial performance improvements.
– VR SLI — provides increased performance for VR applications where multiple GPUs can be assigned a specific eye to dramatically accelerate stereo rendering.
– Context Priority — provides control over GPU scheduling to support advanced VR features such as asynchronous time warp, which cuts latency and quickly adjusts images as gamers move their heads, without the need to re-render a new frame.
– Direct Mode — treats VR headsets as head-mounted displays accessible only to VR applications, rather than a typical Windows monitor, providing better plug and play support and compatibility for VR headsets.
– Front Buffer Rendering — enables the GPU to render directly to the front buffer to reduce latency.

DesignWorks VR
For developers of professional VR applications in markets such as manufacturing, media and entertainment, oil and gas, and medical imaging, NVIDIA DesignWorks VR builds on the core GameWorks VR SDK with the addition of powerful tools, such as:
– Warp and Blend — new APIs that provide application-independent geometry corrections and intensity adjustments across entire desktops to create seamless VR CAVE environments, without introducing any latency.
– Synchronization — techniques to prevent tearing and image misalignment while creating one large desktop that is driven from multiple GPUs or clusters. Various technologies like Frame Lock, Stereo Lock, Swap Groups and Swap Barriers are available to help developers design seamless and expansive VR CAVE and cluster environments.
– GPU Affinity — provides dramatic performance improvements by managing the placement of graphics and rendering workloads across multiple GPUs.
– Direct for Video — enabling VR and augmented reality environments such as head-mounted displays, CAVES/immersive displays and cluster solutions.

AMD has also been more active on VR support lately with the recent announcement of their AMD LiquidVR Technology for Developers. One of the key technology goals of LiquidVR is to reduce unwanted processing latency (reduce motion-to-photon latency) and deliver a consistent frame rate. AMD recently released the Alpha version of its LiquidVR SDK to select technology partners. The LiquidVR SDK is a platform designed to simplify and optimize VR development.

Now the big question that remains is how soon users are going to have their hands on the new VR headset hardware such as the consumer version of the Oculus Rift that should be released sometime in the Q1 2016 or the alternatives such as HTC VIVE and others that might be coming with their own hardware. The developer hardware that has been available with most notable wider availability of the two generations of dev kits of the Oculus Rift has sparked the interest and demand for VR headset in many users that simply cannot wait to get their hands on the hardware and experience the promised great VR experiences as well as play great games in a new more realistic way.

Today Nvidia has launched their new Maxwell high-end GPUs – GTX 980 and GTX 970 and while they offer a nice performance boost over the previous generations at a reduced power usage level making them a really attractive upgrades, thees products came with an interesting VR-related announcement as well. Apparently Nvidia is already working together with companies developing Virtual Reality products such as Oculus Rift in order to provide the users with a better experience. While we may need some more time before VR becomes more mainstream and reaches a really great level of experience it seems that things are really moving at a good pace already. It is also interesting to note that Oculus Rift and other VR headsets may actually turn out to be the saviors of the 3D Vision technology as well, but we’ll have to see about that.

With VR Ditect Nvidia is trying to address multiple things related to the VR experience such as to lower the latency, improve quality and provide more content that will work well on your Oculus Rift or other VR headset without having to be specially made for them. With a VR headset the latency is much more important than when using a traditional display as any perceived delays can throw off the VR experience, potentially causing the user to get motion sickness.

The standard VR pipeline from when you move your head to when you actually see the response on your VR display is about 50 milliseconds. Nvidia’s goal is to reduce this latency as much as possible so gamers feel even more immersed. A large portion of this is the time it takes the GPU to render the scene as well as OS overhead (about 32 ms) and as a result of this effort, they have managed to cut 10ms of latency out of the standard VR pipeline. Using a new MFAA Anti-Aliasing filtering method rather than MSAA for better image quality, they have managed to reduce GPU render time by an additional 4ms, and even further with another technique they are also working on caled asynchronous warp. Rather than requiring the GPU to re-render each frame from scratch, with asynchronous warp the GPU takes the last scene rendered and updates it based on the latest head position info taken from the VR sensor; head tracking input is literally sampled moments before you see it. By warping the rendered image late in the pipeline to more closely match head position, discontinuities between head movement and action on screen are minimized, dramatically reducing latency even further. So they have managed to actually reduce latency from 50ms down to 25ms and that should dramatically improve the VR experience for the user.

Besides latency, another major obstacle that must be overcome to provide an immersive VR experience is performance; not only is a high frame rate needed, it’s also important that the frames are delivered to the user’s eye in a smooth fashion. While one GPU can be used to drive an Oculus Rift, enthusiasts will want two GPUs to ensure the best performance with maximum game settings enabled. That would be especially true if we go from a screen resolution higher than the currently used Full HD on the Oculus Rift DK2 or we have a separate 1080p screen for each eye. Traditionally Nidia SLI relies on alternate frame rendering (AFR) where each GPU renders alternating frames that are presented to the user. For VR scenarios however, Nvidia is implementing a new VR SLI profile where each GPU will render per display; the display responsible for the left eye will be handled by one GPU, while the second GPU will be responsible for the display on the right eye. This solution should provide lower latency and ultimately better performance for the user.

Display resolution is another critical feature for VR. With the displays in a VR headset resting extremely close to the user’s eyes, higher resolution can remarkably improve the VR experience. Dynamic Super Resolution (DSR) can be used to provide an improved resolution quality with today’s existing VR displays by rendering the game images at a higher resolution and then scaling them down to the native VR display resolution preserving more details and smoother image.

Another challenge VR must overcome to be more widely adopted is lack of content. Outside of a handful of tech demos, there aren’t that many applications that support VR headsets. To solve this issue Nvidia is leveraging their extensive experience with the 3D Vision technology to bring VR support to existing games that are already compatible with 3D Vision, and adapt them to work with VR. They plan to use GeForce Experience to optimize game settings and handle configuration automatically, and aside from converting the games to the output format required by the VR headset and providing stereoscopic 3D rendering they will also map mouse and keyboard commands to VR inputs like head movement, similar to what was done with the Gamepad Mapper on the Nvidia Shield Portable.

All of the above sounds very interesting and promising, but what is still missing is information on when all of these features under the VR Direct technology will be actually available to users to try them out – for example developers that already have their hands on the Oculus Rift DK2.

It has been a little over 5 years since the start of this blog, even though I has been a stereoscopic 3D enthusiasts for much longer, the reason for the existence of the blog was Nvidia’s 3D Vision. I was really excited when they have announced it and when the first 3D Vision-ready monitors started appearing on the market I was eager to get one as I really did see the potential that the technology had for gaming. And back then since it was something really new and exciting I decided to share my experience, knowledge and other useful information with other users that were just starting and this is how this blog was born.

Now, five years later we are seeing a deja vu as the situation with 3D Vision is pretty much the same as with the Nvidia Stereoscopic 3D Drivers that were available before the birth of 3D Vision. Nvidia did create a market and strong demand for products at first, however they slowly start to neglect things and in a few years they kill the market. This is exactly what is happening with the 3D Vision now as Nvidia hasn’t done almost anything new in many months. The latest bigger things were the announcement of the G-Sync monitors, some with 3D Vision support, that however were greatly delayed to market. The other more recent thing is the addition of “3D Compatibility Mode”, but the more important thing – supporting and pushing game developers to integrate Stereo 3D support in games has been long gone. It is not about fixing games after they have been released, but integrating proper stereo 3D support while they are being developed.

The stereo 3D gaming community can do the fixing with the help of tools like Helixmod wrapper, however even the game modders are giving up on 3D Vision as instead of helping them Nvidia seems to be doing just the opposite. This is really discouraging and if Nvidia does plan to abandon 3D Vision like they did with their old Stereoscopic 3D Drivers, then they should at least open the project and not kill it completely. The people in different stereoscopic 3D gaming communities that are using 3D Vision have asked numerous times that Nvidia works on something or improves another thing in order for 3D Vision to become better and easier to be used, but we have not seen any results. This is really discouraging and many people have just decided that it is enough for them, some of you might of thing this for me as well due to the last few months with no updates here…

I have not given up on stereo 3D and 3D Vision, at least not yet. I was just busy with other things and have decided to get some rest from 3D, not that very much was happening with 3D Vision anyway, and the Facebook and Oculus deal was a bit of surprise and I wanted to see what will come out of it. So far it seems that things are still good and I’m about to get my hands on the pre-order of the DK2 hopefully next week, so all is not lost. However Oculus Rift is not something that can replace 3D Vision, these are two different things, and even though some of their features and functionality do overlap they are still not the same thing. VR with 3D support is an interesting new concept, but for really good results you need content developed especially for the device, meaning that you not only need a good hardware, but good software support as well. At least here Oculus is doing much, much, much better than Nvidia as Oculus Rift is their only product, unlike Nvidia that has many products and technologies and cannot focus solely on 3D Vision. So interesting times could still lay ahead of stereo 3D technology, so we’ll have to wait and see..