OnQ Blog

Snapdragon 845: Immersing you in a brave new world of XR [video]

Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries.

Imagine what it’ll be like when you can blur the lines of your physical and digital worlds, allowing you to experience unrestrained virtual reality (VR) and enhance what you see around you with augmented reality (AR).

Recognizing this, we’ve launched over 20 XR devices between standalone HMDs and XR-capable smartphones with our customers, and many more are currently under development. We’re working closely with some of the world’s most influential XR companies like Google, HTC Vive, and Oculus and are supporting Facebook, Samsung, ODG and many others. In addition, we have our HMD Acceleration Program (HAP) that supports customers to quickly commercialize using a reference design, qualified ODMs, and technology collaborators.

Adreno 630 also features tremendous power efficiency, delivering 30 percent faster graphics performance, 30 percent better power efficiency, and more than twice as much display throughput compared to our previous generation.

Room-scale 6DoF with SLAM

When you’re experiencing VR, you must be able to create and update a map of your physical environment in real-time, while simultaneously keep track of your location within it.

Utilizing room-scale 6DoF with SLAM on the Snapdragon 845, you can see and avoid walls and obstacles in confined areas through advanced algorithms. You can also understand the room’s size, scan for objects in the room to avoid them, and integrate real-world objects into your virtual world. If you’re a “Trekkie”, you’ll notice some similarities to the Enterprise’s holodeck.

Delivering up to 4 million pixels per eye, SLAM is managed across several heterogeneous engines including the Qualcomm Spectra 280 ISP, Qualcomm Hexagon 685 DSP, and Qualcomm Kryo 385 CPU on the Snapdragon 845.

SLAM is also an essential technology for AR to accurately position virtual objects in tracked scenes, helping to ensure correct visualization and positioning in your real-world environment.

Accelerating hand-tracking and voice commands with the tech ecosystem

We collaborate with companies like Leap Motion to achieve the best hand tracking accuracy at the lowest latency possible on Snapdragon 845. We recently demonstrated that with Think F.A.S.T., a cutting-edge Virtual Reality (VR) application, which showed how VR technology can be utilized to educate people about stroke and ultimately, to help save lives. The app features advanced hand tracking that lets you see your hands as you operate an intuitive interface display and make selections as you progress through your training module.

Snapdragon 845 also simplifies your XR experience via voice commands by listening to your command key words. During Think F.A.S.T. training, your voice inputs allow you and your patient to communicate. Think F.A.S.T. is a great example how education and enterprise training will be taken to the next level with VR using Snapdragon 845.

Adreno Foveation employs a unique kind of graphics rendering combined with eye tracking to understand where your eyes are looking and then drives graphics resources to enhance that area for sharpest visuals. For example, it’ll render objects with progressively less detail outside your fovea region, and then it’ll render objects within your fovea’s field of view with much greater detail.

In the above video, Adreno Foveation reduces the GPU’s workload by prioritizing image resolution based upon the user’s fixation point.

Multi-view rendering - Rather than each eye buffer being rendered independently which is currently the conventional rendering standard for today’s VR HMD, objects are first rendered to the left eye buffer, and then automatically duplicated to the right buffer, with appropriate modifications for position and view-dependent variables like reflections.

Tile-based foveation – We’ve developed this with our software collaborators toconceptually break up your scene into tiles for processing. The level of detail will be greatest in the tiles where you’re directly looking and the surrounding tiles will be less clear, with the tiles at the edge of the scene receiving the least amount of detail.

Fine grain preemption – This is a means of efficiently interrupting a process that’s currently executing to initiate a higher priority process with minimal delay. It also efficiently resumes with the previous process once the interrupting process is completed. This provides improved results, particularly with real-time, latency-critical XR applications. ​

New directions in augmented reality

Voice UI will be important for the next generation of XR experiences, especially for AR. Upcoming AR glasses can have an always-listening component with the ability to recognize key phrases like, “Hey Snapdragon” to either wake up the glasses or invoke a specific action. Future AR glasses may also use Snapdragon computer vision and facial recognition technologies along with a trained, on-device neural network, to identify people in the room. With that contextual information, apps for AR glasses could either recommend actions, or automatically take actions, like displaying useful information about the people in the room.

AR could also change the way you shop and make mobile payments. During last month’s Snapdragon Tech Summit, we worked with ODG to demonstrate the use of glasses in a virtual shopping environment. The demo illustrated how you could add items to your cart by simply looking at them and tapping your glasses. When it was time to pay, the glasses performed iris authentication to scan your eyes, confirm your identity, and release your funds for payment.

Exploring 3D audio

Snapdragon 845 also supports 3D audio to further immerse you in VR environments, allowing you to hear sounds like you’d hear them in real life. For example, 3D audio reflects the physical composition of your environment. If you’re standing in a virtual cave, you’ll hear the reverb and reflections of your voice just as you would in a real-world cave.

Adreno 630 represents a truly dynamic visual processing subsystem and we’re very excited about the new experiences Snapdragon 845 can deliver to you in 2018.

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries ("Qualcomm"). Qualcomm products mentioned within this post are offered by Qualcomm Technologies, Inc. and/or its subsidiaries. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.

References to "Qualcomm"; may mean Qualcomm Incorporated, or subsidiaries or business units within the Qualcomm corporate structure, as applicable. Materials that are as of a specific date, including but not limited to press releases, presentations, blog posts and webcasts, may have been superseded by subsequent events or disclosures. Qualcomm Incorporated includes Qualcomm's licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm's engineering, research and development functions, and substantially all of its products and services businesses. Qualcomm products referenced on this page are products of Qualcomm Technologies, Inc. and/or its subsidiaries.