an Augmented Reality Resource

Menu

News

Earlier this year I wrote about breathtaking it was to don a PSVR headset and stare down massive colossi in Shadow of the Colossus when its remaster released on PS4 in February. Using Cinematic Mode, you can play it and any other non-VR PS4 game inside the headset on a massive screen that nearly encompasses your entire field of view. There is no head-tracking, it’s not 3D, and you can’t reach out and interact using your hands or controller at all — but it’s undoubtedly immersive.

Then when you switch over and play a brand new game like Red Dead Redemption 2 using the feature and switch to first-person view, it’s so close to the real thing it feels unfair. I found myself so engrossed in Rockstar’s meticulously detailed world that I couldn’t help but turn my head side-to-side at various moments expecting my vision to follow inside the game world. If you’re confused on how Cinematic Mode works, it’s like this. Similar to using Big Screen and opting for the void environment.

Just today we published an excerpt from an interview with Joel Breton, GM of Vive Studios, in which he discusses the success of LA Noire: The VR Case Files and he states that Rockstar, the studio behind that game, Grand Theft Auto, and Red Dead Redemption, is “not done” with VR yet.

Obviously this is far from confirmation that anything official is coming, but it got us curious. I received a copy of Red Dead Redemption 2 early (you can watch my review right here) so I fired it up inside the PSVR headset and was honestly blown away by how it felt.

The lower resolution sucks away some of the fidelity when compared to my 70-inch 4K HDR TV, but the all-encompassing immersion helps make up for it. Moments like in the GIF below feel so much more intense when you’re wearing a headset even if it isn’t actually VR:

Anyway, maybe it’ll happen one day. We hope. What do you think? Would you play a game like Red Dead Redemption 2 in VR?

Let us know down in the comments below! Red Dead Redemption 2 releases on PS4 and Xbox One tomorrow, October 26th, 2018.

With the release of the TPCast wireless adapter (for Oculus Rift and HTC Vive), and now the official HTC Vive Wireless Adapter, many have been wondering whether Oculus are working on a 1st party wireless solution for Rift, or whether they plan to incorporate wireless in the future version of Rift. These adapters are, however, very expensive, large, bulky, and the transmitter might have to be mounted on the wall to work well. This is because they transmit a high frequency (60 GHz) signal over a large field of view that generally requires line of sight to the headset.

Narrow Beam Following Positional Tracking

Late last year, Oculus filed for a patent on a technique that would have the wireless transmission system use the positional tracking data from the headset to send a relatively narrow beam to the direction of the headset, instead of all over the room. When the headset moves position, it could inform the transmitter of the new position over regular low bandwidth omnidirectional wireless (similar to Bluetooth) and then the transmitter would direct the high power beam at the new position.

The advantage of this approach is that, because the transmitter only has to send the wireless signal to one spot in the room, less power should be needed overall. This idea could be used to lower cost in future wireless VR setups. The high power transmitters used in the Vive Wireless Adapter and TPCast greatly contribute to the $300 prices, so the need to find lower cost solutions is clear.

The patent mentions that one of the possible protocols for the beam could be 802.11ad, otherwise known as “WiGig”. WiGig is an existing 60 GHz standard widely used for wireless displays such as wireless monitors, and is actually used in the HTC Vive Wireless Adapter.

Fighting Occlusion With A “Relay”

Another Oculus patent application adds a “relay” for when the HMD is occluded. Note that “console” is used in the technical sense, referring to the base transmitter connected to the PC.

But what if the view between the transmitter is disrupted? Oculus applied for another patent for using an assisting “relay” in the room. When the view between the transmitter and headset is blocked, and therefore the signal is blocked, the transmitter would instead send its beam to the relay, which would act as repeater.

Coming to Rift 2?

In his “5 year’s from now” predictions made at Oculus Connect 3 in 2016, Oculus Chief Scientist Michael Abrash said that he expected to see wireless headsets “at the high end”, but that there is “no existing consumer electronics link that’s up to the task”. This may be why Oculus began researching a custom (and patented) wireless solution.

Abrash also mentions that without foveated rendering (rendering at a low resolution everywhere except where the user’s eyes are looking), achieving wireless on PC would be very challenging. In Oculus’ foveated rendering patent, originally filed back in 2016, the company describes a display driver which can handle different resolutions for different parts of the image, noting “the devices may communicate

Maintaining full framerate in VR is so crucial that VR companies have developed special driver-level techniques to compensate when frames are dropped. On Rift, Oculus uses a technology called Asynchronous SpaceWarp (ASW), which, when framerate isn’t being met, forces the app to drop to half framerate and fills in every 2nd frame with a synthetic one extrapolated from the previous. Valve and Microsoft also have similar technologies for their respective Steam and Windows platforms.

Where this technology would be even more useful, however, is on mobile, where due to the much lower compute power it is much harder to maintain framerate. This would be especially useful for the upcoming Oculus Quest all-in-one VR system, where the focus is games, and which Oculus is trying to encourage developers to port their PC VR games to. The Oculus Mobile (Gear VR, Go, and Quest) runtime already performs rotational reprojection (called “Asynchronous TimeWarp” / ATW), but this ignores positional head movement.

At Oculus Connect 5, engineer Jian Zhang discussed the limits of rotational-only reprojection in a 6DoF environment like Quest, noting that it leads to positional tracking judder from the incorrect projection. He then spoke of an experimental fix in the works called ‘Motion Reprojection’, which would use the depth buffer submitted by the app each frame in order to more accurately reproject the view with respect to the user’s head position.

Essentially, if a frame is dropped, the Quest’s software would generate a synthetic alternative frame by skewing the image in the rotational direction (which Go and Gear VR already do), and now positional direction, that the headset moved. Having the depth information from the scene allows this new positional reprojection.

Like ASW 2.0 on Rift, ‘Depth Reprojection’ on Quest would rely on apps submitting their depth buffer each frame

This approach would, however, not be fully equivalent to ASW on PC. Whereas ASW also takes the animation and motion of objects in the scene into account and extrapolates them (by reading the color buffer), ‘Depth Reprojection’ would only be making head tracking feel smooth – objects in the scene would still have animation judder if the app was rendering at a low framerate. CTO John Carmack stated on Twitter that doing color extrapolation like ASW is “actually more work than most mobile frames to render”, so this is unlikely to come to Quest any time soon. Zhang also noted that Depth Reprojection would not deal well with transparency such as a virtual fish tank.

Oculus have not given any details on when ‘Depth Reprojection’ will be coming to Quest, but if it does, it would help the headset to handle complex scenes in games. Developers would still need to put serious effort into optimization to avoid animation judder, but at least a temporary drop in framerate will not make the user physically sick through head tracking judder like was seen in the Oculus DK2 days of PC VR.

Learn how to play piano using just your iPhone and a crisp one dollar bill. Instant Musician, previously only being available as a Microsoft HoloLens app, has now made its way to iOS, offering users a way to learn proper piano skills using their smart phones. Using a dollar bill on your piano mantle as

The leading providers of horror-based VR offer yet another selection of terrifying experiences guaranteed to get you in the holiday spirit. If you’re the kind of person who enjoys binge-watching scary movies during the Halloween season, might I recommend upgrading your seasonal routine with a new series of frightening virtual reality experiences provided by the

Startup Holospark deployed 32 Optitrack cameras to capture the body movements of performers along with helmets for facial capture. The result of the effort is a cast of digital characters that invaded my personal space and maintained continuous eye contact in ways only a very dedicated cast member would at Disneyland’s Haunted Mansion.

Reproducing the general feel of that iconic attraction is no easy feat — the floating candlesticks, echoey voices and memorable music — but Holospark achieved that goal with The Haunted Graveyard. The $10 world opened for Rift and Vive owners earlier this month and Holospark is also offering it at VR arcades.

Holospark took a gamble in launching its first VR title, The Impossible Travel Agency, in late 2016. While we found the reactivity they baked into that world to be both subtle and relaxing, the Fantasia-like experience was dropped onto an unfeeling Steam mob interested in only one thing — hours of gameplay.

The Haunted Graveyard arrives two years later and Holospark is showing two years of maturity in its VR design. The better reviews on Steam for this experience give me hope that folks buying VR software are starting to learn they need to speak up when a world is as rich and surprising as this one, even if it’s only designed to last a few minutes.

Some incredible titles like Accounting+ and Virtual Virtual Reality are hardly games at all, yet they provide deep entertainment packing layer upon layer into a world waiting to be discovered by a mind that is inquisitive, playful and cautious. That last characteristic is the one The Haunted Graveyard manipulates so well, luring the visitor along a spooky path with interaction points along the way carefully staged for maximum…intimidation.

I’m reluctant to spoil more about The Haunted Graveyard (there’s a highlights video below which does that) except to say it is readily apparent the care and craft Holospark took to breathe life into three characters you’ll encounter in the graveyard — building on work Holospark did last year with its striking Seance demo. Each of these characters in the graveyard left an indelible mark on not just the person wearing the headset, but also family simply watching the world mirrored on a 2D screen.

One note I will add is that traversing through The Haunted Graveyard is accomplished with an arm-swinger mechanic. You swing your arms in the real world to “walk” in the virtual one. There’s also a field of view restriction feature included by default for added comfort. I can understand why Holospark went with arm-swinger — it helps you remember the actual trip through the graveyard as opposed to, say, a teleporter system that would destroy a sense of continuity to snap between locations. If you’re scared in The Haunted Graveyard there’s a desire to go faster with this mechanism, but there are also forced stops in the narrative where a person might try to keep moving their arms to no avail. It might create some discomfort for a small percentage of folks,

Speaking with UploadVR, Joel Breton, GM of Vive Studios, talked about how early work from developers like Bethesda, Ubisoft and Rockstar had paved the way to bring more companies into the VR ecosystem. “So now everybody is out there building VR, unfortunately, it’s just not public yet,” he said. “So all of the third parties are working on VR now.”

Turning his attention to Rockstar specifically, Breton praised the work the company did on last year’s L.A. Noire: The VR Case Files. “That one, that was lovingly handcrafted,” he said. “They went through everything, every art asset, every animation. Any part of the game that didn’t work, they cut from the VR experience so things that ended up in those seven cases that are in the VR version, those were very very selectively chosen. They figured that would work.”

HTC worked with Rockstar to launch L.A. Noire first on HTC Vive before it came to Oculus Rift last year. Indeed, we were big fans of the experience and how it tailored many of its mechanics to fit headsets.

“So, you know, they’re not done either,” Breton said of Rockstar’s work in VR. “So we’re very very supportive of these studios both on the hardware side and the software side and the optimization side.”

This week is obviously a big one for Rockstar with the long-anticipated launch of Red Dead Redemption 2 (which, we might speculatively add, features a first-person view mode), but we’re obviously excited by the idea of another of the developer’s games being ported to headsets, if not something entirely new. Could the studio possibly be working on VR adaptations of either Grand Theft Auto or Red Dead that cherry-pick the best content as with L.A. Noire? Or is that still too much to hope for?