Thursday, November 3, 2016

Dev update #24

Regrouping and reorganizing

You may have noticed that our development slowed down lately so we want to explain why it happened.

VRidge userbase grew a lot during this year. When we first shown our prototype proof of concept in March we expected some people to be interested in beta testing but we didn't expect thousands.
We always love chatting about development, receiving feedback, solving user problems. Our devs were always directly responding to e-mails. This includes feedback, suggestions, bug reports, pc upgrade advice, general VR chat. It was taking more and more of our devtime.

Now VRidge's userbase is approaching 100 000 and the old ways that used to work had to be changed. Even if we hear back only from a small subset, let's say a 3% of our userbase - that's 3 thousand people.

That's why our devs will no longer be the frontline of our support system because it simply takes too much time discussing every single case individually. Our support system will of course continue to work but we'll try to write a general FAQ items on each type of issue so people can fix most of the problems on their own. We'll have a dedicated person handling support tickets that will be communicating with our devs on a regular basis.

Dev note: We love solving problems directly but this way we'll be able to fix underlying problems faster instead of developing duct-tape workarounds for each invididual case. :) We'll also continue to hang around reddit. We didn't post too much lately but we read every single discussion.

If you contacted us lately and haven't received a response for a few days, don't worry - we'll try to cach up with everyone soon.

We're also reassigning our dev time. Until now we focused 100% on fixing problems instead of adding new features but there's a good percentage of our users running VRidge without any major problems. That's why we'll have 1 developer assigned to fixing problems and optimizing experience (including "the jitter", read more in the last paragraph of this post) while the other developer will work on adding new features.

Sound streaming

If you have decent ac router and non-crowded 5 GHz band you can usually stream wirelessly with quality, lag and reliability comparable to USB cable. Many people asked us for sound streaming for fully untethered experience. Being able to sit down on your couch without a bunch of wires limiting your movement sounds very nice. You will be able to turn it on/off because there's no need to stream another data channel and eat up extra bandwidth if you want to use PC headphones anyway. We want to deliver this feature in December.

PS Move / controller black screen

We tried to fix it on our side. We exchanged our data and test results with PSMoveService contributors but it seems that the problem is not linked to VRidge itself but to the way SteamVR handles multiple drivers. It also happened with Oculus DK2 with PSMoveService (without VRidge) or with Leap Motion + VRidge (without PSMS). PSMoveService on its own works fine. VRidge does work on its own as well. But it seems that when two drivers are interacting with SteamVR together they may randomly "collide" when they try to send messages at the same time. It's a speculation because it's hard to know what exactly is happening because SteamVR runtime isn't open source. You can read full discussion here. We'll definitely let you know if we have more news on this topic.

Jitter

It's a bit long so there's a short version at the bottom. Scroll down if you want to skip this wall of text.

We tested, we measured, we analyzed rendering with high FPS cameras, we spent many, many hours trying to exclude all possible factors of this problem with various phones and display types. We know that in fact 60 (58 actually but this doesn't change anything) frames are reaching the phone (assuming good connection conditions) but something's not quite right - it feels jittery.

Nearly all (90%) of jitter reports come from S6 & S7 (Exynos chipsets) but there were few Snapdragons reports too. It's hard to tell if those Snapdragons experience the same underlying problem as Exynos devices. Samsung devices also use different type of screen than other mobile phones (AMOLED).

We knew that it's visible the most in Gear VR version but it also happens in Cardboard version on S6 & S7 too. After extensive testing we found that other phones also experience not exactly silky-smooth rendering during head movements but it's not quite as noticeable. It may manifest itself in bluriness instead of jitter.

Human perception is a tricky subject and some people don't see a difference between 30/60/144 FPS. People can play games with lowend GPUs at 20 FPS while some others consider the experience ruined if it drops below 60 FPS. The worst thing about it is that it's hard to objectively judge how smooth our small changes affect the experience. We do some changes, we test them again and compare, we show it to different people but when we try to describe how smooth it feels on a scale of 1-10 we get very random results. We literally have to stare at the distance through the window for a while to "reset" our eyes to see differences between our test versions. That's why it takes so long to achieve the perfect streaming.

We tried to use numeric metrics like FPS or frame latency but even we use high FPS camera to record the screen it's hard to catch everything because human eyes don't see the same way camera does. We don't have a brain refresh rate or eye capture rate. We see motion and we know when it's fluid and when it's not. We can get used to non-perfect rates but some some people may get a headache after a while and that's not something we think is acceptable. We know that even regular VR HMDs can trigger headaches and motion sickness in some people so the problem is not 100% solvable but VRidge has a lot of room for improvement.

Since all of our tools and metrics failed to give our definitive, measurable answer that can be used to compare different versions of VRidge we decided that we need more metrics, tools, numbers, graphs, timings, all kinds of data. We wanted to fix it quickly, to find the the wrong line of code, the mistimed part of the pipeline but it seems that there's no magical, easy way of "fixing" it.

With one of our developers assigned to new features and one free to dig deep into the VRidge internals, we think this approach will be the better way of handling the jitter problem. It takes more time but this we can do it the right way instead of the quick way, which doesn't always work. Will it work? We very much hope so.

You can think of VRidge as a street between two points - your phone and the PC. The problem is that there's a bunch of intersections - sensor reading & fusion, sending sensor data, SteamVR/Oculus driver interaction, game capture, encoding, sending, transmission error concealment, receiving, reconstructing, decoding, rendering. A small traffic jam at any of those intersection can stall the whole flow. Cities have cameras to monitor traffic flow and to optimize urban planning. We need similar tools monitoring flow at each part of the pipeline.

We found one thing that can possibly reduce jitter a little bit in non-GearVR Android version. This will be probably pushed to beta Android channel next week. It's the first step of fixing the problem but definitely not the last one.

TL;DR; We didn't find a quick solution for old problem so we took the long way to diagnose them more thoroughly. Some of you are quite happy with current version. so we're splitting devtime 50:50 with 1 dev assigned to new features and 1 to optimize and fix old problems.

32 comments:

Great work guys, for me vridge is the best tool for phone vr experience.Is possible include HDMI option? I mean, there are many vr android hmd in the market, I have two with 1080p and 2k resolution vr hmd , these includes android and niburu, I can run vridge very good , better frame rate than phone, but these vr hmd includes hdmi port, I love the possibility to add hdmi option on vridge for these hmd. Thank you all vridge team

Let's concentrate on the ONE FEATURE that Vridge needs desperately to compete: The ability to play PC games (ala Trinus). Do this and you will blow the other competitors out of the water. Ignore this and you will founder.

Dolphin VR works well with SteamVR. Download the proper version. Don't fall into the trap of coding non VR games. Every game is different and their none VR UI is a mess. The riftcat crew would only spend their limited resources on broken experiences. By focusing the effort on VR games Vridge will only get better for future AAA VR games.

At the moment I've setup a launch path for dolphin vr in the nvidia driver app (whatever it's called). Then I just simply stream the program through moonlight and play from there. That simple and it works perfectly. Completely cable free with sound and controller support from anywhere in the house.

Good to hear you are working on it. I tested the app with sideload in S7 (gearvr) and I was not able to remove the lag/jiddering. I have really good 5GHz network and even the lowest resolution was not enough. Good to hear that you are working on this so I can retest it again later. For now, I'm just waiting. :)

Are you guys able to make use of the daydream apis / VR mode in android 7.0? On my pixel, the output is so laggy and jittery as to be uncomfortable but having just tried out daydream (on my cardboard headest) the tracking/smoothness was a huge step up from cardboard apps on the same device.

Please add Direct Mode support for SteamVR. Without direct mode impossible to use such important feature as Asynchronous Reprojection which can significantly increase ingame fps on a weak system (laptops).

So probably problem on my side, because my SteamVR start in Extended Mode and if I press "Enable DirectMode" it restart again in Extended or just tell than SteamVR don't work correctly and lost the stream.

P.S. I use it on laptop with external GTX 960, connected via miniPCI-E x1 bandwidth (instead of normal x16), but Driver and Steam saw this as normal GTX 960 and it couldn't be a reason for this problem.

As i guessing, async reprojection works on SteamVR side (directly with game, not with VR headset) and Riftcat just capture and encoding final video stream (already with additional frames, made by reprojection). Probably I'm wrong.

Question, what are the chances of purchasing "just" the vibe controllers or the (my preference) oculus touch controllers with sensors, and using them with vridge??? Will steam vr detect them with the mobile hmd?