Some ideas on using virtual reality for data visualization: I don’t really agree with the details here but it’s all worth discussing

Misleading graphs are one thing you consistently write about, and it seems to me that Virtual Reality has the potential to solve part of this problem — there is no point in sharing a “static” VR experience; instead, by allowing users to change the axes and interact with the data, they have access to many more details about the data.

Anyways, if you are interested, I’ve written a guest blog post about how VR can augment traditional data visualization.

I’ve tried to take a non “Pie Charts, but in 3d!” approach. The piece covers everything from the infamous Is Linda a Bank Teller? problem to the disadvantages of realistic Chernoff faces. I’d love to hear your thinking on the piece or the ideas I’ve mentioned above.

My reply: I don’t really agree with the details of what you’re suggesting but it’s all worth thinking about, so I’ll blog it.

Although VR may well be useful for exploring data in some specific cases, the article’s arguments are confused. The article conflates the benefits brought by interaction (interaction is possible and widely available in 2D) with the potential benefits brought by a 3D representation of data. For cases where a 3D representation is beneficial, we need to further think about the benefits brought by a VR setup compared to, for example, a rotatable 2D projection on a regular display (3D rotation provides strong kinetic depth cues), a stereoscopic screen, or a solid model. Also, there are many other approaches to multidimensional visualization than the ones mentioned, include simple tabular visualizations. There is a strong commercial push for VR and a burgeoning research interest in immersive visualization but if we really want to make progress, we need to think much more clearly and more honestly about the opportunities and limitations brought by this technology.

I agree with Pierre, it is not clear to me what advantage that VR has over the 3D perception produced by rotating scatterplots, essentially what was available in PRIM-9 and PRIM-H in the late 70s early 80s. Furthermore, being inside the 3D display would be a bit disorienting.

Curiously, in 1983–5 when I was working on this, I could get about 4 frames per second of rotation on an Apollo workstation. We needed both rotation and stereo glasses to see a missing center in a 3D normal distribution. I made a movie where this fact was in the narration script, but I used animation to get 24 fps, and the audience could see the hole without 3D classes. On a modern computer, the GPU does these rotations far faster than we could do it in the mid-80s, you can probably get realistic 3D on a smart phone.

Most of the good ideas from that era, and the few good ones that have arisen since are available in ggobi (which is available as an R package).

I’ve never had a situation where 50% more dimensions (i.e. 3D) helped me find something that 2D plot matrices didn’t. I wish it weren’t so, but…

Also, I think we should also say that AR (Augmented Reality) would be a better deployment environment than VR (Virtual Reality). Hundreds of thousands of iPhones now have AR capabilities and it works fairly well. Fewer side-effects than VR for the viewer, already have an AR device in your pocket.

There is an argument against 3D on usability grounds at https://www.nngroup.com/articles/2d-is-better-than-3d/ – dating from 1998 so to the first virtual reality hype and possibly out of date. It does suggest 3D for naturally 3D problems and hopes that 3D might get better in future.

“VR also forces data representation to be experiential, which means that it’s much harder to alter the representation of the data to fit a pre-defined story.”

I design user interfaces for applications and this reminds me of the excitement about how the touchscreen interface of the iPad was going to make software so much easier to use because it would allow people to “literally interact with their data.” (Spoiler: it did not.)

Mapping 15-dimensional data to different attributes of a hypothetical picnic table certainly changes the representation of the data. Unless you’re looking to buy a picnic table, referring to the “true, intuitive, meaning of a table that is twice as tall as another” is hard to justify.

Also, the DeathTools video contradicts the notion that building an experiential UI makes it impossible to fit the data to a pre-defined story.