This article is part four of the series that reviews the user testing conducted on Hubs by Mozilla, a social XR platform. Previous posts in this series have covered insights related to accessibility, user experience, and environmental design. The objective of this final post is to give an overview of how the Extended Mind and Mozilla collaborated to execute this study and make recommendations for best practices in user research on cross platform (2D and XR) devices.

PARTICIPANTS WILL MAKE OR BREAK THE STUDY

Research outcomes are driven by participant quality so plan to spend a lot of time up front recruiting. If you don’t already have defined target users, pick a user profile and recruit against that. In this study, Jessica Outlaw and Tyesha Snow of The Extended Mind sought people who were tech savvy enough to use social media and communicate on smartphones daily, but did not require that they owned head-mounted displays (HMDs) at home.

The researchers’ approach was to recruit for the future user of Hubs by Mozilla, not the current user who might be an early adopter. Across the ten participants in the study, a broad range of professions were represented (3D artist, engineer, realtor, psychologist, and more), which in this case was ideal because Hubs exists as a standalone product. However, if Hubs were in an earlier stage where only concepts or wireframes could be shown to users, it would have been better to include people with VR expertise because they could more easily imagine the potential it.

In qualitative research, substantial insights can be generated from between six and twelve users. Beyond twelve users, there tends to be redundancy in the feedback, which doesn’t justify the extra costs of recruiting and interviewing those folks. In general, there is more value in running two smaller studies of six people at different iterations of product development, rather than just one study with a larger sample size. In this study, there were ten participants, who provided both diversity of viewpoints and enough consistency that strong themes emerged.

The researchers wanted to test Hubs’ multi-user function by recruiting people to come in pairs. Having friends and romantic partners participate in the study allowed The Extended Mind to observe authentic interactions between people. While many of them were new to XR and some were really impressed by the immersive nature of the VR headset, they were grounded in a real experience of talking with a close companion

For testing a social XR product, consider having people come in with someone they already know. Beyond increasing user comfort, there is another advantage in that it was more efficient for the researchers. They completed research with ten people in a single day, which is a lot in user testing.

SUMMARY OF RECRUITING RECOMMENDATIONS

Recruit participants who represent the future target user of your product (identifying user profiles is often a separate research project in user-centered design).

The farther along the product is in development, the less technologically sophisticated users need to be.

You can achieve important insights with as few as six participants.

To test social products, consider bringing in people in pairs. This can also be efficient for the researchers.

COLLECTING DATA

It’s important to make users feel welcome when they arrive. Offer them water or snacks. Pay them an honorarium for their time. Give them payment before the interviews begin so that they know their payment is not conditional on them saying nice things about your product. In fact, give them explicit permission to say negative things about the product. Participants tend to want to please researchers so let them know you want their honest feedback. Let them know up front that they can end the study, especially if they become uncomfortable or motion sick.

The Extended Mind asked people to sign a consent form for audio, video, and screen recording. All forms should give people the choice to opt out from recordings.

In the Hubs by Mozilla study, the format of each interview session was:

Welcome and pre-Hubs interview on how participants use technology (20 min)

Use Hubs on 3 different devices (40 min)

Closing interview on their impressions of Hubs (30 min)

Pairs were together for the opening and closing interviews, but separated into different conference rooms for actual product usage. Jessica and Tyesha each stayed with a participant at all times to observe their behaviors in Hubs and then aggregated their notes afterward.

One point that was essential was to give people some experience with the Oculus Go before actually showing them Hubs. This was part of the welcome and pre-Hubs interview in this study. Due to the nascent stage of VR, participants need extra time to learn about navigating the menus and controllers. Before people to arrive in any XR experience, people are going to need to have some familiarity with the device. As the prevalence of HMDs increases, taking time to give people an orientation will become less and less necessary. In the meantime, setting a baseline is an important piece for users about where your experiences exist in the context of the device’s ecosystem.

SUMMARY OF DATA COLLECTION RECOMMENDATIONS

Prioritize participant comfort

Signal that you are interested in their genuine feedbacK

Ask participants for consent to record them

Conduct pre-test and post-test interviews with participants to get the most insights

Allow time for people to get used to navigating menus and using the controller on new HMDs before testing your experience.

GENERATING INSIGHTS

Once all the interviews have been completed, it’s time to start analyzing it the data. It is important to come up with a narrative to describe the user experience. In this example, Hubs was found to be accessible, fun, good for close conversations, and participants’ experiences were influenced by the environmental design. Those themes emerged early on and were supported by multiple data points across participants.

Using people’s actual words is more impactful than paraphrasing them or just reporting your own observations due to the emotional impact of a first-person experience. For example,

There are instances where people make similar statements but each used their own words, which helps bolster the overall point. For example, three different participants said they thought Hubs improved communication with their companion, but each had a different way of conveying it:

[Hubs is] “better than a phone call.”“Texting doesn’t capture our full [expression]”“This makes it easier to talk because there are visual cues.”

Attempt to weave together multiple quotes to support each of the themes from the research.

User testing will uncover new uses of your product and people will likely spontaneously brainstorm new features they want and more. Expect that users will surprise you with their feedback. You may have planned to test and iterate on the UI of a particular page, but learn in the research that the page isn’t desirable and should be removed entirely.

SUMMARY OF GENERATING INSIGHTS RECOMMENDATIONS

Direct quotes that convey the emotion of the user in the moment are an important tool of qualitative research

Pictures, videos, and screen captures can help tell the story of the users’ experiences

Be prepared to be surprised by user feedback

Mozilla & The Extended Mind Collaboration

In this study, Mozilla partnered with The Extended Mind to conduct the research and deliver recommendations on how to improve the Hubs product. For the day of testing, two Hubs developers observed all research sessions and had the opportunity to ask the participants questions. Having Mozilla team members onsite during testing let everyone sync up between test sessions and led to important revisions about how to re-phrase questions, which devices test on, and more.

Due to Jessica and Tyesha being outside of the core Hubs team, they were closer to the user perspective and could take a more naturalistic approach to learning about the product. Their goals were to represent the user perspective across the entire project and provide strategic insights that the development team could apply.

This post has provided some background on the Hubs by Mozilla user research study and given recommendations on best practices for people who are interested in conducting their own XR research. Get in touch with contact@extendedmind.io with research questions and, also, try Hubs with a friend. You can access it via https://hubs.mozilla.com/.

This is the final article in a series that reviews user testing conducted on Mozilla’s social XR platform, Hubs. Mozilla partnered with Jessica Outlaw and Tyesha Snow of The Extended Mind to validate that Hubs was accessible, safe, and scalable. The goal of the research was to generate insights about the user experience and deliver recommendations of how to improve the Hubs product. Links to the previous posts are below.

In previous research, The Extended Mind has documented [how a 3D space automatically signals to people the rules of behavior. One of the key findings of that research is that when there is synchrony in the design of a space, it helps communicate behavioral norms to visitors. That means that when there is complementarity among content, affordances, and avatars, it helps people learn how to act. One example would be creating a gym environment (content), with weights (affordances), but only letting avatars dress in tuxedos and evening gowns. The contraction of people’s appearances could demotivate weight-lifting (the desired behavior).

This article shares learnings from the Hubs by Mozilla user research on how the different locations that they visited impacted participant’s behavior. Briefly, the researchers observed five pairs of participants in multiple 3D environments and watched as they navigated new ways of interacting with one another. In this particular study, participants visited a medieval fantasy world, a meeting room, an atrium, and a rooftop bunker.

To read more about the details and set up of the user study, read the intro blog post here.

The key environmental design insights are:

Users want to explore

The size of the space influences the type of conversation that users have

Objects in the environment shaped people’s expectations of what the space was for

The rest of the article will provide additional information on each of the insights.

ANTICIPATE THAT PEOPLE WILL WANT TO EXPLORE UPON ARRIVAL

Users immediately began exploring the space and quickly taught themselves to move. This might have been because people were new to Hubs by Mozilla and Social VR more generally. The general takeaway is that XR creators should give people something to discover once they arrive. Finding something will will be satisfying to the user. Platforms could also embrace novelty and give people something new to discover every time they visit. E.g., in Hubs, there is a rubber duck. Perhaps the placement of the duck could be randomly generated so people would have to look for it every time they arrive.

One thing to consider from a technical perspective was that the participants in this study didn’t grasp that by moving away from their companion it would be harder to hear that person. They made comments to the researchers and to each other about the spatialized audio feature:

“You have to be close to me for me to hear you”

While spatialized audio has multiple benefits and adds a dimension of presence to immersive worlds, in this case, people’s lack of understanding meant that they sometimes had sound issues. When this was combined with people immediately exploring the space when they arrived earlier than their companion, it was sometimes challenging for people to connect with one another. This leads to the second insight about size of the space.

SMALLER SPACES WERE EASIER FOR CLOSE CONVERSATIONS

When people arrived in the smaller spaces, it was easier for them to find their companion and they were less likely to get lost. There’s one particular world that was tested called a Medieval Fantasy book and it was inviting with warm colors, but it was large and people wandered off. That type of exploration sometimes got in the way of people enjoying conversations:

“I want to look at her robot face, but it’s hard because she keeps moving.”

This is another opportunity to consider use cases for for any Social VR environment. If the use case is conversation, smaller rooms lead to more intimate talks. Participants who were new to VR were able to access this insight when describing their experience.

“The size of the space alludes to…[the] type of conversation. Being out in this bigger space feels more public, but when we were in the office, it feels more intimate.”

This quote illustrates how size signaled privacy to users. It is also coherent with past research from The Extended Mind on how to configure a space to match users’ expectations.

…when you go to a large city, the avenues are really wide which means a lot of traffic and people. vs. small streets means more residential, less traffic, more privacy. All of those rules still apply [to XR].

The lesson for all creators is that the more clear that they are on the use case of a space, the easier it should be to build it. In fact, participants were excited about the prospect of identifying or customizing their own spaces for a diverse set of activities or for meeting certain people:

“Find the best environment that suits what you want to do…

There is a final insight on how the environment shapes user behavior and it is about objects change people’s perceptions, including around big concepts like privacy.

OBJECTS SHAPED PEOPLE’S EXPECTATIONS OF WHAT THE SPACE WAS FOR

There were two particular Hubs objects that users responded to in interesting ways. The first is the rubber duck and the second is a door. What’s interesting to note is that in both cases, participants are interpreting these objects on their own and no one is guiding them.

The rubber duck is unique to Hubs and was something that users quickly became attached to. When a participant clicked on the duck, it quacked and replicated itself, which motivated the users to click over and over again. It was a playful fidget-y type object, which helped users understand that it was fine to just sit and laugh with their companion and that they didn’t have to “do something” while they visited Hubs.

However, there were other objects that led users to make incorrect assumptions about privacy of Hubs. The presence of a door led a user to say:

“I thought opening one of those doors would lead me to a more public area.”

In reality, the door was not functional. Hubs’ locations are entirely private places accessible only via a unique URL.

What’s relevant to all creators is that their environmental design is open to interpretation by visitors. And even if creators make attempts to scrub out objects and environments sparse, that will just lead users to make different assumptions about what it is for. One set of participant decided that one of the more basic Hubs spaces reminded them of an interrogation room and they constructed an elaborate story for themselves that revolved around it.

SUMMARY

Environmental cues can shape user expectations and behaviors when they enter an immersive space. In this test with Hubs by Mozilla, large locations led people to roam and small places focused people’s attention on each other. The contents of the room also influence the topics of conversations and how private they believed their discussions might be.

All of this indicates that XR creators should consider the subtle messages that their environments are sending to users. There’s value in user testing with multiple participants who come from different backgrounds to understand how their interpretations vary (or don’t) from the intentions of the creator. Testing doesn’t have to be a huge undertaking requiring massive development hours in response. It may uncover small things that could be revised rapidly — such as small tweaks to lighting and sound could impact people’s experience of a space. For the most part, people don’t feel like dim lighting is inviting and a test could uncover that early in the process and developers could amp up the brightness before a product with an immersive environment actually launches.

The final article in this blog series is going to focus on giving people the details of how this Hubs by Mozilla research study was executed and make recommendations for best practices in conducting usability research on cross platform (2D and VR) devices.

This article is part three of the series that reviews the user testing conducted on Mozilla’s social XR platform, Hubs. Mozilla partnered with Jessica Outlaw and Tyesha Snow of The Extended Mind to validate that Hubs was accessible, safe, and scalable. The goal of the research was to generate insights about the user experience and deliver recommendations of how to improve the Hubs product.

To read part one of on accessibility, click here.To read part two on the personal connections and playfulness of Hubs, click here.

I am planning a series of workshops to teach XR design and research. Subscribe to my newsletter to be notified of 2019 course offerings.

In many user experience (UX) studies, the researchers give participants a task and then observe what happens next. Most research participants are earnest and usually attempt to follow instructions. However, in this study, research participants mostly ignored instructions and just started goofing off with each other once they entered the immersive space and testing the limits of embodiment.

The goal of this blog post is to share insights from Hubs by Mozilla usability study that other XR creators could apply to building a multi-user space.

The Extended Mind recruited pairs of people who communicate online with each other every day, which led to testing Hubs with people who have very close connections. There were three romantic partners in the study, one pair of roommates, and one set of high school BFFs. The reason that The Extended Mind recruited relatively intimate pairs of people is because they wanted to understand the potential for Hubs as a communication platform for people who already have good relationships. They also believe that they got more insights about how people would use Hubs in a natural environment rather than bringing in one person at a time and asking that person to hang out in VR with a stranger who they just met.

The two key insights that this blog post will cover are the ease of conversation that people had in Hubs and the playfulness that they embodied when using it.

CONVERSATION FELT NATURAL

When people enter Hubs, the first thing they would do would be to look around to find the other person in the space. Regardless of if they were on mobile, laptop, tablet or in a VR headset, their primary goal was to connect. Once they located the other person, they immediately gave their impressions of the other person’s avatar and asked what they looked like to their companion. There was an element of fun in finding the other person and then discussing avatar appearances. Including one romantic partner sincerely telling his companion:

“You are adorable,”

…which indicates that his warm feelings for her in the real world easily translated to her avatar.

The researchers created conversational prompts for all of the research participants such as “Plan a potential vacation together,” but participants ignored the instructions and just talked about whatever caught their attention. Mostly people were self-directed in exploring their capabilities in the environment and wanted to communicate with their companion. They relished having visual cues from the other person and experiencing embodiment:

“Having a hand to move around felt more connected. Especially when we both had hands.”

“It felt like we were next to each other.”

The youngest participants in the study were in their early twenties and stated that they avoided making phone calls. They rated Hubs more highly than a phone conversation due to the improved sense of connection it gave them.

[Hubs is] “better than a phone call.”

Some even considered it superior to texting for self-expression:

“Texting doesn’t capture our full [expression]”

The data from this study shows that communication using 2D devices and VR headsets has strong potential for personal conversation among friends and partners. People appeared to feel strong connections with their partners in the space. They wanted to revisit the space in the future with groups of close friends and share it with them as well.

PARTICIPANTS HAD FUN

Due to participants feeling comfortable in the space and confident in their ability to express themselves, they relaxed during the testing session and let their sense of humor show through.

The researchers observed a lot of joke-telling and goofiness from people. A consequence of feeling embodied in the VR headset was acting in ways to entertain their companion:

“Physical humor works here.”

Users also discovered that Hubs has a rubber duck mascot that will quack when it is clicked and it will replicate itself. Playing with the duck was very popular.

“The duck makes a delightful sound.”

“Having things to play with is good.”

Here’s one image to illustrate the rubber ducks multiplying quickly:

It could be a future research question to determine exactly what is the balance of giving people something like the duck as a fidget activity versus a formal board game or card game. The lack of formality in Hubs appeared to actually bolster the storytelling aspects that users brought to it. Two users established a whole rubber duck Law & Order type tv show where they gave the ducks roles:

“Good cop duckie, bad cop duckie.”

People either forgot or ignored the researchers’ instructions to plan a vacation or other prompts because they were immersed in the fun and connection together. However, the watching the users tell each other stories and experiment in the space was more entertaining and led to more insights.

While it wasn’t actually tested in this study, there are ways to add media & gifs to Hubs to further enhance communication and comedy.

SUMMARY: A PRIVATE SPACE THAT LETS PEOPLE BE THEMSELVES

The Extended Mind believes that the privacy of the Hubs space bolstered people’s intimate experiences. Because people must have a unique URL to gain access, it limited the number of people in the room. That gave people a sense of control and likely led the them feeling comfortable experimenting with the layers of embodiment and having fun with each other.

The next blog post will cover additional insights about how the different environments in Hubs impacted their behavior and what other XR creators can apply to their own work.

>>>>

This article is part two of the series that reviews the user testing conducted on Mozilla’s social XR platform, Hubs. Mozilla partnered with Jessica Outlaw and Tyesha Snow of The Extended Mind to validate that Hubs was accessible, safe, and scalable. The goal of the research was to generate insights about the user experience and deliver recommendations of how to improve the Hubs product.

To read part one of the blog series overview, which focused on accessibility, click here.

I am planning a series of workshops to teach XR design and research. Subscribe to my newsletter to be notified of 2019 course offerings.

Hubs by Mozilla lets people meet in a shared 360-environment using just their browser. Hubs works on any device from head-mounted displays like HTC Vive to 2D devices like laptops and mobile phones. Using WebVR, a JavaScript API, Mozilla is making virtual interactions with avatars accessible via Firefox and other browser that people use every day.

In the course of building the first online social platform for VR and AR on the web, Mozilla wanted confirm it was building a platform that would bring people together and do so in a low-friction, safe, and scalable way. With her years of experience and seminal studies examining the successes and pitfalls of social VR systems across the ecosystem, Jessica Outlaw and Tyesha Snow of The Extended Mind, set out to generate insights about the user experience and deliver recommendations of how to improve the Hubs product.

BACKGROUND ON THE RESEARCH STUDY

In July 2018, The Extended Mind recruited five pairs of people (10 total) to come to their office in Portland, OR and demo Hubs on their own laptops, tablets, and mobile phone. We provided them with head-mounted displays (HTC Vive, Oculus Rift & Go) to use as well.

Users were a relatively tech savvy crowd and represented a range of professions from 3D artist and engineer to realtor and psychologist. Participants in the study were all successful in entering Hubs from every device and had a lot of fun exploring the virtual environment with their companion’s avatar. Some of the participants in their early twenties also made a point to say that Hubs was better than texting or a phone call because:

“This makes it easier to talk because there are visual cues.”

And…

“Texting doesn’t capture our full [expression]”

In this series blog posts, The Extended Mind researchers will cover some of the research findings about the first-time user experience of trying Hubs. There are some surprising findings about how the environment shaped user behavior and best practices for usability in virtual reality to share across the industry.

BROWSER BASED VR (NO APP INSTALL REQUIRED)

Today, the focus is on how the accessibility of Hubs via a browser differentiates it from other social VR apps as well as other 2D communication apps like Skype, BlueJeans, and Zoom.

The process for creating a room and inviting a friend begins at http://hubs.mozilla.com. Once there, participants generated a link to their private room and then copied and pasted that link into their existing communication apps, such as iMessage or e-mail.

Once their companion received the link, they followed instructions and met the person who invited them in a 360-environment. This process worked for HMDs, computers, and mobile phone. When participants were asked afterward about the ease of use of Hubs, accessibility via link was listed as a top benefit.

“It’s pretty cool that it’s as easy as copy and pasting a link.”

And

“I’m very accustomed to virtual spaces having their own menu and software boot up and whole process to get to, but you open a link. That’s really cool. Simple.”

Some believed that because links are already familiar to most people, they would be able to persuade their less technologically sophisticated friends & family members to meet them in Hubs.

Another benefit of using the browser is that there is already one installed on people’s electronic devices. Obstacles to app installation range from difficulty finding them in the app store, to lack of space on a hard drive. One person noted that IT must approve any app she installs on her work computer. With Hubs, she could use it right away and wouldn’t need to jump that hurdle.

Because Hubs relies on people’s existing mental models of how hyperlinks work, only requires an internet browser (meaning no app installation), and is accessible from an XR or 2D device it the most accessible communication platform today. It could possibly be the first digital experience that people have which gets them familiar with the with the concepts of 360 virtual spaces and interacting with avatars, which subsequently launches them into further exploration of virtual and extended reality.

Clorama Dorvilias and I are thrilled to announce that Teacher’s Lens is freely available in the Oculus store. Clorama and I met June 2017 at Oculus Launch Pad and formed a bond over our interest in de-biasing public school teachers using a VR experience.

40 years of education research has demonstrated that TEACHER EXPECTATIONS PREDICT STUDENT PERFORMANCE[1]. It’s a stronger factor than their level of subject knowledge, their training…even home and family conditions are weaker influences of student success compared to teacher’s belief in them. And secondly, white teachers have systematically lower expectations of Black and Latino students [2] and female students in STEM classes by as much as 30% [3] [4] & [5]. What does this mean when classrooms are projected to grow to 50% students of color by 2020? So we went about trying to make an experience that would increase teacher’s expectations of all of their students. Teacher expectations are a lens to students and it can mean the difference between a student being labeled enthusiastic or disruptive. Black and Latino students are disciplined at high rates and tracked into fewer AP classes so there are very high stakes in teacher decision-making.

Clorama, DEBIAS VR CEO, explains our solution:

“Leveraging VR technology, we believe we can restore equity in the classroom using evidence based training simulations, proven to reduce hidden bias in teachers that lead to better student outcomes.

Teacher’s Lens is the first VR solution to be implemented that allows for measurable training towards diversity & inclusion goals. It’s not only proven to be a more engaging tool to handle these tough topics, but it also allows for corporations and educational institutions to measure impact, track progress, and collect data to inform more effective initiatives addressing these very real problems.

Debias VR is committed to creating experiential learning that promises to deliver research driven positive play to reduce bias and promote inclusive behavior. ”

Teacher’s Lens takes a unique approach to bias training in VR. This app is just a glance of the potential for immersive VR to better engage users with elements of playfulness to address tough topics.

Help us make it better by downloading the app for the Oculus Rift and tell us what you think! Tweet your comments to @debiasvr on Twitter and follow @creativeclo for product updates. You can read the other coverage of the app from the Oculus and Unity developer blogs.

And if you would like to speak with us about creating customized de-biasing trainings for you, contact us here.

Working together with Clorama on this project has been an amazing opportunity and I’m grateful to her for her friendship, technical capabilities, and leadership to make this app happen.

The results of my latest VR research project are live. Harassment and fear of harassment represent an obstacle to the growth of multi-user VR platforms. When I asked people about their VR usage, they told me:

“I avoid social VR because I’m afraid to encounter [harassment].”

“I am not very comfortable interacting with strangers in VR…I go out of my way to avoid interactions.”

“I was taunted and told horrible things the first [time]…Never launched the game again.”

Harassment is commonplace in VR. In past qualitative research, I studied sexual harassment of women. In my new project, in partnership with Pluto VR, I surveyed 600+ people who regularly use VR (Rift, Vive, PSVR, or Microsoft Windows Mixed Reality). It turns out that all genders are subject to multiple types of harassment in VR:

49% of women reported having experienced at least one instance of sexual harassment

30% of male respondents reported racist or homophobic comments

20% of males have experienced violent comments or threats

The majority of respondents prefer to spend time with friends or people they already know instead of strangers. Secondly, being there with friends makes those multiplayer VR experiences more enjoyable.

However, some people have a different perspective:

“Oh grow up. It’s pixels on a screen.”

The net effect of the harassment and then the subsequent denial of there being any harassment is that people are finding ways to spend time with people they already know or escape to single user VR apps.

The full report on this social VR survey is available here and it contains more info on my approach, survey respondents’ VR usage patterns, and their privacy preferences.

FIRESIDE CHAT INVITATION

The Extended Mind and Pluto VR are holding two Fireside Chats in VR. The first one will be on Tuesday, April 17, at 1 p.m. PST. The second is on Wednesday, May 2, at 11 a.m. PST. Both will be held in VR using the Pluto communication service. You are invited to participate live by joining a Pluto conversation with SerahD, Forest, and me during those times. Pluto is available for download on Steam. I’ll also post a mixed reality video recording on YouTube after each of these events. For more info or to sign up to be notified of the events, click here.

See you at the Fireside chat!

About Pluto VR

Pluto VR is a Seattle-based startup that is dedicated to helping humanity transcend physical location. The company has developed a free communication service that allows people to communicate remotely as if they were physically present with each other. Learn more about Pluto VR and the Pluto communications service at www.plutovr.com.

Hello - quick announcement that I'm giving my first talk in VR on Friday. Event will be in High Fidelity on Friday at 2pm Pacific.

The Behavioral Science of High Fidelity: I will be speaking on how open source changes people's psychology, how social norms get established, and why physical movement increases shared presence in VR. Bring your questions about the social science of VR to the Zaru Theater June 2nd at 2pm Pacific.

I was in NYC to give a talk about the behavioral science of immersive technology. While I was there, I went to The Void’s Ghostbusters Dimension Hyper-Reality and a production of Sleep No More. In my next blog post, I’ll write about Sleep No More. First up is Ghostbusters (unrelated to the PSVR game). Spoilers!

High-end VR for the general consumer

The Void’s Ghostbusters Dimension is a VR experience offered inside of Madame Tussaud’s Wax Museum, which is located in the Times Square area of New York City. If you haven’t been to Times Square before, this should give you an idea of what to expect around there.

I should have known given the hyper-touristy nature of Times Square that The Void’s Ghostbusters wasn’t for me. Anything designed for tourists is not going to appeal to someone with specialized knowledge. The Void has made a terrific VR experience for the general consumer, but as someone who has logged hours on a Vive, I was underwhelmed.

It felt like a cinematic experience. The art and animation was outstanding. However, it wasn’t fun, especially when you compare it to a something like Epic Games’ Robo Recall.

Here’s The Void’s promotional video. You can get a sense of the the art and animation.

Want more evidence that Ghostbuster is built for a general audience? Look who they are promoting testimonials from:

Time, BBC, Wired, FastCompany, Forbes, Tech Insider, and Popular Mechanics.

Have you ever looked for an entertainment recommendation from Popular Mechanics?? That’s where people go to compare reliability ratings between Hondas and Toyotas.

No risk and no fun

Ghostbusters felt like a cinematic storytelling experience only. I felt like I had zero autonomy or agency in the experience. There was no potential to fail. Risk is part of what makes games feel fun. Ghostbusters has scrubbed out all threats and automated the experience.

It seemed like there was no interactivity between me and the ghosts that I was shooting.

o My avatar was never damaged by the projectiles that the ghosts threw at me.

o It felt like the ghosts were on a timer and were going to be defeated no matter how much or how little I shot them.

I was shadowed by an employee during the experience.

o He told me before I put the headset on that if I ever needed help I could raise my hand.

o At one point when I had stopped to look at some animations, he lifted the headphone off of my ear and told me to proceed to my right. He assumed that I was stuck.

I was aware of there being other consumers waiting for me to complete the experience so that they could take a turn.

o I suspect that there’s no risk in Ghostbusters because they need to cycle consumers through to keep it profitable. If people need multiple attempts to defeat the boss, that would take too long. Plus, some consumers won’t enjoy having to make multiple attempts.

o Contrast this with Schell Games’ I Expect You to Die, a game with no time constraints. The creators expect you to need multiple attempts to complete a level. And each level gets subsequently more challenging to solve.

It seems like they made a safe bet and decided to provide a terrific viewing experience. I wouldn’t call it a gaming experience.

But they had an elevator

There were a couple things inside of the experience that you wouldn’t get in a normal HTC Vive session.

One, they had an elevator simulation that was cool. You stepped onto a platform and it shakes and rumbles. The HMD animation makes it appear as though you are going up several floors and being approached by creepy ghosts. You are sprayed with water when a ghost touches your face.

There’s also a rickety bridge that you have to cross, which messes with your balance.

Lastly, when the Marshmallow Man is defeated, you have the scent of roasted marshmallow around you.

I see the potential for VR arcades. There is an opportunity to build in robust multi-sensory experiences in an arcade that you wouldn’t have at home. But, my take is that arcades are unlikely to appeal to someone like me who already has unfettered Vive / Rift / PSVR access.

Who wants vanilla?

Compared to a recent VR release like Robo Recall, Ghostbusters is vanilla in comparison.

In Robo Recall, you go at your own pace. I died and respawned. I was confronted with choices constantly. It’s highly immersive — I got surprised by the robots. I punched the desk in my VR room until my hand was numb. I was really into shooting those robots.

Ghostbusters is a pretty flat experience in comparison. It’s over in ten minutes. I made zero choices. I was trailed by employees making sure that I stayed on track. They would have stopped me before I hurt myself.

Takeaways for designers

Consider who your experience is for. Is it going to be for a VR savvy audience? Or, readers of Popular Mechanics?

Chose the level of automaticity that fits your goals.

Kent Bye has created an Elemental Theory of Presence to piece together the different aspects of VR (film-making, gaming, emotional resonance, etc) and how they best fit together.

I would recommend this be used in conceptual discussions of all VR development so at least you are clear on what your goals are.

Want to create something similar to Ghostbusters? Space is not your primary constraint. I believe that the room I was in was only about 20 feet by 30 feet total.

In my next post, I’ll give a rundown of Sleep No More, a theatrical production of MacBeth and takes place in a four-story building in NYC. The audience trails the actors around the set for three hours so I’ll talk about the implications for immersive storytelling.

Facebook released the beta version of Spaces today. After using it for an hour today, here are are my initial comments. This is written for people who haven't had a chance to use Spaces yet.

THE AVATAR IS THE STARTING POINT

You start by logging into your Facebook account and customizing an avatar. Other people see your name and Facebook profile picture above your avatar.

My biggest issue was how to select things. I’ve never had a good handle on using the Oculus Touch controllers. Someone in my office had to coach me to extend my finger and actually touch things and/or extend finger and use the x button. Even after the coaching, it took me multiple tries to get it right.

It was especially difficult at the beginning when I was trying to change my skin tone and couldn’t select my skin. I kept getting a beard instead

Hmmm, what type of beard do I want?

TOOL MENU IS GOOD, BUT LIMITED

The primary tools are a mirror, a selfie-stick and a pencil you can use to write in the air.

There’s also things like stickers and pre-made drawings that you can use to decorate your environment. Our space got a little cluttered with graffiti so we left rather than figure out how to clear it.

Big Screen Beta has acclimated me to being able to share videos and monitor a screen when I’m hanging out in VR. Spaces already feels dated because it doesn’t offer that feature.

THIS IS A BETA

Freeze and crash after I pushed "video call."

Here a short list of the things that didn’t work for me

1. My friend using a Vive was frozen and I could only see him blink and move his mouth. I couldn’t see anything else that he did or made.

2. We could only get 3 people in our space at a time

3. I had transparent Touch fingers and everyone around me had opaque hands

4. When I opened up the menu to change my appearance, everything else in the environment was frozen

5. To change my t-shirt color or my eyewear accessories were entirely different menus to the side of the mirror

6. The setting of the park was unappealing to me. In the park, I felt like a creepy person spying on these couples having a day out. I’d prefer to be solo in a beautiful environment.

7. Making a Facebook Messenger call crashed the application…twice.

8. When Facebook Messenger did connect, only I could see my friend on the “tablet.” No one else connected via VR could see or hear him.

9. I couldn’t find a way to change my the default environment (park, campsite, etc.)

10. I didn’t know how to change the audio from the Oculus headset to the computer speakers so that everyone else in the room could hear my conversation. There were four other people in the room with me and I had to repeat what I heard for their benefit.

11. When the anyone in the room with me spoke, the lips of my avatar moved.

SOCIAL INTERACTION IS THE MAIN EVENT

Despite the technical challenges, I had fun. But why?

Was it because Eva Hoerth was there? Eva could probably cheer me up at my grandmother’s wake. Overall, it validates a belief that private VR spaces where you hang with friends will be appealing, but that’s hardly a new idea.

Was it fun because everyone I interacted with was acting a bit silly? Are goofballs the key to social VR’s success?

The novelty of it made the experience fun. But, will people quickly habituate and the novelty decrease? Perhaps, but I typically hang out in the exact same environment every time I hang out with friends. We spend time in my living room - but it’s often different people and we do a variety of things.

Overall, it’s a good use of the Facebook infrastructure, leveraging friend networks, Messenger calls, sharing on your feed, but it still feels very beta.

In January, I gave a talk on social cognition in VR and two different people asked at the end if harassment will end once your VR avatar really looks like you. No. Higher resolution facial feedback will just let your abuser know when they land a direct hit.

But the comment about facial feedback stayed with me. After thinking about it, I realized that people were talking about an idea from a Louis CK bit in 2013:

Specifically, Louis CK’s comments about kids from 0:22 to 0:55

“You know, kids are mean and it’s because they are trying it out. They look at a kid and go 'You’re fat' and then they see the kid’s face scrunch up and they go ‘ooh, that doesn’t feel good to make a person do that.’” – Louis CK

Or, if VR creators hadn’t seen the actual video clip, they knew about the idea from hearing other people paraphrasing Louis CK. For example, Tim Sweeney of Epic Games in a December 2016 interview with The Verge.

“If you insult somebody and you see that they have a sad look on their face, then you’re going to feel really, really bad about that. And you’re probably not going to do it again.” -Tim Sweeney

Let’s set aside the impact on the culture of VR of having a solo comedian hold this much influence over how people in our industry think. Instead, look at the content: Louis CK’s belief is that people will only behave if they experience getting negative feedback from acting like a bully. This sentiment is highly problematic. It relies on people making the right decision because it feels good for them, rather then it being the right thing to do.

Now, I genuinely believe that the warm glow effect is real. When you donate to good causes, give advice to a friend, lend tools to your neighbor, etc, that feels good. However, I think the reason that we should encourage people to behave well and treat others with respect is because it’s the right thing to do, not because they’ll get something out of it.

Photorealism isn’t going to cure harassment

Throwing enough engineering resources to achieve photorealism isn’t going to solve harassment. Teaching people how to behave and setting social norms has much higher and faster returns. Spend your resources to set up social rules and patterns of behavior in your environment that teach people how to act. Give people activities that teach them how to treat one another.

Once people form social connections and feel like they are part of a group, you’ll see more of the behavior that you designed for.

Photorealism decreases likability

Lastly, people don’t even like photorealism. People like animals and cute robots. See my blog post on the uncanny valley and how photorealism reduces liking.

Takeaways for designers:

Watch Louis CK’s new Netflix special “2017” if you want a preview of what VR creators are going to be saying to each other four years from now.

VR will not inherently make people better people. However, creating strong social rules and norms for positive behavior in your environment will make people act better.

Ever wonder what a historian would say about VR? Yuval Noah Harari, a historian at Hebrew University of Jerusalem and the author of Sapiens: A Brief History of Humankind and Homo Deus: A Brief History of Tomorrow, speculates on VR:

"If you don't have a job anymore and the government provides you with universal basic income, the big problem is how do you find meaning in life. What do you do all day? And here the best answer so far we've got is drugs and computer games. People will regulate their moods with all kinds of bio-chemicals and they will engage with 3-dimensional virtual realities that they’ll be absorbed in [sic], and these things will provide emotional engagement and interest more than anything in the outside world. This idea of humans finding meaning in virtual reality games is actually not a new idea. It's a very old idea. We have been finding meaning in virtual reality games for thousands of years, we've just called it religion until now."

"This idea of humans finding meaning in virtual reality games is actually not a new idea. It's a very old idea."

"You can think about religion simply as a virtual reality game. You invent rules that don't really exist. It's just in your mind. But you believe these rules and for your entire life, you try to follow the rules. If you're Christian then if you do this, you get points. If you sin, you lose points. If, by the time you finish the game (when you are dead), you gain enough points you get up to the next level. You go to heaven. People have been playing this virtual reality game for thousands of years. It made them relatively content, happy with their lives. In the 21st century, we'll just have the technology to create far more persuasive virtual reality games than the ones we've been playing for the last thousands of years. We'll have the technology to actually create heavens and hells, not in our minds, but using bits and using direct brain-computer interfaces."

Whatever your religious beliefs, Harari’s perspective that VR will become a primary source of meaning in our lives is dystopian. Considering that VR will offer higher emotional engagement and interest than outside of the headset, it’s clear that people can easily fall into narratives that they don’t have control over or even choice in. That is what Harari is calling the ‘heavens and hells’ of our own creation.

Digital Addiction

If people have a behavioral pattern to find meaning inside of VR, will they spend more or less time in VR? Obviously, more time.

Adam Alter, a psychologist at NYU, published a new book about digital addictions called Irresistible. You can buy it here.

Alter defines addiction as “something you enjoy doing in the short term, that undermines your well-being in the long term — but that you do compulsively anyway.” He goes on to state that “You never have to remember anything because everything is right in front of you. You don’t have to develop the ability to memorize or to come up with new ideas.”

Now consider the combined implications of Harari and Alter’s research:

VR has the potential to replace religion—defining religion as the stories we tell ourselves to find meaning in life.

VR experiences will be addictive.

It’s unlikely that people will come up with new ideas and stories for themselves.

Takeaways for VR designers

People will become addicted to the heavens and hells that you create for them.

Don't fuck this up.

Additional reading

H/t to Austin Ramsland for directing me to the Vox interview with Yuval Noah Harari. Link to the original podcast is here.

I was reading University of Illinois professor Steve LaValle’s book Virtual Reality and he makes a forceful argument for doing research in any VR experience. LaValle starts out talking about research by talking about the difference between having a first-person vs. third-person perspective.

"When a scientist designs an experiment for an organism, as shown in Figure 1.2, then the separation is clear: The laboratory subject (organism) has a first-person experience, while the scientist is a third-person observer. The scientist carefully designs the VR system as part of an experiment that will help to resolve a scientific hypothesis. For example, how does turning off a few neurons in a rat’s brain affect its navigation ability?"

"On the other hand, when engineers or developers construct a VR system or experience, they are usually targeting themselves and people like them. They feel perfectly comfortable moving back and forth between being the 'scientist' and the 'lab subject' while evaluating and refining their work…this is a bad idea!"

There are concrete benefits to testing an experience outside of your immediate team. LaValle continues:

"The creators of the experience are heavily biased by their desire for it to succeed without having to redo their work.

They also know what the experience is supposed to mean or accomplish, which provides a strong bias in comparison to a fresh subject.

VR disrupts the ordinary perceptual processes of its users…It is hard to predict how others will react to your own writing. Also, it is usually harder to proofread your own writing in comparison to that of others. In the case of VR, these effects are much stronger and yet elusive to the point that you must force yourself to pay attention to them.

It should be clear from this section that proposed VR systems and experiences need to be evaluated on users to understand whether they are yielding the desired effect while also avoiding unwanted side effects. This amounts to applying the scientific method to make observations, formulate hypotheses, and design experiments that determine their validity.”

Takeaways for designers:

You have incentives to believe that first-time users will understand your experience, but testing is the only way to really find out.

Develop a test plan before you bring in people to try your experience. For example, let’s say you are updating version A to Version B. Get very specific about what you predict the improvements of Version B will offer, such as:⁃ reduction in adverse symptoms⁃ improved comfort⁃ greater efficiency at solving tasks⁃ higher presence, or sense of being there⁃ greater enjoyment of the activity

LaValle’s book is a great resource for all things VR. It’s downloadable here.

1919 – Sigmund Freud published an essay called The Uncanny” (translated from the German unheimlich), defining it to mean weird/eerie/unfamiliar.

1970 – Valley of Eeriness (translated from the Japanese bukimi no tani) was coined by robotics professor Masahiro Mori to model affinity for androids as they become more humanlike.

1978 – Uncanny Valley first appeared in English inside a book by Jasia Reichardt called "Robots: Fact, Fiction, and Prediction.”

2007 – Uncanny Cliff was introduced to more accurately reflect the shape of the curve.

What Is the Uncanny Valley?

The uncanny valley is a very precise feeling of weirdness or fear that nearly-humans (but not quite humans) evoke. The uncanny valley is elicited when a robot or AI has some human likeness, but isn’t human. Long time art director and VR/AR insider Spencer Lindsay uses “creepy corpse” in his definition of the uncanny valley.

It’s assumed that liking of a robot increases the more human like that it is, until it crosses into the valley (the grey area). Lindsay’s creepy corpse would appear to be at the nadir of the valley – with zombies!

It is likely that the uncanny valley response is evolutionarily adaptive. Studies done with rhesus monkeys show a similar pattern of likability. Monkeys will look longer at the unrealistic (left) and real (right) faces below, rather than the realistic one in the center.

To categorize is a fundamental psychological process that happens automatically. The very act of perception involves an act of categorization. You haven’t completed the process of perception until you have categorized it and matched it up to other things that you know about. That is, “this object is the same as other things I know about and different from these other things.” We judge new things based on their similarity to previous things. And a human-like puppet easily confuses our judgment process.

“It’s the process that grinds away constantly and generates much of our understanding and response to the world,” says Berkeley psychologist Dacher Keltner. “First of all, it’s how do you categorize things? And that’s everything. Do I sleep with him or not? Is that a boy or a girl? Is that predator or prey? If you solve how this process works, then you solve how you know things. It’s how knowledge about the world is organized. It’s like the thread that is woven through everything in the mind.”

"Do I sleep with him or not? Is that a boy or a girl? Is that predator or prey? If you solve how this process works, then you solve how you know things."

Encountering a creepy android from the trough of the uncanny valley leads to conflicting perceptual cues. Is it human or not? It interferes with the automatic, System 1 processing that we rely on to get through our day.

The Uncanny Valley Is an Uncanny Cliff.

Lastly, there have been empirical studies that map the uncanny valley since Mori originally theorized it. Researchers show 11 different images as an object morphs from a thing to a human, and there is a decrease in liking midway through.

It’s actually more like a cliff, because likability doesn’t fully recover:

"The uncanny valley appears to be more of a cliff than a valley since even pictures of humans do not reach the level [of likeability] of pictures of toy robots. It has to be acknowledged that there is a small upwards trend again towards highly human-like entities, which results in a small valley. However, the most dominant feature in the graph is not the valley, but the cliff preceding it." – Bartneck et al.

Takeaways for designers:

Avoid anything related to the uncanny valley by populating your experience with non-humanoid avatars. People prefer non-humanoids to humanoids

The uncanny valley doesn’t actually exist. It’s more like a cliff. Once you get too humanoid, liking decreases dramatically and never fully returns.

Additional info

If you are interested in the uncanny valley, you’ll like Kimberley Voll’s interview on the fidelity contract. Listen to her full interview with @VoicesofVRKentBye here.

Freud wrote an essay in 1919 arguing that the origin of the uncanny is the loss of one’s eyes. If you are covering a person’s eyes with an HMD, no wonder it brings up feelings of strangeness.

Uncanny is an adjective best used to describe something weird or unexpected. It is defined in relation to the familiar or expected.

I’m over the uncanny valley. When I hear people describe a VR experience by saying, “It’s super uncanny valley,” my first response is usually:

People are overgeneralizing and applying this very precise term to any VR experience that they don’t like. It’s like using “epic” for anything that is even mildly interesting, or “interesting” to describe, well, anything. To elevate discussions of VR, I am going to dissect the meaning and use of “uncanny” in the context of current experiences, so people can confidently describe their subjective experiences of VR & AR with a richer vocabulary.

Let’s Get Freudian

"The Uncanny” is an essay from 1919, written by Sigmund Freud. While most of Freud’s work has been discredited, this essay remains relevant to VR creators who are making new experiences. The first third of the essay describes what uncanny means in terms of aesthetics and psychology. He states that the uncanny is the aesthetic of anxiety and fear, rather than the aesthetic of beauty.

Freud (being who he is) argues that the origin of the feeling of uncanny is a revival of repressed infantile memories or the return of primitive beliefs of the human species. He also spends a long time giving a psychoanalytic perspective on a story from 1817 called “The Sandman” wherein a man who was traumatized in childhood by almost having his eyes removed falls in love with a woman who turns out to be a doll with unusual eyes. At some point he sees the doll’s eyes on the floor, loses it, and is committed for insanity. Freud believes that the origin of uncanny is the loss of one’s eyes. What does this mean for someone creating for an HMD?

The uncanny has to do with our feelings of what is familiar. Digging into the German words of heimlich (canny/homey) and unheimlich (uncanny/unhomey), you find distinct meanings:

2. unconcealed, unsecret; what is made known; what is supposed to be kept secret but is inadvertentlyrevealed. (BTW - This would be related to a Freudian slip, saying something accidentally that reveals a hidden truth).

For good measure, I’ll include the Merriam-Webster dictionary definitions of uncanny:

1. seeming to have a supernatural character or origin : eerie, mysterious

2. being beyond what is normal or expected : suggesting superhuman or supernatural power

Given these definitions of uncanny, I believe that most of the time that people use the phrase “uncanny valley,” they likely mean uncanny. They want an adjective to describe something unfamiliar, eerie, or unexpected. In my next post, I’ll write about the meaning and use of the term Uncanny Valley, coined by a robotics professor in Japan in 1970. Until then, feel free to use “inconceivable” as your general VR adjective.

Takeaways for designers

Uncanny is an adjective best used to describe something weird or unexpected. It’s about the relationship to something familiar or expected.

If you don’t want your VR experience to feel quite so uncanny, amp up the heimlich elements of your experience. What can you do to make it more familiar, tame, or hospitable?

With current devices, you lose sight of your own surroundings, plus you cannot see the eyes of others, or what they are looking at. If Freud was right and the loss of eyes is the source of the uncanny, we might have to resign ourselves to VR being perceived as uncanny for the near future.

A viewer watches the process of factory farming as part of the 360 VR documentary series iAnimal. Photo Credit: Laika Magazine

Empathy is a real buzzword in the world of VR. VR filmmaker Chris Milk has said he aspires to build the ultimate empathy machine. The purpose of the iAnimal series seems to be to raise people's empathy for animals in order to activate their moral reasoning that animals deserve better treatment.

However, while empathy has a role in our lives, I don’t think it should be the end goal of any VR experience, because empathy does not necessarily lead to the fair treatment of others. One study on empathy for blind people woke me up to this. Adam Waytz summarizes in HBR:

Participants were asked how capable they thought blind people were of working and living independently. But before answering the question, some were asked to complete difficult physical tasks while wearing a blindfold. Those who had done the blindness simulation judged blind people to be much less capable. That’s because the exercise led them to ask "what would it be like if I were blind?" (the answer: very difficult!) rather than "what is it like for a blind person to be blind?"

People are so egocentric that even empathy tasks get reframed to be first person perspective. Humans are not terribly good at predicting how others feel. Yale psychology professor Paul Bloom wrote an entire book called Against Empathy that advocates using reason rather than empathy.

Consider learning about a ten-year-old named Sheri Summers who had a fatal disease and was waiting in line for treatment that would relieve her pain. Research participants were told that they could move her to the front of the line. When simply asked what to do, they acknowledged that she had to wait because other needy children were ahead of her. But if they were asked to imagine what Sheri felt, they tended to choose to move her up, putting her ahead of children who were presumably more deserving. Here, empathy was more powerful than fairness. – Against Empathy, p. 25

While there is certainly a role for empathy and compassion in life, it can sometimes narrowly focus us on the wrong details. And empathy does not appear to be a reliable way to activate people’s moral reasoning.

Another example of the underbelly of empathy is its ability to give people a reason or motive to be hurtful toward others. In a study where people were primed to empathize with someone (Person A), the research participants assigned Person A’s competitor (Person B) to eat more hot sauce as a punishment. It appears that creating an intervention that increases empathy toward Person A increased aggression toward Person B.

Rather than relying on empathy to guide fairness, ask people to make judgments based on logic and reasoning. Gather data by actually speaking with people about their experience, rather than just imagining how they feel.

Lastly, here’s a link to Paul Bloom speaking on how empathy blinds us to the long-term consequences of our decisions.

NBA players who touch each other during games (fist bump, high-five, head grab) cooperate more and outperform their prickly counterparts on other teams. Consider the effect of mimicking those high-fives, hugs, and team huddles in your VR experience – people would feel a high sense of trust and liking for others.

Come learn the behavioral science of how body movement influences thoughts, language, and decisions. Psychology and neuroscience research has established that human behaviors are affected by characteristics that you might not think would affect them – just pulling an item toward yourself leads to greater desire and willingness-to-pay for it.

This talk will address how simple physical movements affect people’s decision processes and how to optimize them in VR interaction design. VR designers can create a broad range of emotional responses (empathy, courage, competitiveness, etc.) just by designing the right movements and gestures for users to do inside of their experiences. No haptics required.

Are you interested in boosting player performance? Giving people the right social environment will help them achieve more.

NBA players who touch each other a lot during games (fist bump, high-five, head grab) cooperate more and outperform their prickly counterparts on other teams. Consider the effect of mimicking those high-fives, hugs, and team huddles in your VR experience – people will feel a high sense of trust and liking for others.

I’m interested in how small things influence actions and decisions. Why would a small social gesture like a high-five help a professional athlete perform better? The stakes are extremely high for them so you might imagine they are already maxed out on motivation to win.

Like most things in life, there’s an evolutionary explanation. People who belonged to a strong tribe knew that they could take more risks. In the event of a negative outcome, there were people who could care for you. These small things like fist bumps signal strong social ties.

“I’m not the guy who’s afraid of failure. I like to take risks, take the big shot and all that.” – Steph Curry

Consider how social placebos would change a VR game like Surgeon Simulator: Meet the Medic by Bossa Labs. You are a surgeon in the game and have to perform tasks like heart transplants to save the patient. This is a gaming experience where having people around could boost a player’s performance. Having another person watching you would make you move faster.*

What are the limitations of the social placebo?

Having an audience when doing a complicated task for the first time could sabotage performance. But, if it’s a straightforward action that doesn’t require any particular skill, having supporters would likely help. And it can be a complex activity, as long as the user has already rehearsed.

Also, the encouragement should probably come from the person’s in-group.

Kobe Bryant congratulating Steph Curry on a trifecta

Does the social placebo work when you are surrounded by AI avatars instead of human avatars?

Most likely. It might not last as long or be as effective compared to being around humans you know well and like, but a high five from an AI is likely better than no high five at all.

How many viewers are optimal?

It really depends on your goal. One person might be enough. Building a stadium of AI spectators might be overkill, but athletes do get a buzz from those national anthems, pre-game rituals, and cheering fans.

Takeaway for VR designers:

Usage will increase if you build in social placebos. People will be perform at a higher level and have more fun.

*Human runners go faster when they are under observation than when they are solo. Same effect in cockroaches. Those pests fun faster when other cockroaches are watching them.

The human mind functions more like a coping mechanism than a well-oiled machine. We humans are easily fooled into erroneously believing that we have encountered people, objects, and events before when in fact we have not, especially when they are similar to ones that we have genuinely experienced. So what does this mean in VR?

Presence has a downside in VR

Somehow it has become ideal in VR to achieve a high feeling of presence (“being there”). However, the more real a VR experience feels to people, the more likely they are to confuse it with something similar that they have encountered in real life—something to keep in mind when designing darker VR experiences.

But how does this work? It has to do with the malleability of memory. Any content online or offline can change your memory. Even telling a story at a dinner party about your childhood changes your memory of it. The process of recalling a memory itself strengthens it in comparison to other memories.

As Jeremy Dean puts it, “Many memories which have the scent of authenticity may turn out to be misremembered, if not totally fictitious events, if only we could check. Without some other source with which to corroborate, it is hard verify the facts, especially for events that took place long ago…”

In VR, the high presence and photorealism of Google Earth VR could easily lead to people making mistakes in their memory. For example, it would be easy to fool a native of Canton, OH by putting him in Google Earth VR, telling him he’s looking at Canton (when it’s actually Dayton, OH) and asking him to tell you a story about growing up in this neighborhood. I would guess that almost all people would tell you a memory of the place that did not actually occur there, and by doing so would be confusing their own memory of the event (where it actually happened). This is what can happen when photorealism intersects with the nature of memory—the more similar things are to authentic memories, the more likely one is to be tripped up.

Canton or Dayton? You decide!

Our Memories and Fantasies Intermingle

Here’s a recap of a study where participants were shown images and came to believe they had seen the real art in a museum, but in fact had only ever seen photos. Jeremy Dean summarizes:

"In the experiment participants took part in a self-guided museum tour where they were told to stop at particular exhibits along the way (St. Jacques & Schacter, 2013). These stops on the museum tour are the experimental stand-ins for the events you’ve experienced across your life.

Participants were asked to look at a series of photos showing the stops they visited during their tour, a sort of ‘event-movie’, thus retrieving or reactivating memory. Following retrieval, they were shown a new photo taken at the museum that showed stops that were not part of their tour.

...In the study participants returned for a third session and were shown photos and asked whether they’d stopped at the exhibit or not. Once again, some of the time they were shown pairs of exhibits they had or hadn’t looked at and other times real and false memories were mixed up together.

Across the three sessions, then, the researchers had simulated the recall of the jumble of real and false memories that are likely to be returned to consciousness when we try to recall past events. Real aspects of a memory get mixed up with false aspects and the whole confection gets stirred up each time we recall it.

In the study they found that participants’ memories were both enhanced and distorted by the process of recall. People found it easier to remember those exhibits which they were subsequently shown photographs of. This shows that merely recalling a memory is enough to strengthen it.

This is one illustration of how memory is an active, reconstructive process; recalling something is not a neutral act, but rather it strengthens the recalled memory in comparison to others.

What this is showing is how false memories can grow in the mind. Of course, in real life things don’t happen as cleanly as they do in the psych lab. Our memories and fantasies are intertwined, crossing over and interfering with one another. Thinking about the past continues this process of interweaving…

Choosing to recall certain events rather than others is a way of choosing how we live now and what decisions we make in the future.”

Takeaways for Designers:

Human memory is easily influenced by fantasy.

Giving people experiences in VR can subsequently alter their real life memories.

Presence is not always desirable, especially if you are making dark, realistic content.

On a recent “Voices of VR” podcast, I heard host Kent Bye talking about using Google Earth VR to reminisce and share stories with friends. Now, I had a very happy childhood, but my first reaction was, “I would never reminisce or share stories about my past in Google Earth VR. That’s just going to corrupt those memories.”

Memory is one the pillars of human cognition, in addition to attention, problem-solving, evaluation, and decision-making. It is also extremely fragile and easy to influence. If our experiences and memories are what make each of us unique, then it is sobering to realize how unreliable human memory can be.

Your brain is not a computer.

Human memory does not behave like a computer. Memories are not files on a hard drive just waiting to be loaded. They are not fossils to be re-visited in a perfectly maintained museum. Just by reminiscing about an event in your life, your memory of that event is impacted.

Recalling memories changes them.

The act of revisiting our memories alters them. Here’s a good recap from Jeremy Dean at PsyBlog:

"How can recalling a memory change it? Well, just by recalling a memory, it becomes stronger in comparison to other memories. Let’s run this through an example. Say you think back to one particular birthday from childhood and you recall getting a Lego spaceship. Each time you recall that fact, the other things you got for your birthday that day become weaker in comparison.

The process of recall, then, is actually actively constructing the past, or at least the parts of your past that you can remember.

This is only the beginning though. False memories can potentially be created by this process of falsely recalling the past. Indeed, psychologists have experimentally implanted false memories. This raises the fascinating idea that effectively we create ourselves by choosing which memories to recall."

And Dean writes of another illustration of how easy it is to manipulate human memory:

"A neat experiment by Goff and Roediger (1998) demonstrates how easily our memory can transform fantasy into reality. Participants were asked either to imagine performing an action or actually asked to perform it, e.g. breaking a toothpick. Sometime later they went through the same process again. Then, later still they were asked whether they had performed that action or just imagined it. Those who imagined the actions more frequently the second time were more likely to think they’d actually performed the actions the first time."

You might think - I can remember details about my childhood much better than if a researcher in a white coat asked me to break a toothpick. Possibly, though we tend to have the best memory for the most recent events. Think about the implications of this false memory study for VR. People could easily believe that what they experienced virtually happened in real life. It makes me wonder if creating realistic presence in VR should actually be a goal, given how it could affect people’s perceptions of reality.

Ifyou are interested in memory, it has been studied extensively by researchers in criminal justice. Cognitive psychologist Elizabeth Loftus has done tremendous work in false memory and the inaccuracies of eyewitness testimony. In short, eyewitnesses are not reliable due to poor viewing conditions, brief exposure, being under stress, having pre-conceived expectations, biases, stereotypes, and more.

Takeaways for VR Designers:

The human brain is not a computer.

The act of recalling memories changes them.

Maintain skepticism of the memories that people recount to you (and the memories that you personally retrieve). We are actively constructing our own experiences everyday.

VR and AR rely on interactions from the participant. Being inside of VR/AR experiences requires the user to take action. Users in these open-loop systems have partial control over the experience by moving their eyes, head, hands, or legs. In the PSVR game Headmaster, the player moves his head in order to bounce incoming soccer balls into the net and score points. It’s an easy example of gameplay relying on player movement.

In Headmaster in PSVR, you act like a soccer player and use your head to spike the ball into the net and score points

There are important implications from relying on human movement for gameplay. Here I’ve collected some research on how gestures specifically affect people’s cognitive capabilities. From Science of People:

• "You’re born to speak with your hands. Researchers have found that infants who use more hand gestures at 18-months old have greater language abilities later on. Hand gestures speak to great intelligence.

• Hand gestures make people listen to you. Spencer Kelly, associate professor of Psychology and co-director of the Center for Language and Brain at Colgate University found that gestures make people pay attention to the acoustics of speech. Kelly said, 'Gestures are not merely add-ons to language – they may actually be a fundamental part of it.'

• We can’t help it. Hand gestures come to us naturally. Spencer even found that blind people use hand gestures when speaking with other blind people. After studying native English and Turkish speakers as well as blind and sighted people, the researchers established that people learn gestures from language and grammar NOT from watching others

• Gesturing helps you access memories. Using hand gestures while you speak not only helps others remember what you say, it also helps you speak more quickly and effectively!

• Nonverbal explanations help you understand more. One study found that forcing children to gesture while they explained how to solve math problems actually helped them learn new problem-solving strategies."

• "Gesture for persuasion - The way people’s hands cut through the air while they talk is…more than just a by-product of communication. Maricchiolo et al. (2008) found that hand-gestures help increase the power of a persuasive message when compared to no use of gesture. Most effective are gestures which make what you are saying more understandable. For example, when referring to the past, point behind you."

There are two additional studies on the impact of gestures on thought:

fMRI brain imaging studies suggest that people tend to look for meaning in gestures when observing others. This means that VR/AR designers should be thoughtful when creating characters that gesture and limit gestures to things that are relevant and understandable to the user.

Gestures activate the Mirror Neuron System in the brain, which is associated with mimicry and empathy in social relationships. Mirror neurons are activated when the user makes gestures or when observing others do the same

Takeaways for designers:

Gestures activate cognitive processing at a deeper level than just words alone

The right gestures can make VR/AR feel more immersive, memorable, or persuasive

Don’t add gestures in just for fun. People look for meaning inside of gestures so limit their use in a way that helps people grasp information more quickly

I’d like to persuade you that gestures are a fundamental building block of human language and thought. This begins a series of blog posts on gestures and how physical movement in VR & AR affects cognition.

Part one of this series will deal with why gestures provide a shortcut to human thought.

But first, on the tech front:Devices to capture small hand gestures are already available (like Microsoft Hololens) and more are underway. Project Soli at Google can use radar to track micro-motions and twitches. The radar from the device senses how the user moves his hands and can interpret the intent. Link to the full Project Soli video here.

Why are gestures powerful shortcuts to cognition?

I’m reposting an article from Scientific American here that answers “Why is talking with gestures so much easier than trying to talk without gesturing?” Psychology professor Michael P. Kaschak responds: