Leap Into VR/AR Design

Design thinking in three dimensions.

Virtual reality is no longer in the realm of science fiction. In the last few years, the virtual and augmented reality (VR/AR) market has been heating up with big tech companies like Microsoft, Google, HTC, Facebook, and even Apple jumping in. And the technology is no longer just for gamers but is being used for everything from car sales to corporate training. Forbes predicts VR and AR headset sales will hit $80 million by 2021 and TechCrunch is making a case that AR could become the primary driver of a $108 billion AR/VR market by 2021.

Right now, the VR/AR landscape looks a bit like the wild west. Not many standards or guidelines exist, and many people are still not entirely sure what VR/AR is, and why it matters. For designers, VR/AR offers an opportunity to be in on the ground floor of creating and designing an entirely new industry — developing the design standards, rather than following preset rules outlined by someone else.

Since most designers have spent years honing their craft on flat screens: computers, mobile devices, or even print design, it may seem daunting to jump into 3D design. It may be surprising to learn that many of the skills you already have apply to this new frontier.

As with any design project, it’s essential to know and understand as much as possible about the platform, the target audience, and what you (or the client) are trying to achieve. Be realistic. This is a new frontier, so don’t underestimate how long it will take to grasp the technology —– prepare to make mistakes and learn from them.

Getting Started: Immerse Yourself

The best way to get started is to experience VR/AR for yourself if you haven’t already, and if you’ve tried it before, it’s time for total immersion. Since this space is so new, it’s impossible to start designing without getting as much first-hand experience as possible.

For beginners on a budget, start by downloading VR/AR apps from the Google and Apple App stores onto your smartphone or tablet. Many apps are still free, and there’s a pretty wide variety to choose from. Pick up Google Cardboard which is the lowest cost way to start experiencing VR.

If you’re feeling a little more adventurous and have a little more money to throw around, Google Daydream and Samsung Gear VR offer more integrated and higher quality experiences on the lower end.

For those with bigger pocketbooks, Oculus, HTC Vive, and Sony offer higher-end platforms that focus more on gaming. These are not for the faint of heart, and costs can be high, from $949 for a complete PlayStation VR bundle to several thousand dollars for a Vive or Oculus system with a high-end PC. These systems also require quite a bit of set-up and tech savvy.

As you explore, make notes on what works and doesn’t work from a UX/UI standpoint. Make sure you take into account technical limitations and whether the hardware is helping or hindering the user experience. Also, make a note of what elements you’re drawn to, do you like the total immersion of VR or do you prefer the mixed reality of AR? This will help you determine where to spend your time learning.

First Person vs. Flat Design

VR/AR is all about three dimensions, and the user interface is no longer confined to a flat rectangular screen or surface. So, it’s essential to understand spatial design and spatial UI elements a little better.

Types of spatial UI elements:

Follow-along – UI elements follow the user as they move through the environment and are not fixed. A real-world example of this would be the dashboard of your car.

World-locked – UI elements have a specifically assigned space within the environment and do not move. In the real world, examples of this are street signs, traffic lights or crosswalk indicators.

Start to think about how design in the real world can apply to the virtual world. Make sure you think about sight lines and font sizes and be prepared to do some rapid prototyping in VR to test your assumptions. Sometimes you can make do with a hotlink on an object for simple interaction; for more complex interactions you’ll need to create dynamic menus with secondary or even tertiary interactions. Gamers are already familiar with many of these concepts and grasp the importance of each type of spatial UI element.

New Inputs, New Challenges

For the first time, users are being asked to interact with a UI without tactile interaction. No more mouse, touchpad, or touchscreen. Although some methods will mimic that tactile interaction, in most cases the user will be interacting with a virtual environment through some unfamiliar inputs, so it’s important to understand how this impacts UI design.

Types of inputs:

Gesture

Gaze (eye tracking)

Voice

Hardware inputs:

Platform specific

Touch & click controllers

Game controllers

Haptic

Each platform or device will have their own unique set of input methods and may have all, or only some of the methods explored above; therefore it’s essential to understand the platform you are designing for.

Back to Basics: the Process of Virtual Design

Every great design begins the same way, no matter what the end product. Once you have a good grasp of the project and goals, start collaborating and brainstorming. Use whatever you currently use for early design concepts: whiteboards, pen and paper, or software. Collect feedback and iterate as possible to refine and narrow the focus before moving on.

As with any design project, sketching or whiteboarding at the brainstorming stage is the best way to flesh out ideas. The image above shows initial concepts for a three dimensional AR/VR dashboard.

After initial concepts are refined, the next step is getting into the mindset of the user by creating user flows. By tracing the path that a user would take through the UI, we’re able to make sure that the path is simple and easy for them to understand.

The final stage is wireframes. Designing the way that the UI will be laid out and accessible to the user.

Start Small: Rapid Prototyping

Don’t be afraid to take sketches or wireframes into rapid VR/AR prototypes at an early stage. It will help you work out any unforeseen issues with your ideas before getting too deep into the development woods.

Some simple tools can be used to test out some of these early concepts quickly:

Sketch to VR – a plug-in that enables designers to use Sketch to create Google VR enabled demos. An excellent tool for creating and showing early prototypes to stakeholders.

Vizor – a little more advanced and allows designers to create photo- or 3D scene-based interactive demos. Storyscape walks through how to get started.

WebVR – an open platform that lets you experience VR in a web browser.

You can also use software such as Sketchup, Blender, and Cinema 4D to mockup and prototype VR/AR interfaces and scenes. Alternatively, if you want to jump right into one of the primary tools for design and development in VR/AR, go directly to Unity and set up a free account. It’s got a bit more of a learning curve, but there are plenty of tutorials out there to get you started.

Once user interface ideas begin to take shape, there are a variety of ways to prototype concepts. Shown above are 3D user interface elements that were modeled and animated in Cinema 4D before exporting to Unity Pro.

Keep in mind the device the end user will use to experience your UI. Higher resolution screens and headsets (tablet AR, Oculus, Vive, PlayStation VR) allow for more detailed designs, and smaller, more elegant fonts. Lower end or small screen devices (smartphone VR/AR, Cardboard, Daydream, Gear VR) will need to use larger, thicker fonts and less detailed UI elements.

Keeping it Real

Guide users through your interface using hints, and always provide feedback with each interaction so the user gets oriented quickly. Stick with UX/UI best practices and create clean, simple, user-friendly graphics, allow your users to come to grips with virtual experiences by giving them familiar reference points and visual metaphors.

Believe it or not, after all these years moving away from skeuomorphism in design, VR/AR is bringing it back. Using a gesture to turn a dial or pick up an object feels natural to users. Just ask hardcore gamers, they’ve been interacting with virtual worlds for years.

The image above shows a mini-game made for an immersive children's book. Users grab parts and place them over the blueprint to construct robotic power legs.

Jump in Now!

There are no limits to where creative people can go in VR/AR. It’s an open field with loads of new opportunities. It’s rare that designers can get in on the ground floor of something utterly new, and with so many possibilities. Immersive technology desperately needs designers to jump in, get involved, take risks, and make something new.

What is the difference between virtual reality and augmented reality?

What are AR and VR technology?

VR is an interactive CG experience in a simulated environment, incorporating mainly auditory and visual, but also other types of sensory feedback like haptic. AR layers virtual information over a live camera feed into a headset or through a smartphone or tablet device giving the user the ability to view 3D images.

How does a Google cardboard work?

Google Cardboard works by placing your phone at the optimal distance away from the lenses. Then, by using compatible apps, the lenses create a 3D effect when held up to your eyes.

What is meant by flat design?

Flat design is a user interface design style that uses simple, two-dimensional elements, and bright colors. It is often contrasted to the skeuomorphic style that gives the illusion of three dimensions through copying real-life properties.

What is Skeuomorphism in design?

Skeuomorphism is a term used in design to describe objects that mimic their real-world counterparts in how they appear and/or how the user can interact with them, i.e it makes interface objects familiar to users by using concepts they recognize, e.g. the recycle bin icon used for discarding files.

Ted has a passion for creative problem solving and has brought ideas to life for companies such as Sony, Apple, Nintendo, Comcast, Intel, Microsoft, Disney, Cox, and Virgin Media TV. He has a proven track record creating award-winning designs for virtually any media or platform, including print, live events, broadcast television, web, mobile, and emerging technologies (AR/VR/MR). [click to continue...]

Comments

Here in Brazil very few people are adept at virtual reality because of the high costs, but I truly believe that one day, virtual and augmented reality will be so popular and so common that people will use the streets and public transportation

EvansStudios

Thanks Tim. Glad you enjoyed it. The more creative people that get involved with VR/AR, the better it will become.

EvansStudios

Hi Marco. I agree with you. As more companies embrace the technology and create platforms and hardware that have useful and fun applications, the more it will be adopted by consumers. Over time, it will be as commonplace as the smartphone or tablets are today. Thanks for reading!

Ted has a passion for creative problem solving and has brought ideas to life for companies such as Sony, Apple, Nintendo, Comcast, Intel, Microsoft, Disney, Cox, and Virgin Media TV. He has a proven track record creating award-winning designs for virtually any media or platform, including print, live events, broadcast television, web, mobile, and emerging technologies (AR/VR/MR).