Tools of the Trade: Applications for AR/VR Design by Aaron Faucher

Aaron Faucher, a graduate of the Ivory Cohort, is a UX/UI designer focused on the emerging platforms of AR/VR. Follow him on Medium at @f0sh_.

If you’re a UX/UI designer getting into AR/VR, you might feel overwhelmed. While AR/VR is relatively new in the consumer space, 3D design has been around for a while, so there are plenty of established tools to learn, workflows to study, and terms to memorize. This month, we’re highlighting a few “must-know” applications for aspiring AR/VR designers.

Modeling, Rigging, and Animation
For 3D applications, you need 3D assets. When designing a 3D app, the process of building assets is separate from the process of building user interactions. Typically this asset creation process involves three steps:

Rigging involves assigning an underlying frame, or skeleton, to that mesh

Animation involves creating movement sequences for the mesh, typically by creating keyframes of the rig in different positions

There are myriad 3D modeling applications, and most of them feature rigging and animation toolsets as well. True 3D professionals are well versed in several of these programs, but as a beginner, you won’t go wrong by learning one of these three. The following applications have large, active user communities, offer free editions for students and educators, and have traction within the AR/VR design community.

Maya
Part of the Autodesk suite of 3D modeling software, some would argue that Maya has emerged as the industry standard for creating 3D assets for AR/VR. Its main competitor, 3ds Max, is Windows-only, making it a non-option for Mac-based designers. Maya is noted for its strength in rigging and animation workflows, and has excellent integration features with the Unity and Unreal interaction engines (see below).

Maya is one of the most popular (and powerful) applications for 3D modeling, rigging, and animation. Source: TinyPic

Blender
Blender is free. Once you see the price tags on professional 3D modeling apps, you’ll understand why this is such a big deal. What’s more, Blender is open source, has an active user community, and an excellent set of absolute-beginner tutorials. The downside? If you’re looking for a job in AR/VR, Blender is used less by 3D professionals than heavyweights like Maya, 3ds Max, or Cinema4D. If you’re serious about integrating 3D into your professional toolset, you may want to invest your time in one of these industry-standard applications. That said, Blender holds its own as a modeling and animation application, and the modeling weapon-of-choice for many indie designers.

Cinema4D
For all their power, applications like Maya and Blender are often chided for their impossible user interfaces. While C4D has a learning curve of its own, many have noted that it’s a bit easier to pick up if you’re new to the 3D space. Like its competitors, Cinema4D handles all parts of the 3D asset creation process: modeling, rigging, animation, materials, and textures. There’s a growing community of artists and designers in the UX/UI world who are adding C4D to their toolset, so it’s definitely worth considering.

Cinema 4D is known for its intuitive modeling and animation workflow. Source: FullProgramlarIndir

Texturing
The process of adding textural details and color is typically its own dedicated step in the 3D design pipeline. Once the shape of your model is set, you’ll want to explore using tools like ZBrush and Mudbox for these finishing touches.

Texturing applications like ZBrush are used to model the finer details of an asset. Source: DeviantArt

Interaction Engines
On their own, your individual 3D assets can’t do very much. How do you bring them all together into your interactive AR/VR app? This is where Unity and Unreal come in. Initially designed as engines to support video game development, these 3D interaction suites are workhorses that compute essentials like lighting, object physics, and event logic. You’ll use file formats like .OBJ, .FBX, or .DAE to bring models, animation data, and textures from your modeling program into your interaction engine.

Unity 5
If there is any piece of must-know software for aspiring AR/VR designers, it’s Unity. All major AR/VR platforms currently support the Unity interaction engine, meaning your designs will port equally well to Oculus, Hololens, HTC Vive, OSVR, Playstation VR, or Magic Leap. Unity’s scripting engine also supports Javascript, a boon for those coming from the world of front-end development.

Unreal Engine 4
UE4 also has widespread AR/VR support. One of Unreal’s unique features is its visual scripting environment, called Blueprints, which allows designers to use visual nodes (rather than traditional code) to control animations and events in the scene. If you’re script-shy, this might be a perk for you.

SDKs
On their own, Unity and UE4 offer robust interaction design functionality. If you’d like the rubber to hit the road, however, you’ll be importing third-party software development kits (SDKs) to provide added AR/VR interactivity.

Google Cardboard SDKGoogle Cardboard is hands-down the easiest (and cheapest) way to really get your hands dirty with the world of AR/VR. Take your pick – if you’re interested in designing AR as well as VR, I’d recommend buying a Cardboard with a pass-through slot for the camera. If you’re looking to rapid-prototype a VR app, it’s easy to use the iOS Unity Remote app to pull your scenes right from Unity and onto your iPhone-Cardboard setup (you might need this extra script to make the gyroscope kick in!).

Leap Motion SDK
As soon as you start playing in AR/VR, one of the first things you’ll notice is that you’ll want to touch, move, throw, or otherwise interact with the virtual objects you’ve created. That’s where Leap Motion comes in. Do yourself a favor – grab a Leap Motion controller, and download the Leap Motion Unity SDK. Within minutes, you’ll realize what the hype is about. That sweet VR mockup you created in Unity? Your real-life hands can now push, pull, touch, or grab anything in that digital space.

PTC Vuforia
PTC’s Vuforia is of the premier augmented reality kits with Unity support. Vuforia allows you to create augmented reality apps for mobile, tablet, and digital eyewear using image and object recognition technologies. With headsets like Cardboard and Gear VR becoming more accessible, mobile support means you can build a robust AR prototype using the pass-through camera – without dropping $3,000 on a Hololens. Download the SDK, import into Unity, and start working with the prefabs.

Vuforia is a popular Unity SDK for designing augmented reality experiences. Source: ytimg

The 3D pipeline is complex, but it’s not impossible to pick up – even as an absolute beginner. The best way to start? Jump in head-first. Start by downloading Unity, and follow along with one of many free beginner tutorials (I recommend Roll-A-Ball). You’ll start to absorb the unique vocabulary of 3D interaction design, and understand how the different stages interrelate. Before long, you’ll be well on your way to creating immersive, groundbreaking AR/VR experiences.