A marketer's guide to augmented reality

The years from 2006 to 2008 gave rise to a unique state of mania among brand marketers -- a state by which this author was happily seduced. The virtual world, "Second Life," captured the imaginations of both marketers and consumers alike, but the barriers to entry were high, and the user experience was cumbersome. There is still a place for virtual worlds, but the mania among the marketing community has died down (or should I say, "has come to a screeching halt").

Another phenomenon has gained popularity in marketing circles -- augmented reality (AR). Although the term has been around since the 1990s, marketers began to pick up on the technology circa 2008. A cousin to virtual worlds, AR has proven to fit more neatly into the lives of the everyday consumer, and after years of use by brand marketers, AR appears to be here to stay. As someone who spends a great deal of time studying the cross section of technology and human behavior, I urge you to keep an open mind when considering the potential of AR -- 2012 will be a big year.

Have an understanding of how AR works

The typical AR application takes a tremendous amount of computing power. There are different ways to execute AR, some more processor-intensive than others. It is important to understand that there are parallel processes at work no matter what type of execution you are employing -- all of which are a drain on today's average smartphone. Before we go any further, let's take a look at basic components of traditional AR.

First, a camera has to detect a specific shape, image, or object in order to ascertain what is meant be augmented. There are other triggers for AR such as GPS, but for simplicity's sake, we will deal with computer vision in this article. You may have seen AR executions in which a black and white marker is used; this method is employed because a unique pattern with stark contrast takes less computing power to detect. Older, less powerful smartphones are not capable of detecting more complex, natural features. Markerless AR is growing in popularity, but as one might guess, detecting complex patterns is more difficult than a black and white marker. As you can see in this demo, the first thing the camera is attempting is pattern recognition:

Once an image or object is recognized, a digital overlay is positioned in relationship to what is detected. The complexity of the overlay will determine how hard a processor will have to work. A single core processor is often not powerful enough to get the job done (we will talk about devices and processors in the next section). 3D models with large numbers of polygons (poly-count) make it difficult for an augmentation to render, and often slow down the process, resulting in a poor user experience.

Many of the popular AR applications do not accurately track to the object of origin -- tracking is very difficult to do but generally makes the experience much more meaningful. Tracking means when the camera moves, the augmentation moves relative to what it is positioned to, as we saw in the first video. Metaio, an AR software provider, has taken AR tracking to new heights with what they call "gravity-aware" AR. Have a look.

As marketers and advertisers, we aim to create magical experiences that capture the imagination. The more sophisticated the AR execution, the greater chance we have to suspend disbelief and create a sense of awe in the minds of our customers. Early examples of mobile AR were less than magical. Take Yelp's Monocle for a test drive. Although innovative for its time, the user experience is less than inspiring. As AR becomes part of our everyday lives, the ability to surprise and delight our customers with the mere presence of AR will diminish -- but for now, there is still an element of magic in the technology itself.

Keep up with popular consumer electronics

(Note: We are about to get geeky, but this is important stuff.)

Now that you are experts on the inner workings of AR, you need to think about where your experience is best executed. As marketers in the post-PC era, it is critical to think about the context in which customers will encounter the experience. Due to the current state of device fragmentation, it is important to consider hardware as an indication of context.

The first examples of AR in the advertising and marketing space were executed on the desktop (you can see some examples from a talk I gave at an iMedia Breakthrough Summit in 2009). Smartphones were exponentially less powerful in 2008, and the idea of creating an AR campaign for a mobile device, and have it reach a mass audience, was ludicrous. Today our devices are much more powerful -- still, you need to stay on top of smartphone trends order to know what types of devices can perform which types of tasks.

There are many considerations when assessing a device's ability to execute an AR experience. Processor speed is not the only consideration, but we will use the processor as a benchmark (camera, graphics, and other sensors also play a role). Here are some specifications to use as a frame of reference: a mid-priced Macbook Pro comes packed with an Intel Core dual-core i5 processor running at an average speed of 2.4 gigahertz. There are desktops from Apple packing as many as twelve cores (if you are not familiar, it may be good to read up on the notion of multi-core processors). Meanwhile the A5 chip in the iPhone 4 is a dual-core processor running at about 1 gigahertz, and the latest and greatest in the Android world, the Samsung Galaxy Nexus, is running a dual-core 1.2 gigahertz Cortex-A9. According to a report by Strategy Analytics, multi-core smartphone processors accounted for almost 25 percent of total smartphones shipped in Q3 2011. It is important to keep in mind the sobering statistic that only 44 percent of U.S. mobile subscribers own a smartphone.

While dual-core smartphones are growing in popularity and rumors of quad-core smartphones are beginning to arise, the fact remains that the percent of U.S. consumers packing enough power in their pockets to perform the complex tasks at hand is still limited. If it is mass adoption you want, you may not want to leverage AR. With that in mind, those who experiment early, succeed often.

Consider sensor fusion

Each day most of us walk around with a device in our pocket that has 7-10 sensors. The smartphones we carry around with us are sensor-rich devices. For creative technologists, the proliferation of sensor-rich devices has been both challenging and empowering. Currently, most AR relies on the use of either the camera (computer vision) or the GPS as a means of knowing when and how to display an augmentation. Most AR requires more than one sensor (GPS plus accelerometer), and there are creative ways to use multiple sensors. The use of multiple sensors in conjunction with one another is commonly referred to as sensor fusion. Sensor fusion can employ computer vision, the accelerometer, gyroscope, and any of the other sensors available on a smartphone. There are many creative combinations that can result in incredibly interesting executions. While AR scientists often focus on sensor fusion as a means of making AR appear more reliable and realistic, the creative technologist can use sensor fusion to make incredibly innovative experiences.

Let's take a step away from smartphones for a moment in order to provide content to inspire you to be creative with sensor fusion. The first example is fusing motion sensors with digital overlays:

This second example is not AR, but it is relevant. Numerous sensors are fused together to make the robot in this video a mind-boggling entity:

The creative potential for sensor fusion is vast. Don't be afraid to consider all of the sensors on a smartphone as potential colors on your creative pallet. Here is an extremely creative example of AR fused with the audio sensor:

Don't believe everything you see on YouTube

As someone who runs an agency that creates AR experiences, I am entertainingly inundated with requests for things that people see on YouTube (I don't mind, and it is not a waste of my time, though it may be a waste of your time). It is important to keep a few things in mind when searching the web for cool AR executions:

AR is a hot item for university researchers Just because you see something done in a lab does not mean it can be done on a large scale for consumers.

The videos you see are in controlled environmentsThe technologies used in marketing videos are generally not personal devices. Keep in mind that the typical smartphone user has numerous other processes running and does not know how to turn them off.

Watching online videos is different than being in the experienceBelow is an example of a very cool AR implementation. Keep in mind, the people you see in real life are watching a monitor many feet away from them and are not really standing next to animals. This may seem obvious, but when you consider the actual experience, it is different than how it appears in the video.

Keep an eye on the future (it is almost here)

Unfortunately, AR has relatively little precedent, and researchers emerge with new ideas and technologies daily. It is important to keep up on the future of AR. Companies like Total Immersion, Metaio, Vuzix, and others are constantly releasing enhancements to their products. In fact, Total Immersion and Metaio each announced a partnership with Texas Instruments (TI) -- this partnership will optimize each of the two companies SDKs in order to more seamlessly work with TI's OMAP processor.

Many of you may have heard about AR glasses. Well, at CES 2012 Vuzix showed us how close we are to a practical pair of such glasses with the SMART Glass:

By now you may also have seen the PlayStation Vita and the various ways in which Microsoft's Kinect is taking AR to the next level. While this hardware may not seem ripe for your average marketing initiative, the technology employed will soon be everywhere.

The evolution of AR will take us beyond mere glasses. Take a look at this concept video from Microsoft and the University of Washington:

As you can see, scientists are actively building AR technology to fit into every element of our lives.

AR and post-PC marketingAs marketers, we must recognize that we have entered the post-PC era. In this revolutionary time when digital information will change so dramatically, we will need to completely rethink the consumer experience. Screens are popping up in all shapes and sizes, and with AR technology, screens will become true extensions of our bodies, and media will be embedded in everything we do. The need for well-designed natural user interfaces will be of paramount importance in the way a brand reaches its consumers. Consumers will begin to expect an on-demand brand presence in this new world of ubiquitous access. If your brand is not there to greet them, you may not be there when the time comes to make a purchase.

AR is central to the notion of the post-PC era. It is a key technology and a driver of the future of media and marketing. Are you prepared?

Recognized by iMedia’s top 25 Internet Marketing Leaders and Innovators, Broitman is known for devising effective creative strategies that live at the cross section of technology and marketing.
As Vice President and Senior Business Leader of...

Comments

Commenter: PromoAid LLC

2012, March 01

Really fascinating look at all the new augmented reality technology that will arrive in the near future. As marketers and proponents of nontraditional marketing over here at PromoAid, our brains are churning out all the ways in which this relates to selling a brand message to the consumer. Much the same way marketing campaigns via emerging technologies like smartphone apps and tablets have to be carefully implemented so as to add value to the consumer experience, we're sure this will be the same. It may take some time to figure out what works. What do you think the implications for AR are for marketers?

Commenter: Adam Broitman

2012, February 28

Glad you found it helpful, Anthony!

Commenter: Anthony Green

2012, February 28

Been wanting to learn more about AR for some time and wondered why adoption was so low - now I know why. Until the hardware and software catches up, it will be on the periphery. Good article, thanks