Perhaps taking two disparate assertions, one tacit and one spoken, completely out of their original contexts and re-mixing and re-applying them to a different set of circumstances can be a helpful means to introduce an emerging and potentially prosperous new trend.

First, someone I know has a relatively smart and empathetic dog who will tilt his head from side to side if you ask him (the dog) something that sounds like a question. His owner claims that this is his dog’s way of non-verbally communicating – – to paraphrase (or parabark, maybe) – – something to the effect of “You know, you’re right. I never really thought if it that way”. Second, in an article in the January 4, 2019 edition of The New York Times entitled The Week in Tech: Amazon’s Burning Problems, by David Streitfeld, there is an amusing quote from a writer for WIRED named Craig Mod describing his 2018 Amazon Kindle Oasis as being “about as interactive as a potato”.

So, let’s take some literary license (and, of course, the dog’s license, too), and conflate these two communications in order to paws here to examine the burgeoning commercial a-peel of the rich business data now being generated by augmented reality (AR) systems.

To begin, let’s look no further than the 2019 Consumer Electronics Show (CES) held last month in Las Vegas. New offerings of AR products and services were all the rage among a number of other cutting-edge technologies and products being displayed, demo-ed and discussed.¹ As demonstrably shown at this massive industry confab, these quickly evolving AR systems that assemble and present a data-infused overlay upon a user’s real-world line of sight, are finding a compelling array of versatile applications in a widening spectrum of industries. So, too, like everything else in today’s hyper-connected world, AR likewise generates waves of data that can be captured, analyzed and leveraged for the benefit and potential profit of many commercial enterprises.

A close and compelling examination of this phenomenon was recently posted in an article entitled Unlocking the Value of Augmented Reality Data, by Joe Biron and Jonathan Lang, on the MIT Sloan Management Review site on December 20, 2018. I highly recommend a click-through and full read if you have an opportunity. I will try to summarize and annotate this piece and, well, augment it with some of my own questions.

[The Subway Fold category of Virtual and Augmented Reality has been tracking a sampling of developments in this area in commerce, academia and the arts for the past several years.]

Image from Pixabay.com

Uncensored Sensors

Prior to the emergence of the Internet of Things (IoT), it was humans who mostly performed the functions of certain specialized sensors in tasks such as detecting environment changes and then transmitting their findings. Currently, as AR systems are increasingly deployed, people will be equipped with phones and headsets, among other devices, embedded with these sensing capabilities. This “provides uncharted opportunities for organizations” to make use of the resulting AR data-enabled analyses to increase their “operational effectiveness” and distinguish the offerings of their goods and services to the consumer public.

AR’s market in 2019 is analogous to where the IoT market was in 2010, garnering significant buzz and demonstrating “early value for new capabilities”. This technology’s capacity to “visualize, instruct, and interact” can become transformative in data usage and analytics. (See Why Every Organization Needs an Augmented Reality Strategy, by Michael E. Porter and James Heppelman, Harvard Business Review, November – December 2017.)

To thereby take advantage of AR, businesses should currently be concentrating on the following questions:

How best to plan to optimize and apply AR-generated data?

How to create improved “products and processes” based upon AR users’ feedback?

Image from Pixabay.com

AR Systems Generate Expanding Spheres of User Data

Looking again to the past for guidance today, with the introduction of the iPhone and Android phones in 2007 and 2008, these tech industry turning points produced “significant data about how customers engaged with their brand”. This time period further provided engineers with a deeper understanding of user requirements. Next, this inverted the value proposition such that “applications could sense and measure” consumer experiences as they occurred.

Empowered with comparable “sensing capabilities emerging through the IoT”, manufacturers promptly added connectivity, thus generating the emergence of smart, connected products (SCPs). These new devices now comprise much of the IoT. The resulting massive data collection infrastructure and the corresponding data economy have been “disrupting technology laggards ever since”.

Moreover, using “AR-as-a-sensor” for gathering deep quantities of data holds significant potential advantages. Many AR-enabled devices are already embedded with sensing capabilities including “cameras, GPS, Bluetooth, infrared and accelerometers”. More organically, they also unleash human “creativity, intuition and experience” that cannot be otherwise replicated by the current states of hardware and software.²

What can humans with AR-based devices provide to enhance their experiences? New types of data and “behavioral insights” can be harvested from both SCPs and unconnected products. For example, in the case of an unconnected product, a user with a device equipped to operate as a form of AR-as-a-sensor could examine how the product is used and what are the accompanying user preferences for it. For an SCP, the AR-equipped user could examine how usage affects performance and whether the product is adaptable to that particular user’s purposes.

For additionally needed critical context, it is indeed “human interaction” that provides insights into how SCPs and unconnected devices are realistically operating, performing and adapting.

“Rainbow Drops”, Image by Mrs. eNil

Evaluating Potential Business Benefits from AR-Derived Data

This new quantum of AR information further creates a form of feedback loop whereby questions concerning a product’s usage and customization can be assessed. This customer data has become central to “business strategy in the new digital economy”.

These are deployed in information management operations to process unrefined AR data into “value-rich knowledge and insights”. By then porting the resulting insights into engineering systems, businesses can enhance their “product portfolio, design and features” in previously unseen ways.

AR data troves can also be merged with IoT-generated data from SCPs to support added context and insights. For unconnected devices or digital-only offerings, humans using AR to interact with them can themselves become sensors similarly providing new perspectives on a product’s:

Service usage

Quality

Optimization of the “user experience and value”

“DSC_445”, Image by Frank Cundiff

Preliminary Use Cases

The following are emerging categories and early examples of how companies are capturing and leveraging AR-generated data:

Expert Knowledge Transfer:Honeywell is gathering data from experienced employees and then enhancing their collective knowledge to thereafter be transferred to new hires. The company has implemented this by “digitizing knowledge” about their products only made visible through experience. This enables them to better understand their products in entirely new ways. Further details of this initiative is presented on the firm’s website in a feature, photos and a video entitled How Augmented Reality is Revolutionizing Job Training.

Voice of the Product: Bicycle manufacturer Cannondale is now shipping their high-end products with an AR phone app to assist owners and bike shop mechanics with details and repairs. This is intended to add a new dimension to bike ownership by joining its physical and digital components. The company can also use this app to collect anonymized data to derive their products’ “voice”. This will consequently provide them with highly informative data on which “features and procedures” are being used the most by cyclists which can then be analyzed to improve their biking experiences. For additional information about their products and the accompanying AR app, see Cannondale Habit Ready to Shred with All-New Proportional Response Design, posted on Bikerumor.com on October 9, 2018. There is also a brief preview of the app on YouTube.

Personalized Services: AR is being promoted as “transformative” to online and offline commerce since it enables potential buyers to virtually try something out before they buy it. For instance, Amazon’s new Echo Look permits customers to do this with clothing purchases. (See Amazon’s Echo Look Fashion Camera is Now Available to Everyone in the US, by Chris Welch, posted on TheVerge.com on June 6, 2018.) The company also patented something called “Magic Mirror” in January 2018. When this is combined with Echo Look will point the way towards the next evolution of the functionality of the clothing store dressing room. (See Amazon’s Blended-Reality Mirror Shows You Wearing Virtual Clothes in Virtual Locales, by Alan Boyle, posted on GeekWire.com on January 2, 2018.) The data collected by Echo Look is “being analyzed to create user preference profiles” and, in turn, suggest purchases based upon them. It is reasonably conceivable that combining these two technologies to supplement such personalized clothing recommendations will produce additional AR-based data, elevating “personalized services and experiences” to a heretofore unattained level.³

Quality Control: For quite a while, DHL has been a corporate leader in integrating AR technology into its workers’ daily operations. In one instance, the company is using computer vision to perform bar code scanning. They are further using this system to gather and analyze quality assurance data. This enables them to assess how workers’ behavior “may affect order quality and process efficiency”. (See the in-depth report on the company’s website entitled Augmented Reality in Logistics, by Holger Glockner, Kai Jannek, Johannes Mahn and Björn Theis, posted in 2014.)

Image from Pixabay.com

Integrating Strategic Applications of AR-Derived Data

There is clearly a range of meaningful impacts upon business strategies to be conferred by AR-derived data. Besides the four positive examples above, other companies are likewise running comparable projects. However, some of them may likely remain constrained from wider exposure because of “technological or organizational” impediments.

With the emergence of AR-generated data resources, those firms that meaningfully integrate them with other established business data systems such as customer relationship management (CRM) and “digital engagement”, will yield tangible new insights and commercial opportunities. Thus, in order to fully leverage these potential new possibilities, nimble business strategists should establish dedicated multi-departmental teams to pursue these future benefits.

My Questions

Because the datastreams from AR are visually based, could this be yet another fertile area to apply machine learning and other aspects of artificial intelligence?

What other existing data collection and analysis fields might also potentially benefit from the addition of AR-derived data stream? What about data-driven professional and amateur sports, certain specialties of medical practice such as surgery and radiology, and governmental agencies such as those responsible for the environment and real estate usage?

What entrepreneurial opportunities might exist for creating new AR analytical tools, platforms and hardware, as well as integration services with other streams of data to produce original new products and services?

What completely new types of career opportunities and job descriptions might be generated by the growth of the AR-as-a-sensor sector of the economy? Should universities consider adding AR data analytics to their curriculum?

What data privacy and security issues may emerge here and how might they be different from existing concerns and regulations? How would AR-generated data be treated under the GDPR? Whether and how should people be informed in advance and their consent sought if AR data is being gathered about them?

How might AR-generated data affect any or all of the arts and other forms of creative expression?

Might some new technical terms of ARt be needed such as “ARformation”, “sensAR” and “stARtegic”?

1. Much of the news and tech media provided extensive coverage of this event. Choosing just one report among many, the January 10, 2019 edition of The New York Times published a roundup and analysis of all of the news and announcements that have occurred in an engaging article with photos entitled CES 2019: It’s the Year of Virtual Assistants and 5G, by Brian X. Chen.

3. During the 2019 Super Bowl 53 played (or, more accurately, snoozed through), on February 3, 2019, there was an ad for a new product called The Mirror. This is a networked full-size wall mirror where users can do their daily workouts in directly in front of it and receive real-time feedback, performance readings, and communications with other users. From this ad and the company’s website, this device appears to be operating upon a similar concept to Amazon’s whereby users are receiving individualized and immediate feedback.