Augmented reality: A catalyst for the coming cognitive revolution

By allowing humans to interact with the physical and digital worlds at the same time, augmented reality will extend humans’ cognitive potential, reduce cognitive burden, and provide a new lens with which to view business operations.

Information overload. Information anxiety. Information addiction. In the last 20 years, business executives have used these all-too-familiar terms to describe their world. The near-constant interruptions from ringing telephones, buzzing pagers, the web, and an unceasing stream of email messages were thought to be overwhelming people with information, forcing them to switch attention from one task to the next, again and again. People were becoming multitaskers who couldn’t concentrate for more than a few seconds at a time.

Today, the impact of the information barrage has been termed cognitive overload—the information itself doesn’t cause problems, but having to think about it does. In almost all situations, the amount of information that comes at people exceeds their cognitive capacity to handle it, and their performance could be adversely affected if they miss important details or have difficulty understanding the information. Management science research firm Basex calculated the aggregate cost of cognitive overload in the enterprise to be at least $900 billion annually due to the loss of workers’ ability to “make decisions, process information, and prioritize tasks.” The good news is that technology itself—in particular augmented reality (AR)—could provide a solution to the problem it created.

Yet there’s work still to be done. Challenges remain in the four technology areas that comprise AR—optics for smartglasses, content authoring, the human-computer interface, and 3D capabilities—but each is emerging, as detailed in earlier articles in this PwC Technology Forecast series about AR. Notwithstanding the challenges, PwC anticipates that AR will likely be a catalyst for the coming cognitive revolution in which the physical and digital worlds will converge to allow humans to interact with both worlds simultaneously. This concluding article in the series examines factors that led to the cusp of this cognitive revolution and how AR will facilitate it.

How did humans get to cognitive overload?

Modern society has evolved through two major revolutions, both of which are still being felt: the industrial revolution beginning in the late 1700s, and the computer revolution beginning in the 1960s. The former was a revolution of the physical context. The latter is in the digital context.

Since the advent of the information age, the digital and physical worlds have been diverging. (See Figure 1.) Each step along the way has siloed these two worlds and separated them further, and humanity has needed to learn how to live in both simultaneously. Email, web browsing, and text messaging do not immerse people in the physical realm; rather, they isolate people from it.

This divergence, along with rising volumes of information, has been a key driver of cognitive overload as people find themselves increasingly torn between two modes of existence. In the early days of computing, the impact was minimal. People might spend their workdays in front of a PC (immersed in the digital world), and then shut it off for the night (thereby re-entering the physical world until the next morning).

Over time, that routine has changed, as computing devices have become more numerous and, critically, more mobile. With a laptop, the digital world could be taken home or to a café, allowing for more frequent shifts between the digital and physical worlds. (See Figure 2.) With a modern smartphone, people might jump between these two worlds hundreds of times a day—often checking their phones for a few seconds in the middle of a real-world conversation.

Averting a cognitive crisis

How can a cognitive crisis be avoided? There are essentially three alternatives:

Reduce the cognitive burden. This solution is straightforward and is often implemented in user interface design by filtering and contextualizing the information presented to the user, so the user’s attention is focused on what is important. AR brings the context of the physical world in scope for the interfaces and thereby opens up new and novel ways to filter and contextualize information, therefore reducing the cognitive burden.

Increase cognitive capacity. This alternative is tricky. Can people actually increase the brain’s ability to absorb and process data? Some scientists think it’s a possibility given the neuroplasticity of the human brain. Neuroscientist David Eagleman addressed this issue in a 2015 TED Talk about the possibility of expanding human perception through the creation of new primary senses, noting that all sensory data is merely electrochemical impulses in the brain. His prototype sensory vest allows wearers to subconsciously “feel” abstract information—such as trends in the stock market—through dozens of sensors across the back.

Transfer cognitive burden elsewhere. This potential solution is the concept of offloading the burden to a third party, primarily a technological one. Such offloading is the focus of the majority of today’s research into ways to avoid a cognitive crisis. Artificial intelligence (AI) systems are designed to free people from making decisions and expending cognitive resources when they don’t need to. Research in distributed cognition develops systems and methods by which “cognition is off-loaded into the environment through social and technological means.” For example, a navigation app offloads the burden of planning a route or getting directions.

Enter the cognitive revolution

As the viability of these initiatives—particularly the transfer of cognitive burden—continues to rise, PwC believes society will embark on a third major revolution: the cognitive revolution, a revolution defined by technology’s ability to augment the cognitive potential of humans.

In simple terms, during the cognitive revolution, the physical and digital worlds will converge, driven by a pool of technologies that will bridge these two realms and allow humans to interact with both realms simultaneously. (See Figure 3.)

This converged world is the cognitive one, where digitally driven perception can lead to real-world action. That physical action can in turn inform the digital world, creating an analog-to-digital-to-analog feedback loop.

The catalyst or gateway technology for this revolution is AR, which will be the first experience many people have with a converged physical and digital world. AR works not just by projecting digital data on top of the physical world but also by letting people interact with both worlds together.

This convergence is the most fundamental aspect of the cognitive revolution, reducing the cognitive burden by consolidating the digital and physical experiences into a single, integrated one. In an AR environment, the negative impact on humans caused by constantly shifting between the two worlds is minimized, because the two worlds are one and the same. In the cognitive era, the physical and digital worlds become united, allowing the gap between the physical and digital worlds to finally begin to close, thus easing the cognitive burden and helping to avoid a cognitive crisis en masse.

Augmented reality’s role in the cognitive revolution

To date, digital technology has been limited in its ability to perceive the physical world. For example, a digital camera can capture an image and a microphone can convert audio into a digital format, but capabilities that apply intelligence to those activities have only lately begun to emerge. In recent years, facial recognition in images and video has become commonplace, and technologies such as speech-to-text and real-time language translation services have become increasingly viable.

Looking ahead, the perceptual and cognitive capabilities of technology products will soon reach a tipping point. Intelligent devices like self-driving cars can collect data from their surroundings, process it, and make decisions based on that data, linking the physical and digital worlds.

AR products such as headsets or glasses have similar capabilities, though in most cases the wearer makes the decision and not a machine. But the heavy lifting—the cognitive burden—is handled behind the scenes, thanks to cognitive systems integrated into these devices.

The ways in which earlier revolutions offloaded work from humans can provide analogies. In an industrial-era locomotive, humans are in control and command the machine to move forward, but they don’t do the hard work of providing power to the engine; technology does it. By using a computer, people can generate data and provide structure to it, but they ask algorithms to analyze the information so people may draw conclusions from it. Again, the most difficult tasks are offloaded onto technology.

In an AR scenario, the environment that people are in determines the objectives—for example, to disassemble a motor. Understanding that objective, the AR hardware and software set out the sequence of actions required. Unscrewing a bolt and removing a panel are not the burdensome cognitive tasks. Knowing which bolt to unscrew and which panel to remove, and the sequence of these events, is what requires significant cognitive effort—effort the AR system can offload from the user.

Augmented reality successes in the real world

Any modern commuter is likely well versed with one of the most common (albeit primitive) forms of AR: GPS navigation. While few people have experience using GPS in a true AR environment through smartglasses or headsets, just using GPS on a smartphone can be a great reliever of cognitive stress, particularly when driving in unfamiliar territory. While no scientific studies have been performed to quantify the reduction of stress due to GPS, anecdotally many drivers report that the technology has helped them feel more comfortable and safer behind the wheel.

Considerably more research has been done with early AR devices in industrial settings. For years, Boeing has been piloting AR technologies to improve the manufacturing process. The company has five active pilot projects—four for its commercial and defense business units and one for training.

In the training pilot, a group of factory trainees used instructions from an AR application to assemble a mock airplane wing. The group finished the complicated task 30 percent faster and with a 90 percent improvement in accuracy compared with trainees using standard instructions.

“Such evidence is very promising, and that is what we’re trying to capitalize on in our AR efforts,” says Paul Davies, an associate technical fellow in Boeing Research and Technology.

In another pilot, an AR tablet app helps a mechanic navigate through 50 steps to install a certain part. The mechanic can click on the step number in the app. The display will show the tooling in 3D, such as drills or clamps. Some of the displays are animated and show a mechanic how to attach a clamp. In parallel, the mechanic gets audio cues that explain the task.

Davies says Boeing’s goal is to “help mechanics understand work instructions and the design intent of what they’re building, and to help them build it right the first time.” The AR solutions are essentially cognitive aids for the mechanics. While these early experiments used tablet-based AR, smartglasses-based manufacturing environments are in the works and could drive greater successes. “In the next five to ten years, there will be a big increase in the use and adoption of AR,” Davies says.

Similarly, DHL has been experimenting with workers wearing AR-enabled smartglasses in one of its warehouses in the Netherlands. In the pilot project, wearables improved the efficiency of the picking process (item selection) by 25 percent. DHL is now investigating the economic feasibility of a broader implementation of “vision picking” as well as other AR applications.

In another example, AR headset manufacturer DAQRI partnered with KSP Steel in Kazakhstan to give engineers a real-time AR dashboard for visualizing data related to its steel pipe rolling line. The visualization lets workers in the control room see data as it is generated on the production floor, eliminating numerous trips by engineers back and forth between the two rooms. The overall impact: a 40 percent increase in worker efficiency and a 50 percent reduction in factory downtime.

All of these examples are strong evidence to support the idea that the cognitive revolution is not just in its beginning throes—it is already having a positive impact, thanks to AR.

The cognitive revolution: A new lens on business activity

Augmenting the physical world with contextual digital information, as AR does, will change how people view human cognition. With AR, there is no need to change focus between the digital and physical worlds and no need to disengage from a task to hunt for relevant information about what to do next or how to do it. AR users can keep their full perception focused on the task at hand while offloading mundane cognitive tasks to AI or other technological agents. The upshot is that AR users will likely experience a decrease in cognitive burden while doing their work.

PwC believes AR will likely be a catalyst for the coming cognitive revolution in which the physical and digital worlds will converge to allow humans to interact with both worlds simultaneously.

In the long run, the cognitive revolution represents a new lens on business activity, one in which people look at operations from a combined view of digital and physical operations and externalize the cognitive burden that is inherent in the task. Only then will people find new and novel ways to apply AR, and other technologies that merge the physical and digital worlds, to deliver business results.