Artists deal in both the possible and impossible. Many, especially science fiction writers, play somewhere in the middle. They write about technologies that don’t yet exist. Or they assume the possibilities of technologies far ahead of their time.

Melbourne, AUSTRALIA: Kellie Shaw (R) inspects one of Leonardo da Vinci's most famous designs which is recognised as the ancestor to the modern helicopter and is part of an exhibition featuring 50 models of 15th century inventions by Da Vinci, in Melbourne 04 July 2006. This craft is made of linen, reeds and iron thread and would have been operated by four men rotating a shaft. The exhibition focuses on four themes: mechanical, military, hydraulic and flying machines with each model built according to Da Vinci's drawing and are crafted from materials available in 15th century Italy. AFP PHOTO/William WEST (Photo credit should read WILLIAM WEST/AFP/Getty Images)

Da Vinci’s sketches, writings, and musings were at times centuries ahead of possibility. It’s not an overstatement to say we can attribute the entire aerospace industry to his vision. Da Vinci began a tradition of futurism in art and science and made a strong statement about the relevance of the relationship of imagination, art, and science. It is the ability to conceive and share a vision, to make the invisible visible, and therefore possible.

This art of futurism has been practiced by novelists, who two hundred years ago created the science fiction genre, beginning with Mary Shelley’s Frankenstein in 1818. It is no accident that science fiction is every technology geek’s favorite genre. When a geek’s imagination gets combined with their knowledge of technology, science fiction can, and has, become science fact. Thanks to megahits like Star Wars, Star Trek, Alien, Avatar, The Terminator, The Matrix, and Guardians of the Galaxy, science fiction is well entrenched in popular culture. Through each representation of virtual reality and augmented reality in the past, writers and artists inspired technologists and entrepreneurs to create new things.

Some companies today attribute their success and core technologies to the inspiration of science fiction. Dreamscape Immersive, The Void, Zero Latency, VRstudios and others are using free roam VR to create experiences very much like the Star Trek holodeck, created to relieve the relentless monotony of space travel. VR Games like Lone Echo pit two teams against one another in a zero gravity arena, influenced in no smart part by Orson Scott Card’s 1985 novel Ender’s Game.

The influences of popular culture on AR and VR run deep and will continue to do so. Philip Rosedale, Founder of the massive virtual world Second Life and the recently opened High Fidelity VR metaverse, said he was so inspired by the movie The Matrix, he stepped down from his role as CTO of Real Networks to start Second Life. “I said ‘I’m going to make that, but it’s not going to look like that,” Rosedale recalled in a recent interview.

Snowcrash: Welcome to The Metaverse

The term Metaverse was coined well before it was popularized by Neal Stephenson in his seminal 1992 novel Snowcrash. The word “metaverse” combines the prefix "meta" (meaning "beyond") with "universe", here referring to an infinite number of interconnected virtual or digital spaces. Stephenson’s book was more than just prescient, which is why he’s been called “the tech Nostradomus”. Snowcrash has influenced thought leaders and inventors for twenty-five years, including Jeff Bezos, Google Earth designer Avi Bar-Zeev and Magic Leap founder Romy Abramovitz, who hired Stevenson as a Futurist.

Ernest Cline's novel, Ready Player One, also describes a dystopian future where many people prefer a virtual world, called the OASIS, to the real one. “For my money, Snow Crash is still the greatest virtual reality novel ever written, and it had a profound influence on me when I was writing Ready Player One”. Cline wrote in an exchange with me on Twitter. “The Metaverse is a perfectly realized visual world that Stephenson crafted with a programmer’s eye for detail, and it directly inspired the OASIS in my own novel.” In 1992, Stephenson predicted a world remarkably similar to the one we’re living in today (remember, everyone underestimates the future) featuring smartphones, GPS, VR and AR HMDs, crypto currency and the gig economy. Stevenson is also credited with bringing the Hindu term “avatar” into popular culture.

“There’s so much to love about this book,” Cline continued. “The tech-savvy rapid-fire prose. The super cool computer hacker / master swordsman / pizza deliverator main character named Hiro Protagonist, and the crumbling future America that serves as its setting. It’s still as much fun to read as when it was first published a quarter of a century ago.” Hiro, who makes a crazy living working an Uber type gig gets warned off Snow Crash (a street drug named for computer failure) by a batty acquaintance, who gives him a file labeled “Babel”. Complications ensue.

The Matrix: Do you think that’s air you’re breathing?

In 1999 The Matrix downloaded to theaters everywhere. The Matrix depicts a future where machines have taken over the surface of Earth while most humans are “wetwired” into a system known as The Matrix: A computer simulation of the real world so real its inhabitants are unaware of the outside world.

Users in the matrix have surgically implanted "ports" near the brain stem, down the spine, and on each limb. Tubes implanted throughout the body provide the nutrients necessary to keep the human alive. The unplugged character Morpheus refers to this as being wetwired.

A pedestrian walks past a poster advertising the movie "Avatar" in the lobby of a movie theater in New York, U.S., on Wednesday, Feb. 3, 2010. News Corp. rose the most in almost six months in U.S. trading as profit beat analysts' estimates and the company raised its 2010 earnings forecast after "Avatar" broke box-office records. Photographer: Daniel Acker/Bloomberg

With direct wired connections to a person’s nervous system, The Matrix simulates the same responses on a user's nervous system as experienced in reality. Those jacked into The Matrix don’t need to wear special suits or wear heavy head gear. Users lay in a chair where a cable can be plugged into a socket embedded in the back of their head. Software depicts what the user sees, feels, and experiences.

VR enthusiasts hope for a level of immersion as real as the Matrix, but fear the dystopian future of machines trapping the entire human race within a simulation. Direct neural interfaces are a desired future of many VR enthusiasts, and the required technology is a hard problem to solve given the complexity of the human mind. Futurists, such as Ray Kurzweil, believe human brain computer interfaces are inevitable within the next two to three decades, while others doubt they will ever be feasible.

There are less invasive forms of neural interfaces in pop culture. A recent phenomena, Sword Art Online, a Japanese anime series, presents a technology called Nerve Gear. Nerve Gear is a head mounted display users wear, usually while lying down. The head gear interfaces with a user's nervous system and simulates the feelings of the virtual world directly to their brain. This way no heavy haptics hardware is needed.

With the nerve gear on, users can immerse themselves in a digital world from head to toe with all their senses. Their real body is in a comatose state in the real world. They can only be in VR for a limited time before needing to go back to reality to attend to their bodies and real lives.

Avatar

In James Cameron's Avatar, a crippled marine is sent to the planet Pandora where humans are mining a precious metal known as Unobtainium, which is apparently valuable, but extremely difficult to obtain. To do so, they must destroy the habitat of the native species, the Na'Vi. As a means to communicate and understand these natives, scientists have created "Avatars". These avatars are biological copies of a Na'Vi that human pilots can jack into and use to meet and greet the aliens on their own terms. The presumed Bluetooth type connection between the puppeteer and the avatar is never explained.

Jake Sully lays in a pod and has his mind uploaded into the Avatar's body, in which he can walk again. The avatar body has working legs. His physical body does not. When we are in VR, we are piloting a virtual body in virtual space. The only difference between the marine piloting the Na’vi, and you being a ghostbuster at The Void, is the headset.

Star Trek’s Holodeck

Star Trek's Holodeck first appeared in 1974 and depicted a near perfect simulation of reality. A room used for entertainment, education, training, and all sorts of plot twists. It has been a VR enthusiast’s dream for decades now to create a perfect holodeck like that in the Star Trek series. Tim Ruse, Founder and CEO of free roam location based VR (LBVR) says The Holodeck was his original inspiration for starting a free roam VR company.

The Holodeck is said to use microscopic projectors that projects light directly into the eyes of the room’s occupants. The projectors follow the individual eye movements and position of each occupant. No matter how many occupants there are and no matter where those occupants are standing, the Holodeck appears real to them. While eye tracking and projection technology can accomplish these features in theory, the precision required to simulate reality on the level of a holodeck appears far out of reach.

Everything about the Holodeck can be done today. The Void, Zero Latency, Dreamscape, and others (see our chapter on LBVR) are creating a universe in 30’x30’ rooms you can walk around, as if real.

Star Wars

One of the delights of Star Wars is it’s inventive, unexpected revelations of high technology, often in primitive situations, such as the holographic display of Princess Leia’s message for Obi-Wan Kenobi. Such a projection can be done today. Of course, you would need a projection surface. Oh, and a stereoscopic camera to do the volumetric capture. By the way, did you ever wonder who shot the Leia hologram? It’s not from R2D2’s perspective.

Holochess isn't just for Wookies anymore.Disney

In the first Star Wars movie, we see the characters relieve the boredom of travel with “HoloChess”, of which there are now two excellent versions. Star Wars Jedi Challenge AR, which I highly recommend, delivers a satisfying version of HoloChess with its new Lenovo mobile headset. Phil Tippet also recently released the new mobile game Hologrid Monster AR.

There is a famous scene in Star Wars: Episode I – The Phantom Menace (1999) in which the meeting of the Jedi Council includes members who were in distant locations by volumetrically presenting them in the same physical space as the meeting. This technology is coming to market now. Of course, an AR HMD is required.

The Terminator

The Terminator (1984) is a robot controlled by extraordinary artificial intelligence. This AI can do everything we hope to someday with an AR headset. It does real time facial and object recognition. It is contextually and geospacially aware. It does translation and displays branching dialog choice. And it learns. It adapts. It acquires and terminates targets. In short, you do not want this thing chasing you. The Terminator is sent back in time to track down Sarah Connor before she can give birth to John Connor, who will lead the rebellion against the robot masters.

When the Terminator first arrives in the past, he finds his way to a biker bar. We see from his perspective what appears to be an augmented reality display. The display scans the environment in search for tools, displaying information on vehicle models as well as environmental computations and statistics. With a quick glance around the room, the AR display sizes up a few humans, finding one of appropriate size. Upon picking this human for his body size, the terminator kindly asks for his clothes and motorcycle, which the AR display knows the man has via his boots. The man, a fan of the clothes on his back, responds defensively. After a decisive barfight, the Terminator walks out fully clothed, hopping on the correct bike with the keys he just obtained. Thanks to the initial environmental scan when he walked in, he knows which bike the keys belong to. That is one vision of what AR and AI can do together.

Minority Report

Minority Report (2002) received praise for its depiction of an augmented reality interface and hand tracking gloves. The main character can be seen manipulating images by swiping, flicking, and grabbing digital elements on a see through display. Later we see a pseudo 3D projection of the character’s deceased wife projected into his living room. The image lacks detail when viewed from any direction but straight on, similar to 3D images captured by today's smartphone cameras. At another point in the film, the main character walks into what we might call a VR Arcade. A hallway filled with pods on either side lets users enjoy any fantasy imaginable.

Each of the technologies presented in Minority Report have influenced companies today. Touch screen displays and multi-touch software mirror the hand motions of the Precog displays. Companies like HaptX have created gloves capable of tracking a hands movement, and technologies like Leap Motion allow for hand gestures in three dimensional space, no gloves required. 3D images can be viewed with a VR head mounted display and taken with affordable cameras and smartphones. VR arcades are popping up in cities across the world, and entire VR theme parks are being built such as the $1.5B Oriental Science Fiction Park in China.

Speaking of VR theme parks...

Is she real, or is she a photorealistic AI character? In VR Westworld, we might have tactile touch, walk on a treadmill, and wear a haptic vest, even smell, to be completely immersed in the virtual world.HBO

Westworld

The interesting thing about HBO’s epic new series, Westworld, based on Michael Crichton’s 1973 film, that’s set in the wild west populated by AI driven cyborgs, programmed to provide adventure, pleasure and generally make their guests (who pay $50,000 a day for the experience) feel like they are in another world. VR headset not required. Of course, the humans misbehave badly, which causes the cyborgs to rebel. Needless to say, it doesn’t end well for either the humans or the cyborgs.

The technology is not even close to being invented yet. But of course, in the distant future beyond our lives, who knows where technology will take us. In Westworld, the cyborgs, including dogs, chickens, and horses, are 3D printed, and programmed with complex back stories and narratives. The cyborgs begin to dream and gain self-awareness. Made in our image, like God made Adam, our creations turn out, like Frankenstein, to be very human indeed.

I find the idea that we can’t tell who is human and who is AI very relevant to the conversation about where VR is going. Right now, AI inside virtual worlds is robotic and ridiculous, and couldn’t begin to pilot an avatar. But when they can, which will be soon(ish), AI will be integrated into all facets of our lives, not just video games and VR experiences.

The point is that invention is part inspiration, imagination and artistic vision, and parts hard science. But the scientists’ ideas begin some place. That place is where we all begin our lives. In front of a television. Ideas, vision, use cases and pure imagination precede invention. Stories give us a way of talking about things that have not yet been invented. A lot of what’s happening in VR and AR happened in a dream first. In a fictional story. The writer had to imagine a seamless use case. Every piece of art, story, movie and book we referenced is an example of this.

To be released on Amazon January 9, 2018. Finkmetaverse.comCool Blue Press

Charlie Fink is an AR/VR consultant, columnist, speaker, and author. As a 27-year-old junior executive at Disney, Fink created "The Lion King". For this sin, he was promoted to VP of Story Development for feature animation. In the 90s he oversaw the expansion of Virtual Worl...