ETC Storyline

In this demo I show two sets of objects- an animated gearbox and an engine which can be viewed in X-Ray mode. In both sets of objects I can cycle through selecting individual parts by tapping on the buttons located near the bottom left corner of the iPad.

​Notice that as each item is selected, the corresponding text label appears in the panel.

On June 19th- 20th Chris and I had the pleasure of poring over the ins and out of Hololens technology at the Microsoft HoloHack-NYC.

​We wanted to make a project that took advantage of as many features of the HoloLens as possible. However, the most interesting one for us was the spatial mapping.

In Augmented Reality you want people to explore the space around them and have the application change based on their location. We decided to build a fire fighter simulator.

​We used the spatial processing to discover all the flat surfaces in the area you are in and virtually set them on fire. Each of those fires also had spatial audio on them to help you find them in the space.

Voice commands were used for activating the fire hose based on the keyword "fire" -- which is where the name comes from. The user ends up walking around the space, screaming the word fire over and over to put out the fires around them. For those people that were too shy to scream fire, they could also air tap on the fires to put them out, taking advantage of the gesture recognition.

Here is a sample video capture of the application:

​Those of you who would like to play around with the application can go to our GitHub site.

At the end of the hackathon we had a fun time demoing the App for a bunch of students from Riverdale.

Since last summer we started a collaboration with storyteller Zohar Kfir building VR experiences. Our collaboration eventually led us to do a VR project sponsored by Oculus Studios and produced by Kaleidoscope VR - called Testimony.

Testimony is an interactive documentary for virtual reality that shares the stories of five survivors of sexual assault and their journey to healing. Testimony is an advocacy platform to allow the public to bear witness to those who have been silenced.

The world premiere was at Tribeca Film Festival in April. We published "Testimony" on the Oculus Store for both Gear VR and Rift, on June 1st. As of the date of this post (6/13/2017) It's already topped 15,000 downloads.​

The Made in NY Media Center wrote a nice blog post about this project:

We recently built a virtual tour experience of Markthal Rotterdam - a recipient of the 2017 VIVA International Awards for the most cutting-edge design housing residential apartments with one of The Netherland’s largest food markets. The VR experience was exhibited at RECon 2017 at the Las Vegas Convention Center.

A number of visitors to the VR Zone inquired about how we put together the Markthal VR application. We'd like to give you a sneak peak under the hood.

For clients new to VR and AR experiences we first like to show what can be expected in a finished product. Typically this involves demonstrating our previous work on similar applications. Then we work with the client to outline the key elements for their custom application. Clarity in the VR experience and the kind of user interaction/user experience is what makes it possible to deliver a solid and complete application on time and on budget. For this application we wanted to create an authentic experience of what it's like to visit Markthal Rotterdam and explore its many facets. The concourse alone is over 100,000 sq ft.

To start this project we set out a few guidelines to direct our development. We combine client feedback with our own experience creating VR experiences. For Markthal we compiled a short list of design constraints:

For an application shown on a trade show floor it must me extremely easy to use and not need more than than a minute of instruction

Allow people the freedom to explore and have enough time to enjoy the environment

Set a comfortable time limit of the app so that crowds can flow through

Optimize the app for continuous usage in a Trade Show environment

Optimize application power requirements to run for 5 hours+

This list was used to guide our development over the next 3 weeks. ​

Ease of use was most important. To simplify control in the VR space we decided that all user interactions would either be done through Gaze or a single tap. To provide feedback to the user that the gaze activation is working, each object that can be looked at always reacts to your gaze.

​To better capture the experience of wandering through Markthal we build a simplified model of the space. We combine architectural drawings and observations of the space to map out all of the locations for the experience. Movement through the virtual space in most natural by jumping from one location to the next by line of sight. This avoids visual clutter and encourages the user to explore. ​

Considering the volume of people at RECon 2017 we also didn’t want people staying in the VR experience for too long, creating lines. We designed the App so that it could be easily reset, instantly ready for the next person. To gracefully end the experience for users that have spent too much time, we built in a 10 minute time out.

​Ahead of the shooting we scout out the location so that we can reconstruct in VR all the images, videos, sound recordings and other features.​

We arrived with film crew in Rotterdam for video interviews of the property developer and shop owners.

​

​​After three and a half days in Rotterdam we're on the plane headed back to New York. We now have 8 days to turn all the captured footage and recordings into our VR experience.

We captured approximately one hundred and seventy 360 images. Every image file was curated for color balance, brightness, contrast. Here is a sample of the Before & After.

​With each image we also captured ambient audio for that location. The audio was mapped to a location on a virtual model of Markthal resulting in full spatial audio as you explore the space. ​

To navigate around the space we let you teleport from one location to the next. To allow that we use a gaze activated icon. The icons are placed in the space giving depth, so farther icons appear smaller and closer icons are bigger.

​We use XpressVR - a custom tool designed in house to help us rapidly build the environment and place all of the assets into the space.

To keep users from ever being lost and give them quick access to new locations we made it easy to access the main menu. With a single tap the user can jump back to the main menu and explore a new location.

​Perhaps the most fun part of building the application was creating the arch with the magnificent ceiling art.

​Lastly, it comes time to press a magic button that deploys the App onto the VR device (in this case, the Samsung Gear VR).

​We are always looking for new projects and interesting ideas. If you have an idea for a VR application for your industry or would just like to chat about the possibilities of using VR in your industry, reach out to us.

Developed by Google, Daydream VR is the successor technology to their Cardboard VR standard. It is slated to be released this Fall.

Evolving Technologies Corporation will talk about what’s strikingly different with Daydream, our experience in working with the pre-release version, and we'll showcase some hands-on demos using Daydream.

​

How to build digital vegetation for VR and what Central Park has to do with it

Vegetation is everywhere in many virtual worlds, but at the same time is very hard to do; there is lots of detail and complex light interaction. Furthermore, with the advent of 3D in movies, complex animation and finally with Virtual Reality, more and more tricks that were used to work around these complexities don't work any more. Timm Dapper, co-founder of Laubwerk, has pioneered tools for the creation of digital vegetation for more traditional computer graphics, and is now moving towards Virtual Reality.

​Our speaker for this evening is Iain Catling- cofounder of AudioSalad, developers of a software platform that provides media asset and metadata management and operational functionality for the music industry, particularly for rights holders - record labels, artists and distributors.

Much of the fascination in VR is dominated by products and software targeted at a consumer audience. But what of using VR as an integral element of services for businesses?

Iain has been exploring the use of VR as a tool for music industry professionals. His team at AudioSalad has built and is spearheading several engaging music VR experiences. Iain will talk about their ideas and thought process.

We will also have John Benton- a master storyteller fluent in technology and media talk about his experiences in using VR for storytelling.

Some of you may have developed VR Apps that you demo to your friends and colleagues. They love the App and want their friends to try it out. But they can't; it's not on the store available for download. Chris LoBello (Evolving Technologies Corporation) will outline the basic process to get your VR App on one of the stores.

The evening would not be complete without some fun, lively, and new hands-on VR demos on the HTC Vive, so you can experience room-scale vr. And of course we will also have vr demos on Gear VR and Google Cardboard.

The Made in NY Media Center provides a one of a kind setting for things to happen. I think of it as planned serendipity. A case in point is a delightful collaboration that came about between AXS Lab, Inc. and my company- Evolving Technologies Corporation.

Jason DaSilva, a veteran filmmaker, was diagnosed in 2005 with Multiple Sclerosis. There are two forms of this condition.- relapsing-remitting and primary progressive. Jason unfortunately has the less common but more aggressive form - primary progressive. As if life derailing events didn't matter, Jason pressed on making films including the Emmy Award winning documentary When I Walk, got married, has the most adorable three and half year old child, and pretty much singlehandedly launched AXS Map- a worldwide crowdsourced platform for gathering and rating accessibility experiences around the globe. The site is Powered by Google Places API and currently has over 100,000 crowdsource reviews.

The following provides a brief overview of AXS Map.
​

Jason and I are both grant recipients, here at the NY Media Center. Back in November we struck up an immediate friendship and very quickly began our collaboration to bring VR to AXS Map.

​​VR AXS Map is developed using Unity. We have devised an effective workflow for site planning, 360 immersive image capturing, curating content, and deploying to Gear VR and Google Cardboard..

Our next steps

We plan on introducing a fully developed VR-based accessibility framework that goes well beyond simple gaze control.

Down the road, VR AXS Map will be integrated with the entire AXS Map database. For now, we care concentrating on a truly effective VR experience for individuals with disabilities. We'll keep you posted on our progress.

It is not everyday that you get to take an exciting technology like VR and creatively use it for fundamental social needs that are all too often relegated to an afterthought. We have the right technology and ingredients that could lead to something extraordinary.

If some of you are going to be in or near Dumbo - Brooklyn, you might be interested in attending a VR meeting we're holding at the NY Media Center this Thursday March 17th from 6:30 - 8:30pm. Here are the event details:

Both of these are sure to be exciting presentations. To top it off, we'll be giving away a 3D printed 360 Camera Rig for the GoPro. To be eligible, you'll have to register for the meeting and of course, be present at the time of the drawing.