Mike Seymour talk about the presentation he gave @ SIGGRAPH 2017 in Los Angeles earlier this year. MEETMIKE showcases the latest research in digital human technology, with leading industry figures interviewed live and in real-time by a photo-realistic avatar in a ‘virtual set in Sydney’, presented at the VR Village at SIGGRAPH 2017 in Los Angeles.

Speaker: Mike Seymour, fxGuide. #MEETMIKE

Producing a digital human in CG has been the ‘Manhattan’ project of computer graphics, it is both extremely difficult and has wide ranging implications both commercially and ethically. Digital Actors, Agents and Avatars are all very hot topics, but what few in the industry anticipated is how rapidly this would move from the issue of rendering a high quality human, to being able to do so in real time. Come and see MEETMIKE at ACM Siggraph ANZgraph were Mike Seymour will explain the project first shown that SIGGRAPH 2017 in LA that aimed to not only produce a realistic human but to do so at 90 frames a second in stereo in VR. Beyond realtime at even 30 fps (30 milliseconds), this International team has to render each frame in just 9 Miliseconds. To produce such fast results, the team deployed Deep Learning AI for a markerless facial tracker and solver, and used a custom build of Epic Games UE4.

In this worldwide first, each day of the SIGGRAPH show digital Mike met digital versions of industry legends and leading researchers from around the world. People such as Dr Paul Debevec, Tim Sweeney, Oscar Winners Ben Grossman (Magnopus) and Wayne Stables (Weta) and leading researchers from places such as PIXAR’s Christophe Hery, Glenn Derry (Fox).

Together they met and conducted interviews in “Sydney” via virtual reality which was watched either in VR or on a giant screen.

This project is a key part of a new research project into virtual humans as Actors, Agents and Avatars. The ANZGraph session will provide valuable data and insights for taking digital humans research to the next level, and share lessons learnt from the collaboration of R&D teams from around the world. MEET MIKE researchers span four continents, three universities and six companies including Epic Games, 3Lateral, Cubic Motion, Loom.ai and the Wikihuman global research project. The project aimed to explore best of class scanning, rigging and real-time rendering. Please note that this project aims to give away nearly all the data for non-commercial use and is a non-profit research effort.

Mick also spoke about Matching and Mirroring – Remember that people tend to like people who are like themselves. You will tend to like people who are like you, I will tend to like people who are like me. The most important key to gaining instant rapport with another individual therefore is to make ourselves like them. One way that we can do this is to match and mirror their words (7%), tonality(38%) and physiology (55%).

Affective computingis the study and development of systems and devices that can recognize, interpret, process, and simulate human affects. It is an interdisciplinary field spanning computer science, psychology, and cognitive science.[1] While the origins of the field may be traced as far back as to early philosophical inquiries into emotion,[2] the more modern branch of computer science originated with Rosalind Picard‘s 1995 paper[3] on affective computing.[4][5] A motivation for the research is the ability to simulate empathy. The machine should interpret the emotional state of humans and adapt its behavior to them, giving an appropriate response to those emotions.

Uncanny valleyis a hypothesized relationship between the degree of an object’s resemblance to a human being and the emotional response to such an object. The concept of the uncanny valley suggests that humanoid objects which appear almost, but not exactly, like real human beings elicit uncanny, or strangely familiar, feelings of eeriness and revulsion in observers. Valley denotes a dip in the human observer’s affinity for the replica, a relation that otherwise increases with the replica’s human likeness.

Synthetic datais “any production data applicable to a given situation that are not obtained by direct measurement” according to the McGraw-Hill Dictionary of Scientific and Technical Terms; where Craig S. Mullins, an expert in data management, defines production data as “information that is persistently stored and used by professionals to conduct business processes.” The creation of synthetic data is an involved process of data anonymization; that is to say that synthetic data is a subset of anonymized data. Synthetic data is used in a variety of fields as a filter for information that would otherwise compromise the confidentiality of particular aspects of the data. Many times the particular aspects come about in the form of human information (i.e. name, home address, IP address, telephone number, social security number, credit card number, etc.).

The Wikihuman project is a collaborative by the members of the Digital Human League to advance the study of digital humans. For more information on the Wikihuman project, please visit their blog at wikihuman.org

Chaos Group – We would like to officially announce the wikihuman.org site. This is the start of an ongoing project by the Digital Human League (DHL). Our mission for this project is to study, understand, and most importantly share our knowledge of digital humans.

alSurface Shader Wikihuman project By Mike Seymour October 20, 2016 The “al” in alSurface refers to Anders Langlands (right), a VFX sequence supervisor currently at Weta Digital, who wrote a series of shaders for Arnold.

Surface roughness often shortened to roughness, is a component of surface texture. It is quantified by the deviations in the direction of the normal vector of a real surface from its ideal form. If these deviations are large, the surface is rough; if they are small, the surface is smooth. In surface metrology, roughness is typically considered to be the high-frequency, short-wavelength component of a measured surface. However, in practice it is often necessary to know both the amplitude and frequency to ensure that a surface is fit for a purpose.

Emotions, immediacy, we care about faces, smiling and the effects on a digital person.

Creating an optical flow for the merge between the two expressions with no texture loss, blend shapes on his face and needing something to drive it. Fluid simulation using training data, learnt from an example, we can make the data. Using synthetic and training data for more powerful interactions creating a nighter affinity, it is more trustworthy.

Look at the pores of the skin, the Diffuse, Specular and Normal Map and line up the textures for a smile and not a smile. Higher resolution specular maps for the face to be real. Change the pings off it to read the surface texture.

VR, no 30fps, 90fps stereo vertically because of our noses. UV lighting. 75% of the geometry went on the hair.

Is a creative project aimed at showcasing the new wave of virtual reality works. Pushing the boundries of the medium and assisting experienced content creators to create VR works. By having a VR cinema space to screen, network with other vr creators. Virtual Reality is upon us, its a new frontier, a new medium. For the newcomers, VR is an immersive experience in which your head movements are tracked in a three-dimensional world, seen in a 360° perspective, thanks to mobile VR headsets. This cinema brings Australia’s New Wave of VR Creators who are pushing these new boundaries.

The SIGGRAPH conference is a year event that focuses on the latest in computer graphics and other interactive arts. It is a multidiscipline show, mixing computer imaging with other physical creations. It is also a crossroads for the film, video game, commercial, research and education industries. It has a forward-looking vision while celebrating the recent and not-so-recent past.

The beginning of August saw the 44 year old ​SIGGRAPH ​conference​ (​S​pecial ​I​nterest ​G​roup on Computer ​GRAPH​ics and Interactive Techniques) return to Los Angeles for the 11th time, and this year the topic was almost exclusively VR. As this is ProVideo Coalition and not VirtualReality Coalition, I’ll be taking a more filmmaker-centric approach to this recap. That being said, the word of the week was “immersion”, and at this point I’m confident we’re on our way to ​The Matrix ​within the next 5 years.

MEET UP Sydney Augmented and Virtual Reality. We are Augmented Reality enthusiasts of all descriptions – technical, arts, sports, industrial, commercial, investors, entrepreneurs and simply any end user. There is a lot happening in this tech niche and in particular locally which beckons more co-ordinated face to face interactions and formal/informal meetup gatherings. Jump on board with us!

ALIVE: live-action lightfields for immersive VR experiences is an eighteen-month industrial partnership between Foundry and Figment Productions together with University of Surrey and funded by InnovateUK. The collaboration aims to develop the technologies required to capture live-action content as a fully immersive Virtual Reality experience, where the viewer has the freedom to move inside the content.

From dystopia to utopia: the new hyperreality Online shopping, led by Amazon, is hugely popular. But imagine being able to strap on a VR headset and ‘walk’ through those digital stores, getting all the best bits of visiting the shops without suffering the crowds of people or lugging around heavy bags. An obvious application for the medical industries is hyperreal training scenarios. Instead of operating on plastic models or cadavers, budding surgeons – or senior doctors updating their skills – could use VR or AR to retrace the steps of a recorded real-life surgery.

Ana Serrano is the Chief Digital Officer at the Canadian Film Centre and Founder of CFC Media Lab, the world-renowned institute for interactive storytelling. We spoke with Ana who spoke at our 360 Vision event about her experiences with VR and what trends she’s noticed over the last year.

The now familiar pattern for AR/VR investment saw the highest volume of investments from pre-seed through Series A, with chunky later stage deals accounting for most of the dollar value in the quarter (Improbable over $500 million, Unity $400 million – reported as half investment and half secondary share sale by employees).

SAN FRANCISCO — Dawn Jewell recently treated a patient haunted by a car crash. The patient had developed acute anxiety over the cross streets where the crash occurred, unable to drive a route that carried so many painful memories.

We’re all aware of how rapidly virtual reality is gaining ground, but just how difficult is the VR production process?

FIRE PANDA LTD is a Virtual Reality development studio creating exciting and innovative VR Content, Experiences and Games. We are proud to work with some of the most talented people exploring everything VR can offer.

VR will definitely continue to grow, as long as international markets exist and clients are open to it. Currently at FKD, it’s just another addition to our workflow and another way to add strength to the collective idea.

In the early 1990s, many people gawked at ugly and bulky mobile phones, unable to see just how much they’d change the face of communication. In this piece, Kaga Bryan argues we’re at the same point now with virtual reality, and he’s having a bit of déjà vu.

Seven out of 10 early adopters of virtual and/or augmented reality hardware have bought into the buzz and firmly feel that the technology is set to disrupt media, education, work, social interaction, travel and retail.

A documentary about a clinic in Los Angeles that uses virtual reality simulations to treat war veterans with PTSD. Hosted by a former master sniper for the Canadian military, this documentary short explores the efficacy of these systems to treat a still-mysterious mental condition.

VR’s first scripted series sheds new light on medium’s unknowns by April Robinson

The Molecule, says CEO Chris Healer, is very good at solving unique challenges. So when 30 Ninjas came calling with a scripted VR series to be directed by its principal, Doug Liman (The Bourne Identity, Edge of Tomorrow) and shot by Jaunt VR, Chris and his team were all in. In his words, “It made so much sense.”

Based on a screenplay by Julina Tatlock and Oscar-winning screenwriter Melisa Wallack, the mystery series centers on an heiress’ quest to guard her recently deceased grandfather’s estate and the family’s supernatural secrets. The story, told over five, roughly six-minute episodes, marks the first attempt at using virtual reality as an episodic-friendly, storytelling format while challenging many of VR’s current known truths.

Check out our six new videos with screen producers and VR pioneers sharing their thoughts on where VR is heading, what the future could look like and what could make the next great VR project, now available for your inspiration. The videos come from our 2016 virtual reality event, 360 Vision, which was presented by Screen NSW, ABC, Screen Australia, Event Cinemas and AFTRS. The videos capture the conversation from the day and include talks by Collisions creator and 360 Vision Mentor Lynette Wallworth, Rose Troche: internationally renowned director, producer and screenwriter, Ana Serrano, Gabo Arora and Barry Pousman to name a few.

Virtual reality films are fun to watch, but there’s a ton of work that goes into creating them. We sat down with Here Be Dragons to understand the effort behind their films. Watch the behind the scenes video here.

CARA VR™—the much-anticipated new plug-in toolset for the NUKE® family of compositing, editorial and finishing products—helps you to create incredible live-action virtual reality content.

With a specialised toolset that includes headset review, CARA VR dramatically speeds up the challenging and often tedious process of stitching and compositing 360° video footage, so you have more time to focus on creating high-quality immersive experiences.

AUTODESK

3D to VR: The Essentials. We’re kicking off our journey from 3ds Max to VR, and we want you to come along. Join us for a daily video tutorial as we walk you through the fluid workflow of 3ds Max’s powerful 3D animation tools combined with a new interactive toolset. 3ds Max 2018.1 features 3ds Max Interactive, a real-time engine that extends the power of Max with tools to create interactive and virtual reality (VR) experiences.

Reforming the Workflow Resident Evil 7: Biohazard in Virtual Reality. We talked to the Resident Evil 7: Biohazard (RE7) Development Team, Mr. Kawata (Producer), Mr. Fukui (Technical Director) and Mr. Tsuda (Art Director) about overcoming the various challenges in creating RE7, the transition to physically-based rendering, the development of their company game engine and bringing the franchise to VR.

3ds Max is now combined with 3ds Max Interactive, a powerful VR engine that gives you the ability to go from Max to VR in just a few clicks. If you’re a current subscriber, you’ll notice a new Interactive menu when you open 3ds Max 2018.1: this will launch 3ds Max Interactive, our new 3D to VR creative workflow for design viz artists like you.

VR PRODUCTION

Staples VR is a complete VR production studio providing both end to end solutions and consultation services for a variety of clients and industry leaders. We operate with the newest and highest quality equipment for image capture and 360 degree VR capture such as the Jaunt, Nokia OZO, Red, Arri, Blackmagic, Z-Cam, Sony, Panasonic and custom built solutions such our world fire fire proof VR capture system, and our underwater and aerial solutions. We specialise in dynamic camera movement, from aerial drone, cable cam and crane to dolly and underwater, enabling us to add that something special to your production. We pride ourselves on our ambisonic audio capture and offer full spatial post production mixing and sound design.

Jaunt ONE is a professional-grade camera system specifically designed for capturing high-quality stereoscopic 360º cinematic virtual reality experiences. Built from the ground up with visionary VR creators in mind, Jaunt ONE has proven itself in the hands of the world’s top filmmakers, studios, and networks.

Jaunt ONE offers industry-leading stereoscopic capture quality and a suite of tools for camera control and data management. It features 24 camera modules with frame-sync, global shutter, 10-stops of dynamic range, and custom 130º FOV lenses with a fixed f/2.9 aperture. Additional features include support for 120 frames per second capture and a live viewfinder. Jaunt ONE and its complementary workflow present content creators with an end-to-end solution for filming high quality cinematic virtual reality content.

The Jaunt One is a professional grade, stereographic cinematic VR build from the ground up and designed with visionary VR creators in mind. Staples VR will put the camera through its paces and explain the good the bad and the amazing when it comes to the system. Client support to visualise, post production, time line cost, testing, consulting, access to equipment. and do well in collaboration.

Staples VR lead the way when it comes to live action 360 video and your content creation. What can you do with the technology? They have worked with clients to make high end quality VR including live action capture, gameified interactivity, photogrammetry, lidar scaning, installations, training, equipment resource, live streaming VR that can be mix in with prerecorded material and swapping between different camera systems. They have build a huge range of skills and techniques to get the most of your capture systems and post workflow for the entertainment, medical, architectural, OH & S, forensic training, military training, building installations, airlines, factories, education and telecommunication industries to name a few and are on the fore front of this rapidly expanding industry.

New Zealand Fire Service today launched a world-first initiative – a 360 degree and virtual reality (VR) experience – Escape My House available online now. For the first time ever, the public can experience a real house fire first-hand and, along the way, learn why they need an escape plan.

LYTRO CIMEMA Imagine working on footage in visual effects and having the ability to change the depth of field or focal point of a scene on the fly. Change the frame rate and have physically accurate shutter angle that would have been appropriate for the rate. Reposition the camera in the scene. Drop 3D objects in your scene and have them properly occluded by the live action content. Pull mattes by setting a near depth and a far depth and extracting an object. Effectively, think of having many of the benefits of deep compositing for live action scenes.

Proof of Concept (PoC) is a realization of a certain method or idea in order to demonstrate its feasibility or a demonstration in principle with the aim of verifying that some concept or theory has practical potential. A proof of concept is usually small and may or may not be complete.

light field systems such as Lytro recording refractions and reflections of the entire area of space. Light field camera, also known as plenoptic camera, captures information about the light field emanating from a scene; that is, the intensity of light in a scene, and also the direction that the light rays are traveling in space. This contrasts with a conventional camera, which records only light intensity. Lytro is building the world’s most powerful Light Field imaging platform enabling artists, scientists and innovators to pursue their goals with an unprecedented level of freedom and control. This revolutionary technology will unlock new opportunities for photography, cinematography, mixed reality, scientific and industrial applications.

volumetric capturing an area, volumetric video with large storage and at the moment no suitable way of playing it back.

JauntONE camera has 24 independent camera systems, resolution 8K, minimum 4K in a stereoscopic field of view. It uses every second camera module to give a slightly parallax view for the left and right eye giving a sense of depth and only playable back through head sets. Can shoot different frame rates up to 120 for slow motion which can only do 4K output. Can preview the camera system, change the ISO values, 12 volts, workflow with changing batteries and cameras. The parallax information becomes your depth map in the full 360 arc to incorporate in vfx elements.

Can get artefacts in the stitch line when something is close or further away from the camera, looking for the ability to stitch together with only one stitch line or as few as possible. Optical flow form of stitching, analyses every pixel of the frame and comparing to the frame before and after to track the movement to know what is moving and what is staying static. Not a perfect system, issues such as the algorithms not being able to recognise pixels from one camera to the next when things getting to close to the lenses within the minimum parallax distance, when two lenses cannot see the same material. The other one is motion blur, going to quickly through the frame. It is harder when the camera cannot see the image in its entirety is harder to get the stitch points. Shooting a higher frame rate, will give clearer pictures even if playing back at 30.

Needs to be level, stereoscopic around the middle and tapers off top and bottom. Assume the audience is on the same eyeline as the camera, if the audience choose to look at the skyline then you cannot control the angle the are looking at that object and will not be able to play back correctly. Create depth through the left and right eye and our brain focuses somewhere in the middle.

Some issues when considering which camera to use: safe distance from the camera 2.5 meters, not too fast moving images, camera moving to minimise and avoid haloing, artifacting, morphing, repeating textures, semi-transparent, moiré effect (are large scale interference patterns that can be produced when an opaque ruled pattern with transparent gaps is overlaid on another similar pattern. For the moiré interference pattern to appear, the two patterns must not be completely identical in that they must be displaced, rotated, etc., or have different but similar pitch.) Use 8 modules out of the camera, does not like single orbit.

It pulls up every camera module to see the placement, exposure and correct accordingly. The camera has up to 10 and 1/2 stops, the system can go up to 18 stops of dynamic range in a gradient across the system. Situations such as the camera placed between a window and action, you can expose correctly for skin tone and then for the window, outside exposure and the cameras in-between will incrementally so it works together accurately and doing multiple exposures.

AUDIO FOR VR

George the dummy head microphone, two microphones and does not do top, bottom, front and back very well and does not change the perspective. Humans have two ears, notes the difference in levels between the ears, the difference in time that it takes for the sound to travel to the ears and position the sound based on those things. Giving the recording that sounds real, suitable for atmos, two track stereo. The head has been used in radio for many years and during an Opera on the steps of the Sydney Opera House, George was used for the orchestra inside in the concert hall.

The dummy head is designed to replicate average sized human head and depending on the manufacturer may have a nose and mouth too. Each dummy head is equipped with pinnae and ear canals in which small microphones are placed, one in each ear. The leading manufacturers in Dummy Head design are: Neumann, Brüel & Kjær, Head Acoustics GmBH, and Knowles Electronics.

The term “binaural” has frequently been confused as a synonym for the word “stereo”, and this is partially due to a large amount of misuse in the mid-1950s by the recording industry, as a marketing buzzword. Conventional stereo recordings do not factor in natural ear spacing or “head shadow” of the head and ears, since these things happen naturally as a person listens, generating their own ITDs (interaural time differences) and ILDs (interaural level differences). Because loudspeaker-crosstalk of conventional stereo interferes with binaural reproduction, either headphones are required, or crosstalk cancellation of signals intended for loudspeakers such as Ambiophonics is required. For listening using conventional speaker-stereo, or mp3 players, a pinna-less dummy head may be preferable for quasi-binaural recording, such as the sphere microphone or Ambiophone. As a general rule, for true binaural results, an audio recording and reproduction system chain, from microphone to listener’s brain, should contain one and only one set of pinnae (preferably the listener’s own) and one head-shadow.

SENNHEISER The elegantly designed AMBEO® VR Mic has been developed in cooperation with VR content producers and fine-tuned through an extensive creators’ program with participants from across the audio and VR communities. The mic caters exactly to the needs of VR content creators, letting you capture the experience and spirit of any location enabling the listener to be immersed as if they were there.

Four mono channels, matrix into one mono in the middle, up down stereo and a left right stereo and front stereo. When you move around the sound stays with the visual experience, losing the illusion.

AMBEO A-B format converter plugin. The capsules of the Sennheiser AMBEO VR Mic deliver A-format, a raw 4-channel output that has to be converted into a new set of 4 channels, the Ambisonics B-format. This is done by the specifically designed Sennheiser AMBEO A-B format converter plugin, which is available as free download for VST, AU and AAX format for your preferred Digital Audio Workstation for both PC and Mac. B-format is a W, X, Y, Z representation of the sound field around the microphone. W being the sum of all 4 capsules, whereas X, Y and Z are three virtual bi-directional microphone patterns representing front/back, left/right and up/down. Thus, any direction from the microphone can be auditioned by the listener during playback of Ambisonics B.

AUGMENTED REALITY

Overlaying a digital canvas onto a person’s view of the real world, to enhance the design visualisation process, has vast potential.

AR Experiments is a site that features work by coders who are experimenting with augmented reality in exciting ways. These experiments use various tools like ARCore, an SDK that lets Android developers create awesome AR experiences. We’re featuring some of our favorite projects here to help inspire more coders to imagine what could be made with AR.

Augmented reality project adds a little road system to your living room floor. Invisible Highway is an experiment in controlling physical things in the real world by drawing in Augmented Reality. Simply make a pathway along the floor on your phone and the robot car will follow that path on the actual floor in your room. A custom highway with scenery is generated along the path to make the robots a little more scenic on your phone screen.

360º

Vertical video transformed from a syndrome to an accepted form of viewing the world. Agencies, film festivals, online groups, promote the portrait format. But there is a new kid on the block: 360 video.

*Skybox 360/VR Plug-ins have been acquired by Adobe and will be integrated into After Effects and Premiere by the end of 2017. They are available now for free if you’re an Adobe Creative Cloud subscriber.

Microsoft created waves at E3 this year with the announcement of their new gaming console, Xbox One X. Kicking off the release was a spectacular trailer created by the crew at Blind. The minute we saw it, we knew we had to find out more from the team about how it was made. Below, Creative Director Matthew Encina offers an exclusive look inside the inspiration, process and tools of the momentous trailer.

If You Love Something, Set It Free. You can download the engine and use it for everything from game development, education, architecture, and visualization to VR, film and animation. When you ship a game or application, you pay a 5% royalty on gross revenue after the first $3,000 per product, per quarter. It’s a simple arrangement in which we succeed only when you succeed.

Who says the creativity well has run dry? We’re still seeing a number of innovative entrepreneurs making waves in 2017. You’ll find a lot of them pitching their ideas on crowd-funding websites such as Kickstarter and IndieGoGo, where anyone can pledge some money to support new products and business ideas.

WORDS OF WISDOM FROM SUCCESSFUL INDIE DEVS Looking to see what it takes to become a successful indie developer? You’ve come to right place. We spoke to ten game devs and asked for their words of wisdom on making it in the indie world.

MAME ENTERTAINMENT: originally stood for Multiple Arcade Machine Emulator. MAME’s purpose is to preserve decades of software history. As electronic technology continues to rush forward, MAME prevents this important “vintage” software from being lost and forgotten. This is achieved by documenting the hardware and how it functions. The source code to MAME serves as this documentation. The fact that the software is usable serves primarily to validate the accuracy of the documentation (how else can you prove that you have recreated the hardware faithfully?). Over time, MAME absorbed the sister-project MESS (Multi Emulator Super System), so MAME now documents a wide variety of (mostly vintage) computers, video game consoles and calculators, in addition to the arcade video games that were its initial focus.

MACMAME: is part of the MAME project, a non-profit organization dedicated to preserving the history of arcade videogames via emulation. MacMAME achieves this by running the original program code found in the arcade games. As such, it is much more than a reproduction, it is essentially that same game running via an emulated layer inside your Macintosh. On this site you’ll find the most current build of MacMAME, information about upcoming versions and instructions for using it. Please look around and enjoy reliving some of the games that made going to video arcades an enjoyable part of our pop culture. You can move to various parts of the site by clicking on the headings to the left.