Immersive Pipeline: Production pipelines and translators for the authoring, sharing, and touring of immersive media performance works

Abstract

Immersive media and virtual reality are not the same thing. While VR brings with it the image of experiencing virtual worlds through 3D headset displays, the actual use of such displays can be physically disorienting and socially isolating. Immersive audiovisual experiences can be created in public space and in performance by the seamless projection of data, graphics, and video on all surfaces of a performance space, and by the use of multichannel surround sound. Immersion can take place in real space and be shared with others as an embodied collective experience. Goldsmiths' Sonics Immersive Media Labs (SIML) is part of an international network of immersive projection venues including rectangular surround video facilities like the RML CineChamber (San Francisco), domes such as the SATosphere (Montreal), or new formats such as the ISM Hexadome (Berlin). Whilst these facilities exist around the world, there has yet to be established a standard working method for the creation of new cultural works for them. Artists find themselves dedicating an inordinate proportion of their creative time understanding how to configure their work to fit an immersive space, often reinventing the wheel. Once a work has premiered, they find it difficult to tour it out of lack of compatibility across these venues, resulting in inadvertently site-specific work.

We are experiencing a teething period in immersive media not dissimilar to the early days of cinema, before a lingua franca of authoring method and technical process was established. The Immersive Pipeline project will partner with these leading cultural sector institutions above, as well as with immersive technology companies Derivative and NSC Creative, to form a partnership to design and develop an artist-facing immersive media production workflow. The project will engage with artists, scientists, presentation venues, and technology companies in participatory design sessions to define best practice for creative immersive media production. We will produce an immersive content authoring pipeline and make it available to artists as a method and process with which to create new works for multiscreen spaces. By partnering with the Elizabeth Garrett Anderson Institute for Women's Health, the project will demonstrate the pertinence of artistic immersive visualization and sonification in the health and medical sector.

Finally, we will work with our partner venues to define the specification for, and prototype, a translation and format conversion technology allowing work created for one venue (for example a dome) to be automatically translated for projection in another (a rectangular venue). The results of the project will: 1.) Empower artists to produce works for a range of immersive venues, 2.) Advance the state of the art of projection mapping technology, 3.) Bring together a diverse group of cultural institutions through shared vision and technical interoperability, and 4.) Bootstrap an international consortium for future large research projects. The results of the Immersive Pipeline project will be showcased in a public performance/screening event.

Planned Impact

As a pilot project, the impact of the 6 month project will be preliminary and indicative. As the artist and cultural venue beneficiaries described above gain the knowledge to produce and present work for immersive spaces, the potential for broader impact on industry and culture will become apparent. This includes impact in mainstream cinema, responding to interest expressed by large organisations like the British Film Institute (BFI) and industry companies like IMAX (https://www.wired.com/2017/01/imax-vr-theaters/). This has the potential to trigger a sequence of impact from practitioner and industry bodies leading to influencing policy at the governmental level. This upward arch starts with Immerse UK (www.immerseuk.org), followed by the Creative Industries Federation (https://www.creativeindustriesfederation.com/) to future policy briefs following on from NESTA's 2016 report, "The Geography of Creativity in the UK" and fulfillingthe creative economy elements in HM Government's 2017 green paper, "Building our Industrial Strategy."

Our work with the EGA Institute for Women's Health's Sonic Womb project will have an impact in society beyond the academic benefits described in the previous section, and point out the potential for immersive audiovisual projection to aid in the public communication of science leading to greater public understanding of health and well being.

A specific process of identifying and fulfilling impact is described in the Pathways to Impact section.