In 2017, Kay Voges directed the legendary first opera by Philip Glass, Einstein on the Beach – a milestone of the American avant-garde and of Minimal Music – at the Dortmund Opera House.

Already before launching the first preparations, a basic principle of the staging was established: everything will be steered and controlled by the music. Video projections, stage action or use of lighting should not only illustrate the music but should be linked with it in real time: the continuous, subtle variations and phase-shifts of the music, as well as its individual layers and elements, are translated into visual impulses. The production, therefore, gains an element of proper synesthesia and allows the perception of time and space to become relative – aptly fitting to the hypnotic music and to the character of Albert Einstein.

To create this unity of music, language and visuals, special software was needed. For this purpose, several shader-applications were programmed. Computing with up to 100.000 simultaneous processes, these shaders gave effects, outlines, shapes and textures to the projected video data – specifically timed to the rhythm of the music. For example, a Kinect camera was positioned close to the conductor’s desk to scan the movements of musical director Florian Helgath during the pieces Train and Knee Play 3. This scanned footage was transformed into distorted 3D-video signals, its visual appearance always paired with the music.

The same philosophy applied to the software developed by Ullrich, which synchronized the text of Knee Play 3 sung by the choir (ChorWerkRuhr) concurrently and autonomously to the outlines and shapes of projected film footage. Furthermore, the visuals of the Bed aria towards the end of the opera (Soprano: Hasti Molavian) were generated only according to volume and intensity of the sound sung by Molavian.

Together with mathematician Dr. Frank Génot, Ulrich developed an OpenGL-Renderer which allowed to project virtual landscapes and point clouds with more than 50 million data points in real time. Real landscapes are scanned by 3D lasers, so that they resurface as virtual point clouds (consisting of so-called LIDAR data sets) by means of enormous processing power. In Einstein on the Beach, virtual effigies of Italian landscapes (in the piece Train) or an apparent helicopter flight through the giant, virtual skyline of the Japanese city Osaka appear – these apparitions are always triggered by the structure of Glass’ music as well.

Fraying out the video image of the conductor, the genesis of exclusively digital cityscapes, as well as the costume design with its evocations of animals or post-human cyborgs (costume design: Mona Ulrich), give a special aesthetic and philosophical spin: the humanist era of being Human is fading. The notion of human art in mechanical, non-human fashion (fitting to the score and libretto of Einstein on the Beach) serves as a common theme in the multi-hour performance. It is also employed in regard to the “Nano Flight” videos. These are filmed sequences of electron-microscopic footage (conception and production: Stefan Diller). A specially coded software reconstructed Diller’s 2D-video material to 3D, based on pixel movements and colour changes. Thus, a virtual camera presents close and crystal clear 3D footage of insect wings or chemical material, exact to the nanometer. This footage, shown in the piece Trial 1, seems concurrently technical as well as organic, concurrently abstract as well as specific.

These video signals are projected onto a big screen on stage, as well as onto six transparent curtains of surgery tubes via front projection. These surgery tube curtains are embedded into the stage. A control interface was engineered by Lucas Pleß so that individual curtains could be moved spontaneously as well as along predicted, programmed directions in the stage’s rails system.

In the 2017 critics poll of Welt am Sonntag NRW, Einstein on the Beach was awarded as the best production of contemporary opera in the state. During its six-week run, it was nonstop on position #1 in the renowned nachtkritik.de-charts. The WAZ newspaper likened the performance to “a perfectly built, non-addictive but tremendously well-working drug trip. It’s a happening that’s diving through countless cultural layers and comes to its end with a wonderful place of rest.”