Imagine you can use your body as the start for a search in a movie. Imagine searching for similar films scenes as Rocky’s victory dance on top of the steps just by raising your arms.

A non-linear way of experiencing moviesFilm, by definition, is linear, the narrative unfolds itself over time. However, using motion tracking methods we can introduce a new way to interact with film-based footage. Rather than surrendering to the screen as a passive viewer, users can penetrate an archive of movies from a new perspective. This subversion of the traditional film/viewer relationship brings unexpected juxtapositions and a more engaging interaction.

The unpredictable nature of physicality brings an element of freedom into an often static and methodical activity. In its most complex form, after an analysis has been carried out on archival footage, the users body position is recorded in real-time, and matched directly with film clips.

Documentation of the Camera Postura project

While films tend to be linear, databases are more unpredictable. Unforeseen associations emerge as one navigates them, recalling the action of strolling across some archive with its associated lot of discoveries. Yet, databases precisely lack this embodied, strolling dimension. Each path within the database is amplified by cinematic expansions of color values and acoustic enhancements that trigger nothing but a fiction of mastery – fiction that obnubilates the actual experience of the archive.

The point of departure was the idea to explore the inherent cinematic quality of the database, while pushing film away from its linear self. By transforming the ‘filmbase’ into a mirror that is responsive to the body as a whole, the experience of the films and the one of their selection are “folded” together. The database is reincarnated — becoming endowed with the archive and cinema’s bodily dimension — while films are endued with the archive’s collage-like structure — jumping across all scenes and shots alike. A posture initiates a search request, a movement a transition between two films and information then reveals itself insofar as it is given a body — the user’s.

NavigatorIn a more functional form, motion tracking can also be used as a navigation tool. You use your hands and body to search through the archive. Essentially you are the playhead, cutting a path through vast amounts of imagery. Context is challenged and new narratives are pieced together from fragments, triggering the mind in new ways.

Examples of different postures

After an analysis has been carried out on archival footage, the user’s body position is recorded and matched with film clips. You make a movement and that movement is repeated in film. These scenes are then augmented with additional information, such as actor names, locations, tweets and film reviews of the movie. Each pose results in different matched scenes creating unique film posters at each visit.

Studio

RNDR is a design studio for interactive media that develops ‘tools’ that are only finished by how they are used.

To achieve this, we develop processes, create structures, design visualisations, code programs, and create interactions. The end result can manifest itself across different media, ranging from interactive installations, data visualisations, generative identities, prints and everything in between – often real-time. We are triggered by how information and technology transforms networks, cultures, societies, relationships, behaviours, and interactions between people.

RNDR was founded in 2017 in The Hague, (NL). Its main members have years of experience as partners, computer scientists, designers, and developers at LUST and LUSTlab.

One of our core projects, and basis for most of our projects, is OPENRNDR, an open source framework for creative coding –written in Kotlin for the JVM– with over seven years of development, that simplifies writing real-time audio-visual interactive software. OPENRNDR is fundamental for the work and software capacity of RNDR as a studio, as it allows us to realize complex interactive works.