The Data Sensorium is a multidisciplinary initiative bringing together scientists and artists from Stony Brook University and Brookhaven National Lab to explore how to sort, categorize, interpret and analyze data. This data, especially the massive scale of data collected on a daily basis, is one the problems of our era—we have far more than we can decode and unravel using traditional methods. This introductory colloquium will showcase different models for interpreting data; models that use a multiplicity of senses—optic, haptic, sonic.

One of the large projects we are starting this year is the “Data Sensorium,” focusing on the promise and problems of large data sets, working on ways to make them more useful by visualization, sonification or the addition of haptic feedback. cDACT was recently awarded a Brookhaven National Laboratory/Stony Brook University Seed Grant for this project. Our “October Event” is a kick-off for this year’s activities: a colloquium with four speakers discussing their work approaching large data sets, from different points of view.

The Data Sensorium Project addresses one of the central problems of dealing with digital information: that of large data sets. Research has shown that parsing of some sorts of data is more effective with the integration of multiple senses. Thus, the Data Sensorium project considers how best to investigate large data sets: with visualization, sonification, and/or haptic feedback. One unique aspect of this Project is the collaboration between artists and researchers – it is our contention that a trained, experienced sound artist, used to dealing with abstract sonic information, can best guide the design of sonic representations of massive data sets; a similar situation would exist with visual artists and images. We also hope to serve as a clearing house/nexus/generator of and for collaboration.

The event will feature presentations from the following guest speakers:

NYC Based Software and Visual Artist
Scott Draves is best known as the creator of ElectricSheep.org, a distributed art initiative with over 450,000 participants worldwide. Draves’ work is currently on display at LACMA and has been commissioned by Carnegie Mellon University and the state-of-the-art Adler Planetarium in Chicago. Draves’ work is permanently included on MoMA.org as part of the Design and the Elastic Mind exhibit. He won an honorable mention at Prix Ars Electronica in 1993 and his work has garnered notice from such competitions as Lumen_Ex in Extremadura, Spain; the Genetic and Evolutionary Computation Conference, Montreal; ACA Media Arts Festival, Tokyo; Life/Vida Madrid and File Prix Lux in Brazil. He has an undergraduate degree in Mathematics from Brown University and a PhD in Computer Science from Carnegie Mellon University.

Algorithms at Attention and Mental Map
Human attention is subjective and ineffable as we create it. But squeezed through a user interface and collected on a large scale it becomes data, and hence fodder for algorithmic processing. We will explore three rather different takes on this idea: Electric Sheep, Rich Cartograms, and the Video Riot. In the Electric Sheep, a cyborg mind composed of 450,000 computers and people creates an infinite abstract animation by attention-directed evolution with a genetic algorithm of a large population of designs. In the Rich Cartograms, attention is collected and used as a lens to create maps that match our mental scale system where size stands for importance. Finally, we relate YouTube and its culture of viral memes to a Nam Jun Paik inspired interactive “Video Riot” processor. After this survey we look back to discover what this tells us about what attention is, and ourselves.

Audio Systems Coordinator and Developer KAUST Visualization Lab
Zachary Seldess, a composer, media artists, teacher, and programmer, is the Audio Systems Coordinator and Developer at King Abdullah University of Science and Technology. He also works as programmer and teacher at Harvestworks Digital Media Arts Center in New York City. Zachary is co-director of the New York City Electroacoustic Music Festival.

Immersive Audio and Auditory Display within Multi-Media Systems

In this talk, Zachary will discuss the potential roles and applications of audio within visualization contexts, beginning with an examination of dimensionality and interplay between auditory and visual perception in media systems. Zachary will explore how spatial sound and auditory display can enhance visual-spatial perception, increase fluency in search and wayfinding, and generally improve on single-modal approaches to exploratory data audition and analysis. A selection of ongoing sonification/visualization research projects from the KAUST Visualization lab will be presented.

Chris Bregler is an Associate Professor of Computer Science at NYU’s Courant Institute and director of the NYU Movement Lab. He received his M.S. and Ph.D. in Computer Science from U.C. Berkeley and his Diplom from Karlsruhe University. Prior to NYU he was on the faculty at Stanford University and worked for several companies including Hewlett Packard, Interval, Disney Feature Animation, and LucasFilm’s ILM. His motion capture research and commercial projects in science and entertainment have resulted in numerous publications, patents, and awards from the National Science Foundation, Sloan Foundation, Packard Foundation, Electronic Arts, Microsoft, Google, U.S. Navy, U.S. Airforce, and other sources. He has been named Stanford Joyce Faculty Fellow, Terman Fellow, and Sloan Research Fellow.

Crowds and Clouds and Motion Capture

This talk will cover several research themes centered around the use of vision and motion capture for animation, analytics, and gaming. This includes human movements as diverse as subtle eye-blinks, lip-motions, spine-deformations, human walks and dances, sports and street games. This is demonstrated by sub-pixel tracking in Hollywood productions, reading the body-language of politicians, visualizing the pitches of NY Yankees Mariano Rivera, crowd-sourcing pose spaces from Academics as well as fans from a Dutch progressive-electro band, the production of the largest motion capture game to date and its reception in various cultures.

Working across media and disciplines, Brooke Singer creates platforms for local knowledge to connect, inform and conflict with official data descriptions. She engages technoscience as an artist, educator, nonspecialist and collaborator. Her work lives “on” and “off” line in the form of websites, workshops, photographs, maps, installations and performances that involves public participation in pursuit of social change. She is Associate Professor of New Media at Purchase College, State University of New York, a fellow at Eyebeam Art + Technology Center and co-founder of the art, technology and activist group Preemptive Media.

Data Visualization: Strategies for Building Interfaces
Digital technologies have spurred an explosion in data collection, but these same technologies also allow people to wrangle, make sense of, represent and redistribute the data too. I will discuss strategies for building interfaces to the data stream and what artists specifically have to contribute. As an artist and educator, I use data visualization to create platforms for local knowledge to connect, inform and conflict with official data descriptions.