Rainbow Resonance

Rainbow Resonance is a computer vision installation

By sofia

17 Apr 2016

Rainbow Resonance is a computer vision installation that generates colors and musical sounds of the equivalent sound frequencies in the electromagnetic spectrum, according to the motion of the participator. The audience is encouraged to participate in a playful performance that engages the body to produce colorful images and sound resonances through simple movements. Anyone can potentially create a personal experience within the installation. The color scheme is based on the visible light spectrum.

Rainbow Resonance at the New York Hall of Science, October 2008, Spring 2009

Light wavelengths are transposed 40 octaves below, into audible sound frequencies and mapped to an 8 tone musical scale based on Pythagorean mathematics. The resulting eight-tone scale corresponds to the “white keys” on the piano, and represents the seven predominant colors of the light spectrum.
An all encompassing composition of the frequencies occurs when the arms extend upwards towards the sky. The movement is reminiscent of the meditation exercise “Carrying the Moon” in Chi Kung, an ancient Chinese method of healing and balancing energy in the body.

User Experience

Rainbow Resonance utilizes technology to engage, entertain and empower the participant by enabling them to playfully interact with a software mirror image to produce sounds and visuals.

A lingering sound invites the visitor to enter the space. Upon entering the installation area, the participant generates his/her mirror image mapped to the colors of the rainbow, projected on a life-size screenwall. Simple motions generate harmonic intervals. Movement along the vertical axis (upward or downward) iterates through the color musical scale, while movement along the horizontal axis (left or right) or the z-axis (forward or backward), adds richness to the sounds with additional chord variations. An all-encompassing composition of the frequencies occurs when the arms are extended towards the sky. The movement recalls the meditation exercise “Carrying the Moon” in Chi Kung, an ancient Chinese method of healing and strengthening the body, mind and spirit.

Technology

The Microsoft Kinect 3D motion controller is used to detect motion in space and motion tracking processing maps “zones” of movement in virtual and 3D space. The “zone” with the most vivid motion triggers the sound generated by the participant. Artificial intelligence algorithms are utilized to visualize and sonify the participant’s movements. MAX/MSP/Jitter interprets body movements into a mapping of color and sound.

Musical Sounds

Rainbow Resonance sounds are inspired by the ancient Pythagorean symbol of the Tetractys. The tetractys consists of the first four numbers 1, 2, 3, and 4 which reveal the primary intervals 1:2 (musical octave; geometrical proportion), 2:3 (musical fifth; arithmetic proportion), and 3:4 (musical fourth; harmonic proportion), upon which musical harmony is based.

In ancient Greece the Tetractys was a metaphysical symbol that embodied the principles of the natural world, the mysteries of the divine realm and the harmony of the cosmos. The sacred Tetractys informed color and music theories prevalent in ancient Greece.

The top first three dots represent the threefold White Light, containing all sound and color.
The remaining seven dots represent the colors of the spectrum and the notes of the musical scale.

Chi kung (Qigong) – A healing exercise

Chi kung was discovered when prehistoric man learned how to manipulate breath, as a form of cosmic energy, used for various purposes. By 2700 BCE chi kung had become an important aspect of chinese medicine, according to Chinese records. The earlier type of chi kung was probably a form of meditative dance which encouraged energy balance in the body. Chinese martial arts, especially Shaolin Kungfu make use of chi kung to enhance fighting abilities.

“Carrying the Moon” is an exercise that focuses on enhancing youthfulness by keeping the spine young. Carrying out the exercise daily enhances youthfulness and health at 60 and beyond, strengthens the kidneys and enhances sexual vitality.