Tag: synesthesia

I ran Qi again last night on the walls of Kennedy Theater at UH Manoa as part of the preliminaries to the fiftieth anniversary celebration/performance of the dance program. Here is a short video of a few of the people who used it. Don’t miss the little person at about 2:40. I didn’t ask permission from the people in here and don’t know who they are, so if you see this and you are in it and don’t want to be, let me know and I’ll edit you out.

Performance can take some interesting turns. Performativity comes from the idea that your performances and utterances do not just say something but do something in the world. The past two Friday nights, as part of the preliminaries for the fiftieth anniversary dance concert at UH Manoa’s Kennedy Theater, dance professor Kara Miller invited History professor Richard Rath to set up his interactive motion-to-music installation “Qi” projected on an outside wall of the theater before the show on two of the Fridays of the show.

In it, dancers, cars, and passers by “do something” with their motions, namely make music. The results were projected on the side of Kennedy theater and music made by the audience filled the air outside the theater the two nights of the installation. A couple of ideas are at work in the installation in conjunction with performativity. One is that music is often synesthetic, transforming actions in one sense modality, say vision — as in sheet music — or touch, as in the fingers on a guitar, into another, hearing. In this case the transformation is somewhat direct, as music gets made by moving the balls of green energy, the “qi,”around the screen with one’s motions in order to make music. The installation is also meant to break down the distinction between audience and performer, as often times people discover the instrument through being in the picture on the wall that they are looking at and hearing their motions come out as music. In other times and places, especially before the advent of recorded music, music making was something that everyone participated in, without the formal distinction between audience and performer. Qi points to the artificial nature of that divide while looking forward in new ways rather than back to some romanticized version of the past.

Two highlights that I did not capture were a little girl about age ten who just had a blast with it, and an older couple who did Tai Chi, which worked really well. I did not get video of either though.

There has been a great series of articles on motion to music controllers the past week or so over at Create Digital Music. All of them are cool, but they require some extra piece of hardware like an iPhone attached to your wrist or a Leap Motion, or a kinect. Qi is a max/msp patch that I wrote using the cv.jit computer vision modules. the only hardware is the video camera that comes with the laptop (ok, I used a cheapy USB cam so I could aim it away from me, but the onboard cam works too). People seem to get the idea immediately and have fun doing it, although the occasional detached arm waving in the picture is me giving the two second tutorial: “up for higher notes, down for lower, different synths on left and right sides, bigger circles=louder.”

Just a reminder that my motion-into-music contraption will be making music as long as somebody is moving outside of Kennedy Theater before the performance tonight. UH Manoa. 7-8 PM. Outside stuff is free. If you are around come by. Last week was a blast, come by this week if you can. To get an idea of what it is, see the videos (note to self: smile next time, its fun).

My long term motion to music software, now named “qi” is going to be available for everyone to move to and create music on Friday November 15 and 22. It will be part of the dance program’s 50th anniversary show preliminaries, projected on the wall outside Kennedy Theater starting about 7 PM until the show starts at 8. This Friday and next. Hope you can make it! As usual, facebook people need to come to way.net to view the video I think. Qi motion to music demo

Synesthesia is the transference of one sense to another, such as seeing sounds. I have a theory, that has been provisionally working out, that during times of flux in the senses, such as when the possibilities of a new medium are first being fully explored, synesthesia becomes more prevalent. If you can see what I am saying, you have one version of it, though cogsci types try to seperate metaphors from the actual “affliction.” For some reason, it usually maps sound onto vision, but I have instead mapped vision onto sound with PRISMM.

Here is another video of PRISMM, this time it is just me. If you missed out last time, this is a max/msp/jitter patch I wrote that makes music from motion. Up and down is pitch, left and right changes the mix of the two synths. Let me know what you think!

Synesthesia is a favorite topic of mine. It is the transference of one sense to another

Arrived in Kolkata after 35 hrs of flying and waiting in airports…spent most of the day watching web pages load letter by letter on my thrilling 56k web connection. Aah, longing for the good old internet of the 90s. Monisha got back about half an hour ago with a 1mbps modem to replace the other one, so I am a bit cheerier.

As promised the other day, here is a very poor quality video of the people’s revolutionary insurrectionist synesthetic music machine, or PRISMM for short. Everyone at the Revolution Books show had a grand time with it. This section is set to a whole tone scale (a spacy thing with no tonal center that Claude DeBussy favored) and uses the piano sound only. Moving the green blobs up makes higher notes, down for lower. The bigger the green blobs, the louder they are. Moving them left and right moves them in the stereo field. And here is a music clip that I made with it using a pentatonic scale and a more complex mix of synth sounds. If I get time, I’ll spring it on some folks in Kol, try to record the output, and post that.

I’ve been thinking about a name for it for a long time, thus:

Whereas it takes people moving, sometimes as little as an eyebrow, for it to work, and it works whether they want it to or not, thus preventing wallflowerism and related ways of non-participation, and

Whereas it got its public debut on the 12th at Revolution Books, and

Whereas adding “insurrectionist” to the name adds to the general gist of of the thing, and

Whereas adding said word causes the acronym to be nearly meaningful, and

Whereas it is a vision-to-sound sensory translator (i.e. a machine that performs synesthesia), and

Whereas all the possible names with variants of synesthesia in them sound hopelessly lame, and

Whereas the results sound vaguely musical, and

Whereas computers and max patches are basically machines, and

Whereas the resulting acronym nearly makes sense in the context,

therefore

Be it resolved that henceforth the max patch I spent a semester working on instead of writing like a good historian be so named: the People’s Revolutionary Insurrectionist Synesthetic Music Machine, or PRISMM for short. I figured I better let it out of the bag because pretty soon MS Kinect and the like will be all over this idea, but I’ve had this working since 2007, so remember where you saw it first!

PRISMM works on an object oriented music and vision platform called Max/MSP/Jitter. The patch, as max constructions are called, is only slightly more complex that the Tokyo public transportation system.

I’d love to compile a version of PRISMM to distribute, and in theory it is easy, but the patch relies on some non-standard jitter parts that no one is maintaining anymore on a PC platform, and so I have been unable to package it up as a standalone yet. There are surmountable technical problems, so if anyone knows how to compile (not write) a max extension in C++ so that it will run as a dll under windows, please let me know in the comments or else contact me.