I’d like to raise the question of the extent to which a new documentary technique or form might document itself as opposed to the subject of the document. I’ll illustrate this question specifically within the context of new technological means, but it may warrant consideration in other forms of new documentary not facilitated by technology, such as the re-enactment of historical performance art already blogged at this event. Can the form used to document an event result, to whatever extent, in documenting itself?

The Oxford English Dictionary traces the root of the noun ‘document’ to both the old french and latin words for ‘lesson’ [1]. A lesson, itself defined as ‘the action of reading’ [2], does not pretend to be an objective record. The perspective of the teacher and the biases of the teaching techniques are implied, if not unavoidable. What traces do these perspectives and biases leave on the completed document? Given the benefit of historical hindsight, might these traces overpower the actual subject of the document?

To explore this argument I will begin by pilfering examples from a site already mentioned on this blog. The New Aesthetic, a tumblr blog identifying the incursion of digital perspectives into every day life, has already received an introduction by Caleb Kelly. I am not so concerned with the articulation or identification of said ‘new aesthetic’, as I am in understanding how the digital medium might affect how we perceive the world outside of that medium. The New Aesthetic blog provides a few salient examples. These examples throw light on the question: Has the digital medium affected how we perceive the subject being documented? And if so, would the document not act as a record of the documentary medium itself?

The below image of a blurred photograph on a billboard casts the viewers gaze … placing it behind the lens of a digital camera positioned within a speeding car. This is what a photo would look like if taken with a portable digital device pointed out the window of a moving vehicle. It is a captured vision characteristic of a ‘smart phone’. Within the context of a ‘smart phone’ the image represents a poor photograph of scenery. Within the context of a billboard, the image can represent the medium of portable digital devices. Is this work by Ben Long a document of random scenery produced in transit? Or is it a document of the documentary form of portable digital photography?

A Flickr search for Broken Kindles sparks a curious fascination with the aesthetics of the mechanical breakage of e-ink screens. Geometric patterns interfere with images in random yet occasionally intriguing ways. A glitch moment, certainly.

Having experienced a broken Kindle screen myself, these images remind of the moment of collapse of the (perhaps not so) futuristic idealism of e-readers. A printed text would not cease to be readable upon such a minor incident. These images offer a compelling aesthetic, but they also document some of the fundamental differences between digital text readers and printed texts. The screens are fragile; they break in such a way as to reveal some sort of underlying cartesian structure based on rows and columns. Perhaps most importantly these images represent an infuriating interruption to the engagement with a text, a kind of rude awakening to the trade-offs imposed by digital devices. The devices solve certain problems, but introduce entirely new ones foreign to printed texts. The interest in the aesthetics of these broken screens is paralleled by their action as documents of certain characteristics of e-readers.

Google maps, a detailed document concerning the sub/urban landscape we live in similarly documents itself. Google has a legal requirement to obscure the face and thus identity of passers-by. The image on the right shows a painted mural in which certain faces have been blurred, and others have not. An examination of this image forces the viewer to consider how the face detection algorithms might function. We thus attempt to cast our eyes into those of Google’s algorithms, trying to understand which facial characteristics of the non-blurred faces have escaped the face-detection logic. In so doing, we engage with Google Maps as a document of itself.

This task takes on an other dimension when considering this next image. Flowers in a window box have been blurred. It is evidence of some form of underlying stupidity in the system. Google Maps, the document of our sub/urban environment, is now documenting the chaotic inconsistency of its underlying digital detection algorithms.

I see these artefacts as evidence of the characteristics of particular mediums: as documents of those mediums, documents of documentary forms. These artefacts teach us about the nature of those mediums; how they frame the world. Currently, the artefacts described above hold much aesthetic interest, and for good reason. But I wonder if, with the passing of time and changing documentary forms, these artefacts might hold a significance equal to the documentary subject, in that they reveal aspects of the world view of the documentary technique.

One of my pet life-projects is testing the postulate that we go about our lives perceiving ourselves through the eyes of others. Its a physical absurdity, of course … might be better to say that we perceive ourselves through how we think others perceive us. The point being that when we gaze at someone else, a part of that gaze is trying to understand how they are gazing back at us.

For my thesis, I’ve been researching notions of the illusion of reality, including spatial realities… and how they are achieved technically. In the visual arts, Velasquez’s Las Meninas harbours voluminous discourse on this topic. I havn’t gone too far into the analyses of this painting — nor do I really want to — but cripes almighty its full of people gazing at others gazing back at them. Let me try to elaborate. Note: the King (Philip IV) and Queen of Spain are the eyes of the painting and they can be seen in the mirror in the background.

The painter is looking at the subject of his painting: the King & Queen. — who are looking back at him. So the Painter is actually painting himself through the eyes of the King and Queen

The mirror in the back is apparently not reflecting the king and queen’s image, but rather their image as the painter has painted it (as per a paper I couldn’t be bothered referencing). In other words, the King and Queen are seeing themselves in the eyes of the painter.

One suggested central subject, the daughter at the centre of the image is looking either at her parents, or the painting. If she is looking at the painting, then she is looking at a portrayal of her parents looking at her.

The same could be said for the gentleman in the back door way. As I said, I dont know exactly how this painting is critically interpreted, but it documents at least 4 instances of people looking at themselves through the eyes of others. I’m not suggesting that this is what the painting is about. It might be one of those paintings that has an inherent complexity that caters for whatever perception the viewer might happen to be inclined towards. One could say that my interpretation of this painting is more a reflection on me, than on the intent of the painter.

Either way, I’m starting to understand why this painting is considered important. Its thoroughly engaging. How does he do that?

Serial composition — or the twelve tone technique — is considered one of the most important aesthetic developments in 20th Century compositional thought. What bowls me over is the banality of the technique. In its most generalised form, as devised by Arnold Schoenberg, it is this: Take all 12 notes in the chromatic scale, organise them into any order (a series), then stick to that series for the whole composition.

That’s it … that’s the crux of the twelve tone technique. What it amounts to is: Here’s an outrageous limitation, now see what you can come up with. Unlike other compositional methodologies which provide guidelines for ‘what works’, the serial technique doesn’t help, it challenges. The composer is challenged to find something that works, given a set of imposed limitations. In other words, it is a constraint system, and its success clearly supports the notion that restriction can lead to creative liberation.

What is interesting is that technology is also a form of constraint system. In the essay entitled “The Question Concerning Technology”, the 20th Century philosopher Martin Heidegger characterises technology not as ‘equipment’, not as a means to an end, but rather as something that holds biases. Engagement with it, such as using it to create music, exercises those biases; that is, it reveals certain things in certain ways and it hides certain things in certain ways. Dreyfus, noted Heidegger scholar, paraphrases Heidegger’s warning about the danger of technology as “a restriction in our way of thinking” (1997, p. 99).

A restriction in our way of thinking. It is a phrase that just as well describes serial composition. Serial composition restricts how the composer thinks about music. Through that restriction there is a form of creative liberation. Of course, Heidegger’s writing on technology did not consider technology’s restrictive tendencies when engaged by opportunistic creative exploration.

Yet, through the use of technology (that not designed to create music), musicians have found creative liberation. The turntable is a fine example. To the uninitiated, the suggestion that the humble turntable can be used as a musical instrument of great creative potential must seem ludicrous.

I’m arguing that if the turntable had been designed as a musical instrument, I’d guess that it would have included all sorts of knobs and sliders and other controls to ‘frame’ a much broader set of musical possibilities. Yet it is precisely the narrowness of possibilities, the lack of controls, that have led to a creative exploration now iconic in contemporary musical styles. The turntable is a technological equivalent of serial technique.

The difference between the turntable and serial composition is that the turntable was not a designed constraint. It is an accidental constraint. It is a constraint imposed by using a technology for doing things it was not intended to do. The serial technique is a consciously designed limitation. For me, this highlights two points.

Arnold Schoenberg was a very smart man,

Perhaps music technology (software or otherwise) should focus not on empowering musicians but rather on restricting them. Interestingly, this is a notion that is contradictory to the ‘equipmental’ conception of technology. Adhering to Heidegger’s conception that technology is a form of bias … the design of technology to creative ends should aim to design that bias, and that includes designing the restriction — instead of allowing the restriction to be ‘the stuff that wouldn’t fit in’. It is this restriction that will both drive creativity, but also ‘colour’ the aesthetic. I suspect design ‘biases’, in technology, is a difficult thing to get right.

Karlheinz Stockhausen, considered an important pioneer in electroacoustic composition, predicted in 1977:

space will become as important as pitch in the traditional music (Stockhausen cited in Worrall 1998)

Despite increasing technical means the role of space in music still requires exploration … at least in academic / classical / electroacoustic / art-music circles. One could easily argue that Stockhausen’s prediction has not materialised. Some, such as Emmerson (2007, p. 143), are unconvinced that having the ability to produce almost-real 3D soundfields is an ideal musical aim at all. Of course, Stockhausen hasn’t clearly stated whether his notion of ‘space’ in music involved the realistic simulation of 3D sound, or whether it was just the use of independent spatial attributes like reverberation or location. One thing that is clear is that much of Stockhausen’s explorations of space involved cartesian location … the manipulation of the direction of origin of sounds.

I would like to argue that Stockhausen’s prediction has come true. Space has become as important as pitch. Although it may not be in the form he had envisaged it.

Let me explain. Location is just one aspect of spatial audio. In the stereo field, location is known as ‘panning’. Other aspects of spatial sound include reverberation, delay (which is echo), volume changes can simulate distance, low pass filters can simulate both distance and obstructing objects, stereo width simulates the physical size of the sounding objects etc.

All these aspects are very much present in contemporary music produced in Digital Audio Workstations like Protools. Typically, they are used to help perceptually separate individual sounds to achieve a clear and well balanced overall music (this is called auditory-stream-segregation in the scientific literature). Those who do this well command my respect. Producers do this. Whilst they may not consciously realise it, producers are, to me, spatial audio practitioners.

Some not only do it well — that is: the sounds are clear and balanced — but they even use spatial attributes to sophisticated compositional effect. Aphex Twin (Richard D James) is an example here. Sure, Aphex Twin is dance music, it is not concert music like Stockhausen’s work. It requires a different listening mode.

Now have an other listen to the section from 4:20 to 4:60. Here, Aphex Twin builds tension and releases it, over a series of clear steps, by using spatial devices. Here is a summary of the steps … these steps can effectively be considered a spatial audio score;

All reverb is stripped, to create a dead, centred sound.

He throws the sound into a small room (I get a back of the head sensation). This creates a kind of acoustic tension, enclosing the listener into a confined space with claustrophobic characteristics.

Spatial tension is released by applying a very deep and long reverb that throws the sound way out in front (of the listener). Here the reverb acts as release. One could say that the reverb is also the ‘release’ of sound in space.

The beat comes back — return to subject — but it sits exactly between the listener, and the far-in-front reverb.

Within this small series of steps, a compositional gesture has been created with very little reliance on pitch, harmony, rhythm and timbre. Spatial audio is the predominant attribute. This may not have been Stockhausen’s conception of space, but in this piece, and in much of Aphex Twin’s work spatial attributes are as important as pitch. The two key points, here, are that a) contemporary dance music is delivering Stockhausen’s prediction, and b) working in mere stereo does not at all preclude the composer from exploring spatial audio.

Ambisonics, a technology used to create spatial audio experiences, is bad technology for the home (and most public performances). Before arguing this point, let me define how I am judging technology:

A technology is only as good as the experience users have using it.

If someone were to argue that ambisonics is potentially good technology, then I would agree. But there are so many ways that ambisonic playback can be compromised that it is more than likely going to result in a poor spatial audio experience. It is the experience that the user actually gets that counts… not what the technology is theoretically capable of. And that is where ambisonics fails dismally. It is a fragile technology that insists on a slew of preconditions.

Here is a list of just some of the things that can go wrong:

The user cant easily find/install a software decoder

When the user does eventually install a decoder, there is no guarantee that it is a proper one (includes shelf filters etc.)

Connecting one’s computer to a multichannel sound system can be problematic

The user’s speakers may not be matched

The user may not be sitting in the centre of the speaker array

The acoustic of the playback room may interfere with the ambisonic image

Compare all that to the experience of a high def flat screen TV. Buy it, take it out of the box, turn it on. There it is. For ambisonics to become good technology, it must achieve a high level of consistency of quality experiences … not just have the mere capacity to deliver good spatial audio.

I’ve sat in concert halls that are particularly reverberant. Seeing an array of speakers my first thoughts often concern how the reverberation of a hall will interfere with the spatial image. Recently I was surprised when the spatial imaging was far better than I had anticipated … I later learned that spatial location had been implemented in VBAP (Vector Based Amplitude Panning), an alternative spatialisation technology. Here’s an example of how a technology with far less potential can actually be more robust and therefore ultimately deliver a better experience. I would say it is a better technology.

I think the future of ambisonics is in fixed installations in concert halls; where the installation can be tightly controlled, and doing head-tracked HRTF decodes on portable media devices. Both would allow ensuring a consistent quality experience. Making ambisonics become good-technology in the home is possible, but I think it would require all sorts of funky automated things such as speakers that know where they are relative to each other; built-in mics that can measure and compensate for room acoustics; and mechanical robot devices to push listeners into the sweet spot when they’ve inadvertently decided not to sit in the middle of the room!

As the integration of technology, into the arts, matures I see a pattern forming. Technology does not give us access to things other-wordly. It does not expose objective realities about our material world that help us understand it in a new light. Rather, it allows us to see ourselves in a new light.

But this isn’t my key point. This new way of seeing can then stand independently of that technology. So technology opens a door that it didn’t create. It just opened it.

Consider Chunky Move’s Glow (2010). A dancer’s form is analysed by a computer, interpreted as ‘outline’ then re-projected into that dancer. Add a bit of digital delay to the imagery and what you get is a kind of human spirograph. The inherent geometry within the dancer’s organic movements is extracted and displayed. The computer’s vision acts as a kind of partner dancer, perfectly in-sink yet utterly different. Its a relationship that gives the dancer’s movements a transient permanence that reveals both a pattern of movement but also the frailty of human physical inconsistency.

Now consider the work of artist Tony Orrico. Penwald Drawings (2011) take a leaf out of Chunky Move’s appropriation of technology, but delivers it analogically. The drawings take a similar form. BUT the movements of the dancer are significantly different. Without the aid of a computed digital delay and re-projection the dancer must himself physically perform much repetition. Here, the human form is cast into the iterative repetitions natural of computed logic. When considered in the same breath as Glow, this is an example of the human form mimicking technology’s eye appropriated to reveal human form. To produce this work, the dancer must cast themselves into a formalisation characteristic of machines.

My hypothesis is that this work of art would not have materialised without technology… even though there is no technology involved.

So whilst technology is *not* involved in Tony Orrico’s work, it remains central. I think we will see more of this. Technology’s way of seeing will infiltrate the non-technological arts to create a kind of post-digital practice; where what the digital medium has exposed remains but without the digits. One characteristic of this, I suspect, will be the human simulation of the iterative and repetitive tendencies of technology.

Tony Orrico’s work also reveals something very different … through repetition, the frailty of human physical inconsistency is no longer the subject of the work, instead providing the work a textural depth that reveals a kind of knowing commitment that geometries will appear.

[Addendum: would I be right to suggest that photography had a similar effect on painting? Photography introduced a new way of looking which then infiltrated how painters worked? If so then it might be better to ask specifically; how does contemporary technology differ from older technologies? Maybe, in essence, it doesn’t differ from older technologies. At which point one might say that art is about finding new ways of looking, and technology serves as just one means for discovering new ways of looking…. in other words … there is no special relationship between art and technology]

A little while ago, a coder by the name of Kyle McDonald put together a bunch of code and libraries that enabled mapping photos of different faces onto one’s own face.

The question being, ok but what can you do with it. What is the work of art, what insight can be drawn, what new way of looking at the world can be exposed. As Heidegger might say, what truth is being un-concealed?

As it stands, I dont think it is much more than “look what you can do these days with just a camera and some well crafted code”.

BUT … yesterday I came across a project that had taken this code, and with just a little bit of extra work, had managed to un-conceal a little truth. This was done in a way that very much echos Jaron Lanier’s take on virtual reality. I quoted Jaron in that blog post I just linked… but I’ll quote him again here:

The most important thing about virtual reality isn’t the idea that you’re seeing this dramatic 3D thing. It’s that you, yourself change. That you experience yourself in a different way than you ever have before

Look at that video, above, again … Kyle McDonald is pulling faces trying to, in passing, imitate the kinds of expressions that one might expect from the different faces he uses. So for Paris Hilton he puckers up, for Brad Pit he swoons, etc. Jaron Lanier’s point is not that Kyle McDonald can look like Brad Pit, but rather, that Kyle McDonald behaves differently when Brad Pit’s face is mapped on top of his own. That’s what Jaron Lanier is saying is the power of virtual reality … it changes the way you experience yourself.

This is exactly what the project Apesnake Photobooth has recognised. They have taken the leap from displaying images of the photo overlay, to displaying photos of the individuals *without* the overlay … that’s where the meat is. These photos are then automatically uploaded to a Facebook group. What is revealed is random individuals acting like apes (the face overlay is of an ape).

There are lots of things that could be said about this… lots of things that are revealed. One of them is about inhibitions. By mapping an other face ontop of one’s own face, and seeing only that mapped face … one behaves in ways that one would not normally behave in (that’s bang-on what Jaron Lanier expresses). There’s something about self-perception in there too. There’s something about social norms.

I think this project is worth evolving. Most of the facial expressions people are using involves opening their jaw as wide as possible. I’d love to see something that is more generic, or more open to individual possibilities. I’m wondering if a clown face might have potential … because of the possible happy/sad ambiguity.