Some Thoughts on Interactivity

In The Language of New Media, Lev Manovich explores the topic of interactivity with new media objects. I have attempted to summarize his claims concerning this topic:

Contrary to her impression upon use of the new media object, the user is not a co-author; she is instead forced to follow a predetermined path, stripping her of agency and, eventually, her ability to think for herself. The user is allowed to select chunks of content, offering her the illusion of interactivity. The only interactivity that is occurring here, however, is that of the utilization of the user’s cognitive output as she uses the structure of the program as a program input. The user, through this structured selection, sees the program as fully customized and reflective of her personal preferences and ideas, assuring her of her uniqueness, and thereby supplanting her need for personal associations with hyperlink associations generated by the program that are then accepted by the user as an externalization of her own thought process. The user then learns to prioritize selection within the context of any program over that of personal evaluation, and the line between information access and psychological engagement is blurred, making both navigation and immersion difficult and leaving the user dependent on the program for navigation as well as for providing the path to a finished creative product that once would have been the result of her own psychological engagement.

My purpose is to explore and also to contest these ideas in light of personal experience with and knowledge of a few current new media objects.

A new type of interaction with new media objects has begun, with applications that perform a task so specific that the act of simply activating them is a declaration of intent. Instead of searching and directing our own navigation on an all-purpose search engine, we search a library of applications in order to find one that will navigate these types of queries for us. We navigate the world of many mini-navigations. Instead of trying different combinations of keywords in search of the phrase that will yield “good lebanese restaurants within 20 miles that are within 5 miles of a Target” we search for “iPhone applications restaurant locator augmented reality,” download it and then activate it by touching the Yelp! icon when we are in need. In the most absolute sense, the user is following a “pre-programmed” course of “objectively existing associations” (61)–in fact the only user input here (once the application has been downloaded) are the actual choice to activate the app and the automatically determined GPS coordinate location of the user.

In this way, the user’s direct engagement with the application is structured and objectively orchestrated, but on a different level, she is an agent. Yes, she has selected a certain group of smartphone applications from a library or database that she has somehow decided will serve as her tool set for performing daily tasks, thereby literally manifesting Manovich’s ‘selection logic,’ but she has consciously chosen the structure of her tool. In the case of Yelp!, she has chosen an application that uses the method of collective filtering as the primary structural element directing navigation; in choosing, this application she has also chosen collective filtering as her own method of addressing the task at hand. In this case, the method of collective filtering is transparent–it is the marketed feature of the application. Niche-use applications such as Yelp! compete for users not based on what they do but how they do it. Featuring the way by which the program sources information has become essential, as the usefulness of an application is dependent not on what it does, as the user is assumed to have already bought into this concept, but on how well the method of doing what it does works. There is a clear goal that the program is designed to reach and its ability to do so relies on its method. The user chooses to outsource navigation of a specific kind to a program that uses a certain method for anticipating the user’s desires and access/engagement needs. The user could have chosen other applications serving the same information access purpose that run on completely different but equally apparent predetermined methods of data processing. Other niche-use applications, such as Pandora, rely on algorithmic analysis of program data combined with user-history filters that generate “suggestions” based on certain matches of specific data characteristics–i.e. “if you liked this, you probably will like this similar media object.” Thus, in consciously choosing her method, the user is providing for at least the possibility of conscious cognitive synthesis, considering the output of an application one factor in a much more complicated network of associations, interpretations, and information leading to the formation of a decision.

Given my lauding of the agency inherent in the opportunity to select an application based on its method of processing data, it is easy to assume that I address only applications with a single purpose and scope. This claim of agency in transparency of method is problematized by applications that have developed multiple functionalities with different methods of processing and presenting data as well as different methods of structuring interaction amongst the different functionalities. UrbanSpoon (also a restaurant locator) is an application with the singular function of providing access to an navigation of information. Other applications, including now Yelp!, have developed, often through a series of ‘upgrades,’ additional functionalities including that of social and psychological engagement. In Yelp!, the user can review restaurants herself as well as accessing the reviews of others, and, using the GPS coordinate location input, can view other users’ current locations. The user now has the ability to initiate a real-time online chat with other users. This is often used to initiate online conversation with a user at a restaurant or bar with the intent of requesting current information about the restaurant such as how crowded it is or if the specials are any good that day. This function can also be used to organize ‘spontaneous’ get-togethers of friends who happen to be in the same area near a favorite location or even as an advanced online dating tool, allowing users to view one another’s profiles to determine compatibility before initiating chat and utilizing mutual knowledge of geo-location to orchestrate a meet-up.

While these applications may not encourage the same conscious choice of method on the part of the user, the user still forms the personal associations particular to internal cognition and agency. Manovich’s notion that “before we would look at an image and mentally follow our own private associations to other images, [whereas] [n]ow interactive computer media asks us instead to click on an image in order to go to another image” (61) excludes the possibility that personal associations might be interwoven with the hyperlink associations generated by the program. Upon activating Yelp! while driving, the user might see highlighted a Moroccan restaurant coming up on her left. This result may be highlighted as a result of her past frequently high ratings of Lebanese restaurants and of the consistently good ratings this restaurant has received from other users or even of the presence of a friend at the restaurant at that time. However, the user is by no means guaranteed to unquestioningly turn left and enter the restaurant and enjoy the food. The user may have a particular dislike of Moroccan food or a desire to eat alone. She may even choose to disable the app in order to listen to a podcast of which a personal association with Moroccan food reminded her. The restaurant app did not anticipate, nor does it benefit, from this kind of tangential personal association. The limited purview of many new media applications may actually reinforce the user’s consciousness of the role of her personal evaluation, as well as an awareness of her intent in using the application, whether for navigation and access or psychological engagement or both. The user may find psychological engagement with another user via chatting on Yelp! difficult, but it may be the possibility of access to real life initiation of psychological engagement with fellow restaurant-goers for which the user has selected the application. The user may fully intend to use the information gained from this application to generate and control an immersive psychological experience with another person during which she would be assessing compatibility based solely on the interaction itself and not on the person’s user profile. The user’s choice in managing and layering the influence of the program data and her parallel real life informational input in order to create a mixed reality represents a different type of user agency

This user has selected an application and will process the application’s output in a way that utilizes personal associations and occurs as some form of internal cognition, but through what process does the user do so and given this what is the user’s output? The application, here Yelp! again, is a hypermediated environment in which the user has agency through the act of remediation. She imports the application’s output, separating the spatialized augmented reality model, the user reviews, the real-time interactions, the representation not only of physical locations but also of possible physical experiences (the implied act of eating, enjoying entertainment, etc.), and then processes it in an internal cognitive space in which personal associations augment and complicate the output which has become user input and then eliminate, translate, link, problematize, resolve, analyze, and refashion this complex media input and her associated information. The result is an action or decision that is much larger than the output of the predetermined structure of the program itself. This action or decision may manifest in a future rating, review, or locative event that will feed back as input to the application. However, the user willingly offers up her cognitive labor as program input with the goal of receiving more efficient and organized access to a vast database of information previously unknown to the user and of such a great quantity that the user could not sort on an unguided trajectory (or does not wish to) thereby making accessible information she would, in all likelihood, not have been able to access without a programmatic structure. This then informs her personal cognitive processes, which she can then choose to offer up to the application as input, with the goal of increasing the future usefulness of the application for her own purposes.

The user also engages in a kind of macro creativity of association, combination, and, sometimes, advanced augmentation. This kind of augmentation occurs on a spectrum. A user may choose to use a variety of applications in order to foster real life collaboration. The user and her collaborator may employ a mixture of new media applications, old media objects, online interactions, and real life interactions. This collaborative project does not preclude immersion in either engagement with the collaborator or the process of creating the project. The user switches between accessing information, identifying and sharing tools, immersing in work, and immersing in the collaboration, all the while layering and selecting the tools most suited to the task, time, or location. This example of the creation of a mixed reality (as discussed above) has not caused the users to “to mistake the structure of somebody else’s mind for [their] own” (61). It has allowed them to utilize the work of the minds of others to engage in a different way with work of their own.

On a different end of the spectrum of augmentation, the user may engage in hacking or re-programming of a new media object itself. This may involve exploiting the hardware of a device through re-programming it to run custom software that allows it to perform a different extended function that the user deems more personally useful. Or it may involve creating a personal software application complete with the desired tools or functions through the utilization of both/either the framework/structure of the program itself (algorithmic processes) or the ideas behind the structure of the program (input and output formats, means of access, interface style, method of acquiring user input or external input, etc.). As Manovich points out, “instead of identical copies, a new media object typically gives rise to many different versions” (36). However, these versions are not necessarily generated from a selection of templates or by “simply clicking on ‘cut’ and ‘paste,'” (130) as Manovich has claimed are the operations possible in new media applications. Here new media inspires a kind of variability not generated from a predetermined tree of selection options but from genuine creativity on the part of the user.

While “[p]ulling elements from databases and libraries” may be the default method of operation within a particular application, it is misleading to say that “creating [elements] from scratch becomes the exception” (130). The user may select applications from libraries and use discrete ‘elements’ from databases which are processed according to the specific method of structure of the application, however, the agency and creativity have not necessarily been supplanted. The user’s agency and creativity lie in conscious choice of program structure, personal associations and interpretations of program output data, re-programming of the applications themselves, and immersion in experiences which are inspired by and interwoven with these applications.