Month: September 2015

Enchantment, and the Complicated Landscapes of Ambient Intelligence in Cities

Introduction: On algorithmic poems

The haiku that became the title of the thesis was written by Ray Kurzweil’s Cybernetic Poet; an algorithm that – when fed the works of a certain author (in this case, the poet Kathleen Frances Wheeler) – produces a poem that matches their language and style.

The question is not so much whether the poem will pass a Turing test for poetry – will Wheeler’s readers recognize the work as her own? – or whether the lines of authorship and creativity have become dangerously blurred. It is a beautiful poem in itself, but its curious history touches on another conversation that I hope to continue through the thesis.

While Kurzweil’s algorithm is proprietary and thus can’t be examined in detail, one can speculate – based on his previous work on language and computing and contemporary agents like Siri – about the workings of the algorithm. When fed a poem, it analyzes it according to a number of parameters: number of lines, most commonly used words, verbal structures, sentence types, rhyme, stress, form, and so on. It analyzes those in relationship to one another (perhaps she rhymes more freely when talking of love?), and if fed a sufficient number of works, it can start outlining more or less strict rules about what’s ambiguously called the style of the author. Perhaps Wheeler uses a lot of imperatives (hide), perhaps she refers to the reader as “you”, perhaps haiku is her form of choice?

But the important part, I think, is that we in the first place have constructed the methodologies and tools to deconstruct almost everything as quantitative. Since the 19th century, data and statistics have become a tool for claiming the chaos of urban life; but now this quantification feeds back into itself, and produces an output, a solution of sorts.

This thesis is about ambient intelligence technologies, which are based on exactly the same premise as Kurzweil’s Cybernetic Poet: feed the algorithms enough data about – in this case – urban life (and the question as to which and whose data and under what circumstances, constitutes urban life remains open) and the algorithms will respond in real time by adjusting their operation. This is exactly the promise of smart cities, which are a small segment of the ambient intelligence technology, which can loosely be defined as “the field to study and create embodiments for smart environments that not only react to human events through sensing, interpretation, and service provision, but also learn and adapt their operation and services to the users over time”. Ambient intelligence is the term preferred in Europe, and I use it, because the thesis uses as a starting point a number of documents produced by a branch of the European Union; in USA, the term of choice is “ubiquitous computing”, and Japan has recently embarked on an ambitious “ubiquitous networking” campaign which aims to equip cities with technologies allowing for seamless integration of many services in real time.

But this thesis is also about something else that the poem touches upon, and that took me much longer to put into words and thoughts. It is perhaps the reason that I had a hard time situating my narrative among the many existing ones that deal with ambient technology – it is a trendy topic! – and the reason that I ended up exploring it through something as dubious as hand drawn illustrations.

There is a beautiful story by Borges, “The Garden of Forking Paths”, in which a Japanese spy in Britain during the Second World War has to send a message to his people. The only way for him to do so, is by murdering another person, while simultaneously being chased by a British spy who is intent on killing him. The Japanese crosses the city, catches a train, goes to the province, finds the man, and kills him, but the story is only wrapped as a thriller. The real theme, the way I understand it, is the existence of many simultaneous realities in which everyone and everything goes through all possible permutations of any given situation. Simultaneously you turn left, right, turn around and walk back, or stay at the same spot: all of those happen at the same time, in an almost algorithmic manner. Yet each of us, bound to their inevitable physical body, is locked into only one moment at a time: you might catch a glimpse in a dream or a thought, but eventually, you are doomed to being only one thing at a time, to know only a thin string of all the possible permutations of the things that could have happened.

Lofty as that is, I see something of it in the algorithm: Kathleen Frances Wheeler didn’t write the poem, but she arguably could have written it; and I could write many more and smarter paragraphs that I will never write, but they exist as potentialities; as the changing, evolving output of an increasing body of work that changes as we change. And while I don’t want to talk about metaphysics, I do want to talk about the fascination with technology, about the strangeness of its workings, and the curious, ambiguous, and weird ways in which it positions itself against and in relationship to humanness, as eroding as a word that is. Historian of science Jessica Riskin argues that in Western thought, technology and humanity have traditionally been placed in opposition to each other, with each defining and being defined by the limitations of the other (Riskin, 2003), and “major technical advances in the capability of machines to replicate something considered human, in turn, shift our understanding of what it means to be human” (Bernius 2012). This is a strange dance: the Cybernetic Poet, and a big part of Ray Kurzweil’s work on language and computing, is based on his understanding of how human brains work, and how learning occurs in neural networks. Yet the relationship is never one sided: in the 17th century the brain was likened to a clock. In the 19th, its working were explained as similar to those of a locomotive. And in the 20th, it is of course a computer: our imaginaries and understandings of humanness and technology are interlinked and feeding off each other constantly.

I have always been fascinated by those imaginaries and interplays, though only recently did I start finding words and references, as well as was able to put them together in my mind and somewhat in my thesis with, well, ambient intelligence technologies. Moreover, how do you bridge them in a way that implies not only philosophical contemplation of lofty stuff, but actual engagement and dedication to that one particular permutation that one belongs to?

The work that helped me most was The Enchantment of Modern Life: Attachments, Crossings, and Ethics by Jane Bennett: from it, I borrow the term enchantment that is central to the work. Bennett uses it to denote “… a state of wonder, and one of the distinctions of this state is the temporary suspension of chronological time and bodily movement. To be enchanted, then, is to participate in a momentarily immobilizing encounter; it is to be transfixed, spellbound.” Also, enchantment: “…involves, in the first instance, a surprising encounter, a meeting with something that you did not expect and are not fully prepared to engage. Contained within this surprise state are (1) a pleasurable feeling of being charmed by the novel and as yet unprocessed encounter and (2) a more unheimlich (uncanny) feeling of being disrupted or torn out of one’s default sensory-psychic-intellectual disposition. The overall effect of enchantment is a mood of fullness, plenitude, or liveliness, a sense of having had one’s nerves or circulation or concentration powers tuned up or recharged —a shot in the arm, a fleeting return to childlike excitement about life.” I choose this term because of its being broad, almost philosophical, and thus can easily swallow both algorithmic poems and smart cities, but also brave: because who would be a fool enough to be enchanted with a world as the one we have now? But her interest – and mine – is less in the poetic potential of the term and the state it refers to, but more of the ethical implications of existing through it; enchantment as an ethical strategy, as a form of engagement with the world. Throughout the book, Bennett stresses on many occasions – as if anticipating critique – that being enchanted does not mean uncritical acceptance of the world, or the refusal to engage with it beyond mere observation. In her words, “…it seems to me that presumptive generosity, as well as the will to social justice, are sustained by periodic bouts of being enamored with existence, and that it is too hard to love a disenchanted world.”

1. Thesis argument and summary

Enchantment as a strategy for engagement

And this is where my tale with technology and cities goes, somewhat awkwardly, as there doesn’t seem to be an established narrative for this particular direction. Urban technology, and the technology of ambient intelligence in particular, has been discussed and critiqued on many occasions: for its militaristic origins and implications about the loss of privacy, about surveillance and the cybernetic tale of communication and control (Halpern, Singer, Weizman); for its proprietary nature that through hardware and software perpetuates corporate agendas and interests (Lessig, van Kranenburg); for its solutionist and developmentalist underpinnings; for its becoming a “black box” for citizens which no longer have any grasp on what is happening, and how exactly they are participating in it; for the tense interplay between transparency and opacity, and the grey line that runs between them, as they both come with a price. All of those are very legitimate, and my argument feeds on them, and tries to place itself among them without either negating or agreeing to either. If I need to situate my theory and practice among those narratives, they would perhaps come closest to the works discussing the “black box” of contemporary urban technology (Mattern, Slavin), and the loss of civic agency conditioned by the fact that people just don’t understand what’s happening anymore; that so many of the processes and the devices are blurring into the background, disappearing, while becoming increasingly complicated, that not even professionals (Slavin) can fully understand what’s happening, let alone citizens, whose lives are increasingly managed and influenced by technologies and processes out of their reach.

Thus my interest lies in enchantment with technology as a critical ethical strategy that can mobilize forms of understanding, engagement, and resistance that can, in a way, break the black box, by firstly, turning the eyes towards the multiple sites where technology resides in hardware and code, and secondly, by critically illuminating the many strange, and often contradictory ways in which it works, and the multiplicity of agencies that it embodies. Ambient intelligence technologies are by their very nature complex and consist of a multiplicity of hardware and software components. All of those come with their own story, particularity, and with embedded agencies – those of producers, legislators, users; even the agency of the materials that they are made of. Through this, I do not so much seek to offer a route of action – many others have done so, and in a much better way than I ever could, as to provoke a reexamination of this complexity through seeing it for what it is, because the necessity for a new discourse exists. An evidence for this is not only the multiplicity of academic writing in this regard, but also the practical importance that the AmI narrative has taken across the world: only in the past 10 years, the European Union allocated over 4 billion funds for sponsoring the research, development, and integration of AmI technologies in European cities; and Japan has embarked on an ambitious programme called “ubiquitous networking” that aims to do the same. In US, the preferred term is “ubiquitous computing”, which has given rise to the big data movement, and speculations and practices on smart cities and real time feedback optimization. And as Rob van Kranenburg says, “The vision has caught fire, wild fire.”

Thesis Summary

The thesis is divided in two parts. The first one is textual and asks the following questions:

What does this vision mean to regular citizens? What current discourses are being constructed, and for what rhetorical purposes? What is their role, and who are they meant for? And how do they envisage the spaces, practices, and people in ambient intelligence environments? Moreover, what agencies inhabit and are mobilized through those technologies, and among those, how can one locate human agency?

To examine those, I use as a starting point a series of documents produced by the European Union’s research branch that deal with ambient intelligence and its integration in cities. They are part of the policy shift mentioned earlier in which the European Commission allocated a large amount of resources towards the development and integration of those technologies. The most curious and one of the most influential among them is a report from 2001 speculatively titled Ambient Intelligence 2010. It was prepared by ISTAG, a composite body of experts from government, educational institutions and leading corporations like Siemens, Nokia, etc, and which resulted in four fictional scenarios that aim to describe what living with AmI might look and feel like for ordinary people. I use the report to examine the visions and discourses generated, and the types of spaces and interactions that they imagine. I analyze the rhetorical purpose of the report – who was it created for, by whom, and with which implications? – and the discourse that it establishes. Arguably, its purpose is not so much to predict exact developments, but rather to provide “food for thought”, to set the terms of the conversation that for most people is a very new one: what we should be talking about when we talk about AmI. My main argument is that the four scenarios and the discourse that they are part of, envisage citizens as hardly anything more than passive consumers and workers that communicate and enjoy themselves within strictly established frames. Moreover, it presumes the flawless, unproblematic functioning of all systems and devices, and thus it completely removes the question of competing or orthogonal agencies that complex systems by necessity embody. When those are surgically removed, the resulting picture is an overly simplified one: people have neither the impetus nor the opportunity to engage creatively with those technologies, to experience any glitch, interruption, or malfunction; to use them for anything else than the delineated activities.

My argument is, briefly, that ambient intelligence environments are by necessity, extremely complex because they entail a multiplicity of devices, networks, software; they are very much physical and thus a subject to the glitchiness and irregularity of physical things; they exist often within very dense places like cities; and because of their all-encompassing premise, they need to embrace a large variety of activities and purposes. Thus, they are far from straightforward; rather, the landscapes they comprise, are ones of very complicated and often contradictory agencies. And among those, it is not easy to locate that of humans.

The second part of the thesis consists of series of illustrations and collages that use the EU documents as a starting point as they reference some of things imagined in the fictional scenarios, yet ultimately they take off to explore through illustration different, often competing aspects of those ambient technology spaces. Because I am not a computer scientist, and because a lot of those things are still very speculative and haven’t been tested in practice, it was impossible for me to go much further than artistic speculation. But enchantment is just this, I think: instead of assuming that the narrative of flawless functioning in which one’s agency (as a consumer and worker) is transmitted without distortion and responded to frictionlessly, looking at the complexities that sustain those emergent environments.

To do this, I made three large collages referencing spaces and situations imagined by the report, and five small books with illustrations and writing that touch on five different aspects of ambient intelligence technologies: Hardware, Body, Network, Algorithm, and Readable. The illustrations range from more or less descriptive schemes of existing/proposed technologies (personal agent devices, antennas, cables) through more poetic representations of body and gesture communicating with technology to theoretical snippets from the texts that influenced the work, as well as my own writing.

The books

Illustration from Hardware: The Carrier Hotel in Manhattan

Hardware deals with the materiality of devices and networks that sustain those environments, as well as their embeddedness in a physical, glitchy, and contested world. Where is the line between nature and technology? How is fragility expressed and experienced? What are those devices, cables made of? Who installs them, and where?

Body addresses the relationship between the human body, its practices and gestures, and technology: how do we make ourselves understandable to technology? What bodily gestures do we learn to express ourselves in a mediated world? And where is the line between human and technology when it comes to prosthetics, brain computer interfaces, cloud computing?

Illustration from Network: Radio waves and their distribution in space

Network explores the invisible world of networks, connectivities, connection and disconnection; a world, that in spite of the claims of global connectedness, is still one of fragmentation; still a world that is very much contingent upon physical spaces. A famous example is the Carrier Hotel in Manhattan which provides Internet for the whole city. And because in algorithmic stock trading, the few milliseconds extra that you get if you are located closer to the source, make all the difference, many stock trading firms have emptied out and relocated to the buildings immediately adjacent to the Carrier Hotel.

Illustration from Algorithm: Bruno Latour on agency as making a difference

Algorithm is perhaps the trickiest one, because every software that conditions those environments, can be thought of as an algorithm. Yet I use it mostly to denote and look at the agency embedded in the functioning of the software; sometimes as straightforward as, for example, the software of Target which determines which customers are pregnant, and sends them relevant brochures; and sometimes as elusive and even scary as the 2010 stock market crash in which 10% of a large company’s stock simply disappeared in an algorithmic glitch that even today has no meaningful explanation.

Readable turns the lens around: how do machines make sense of the human world? The book explores the ways in which machines perceive and make sense of human data through multiplicity of sensors. Inspired by a short film, A Robot Readable World, it tries to imagine the ways in which, ambiguous as we are, exist for a machine: as a constellation of pixellated points of heat; as a black blob set against a white background that denotes the part that moves; as a shape outlined in a rectangle that is being labeled as human, car, dog, unknown.

The collage

…is quite large! – approximately 1 / 2.4meters.

It picks on three spaces described in the EU scenarios – Tokyo Airport, an office space, and the subway/car network of an anonymous large city, and combine them together in a somewhat messy way. My initial intention was to map through some simplified signage the multiplicity of agencies in those spaces, but felt that this would be unnecessarily descriptive, and moreover, inaccurate, as I don’t think there is a meaningful way to map technologies that are in the process of emergence. So I just used the collage as a uniting strategy for the illustrations, but in the process of work, they became detached from it, and working on bringing them closer together graphically and conceptually is something that I would like to do in future. It however attempts to present those seemingly familiar and straighforward spaces as confusing, complicated, and perhaps a little bit fascinating: the advocates of technology and connectivity argue that it no more matters whether you are in an office in New York or in Tokyo airport. I believe this is still far from true; but there is an overlap and blurring between place enabled by the new ways it’s being used, imagined, contested.

Another sea, another wind, another world : The allure of AmI in a rapidly changing world

The premises of ambient intelligence have been critiqued on many occasions. In his essay, The Internet of Things, Rob van Kranenburg outlines four “catches” of the idea. The first one is the almost explicit naturalization of the narrative (naturalization in the sense that “it cannot be otherwise”); that computational processes must “fall into the background and become a medium as pervasive and as invisible as electricity” (RvK). I believe that this narrative is so convincing for a number of reasons.

Firstly, it picks up on a process that began as early as the 19th century with the employment of statistics as a tool for understanding and managing the complexity of urban life (Halpern, Beautiful Data). That data, quantification, became something not only operatable and trustworthy, but also beautiful and valuable in itself, is explored in detail by Orit Halpern in her book Beautiful Data in which she traces the genealogy of the idea, and the ways in which it reached present time in embodiments like smart cities and the big data movement. In the 20th century it reached another peak as a military strategy coined as Cybernetics by Norman Weiner, who defines it as “the science of communication and control”. In cybernetic thinking, the agents (be it an enemy airplane, or a citizen) are seen first and foremost as sources of data rather than through their internal mechanics: you are what you emit into the environment, and the correct response has to be calculated accordingly. Similar principles underpin the current fascination with smart cities in which citizens are envisaged as sources of data, and most of the technological and infrastructural effort is put towards equipping them and spaces with devices and strategies for emitting and capturing data – sensors, bioidentification devices, cameras, etc.; – and the development of algorithms to mine and make sense out of this data.

A significant part of this effort is quantifying the seemingly unquantifiable: once you reduce things as vague and elusive as “quality of life”, “engagement”, “desire”, etc. to a common denominator, a shared language of binary code, they become mutually calculable; in a very Enlightenment fashion, chaos becomes tamed by reason (Harvey).

Secondly, and very much as a consequence, ambient intelligence (or ubiquitous computing) comes as an all-encompassing ideology; a technological strategy that can bridge production, legislation, consumption, and can easily be conjured as making the daily lives of people simpler, easier, and more responsive, as demonstrated by the ISTAG scenarios. Such narratives have always been alluring, but now they almost seem necessary. Cities in particular, and the world as a whole, are changing at rates unprecedented before. In the introduction text for Making the Geologic Now, Elisabeth Ellsworth and Jamie Kruse write that “the recent accelerations of change have turned what once appeared as lines with slight slopes upward into near vertical spikes. As [Steve] Denizen put it, at the points in time and speed that we now are approaching:

The world becomes defined not by a time, but by a speed. This is the point at which the world can no longer be merely an extension of our own, a difference in degree, but rather something which takes on a difference in kind: another sea, another wind, another world at right angles to our own.” (Ellsworth and Kruse, 2014; emphasis orig.)

It is no surprise that in those conditions of profound change – albeit without necessarily a shared understanding of the nature, causes, or consequences of this change, large scale narratives which can easily span from the technological to the commercial to the legislative, and to the social, will have a prevalence, and an attraction. AmI offers a vision for the management of increasing complexities: enabled and conditioned by large, real-time streams of data about anything from air quality through traffic conditions to the dailiness of individual lives – availability of products in the fridge or cardio-monitoring of individuals.

But this is still a vision of an emerging world that no one yet knows how exactly will look like; and there has been a proliferation of visions dealing with this: from sci-fi writings that somewhat touch on the topic (David Brin’s Transparent City deals with it mostly as an issue of privacy and transparency) to promotional images like those of New Songdo, to the scenarios produced by ISTAG, the panel gathered under the European Commission to prepare “food for thought” about the future of ambient intelligence in European cities. Something is happening, and many are trying to imagine how it will look like; and because it is a vision of the world as we know it – the cities, the streets, the lives – the visions have to first and foremost reinvent those. And they do so, in a very particular way.

A brief look at New Songdo’s promotional images tells a lot: ambient intelligence comes as a complete package, a lifestyle. Renderings and promotional images of smart cities use the same visual strategies (Shannon Mattern): mid-age, upper middle class families and professionals, occasionally holding a screen and smiling at it, or jointly laughing at a nearby large display. They are thin, neutrally but tastefully dressed, smiling, reading a book or stretching on a bench among abundant greenery in what seems to be an endless weekend or an indefinitely prolonged lunch break, as the occasional suitcase or power suit suggests a successful white collar career. The aesthetics are curiously tied to contemporary architectural visualization and its recurring images. It is always sunny. However, as Shannon Mattern points out, those are promotional images and they cannot be expected to serve another purpose, or be critiqued for doing exactly what they are supposed to do.

Yet strikingly similar visions appear in narratives that supposedly serve different purposes: the ISTAG report that the thesis uses as a starting point, and which I will now address in more detail, often employs imagery that very closely evokes the one used in promotional materials.

3. ISTAG’s Ambient Intelligence 2010

Scenarios summary and analysis. Cut because boring.

4. Theorizing non-human agency

Latour attempts to expand the concept of agency beyond the intentionality of human agents, insisting that the defining quality of agent is not their perceived intentions for acting, but acting itself: “agencies are always presented in an account as doing something, that is, making some difference to a state of affairs (…) Without accounts, without trials, without differences, without transformation in some state of affairs, there is no meaningful argument to be made about a given agency, no detectable frame of reference.” Speaking of objects’ potential to embody, trigger and respond to action, he distinguishes between intermediaries, passive tools which fully appropriate without distortion the agency of the user simply “…transporting meaning or force without transformation”, and mediators who “transform, translate, distort, and modify the meaning of the elements they are supposed to carry” (Latour 2005).

Alfred Gell refines the concept of non-human agency by proposing the idea of primary agents, possessing intentionality, and secondary agents, “artefacts, (…), through which primary agents distribute their agency in the causal milieu, and thus render their agency effective” (Gell 1998). Yet the relationship between those is not one-sided. To Gell, the objects through which the primary agent exercises his or her agency are more than simple tools; they become a part of his or her performative personality, to the extent to which their presence and functionality is critical for the agency being exercised. Like Latour, he concludes that the proof of agency lies only in its manifestation: “We can recognize agency, ex post facto, in the anomalous configuration of the causal milieu – but we cannot detect it in advance, that is, we cannot tell that someone is an agent before they act as an agent” (Gell 1998).

In the four ISTAG scenarios the devices and networks underlying the AmI world are seemingly mere intermediaries: they produce no effects beyond the desired ones, and with perhaps the exception of Dimitrios, they don’t seem to significantly affect the personalities and behaviors of their users: it is a seamless, inobtrusive world where people are undoubtedly in charge. This is the premise and promise of technology “[falling] into the background and [becoming] a medium as pervasive and as invisible as electricity.” (RvK 2006). However, I don’t agree that electricity is in any way invisible even when it functions perfectly; it becomes especially visible when it doesn’t. In her article, “The Agency of Assemblages and the North American Blackout”, Jane Bennett explores the 2003 North American electrical blackout which she pictures as a result and an example of the “distributive and composite nature of agency”: Bennett’s account traces the genealogy of the crash through a complex patchwork of human and non-human agents, which, she claims, is only made visible when the functionality of the system is compromised, and goes on to theorize the agency of what she chooses to call assemblages as “the distinctive efficacy of a working whole made up, variously, of somatic, technological, cultural, and atmospheric elements”.

I argue that by their very nature, Ambient Intelligence environments are complex assemblages, and if the four narratives seem rather simple and straightforward, this is because they completely omit talking about a big part of the picture: the networks that condition them, the minutae of integration, the complexity of creating a trustworthy software across a multiplicity of platforms, the legal intricacies of data-mediated identities crossing borders, the underlying code and protocols, and even the integration of the devices with the bodies using them as a complex interplay between what Gell calls primary and secondary agents.

The extent to which crashes reveal the underlying agencies behind assemblages of any sort is evident also in the recent case of the Stuxnet virus. A diagram in the NY Times makes evident the multiplicity of agencies, both material and immaterial, and human and non-human, and their intricate interplay through the workings of what was deemed “the first weapon made entirely out of code”. But more than anything else, the story demonstrates that any system, and especially systems as complex as those operating a nuclear plant, or bridging the multiplicity of devices, sensors, and public-private entities and networks enabling AmI environments, has many potential vulnerabilities: from bugs in the code to human agents sneaking a USB drive carrying the virus to the unpredictability of electrical currents and plant operations, and so on. In the ISTAG narratives this complexity is hidden; and so are its vulnerabilities.

But beyond failing to account for the underlying complexity and the multiplicity of contesting agencies enabling such a world (private and public entities, legislation, body politics etc), the devices that all of the characters use and rely on to navigate through their daily lives possess their own particular agency which goes beyond that of secondary agents: they are granted certain decision-making powers, and they are able to self-optimize through data accumulation and feedback. They are examples of artificial intelligence, understood less anthropocentrically as the ability to read and respond to context (explain). D-Me grows and changes as it gathers more information and understanding of its Dimitrios, and Carmen’s personal agent device suggests that she buys a bottle of Chardonnay that she likes, or that she stays home the next day to avoid the scheduled demonstration. This is a form of agency that goes beyond an intermediary or a secondary agent, and the way it is constructed, acted, and situated in particular contexts, is a complex function of the interplay between a multiplicity of underlying agendas. The agency of artificially intelligent devices and systems – understood as the ability to self-optimize through feedback loops – is perhaps a fundamentally new thing (at least in the extent to which it is happening now; otherwise cybernetics started as early as WWII); one enabled by the increasing ability to gather and interpret data, and the construction of intuitive and integrated interfaces; all the things that enable AmI. It is contingent not simply upon the algorithms that drive it, nor the availability (or selectivity) of data which they process, but also their very own – sometimes unpredictable – logic of self-optimization. I would like to elaborate on it through a brief and recent example.

In 2014 Google made one of its largest European acquisitions by buying DeepMind technologies, a London-based artificial intelligence firm which specializes in machine learning, advanced algorithms and systems neuroscience. DeepMind is the latest in a long line of artificial intelligence acquisitions from Google that recently included the learning thermostat and smart fire alarm company Nest. “DeepMind was generally interested in reinforcement learning, and in deep learning, which is very useful in mining so called ‘big data’, something Google has a lot of and is interested in processing,” said Murray Shanahan a professor of Cognitive Robotics at Imperial College London.

DeepMind has so far never released a product on the market, but among its most impressive achievements is training an AI gamer to play Atari video games, outscoring professional human players in most of them. The twist is that the software isn’t told the rules of the game; instead it uses an algorithm called a deep neural network to examine the state of the game and figure out which actions produce the highest total score. Deep neural networks are often used for image recognition problems, but DeepMind combined theirs with another technique called reinforcement learning, which rewards the system for taking certain actions, just as a human player is rewarded with a higher score when playing a video game correctly. [rephrase]

Google’s whopping investment possibly rests on the belief that AI researchers increasingly share: that advanced synthetic intelligence can be reached not through programming the software to perform specific tasks, but rather, letting it figure out the context on its own, and through trial and error, building strategies for understanding and action. However impressive, there is an important shortcut: what is encoded is the principle of rewards (high scores) and punishments (say, dying in the game). Learning never occurs in a vacuum: however advanced, an intelligence – be it a synthetic or an organic one – needs a set of criteria through which it chooses its possible actions; it needs to have a more or less concrete idea of what is good and what is bad. In a way, it needs to have encoded values.

And while this seems rather straightforward within the flat, pixellated landscape of video games –- high scores are good, and dying is bad — it is worth asking what sort of motivations will drive artificial intelligences that deal with less arcade things such as urban life; a question that becomes evidently extrapolated in the case of smart cities where large amounts of data are processed in real-time within a self-optimizing system. Optimization is not a self-evident concept; and as we become able to acquire larger, more precise and more diverse amounts of data, and integrate it within increasingly complex systems accounting simultaneously for things ranging from air quality to traffic navigation, the question of the values driving optimization becomes much harder, and about ethics.

Locating human agency

But what values underpin the functioning of those infrastructures and devices? And among those already complicated relationships, how is one to locate the agency of humans? What is really up to citizens, and how is it addressed in the discourse? In the ISTAG’s narratives, the five characters’s existence in the AmI world renders them most of all as workers, consumers, and social networking beings. Thus the decisions that the devices make for them are directly based on premises which condition and enable their existence as such: Maria’s trip to an Eastern city is made as easy as possible so that she can focus on her presentation (with which her P-Com also helps), Dimitrios’ avatar tackles the calls it deems “not important enough” to distract the physical person with, Carmen’s personal agent prepares her shopping list, and helpfully suggests a Chardonnay, and one can easily guess that the learning environment in which Annette and Solomon participate is not a cheap one.

This is not to say that those are necessarily bad things: human lives are to a large extent structured around work, consumption, and socialization, but seeing the AmI devices as mere helpful intermediaries renders the picture incomplete. Any technology is built with a number of applications in mind, which to an extent condition the ways it is used; the ISTAG report itself has as its purpose to outline possible directions for business development. Moreover, the networks, devices, and structures that enable it are mostly proprietary, and have their own particular histories and agendas in different places.

To think that those won’t play any role, that the personal agents will assume the role of a helpful, invisible assistant able to “guess” its master’s desires and needs, is shallow and dangerous. It will be necessarily biased towards certain functions, and considering the history and the structure of the enabling technologies, one can guess that the bias will be towards work, consumption and mediated socialization that falls within certain norms (Will D-Me connect Dimitrios to, say, nearby individuals who share his interest in anarchist ideas, if he had one?). This is not to say that they will necessarily be used in those ways: the history of technological inventions has proved that very often, innovations find a completely different trajectory in societies than the intended ones. But as RvK and many others point out, we are reaching a point where the devices, networks, and algorithms, underlying those environments are increasingly seen as black boxes, as their complexity and proprietary nature goes beyond the citizens’ ability to hack and twist them. So in an increasingly complex and rapidly changing world conditioned by intelligent infrastructures and devices imbued from their very creation with a multiplicity of agencies and values, what agency is left to the citizens which have little to no clue about their inner workings?

Moreover, the systems’ functioning is conditioned upon their ability to self-optimize through feedback loops, and crucial to those feedback loops is the constant, real-time availability of data, and the translation of what were once non-quantifiable things into binary code. Communication always happens halfway; as much as artificial systems adapt – through either programming or self-actualization – to our humanly world by learning to read and make sense of it, we – as conditional as this word is – are also increasingly rendering ourselves readable. We construct and inhabit environments in which space and thought is allocated for the technologies of reading: sensors, data transmitters, broadband, servers; environments imbued with algorithmic thinking and fascination with data and its visualizations: the hosts of spirits made visible; the belief that the binary, quantitative models are an adequate representation of a messy world, and moreover: that messiness can be tamed and overcome by reason, even if this reason is ambiguously outsourced to synthetics. Moreover, we also adjust our gestures, movements, language, etc. to become “machine readable” (Slavin).

And all those agencies – of materials, networks, algorithms, and humans, are embedded in a world changing at rates that perhaps suggest or demand a fundamentally new approach, a new strategy of engagement.

5. Back to Enchantment

Through all those, my main interest is agency; and in particular, the agency of humans which seems so fragile in this complicated landscape: because so little seems up to ordinary citizens who exist simply as data, because no problem is evident yet per se that can trigger mobilization, and because things have become so complex that understanding them, hacking them, and subverting them, requires knowledge that normal people don’t possess anymore. And why would they? – in spite of this complexity, their roles in these landscapes are rendered conveniently only through production and consumption, as conflict and cracks are entirely erased from the conversation.

Again, I don’t intend to propose a course of action, or to say that one framing of the issue is more legitimate than another. My argument is simply that the landscape of agencies that condition the environments – emerging, imagined, or functioning – is very complex, and can open up a multiplicity of conversations and inquiries that go much further than the question of, which is the shortest route that Carmen takes to work? Or, should Dimitrios direct the stranger to the nearest pharmacy? And perhaps further than even the binary distinction that Brin does in his novel: transparency vs. surveillance, to which I wouldn’t subscribe easily.

I started imagining some of those only through drawing, because it forced me to look at the things themselves in a way that just reading or talking never does. Observing an algorithm change over time, or studying the inner mechanics of a guided missile, in order to reproduce and interpret it gives you a perception and fascination with it that I am not quite sure what it translates into. But what I am hoping to achieve with this thesis is to hint at and suggest the complexity of those technologies and the spaces and relationships they will create, as well as the much broader array of themes to talk and think about related to ambient intelligence.

There is one last reason for which I like the poem. In a way, in a creepy and cybernetic way, it suggests resistance, reluctance to play along, desire to go gentle into the night. And while this goes completely further than the topic of the work, it is a very beautiful thing.