A coltan collar is a device consisting of a motor-driven mechanism that ends in a moving head covered with sandpaper that slowly erodes a zinc plate painted in blue. The machine needs an internet connection to work, as it detects tweets sent from devices located in an geographical sector in central Africa, the one where most of the minerals needed for the hardware inside digital systems are extracted from, with barely no revenue for local economies but extremely hard working conditions.

Whenever a tweet sent from this area is detected, the machine performs a movement, transforming the digital act into the mechanical work of eroding the colored plate. This erosion recalls an extraction process of rare earth minerals, as it unveils under the bright blue colored surface its hidden mineral support.

While the production of digital hardware continues to depend on labour exploitation, the affective economies of social networks rely on the exploitation of data exchanged by users within their free online services. A coltan collar plays with the interweaving of these two different types of exploitation, as users tweeting from the selected region in central Africa are the workforce in a system that recalls the illegal mining labour embedded in every digital device. It is a domestic image-machine, whose presence brings to the everyday space this dual notion of exploitation diffused in digital systems.

Increasingly complex interactions with machines blur the differences between matter, life and conscience.

Following the concepts discussed in the presentation of the documentary "Hello World! Processing ", this workshop will explore in detail the creation of living-like responsive environments. Through the introduction to algorithms coming from the simulation of natural and artificial phenomena, such as Vector Fields and Graphs, this workshop will be focused in the complexification of interaction and the composition of systems, two key issues to explore critically and creatively in our increasingly algorithmic driven world.

Text of the conference delivered during Interactivos?'14: Rethinking collective behavior and action in Medialab-Prado, in November 2014.

In this text I will try to present the work I have been doing during the last years in relation to the idea of collectivism and, in particular, when this concept is considered to be large enough to entail not only humans, but also entities such as the continuous flow of interactions between humans, humans and things and things and environment. That is, on the one hand, considering the chemicals we are constantly exchanging and their support structures, for example, our bodies, surfaces, the air in between… and, of course, in the other hand, the whole ensemble of creatures we live with nowadays in our digital times: the swarms of sensors that surround us or that are directly carried by us, the database records we leave as trails and the algorithmic processes that deal with them.

So, if I’m going to speak about chemicals and about the digital, and the interactions they give rise to, and within a context of art practices, what I’m going to talk about is about media. Media as in film, or as in Media Art, or as in New Media. Media as the set of tools and channels that put us in communication, but in an extended way, so to speak. I will consider media not only as the spaces of potential interaction between humans stemming from the proliferation of communication devices, but media in a broader sense as the spaces of interaction that exist also outside us: that is, a sheet of paper as media, of course, but also, the lawn extensions in our cities, for example: lawn yards as surfaces that somehow register and mold a specific set of public and private space human leisure activities, together with the dynamics of living grass, the presence of insects and other animal forms, as well as chemicals -chemicals against insects, fertilizers, pollutants in water, and so on. Lawn yards in our cities are spaces where we interact with the city, with a neighbourhood or a specific private property as a whole, being their grass a very particular species of media, a kind of organic screen where all the processes leave traces, and an interactive device, also, that encourages certain exchanges between humans and nonhumans inhabiting the city, or a specific area.

Media in this broad sense are a very interesting way of understanding collectivities of interactions, and if you’re interested to go deep into this view, I’d recommend you the bookInsect Media: An Archaeology of Animals and Technology, by the extremely insightful author and media art theorist Jussi Parikka, a book that is at the core of lots of ideas of this presentation.

I won’t continue with the example of the lawn, although it is something I’m working on now, and I’ll switch to a different media, which is the synthetic production of colors, an area I’ve been working on last year. It is a particular case that I hope will raise some questions regarding the relation between the growth of the collectives, on the one hand, and the profit of the digital industries, on the other.

In 1834, and thanks to a research carried on by a company that was producing the gas needed for the public lightning in german cities, philosopher-scientist Friedrich Ferdinand Runge synthesized the first artificial color from the wastes of coal, that is, from coal-tar. It was aniline synthetic blue, the first of a large series of aniline based colors. Art historian and writer Esther Leslie has described the unfolding of this discovery within the industrial context of the XIX and XX centuries, in her book Synthetic Worlds: Nature, Art and the Chemical Industry.In her words, “Nasty black waste matter could release from its darkness a world of color”, and that’s a key idea. Apart from the commercial applications of synthetic colors, that is, apart from the industrialization of the production of colors this discovery allowed, since its beginning it inspired the possibility of dialoguing in new ways with nature. These are, for example, images obtained by Runge, published in this book, The Driving Force of Formation of Substances, Visualized by Self-Grown Pictures, where chemicals become expressive technics to make in some sense Nature speak to us.

That is, with this book, and with a handbook of chemistry for children he published also, Runge worked with the idea of the inner activity of lifeless nature, of minerals as well as waste materials, an inner activity that could be brought to our scale thanks to the new aniline chemistry. Matter is, in some sense, alive, speaking its own chemical language, and open to speak to us in chemical terms.

The idea of Self-Grown Pictures is eloquent enough by its own, and will probably nowadays bring us to the creative coding scene, where algorithms and data are worked in a very similar way: a scene and a process aesthetics where coding is not meant to achieve specific goals, but rather to visualize the inner activity of coding systems themselves, that is, to bring to our senses the intensive, creative character of the digital substance, in some way. So, these Self-Grown pictures came already in the XIX century as a reminder that creation does not belong to humans, not to living processes only, but to matter also, to things, to ecologies of things, and to the molecules that compound things. Creation happens, we are surrounded by creation, and before the creative capabilities of the human individual, there is this continuous flow of creation all around us.

Runge’s personal project of extending this poetic and romantic view of chemistry, however, did not spread in the German education system, and the far more pragmatic view of the discovery was developed instead. The chemical industries adopted the synthesis of colors as their main object, and slowly, a whole new surface of synthetic colors began to cover the whole planet, an extension that spread over painted walls, clothes and objects, as well as a line of new synthetic materials and products: fuel, rubber, plastic, fibres, foams, fertilizers, pharmaceuticals as well as chemical weapons. And, with them, the postulation of an enhanced new world, synthetically produced by controlled systems, built upon substitutes, which was marketed as “more real” than the original one, at is was supported by a greater economic value in a commodity fetichism utopia.

Additionally, all these colors were fabricated and sold by companies whose importance and scope steadily grew to become the infamous german industrial cartel IG Farben, a prototype of the new organizational methods within the large corporations of the XXth century, and a symbol also of the dangers of these corporation monsters: they developed the Zyklon-B gas for the concentration camps, fuelled experimental rockets and, of course, supported the nazi economies.

If with Runge’s Self-Grown pictures we see parallelisms with our digital creative coding scene, these corporations should remind us the huge power that corporations, as well as their new distributed organizational principles, are increasingly having within the digital ecologies. Facebook, Google, Apple or Microsoft, among other, are the IG Farben of our times, turning this intensive and creative character of the digital into a big business based on the promise of an utopia of a bright digital and connected future for the humanity as a whole.

In my work I’ve been dealing with this parallelism -synthetic colors as digital media-, focusing in particular in how an underlying ideology of ubiquitous control is being codified into algorithmic creatures that are brought to existence among us.

Let me show you briefly an installation, which is called Vividness, a work on the digital uncanny in relation to its appearance in vivid colors. It is an installation that consists of two elements: a table with a vase with dried flowers, and a projector mapping the flowers. Flowers are arranged as a classical knowledge device, as they appear in XVIII-th century botanical illustrations, as ensembles of colors and species, following the traditional image of the fertility of nature when approached by science. In the installation, it is not science but a digital elaboration of the flowers, a photograph with the colors inverted, what is sent to the flowers from the projector, with all its implicit digital creatures flying around. The original color of the flower, then, mixed with the complementary one that has been sent by the projector, result in the viewer’s eye in a desaturated color, that is, a grey tone.

The idea is that color information is being extracted from the flowers, it is processed, and finally sent back to the world, miniaturizing how digital industries work on us and on everything. If the former industry of synthetic colors spread over the surface of the world a layer of artificial dyes, the characteristic flux of the digital one goes in the opposite direction: rather than spreading, it absorbs and extracts information from the surface of the world; it takes the real, the material presence of the real, in order to operate with its distinctive smartness.

Colors are then removed from the visual surface of the flowers, in real time, and with a rhythmically oscillating color pulse, in a way that makes the dried flowers seem to come to life with this uncanny, sustained digital breath. Living flesh, so to say, to elaborate on a simple question: what does it mean to be perceived, captured by the digital?

Perception is not a one-way relation. The perceived entity is affected, in some way, it is brought into a new existence, within the perceiver world; inside the digital ecosystems, in this case. We are constantly becoming images when we are perceived by these automated surveillance systems that film and monitor us, and we-as-images extend our lives within the relational databases we become part of. If machines perceive, it is because they are able to act. To perceive is somehow to measure the capacity to act.

And the point is that they act, certainly. Take for example this work by Sebastian Schmieg, Search by Image, Recursively, Transparent PNG, #1. An image is fed to the “search by image” google algorithm, and the first result of the query is taken to feed again the algorithm and so on and on. The creative force of the algorithm itself explodes in front of us, as in a digital Big Bang, where, why not, an image of ourselves, or of the Vividness flower arrangement, could be part of.

* * *

Media artist Nam June Paik wrote that video does not imitate nature, but time. Not nature as we perceive it, but the way nature makes it for us to perceive it. That is, through time, aging and irreversibility.

Two years ago, being part of Ultra-lab, a company devoted to open hardware and creation with technology, I was involved in the process of making a documentary on the coding platform Processing: the documentary Hello World! Processing was the final result. After a first phase of interviewing some of the most recognizable people within this community, I had the opportunity to work during several months with filmmaker Raúl Alaejos to enlarge the content of the interviews with online documents in order to be able to speak about different issues regarding the relation between creation and code. And during our research, we found the text by philosopher Henri Bergson,Creative Evolution, and this particular quote that opens the documentary:

"What is time? Time is what hinders everything from being given at once. It retards, or rather it is retardation. It must therefore be elaboration. Is it not then the vehicle of creation and choice? Does the existence of time not prove that there is indeterminacy in things?"

So, in the documentary, we wanted to start from this idea, that working with code and algorithms makes you aware of their own inner intensive, creative capacity, outside your own, subjective, forces. That there are processes, outside us, humans, that autonomously create, through emergence, complexity and these other forms of intelligence we begin to see outside our human realms. And we wanted to connect this awareness with the somehow related disposition to create collaboratively. That is, once you begin to work with algorithms, you are already working with others, and then it is very easy for you to open your creative processes to other humans as well, and then switch to long-term collaborations and collective modes of creation.

A very interesting thing we learnt while doing this documentary is that when we approach programming from this perspective, willing to connect to alien forms -so to speak- of creativity, we don’t follow the standard, engineering-like, mode of learning how to code. We exchange instead examples, snippets of code, already programmed agents and behaviours, and start to combine them and change their parameters, in a playful, non transcendental, way.

It is as if language, instead of being a carrier or a container of information, became an embodied notion within this creative environments, consisting of object-like bodies enacting perceptions, movements and behaviours. A modular, LEGO-like, set of infinite pieces. Think on the idea of learning a human language through snippets of literature, poems or songs. For our individual adult brain it is a really complex task, but in terms of code, and of collective learning, is a wonderful way of learning to create through code -which might be, by the way, a different thing to programming.

I’ve been working with this idea -the embodiment of language operations- in several workshops. And I’d like to share with you two very special ones we had the opportunity to extensively prepare artist María Andueza and I, in collaboration with the Culturambiente association, two workshops which took place in Nicaragua and El Salvador two years ago. This was a large project to think on complex local conflicts that deal with ecological as well as social concerns. Briefly stated, the project was split in two phases: first, a seminar was proposed to discuss and collect views, arguments and methodologies on a particular eco-social conflict, and then, afterwards, in a second phase, we -María and I- worked together with students and researchers to collectively build from scratch a videogame with the materials resulting from the first stage.

As we had ten working days for each workshop, we had to propose one of these embodied language strategies to focus more on the creative and discursive possibilities, rather than on the technical normative knowledge itself of coding and designing. We proposed then a working prototype of a simple new language, where the key idea was aggregation. Instead of sequential specification -building a first object (program) to be elaborated procedurally in a sequential way, introducing gradually more details and functions into it- we proposed a data-driven language, where interactive elements could be created as aggregations of pre-existing behaviours, rendering states and parameters. The set of possible behaviours could be extended by the more proficient programmers among the participants, but simple mechanics such as interactive dialogues or resource managements could be learnt by everyone, just creating different lists of interactive blocks.

For programmers in the workshop, working with this language was at the beginning an unpleasant experience. They were forced to approach programming in a different way, and not a really efficient one, as we were dealing with a prototype of a pedagogical tool. And for designers and researchers, the language proved to be simple enough to begin to think with playable results since the first day. This way, we deactivated somehow the traditional power divide between programmers and designers within these workshops, at least during the first days. In the end, in both workshops, and with the pressure to finish the videogames, disciplinary roles re-appeared as in the standard production chain. But these first days were crucial for the success of the workshops, from our point of view, as they allowed to know each other deactivating inherited productive roles, and to discuss since the beginning the conceptual concerns we wanted to deal with during the whole workshop.

The idea of language as an interaction away from informational or representational purposes seems to me very interesting when dealing with certain types of collectivities. If we think on situations where we easily engage in collective behaviours, such as dancing, team sports or simply producing sounds or noises, it is easy to understand the importance of such embodied communications. In more abstract processes, such as the elaboration of a videogame or an interactive project, it is not as clear to find such interaction modes.

* * *

Baruch Spinoza wrote in one of his treatises this impressive thought: “It is never we who affirm or deny something of a thing; it is the thing itself that affirms or denies something in us.”

I don’t know if aware of Spinoza’s philosophy or not, but most of current big online stores seem to agree with this quote. Parallel to the huge increase in the available information of customers’ habits and preferences, in the last years we’ve seen that when a customer accesses a store, he or she is offered a personalized display of products, that is, the recommended products panel, which conveniently tuned to excite his or her willingness to buy some of them.

Specialized algorithms are in charge of displaying the appropriate selection of products to a particular customer. They are able to compare customer information databases with catalogs of products, and produce relations that neither the seller nor the customer would be able to expect. Experts -and here I’m quoting the well-known TED conference by Kevin Slavin- say that algorithms are responsible of the 70% of the sales produced in large online stores.

If this is true, algorithms have then become their most valuable asset. The confidence on their extractive and predictive capabilities transforms customers free will to choose, or their artificially excited desire for a product, into a less important issue for retail managers. In some sense, once you are in the store, products will choose you, and not otherwise.

I've been playing with these ideas in an ongoing prototype of device. It consists of a bot that prints faces recognized and extracted from the database of books of Amazon. A specifically programmed script crawls the Amazon website, extracting those collections of books that appear together recommended at the same time. Those collections of related books are visualized graphically, and then a face recognition algorithm looks for faces within the resulting images.

The resulting pixelated faces are funny, as well as the idea of providing to the books with a portrait, or a composite picture, of their future buyer. It is scary, however, to think on being predictable, and not in a statistical way, but on an individual basis.

The project, of course, does not pretend to affirm something on the current state of algorithmic businesses, but rather to play, fictionally, with this possibility. I’m interested in the idea of the becoming of the subject, in the becoming of the me-as-entity, not in terms of dissolution or immateriality, but rather in terms of constant reassembly.

To finish, three months ago María Andueza and I were invited to propose a participatory performance in the public space in Stockholm. We prepared a set of boxes with electronics that produced a cicada-like sound, and distributed them to the participants, with a series of written questions, mainly related to the fact of becoming immigrant to a space because of the unpleasant sound of the boxes and the fact of being related with a community, the rest of the sounding participants. The action took place in a crowded square, which became immediately filled with the pulsating noise of the electronic insects. As bodies in a social space, we explored the transitions between the noisy individual and the teeming, vibrating multiplicity: the comfort space of the community against the violent isolation of the single emitter. Always in terms of becoming. Becoming immigrants, becoming media.

“We do not so much have media as we are media and of media”, affirms Parikka in the introduction of his aforementioned Insect Media book. Constantly disassembled and reassembled in our acts and perceptions, maintained by the collective, the unplanned and non-human ensemble of interactions that build and perpetuate the illusion of being.

An Atlas of Color Mediations is an interface to Google Image Search which provides simultaneously with two sets of results for each user's query: images with color, displayed on the left side of the screen, and images in black and white on the right side. Images appear in the same order they appear in a google search.

Once you’ve discovered by yourself the issues of the interface itself -related with polysemy, proper nouns, etc- soon you will find some enlightening results. Color mediates between ourselves and the synthetic contexts we live in.

The use of colored water cannons against participants in a demonstration is an old police force technique increasingly widespread. Apart from dissolving the group, it aims to tag the individuals for subsequent identification. It is a primitive form of current digital tracking technologies, as the ones that have come to light parallel to the massive demonstrations of citizenship all around the world these last years.

The installation stems from this double nature of the digital: on the one hand, it helps collectivities to emerge; but on the other, it has an unusual power to track the individuals. Next to an image of a colored water cannon against groups of people, a vertical projection lightens an aerial photograph of a demonstration. Over it, a layer of fragments of colored petals cover the area occupied by the people in the picture. Their color, due to the projected light, is oscillatingly being desaturated. A hovering elliptical shadow, finally, is projected over the crowd.

The installation may consist of one or several projectors, depending on the space and the technical means. Each projector transforms the image of a demonstration in a different city, with layers of petals of a different color in each case.

A photograph lays on a table. It is lightened by a projector heading downwards. When we get closer, we distinguish a coloured layer of glowng petals. At the installation, we realize that the image shows a crowded Taksim Square in Istanbul during the massive demonstrations last year. Bits of coloured petals placed over the citizens, however, appear as a glowing plasma spread on the Square, which reveals also a disc-shaped shadow hovering the coloured crowd. The projector, with the aid of a computer, maps the petals over the image and, while making them glow, fictionally introduces the surveillance-like moving shadow.

A second image is placed close to the table. It displays a set of photographs were groups of people are dispersed with the aid of water cannons, being the liquids used remarkably coloured. Dyed water is employed to tag people, as a primitive monitoring technique, a material analog of the invisible digital surveillance networks.

Once demonstrations are over, even with no blood spilled, streets remain covered with coloured fluids, revealing the clash between different networked collectivities. In the digital era, of crowd and collective phenomena, are these analogs to blood spills, unveiling an ubiquitous violent and surveillance state of coexistence?

This installation is part of the project L'image est une machine malade (The image is a sick machine), an exploration on the materialization of mechanisms of contemporary visuality through automatic movements and sound. The research shows the image -considered as a factory, as an active subject that is able to transform and mold- as a hybrid cyborg, an assemblage of visual resources and tricks used to catch the viewer's attention.

The installation receives the title from the homonym book of George Simon Winter of Adlersflügel, a treatise on taming and crossing of horses of 1672, where the picture of the horse-human in the piece is found. Two mechanical arms traverse the surface of the image and slowly erode its surface, leaving as a result two tracks, trace of the encounter between the image and the machine. The installation presents then a triptych where the image is completed with the movement and sound of the articulated arms and the tracks opened in its surface.

Bots of trust (aka Facebots) is an installation that consists of a grid of devices similar to points of sale terminals. The bots are specifically designed machines displaying the mechanisms of customer engagement in large online retail stores, with the aim of rendering them experienceable to visitors.

Each bot downloads a list of "Recommended items" from a popular e-commerce platform. The list is then codified into an image, where human-like faces are detected. Faces are printed, together with the articles they come from, mimicking how customers are gradually being codified inside the databases of algorithm-driven online stores.