Algorithmic Facial Image, Regimes of Truth and Datafication

Mitra Azar, Artist and PhD fellow, School of Communication and Culture, Aarhus University

There is the first very uprightness of the face, its upright exposure, without defense. The skin of the face is that which stays most naked, most destitute. […] There is an essential poverty in the face, the proof of this is that one tries to mask this poverty by putting on poses, by taking on a countenance. The face is exposed, menaced, as if inviting us to an act of violence. (Levinas, Ethics and Infinity)

From Selfie to Algorithmic Facial Image

This paper examines the political implications of new technologies for facial recognition, and proposes a new type of selfie aesthetic characterized by new forms of human and machinic agency. The paper argues that when the selfie becomes mediated by new tracking technologies for security system and entertainment based on face-recognition algorithms, the selfie becomes an ‘Algorithmic Facial Image’ (AFI).

Facial tracking technologies have been incorporated in digital cameras for many years, and are offered to users of social networks such as Facebook to facilitate and automatize tagging (the process of recognizing one’s face in a picture and associating it with a user’s profile) and image sharing. Nevertheless, in recent times, facial recognition technologies seem to have taken a new turn, and from the simple recognition of faces with cameras and social networks they have become embedded in mainstream security technologies as much as in entertaining ‘face swap’ apps, transforming the social and cultural implications of the selfie.

The new status of the selfie is evident in a number of examples. The most recent iPhone X unlocks by recognizing the face of its owner despite make-up, glasses, and haircut changing (Face ID Security Guide). New Mastercard technology allows payment by tracking unique bio-metrics features of the users, namely fingerprints and/or faces (Lomas). At the same time, apps such as MSQRD (Masquerade) or Face Stealer allow users to ‘face swap’ in real-time, that is to modify their facial traits by assuming the those of somebody else – either friends, monkeys, or well-known public figure (Dredge). Other apps simply ‘cartoonize’ facial features: in the case of Snapchat, and Meitu – a viral Chinese app that has been regarded by security experts as a privacy nightmare, in relation to the rapacity with which it is capable of extracting data from user’s phones (Fried).

Figure 1: MSQRD app. Screenshots from the Internet.

Lately, the 2017 Deepfakes online phenomena emerging on the online community Reddit (Romano) – where faces of celebrities are swapped over pornostars’ bodies while performing in adult movie – proves the algorithmic precision of neural networks behind facial recognition technologies, able to function not only in real-time but also with moving images.

Thus, if in the early 2000s the selfie seemed to be characterized by a certain degree of (calculated) spontaneity, an analogically constructed liveness and a form of human agency, this new form of selfie is rather defined by its trackability, its algorithmically constructed liveness, and its non-human agency. It is in this new technological context that this paper aims to highlight the underlining aesthetic, political and epistemological implications related to face tracking technologies, and argues that this new phase of the selfie culture can be framed by introducing the notion of the ‘Algorithmic Facial Image’ (AFI) inspired by the notion of ‘Digital Facial Image’ (DFI) (Hansen 205-228), and the concept of ‘faciality machine’ (Deleuze and Guattari 167-191). The paper, indeed, draws a ‘line of flight’ (Deleuze and Guattari 9) from Gilles Deleuze and Félix Guattari’s faciality machine to Mark B. N. Hansen’s transformative appropriation of this concept into the DFI, to the newly crafted AFI, arguing for the need of a new theoretical tool to understand the new type of interaction between the user’s body, affects and algorithmic technologies produced by contemporary selfies. This interaction seems to hybridize the features of the faciality machine and of the DFI into this new type of image which the expression ‘Algorithmic Facial Image’ seeks to describe.

From Faciality Machine to Digital Facial Image

The DFI is usually a type of computer-generated face recognized by Hansen in the domain of media art. He senses the shift from a HCI (Human Computer Interface) paradigm to a DFI (Digital Facial Image) paradigm, and it is here that the face becomes the “medium for the interface between the embodied human and the domain of digital information” (Hansen 206). In the artwork Dream of Beauty 2.0 by Kirsten Geisler (1999), for instance, a digital autonomous face addresses the audience’s affective body, turning it into the framing device for the interaction between the digital and the embodied human: “an interactive, voice activated installation with a digitally generated female persona” invites the audience into “an intense affective experience that forms a kind of human counterpart to the potential autonomy of the digital, a new domain of human embodiment that emerges out of our response to digitization” (Hansen 207). Thus, according to Hansen:

whereas the currently predominant model of the human-computer-interface (HCI) functions precisely by reducing the wide-bandwidth of embodied human expressivity to a fixed repertoire of functions and icons, the DFI transfers the site of this interface from computer-embodied functions to the open-ended, positive feedback loop connecting digital information with the entire affective register operative in the embodied viewer-participant. (Hansen 207)

Hansen defines the DFI in relation to the concept of faciality machine elaborated by Deleuze and Guattari: “this machine is called the faciality machine because it is the social production of the face, because it performs the facialization of the entire body […]. The deterritorialization of the body implies a reterritorialization on the face […]” (181). According to Hansen, Deleuze and Guattari’s faciality machine produces the facialization of the entire body and by doing so it prepares the emancipation of affects from its ties to the body. The faciality machine simply requires a receptive surface, characterized by intensive micro-movements:

the face is this organ-carrying plate of nerves which has sacrificed most of its global mobility and which gathers or expresses in a free way all kinds of tiny local movements which the rest of the body usually keeps hidden. Each time we discover these two poles in something – reflective surface and intensive micro-movements – we can say that this thing has been treated as a face. (Deleuze 87-88)

Exactly because the faciality machine can potentially turn anything into a face, it can produce affects in the absence of a body. Close-ups of objects framed as face in this sense are common in the history of cinema (Deleuze 89), and possess “the power to tear the image away from spatio-temporal co-ordinates in order to call forth the pure affect as the expressed” (Deleuze 96). According to Hansen, Deleuze and Guattari subsume the bodily activity into the perceptive quality of the close-up, and as a consequence affect becomes related to the framing function rather than to the body, and subsumed from perception.

Hansen criticizes this position, and follows a more orthodox approach to Henri Bergson’s theory of affect (on which Deleuze and Guattari’s reflection is partly derived) by locating affectivity as the structuring device for processes of embodiment. Thus, the DFI produces the audience’s embodied affective reaction, while affects operate (or structure) the mediation between informatics and the embodied human. According to Hansen, this change in perspective from Deleuze’s understanding is not trivial because it allows us to keep the human (and the body) as a key element in relation to digital technologies, avoiding a “more nihilistic posthumanism of, say, German media scientist Friedrich Kittler, who has infamously pronounced the structural irrelevance of the human in the face of digital convergence” (Hansen 207).

From Digital Facial Image to Algorithmic Facial Image

Nevertheless, the functioning of new face tracking technologies seems to work differently from the functioning of the DFI described by Hansen, and the notion of the Algorithmic Facial Image (AFI) tries to grapple with these changes. It is necessary to investigate the different functions of these two types of images closely as they have different political implications. On one side, according to Hansen, the DFI produces the “dynamic re-embodiment of the interface, [and] reverses precisely this process of facialization that comprises the very principle of the HCI as an instrument of capitalist semiotics” (208). HCI seems, in other words, to exploit the separation of affects from bodies described by Deleuze and Guattari as the defining feature of the faciality machine; separation which allows the capitalization of everything and makes use of facialization as the mechanism producing the movement from “the organic strata [of the body] to the [the HCI] strata of capitalist signifiance and subjectivation” (Deleuze and Guattari 181). The DFI, according to Hansen, seems to resist this process of facialization and transforms the face into “the catalyst for a reinvestment of the body as the rich source for meaning and the precondition for communication” (208). On the other side, when it comes to the politics of the AFI, it is possible to see how its functions are consistent with capitalist semiotics – indeed with the faciality machine – and yet some of the working mechanisms behind it echo the DFI. With AFI, I argue that the faciality machine hybridizes with the DFI.

Algorithmic Facial Image as Hybrid

To understand how this hybridization comes into being, I propose to look at the differences between the DFI and AFI, to then relate them to the functions of the faciality machine. First of all, there’s a change of context to register: if the DFI is understood in relation to media art, the AFI appears in more mainstream and vernacular contexts (for example in security systems and entertainment apps). Moreover, if in the case of AFI the user’s face is simultaneously the subject and the object of the interface (as it happens with face swap apps), in the case of DFI the face is always the face of a digital avatar. Furthermore, the user’s affective reaction generated by the DFI is overwritten by the algorithmic processes produced by the AFI while processing the user’s affective reaction gathered through the user’s face. If the faciality machine of Deleuze and Guattari “overcode[s] the body on the face” (Hansen 208), and the DFI decodes the avatar’s face into the user’s affective embodiment, the AFI decodes the user’s affective embodiment (in the form of the user’s face) into algorithmic data. Indeed the AFI echoes the functioning of the DFI but works as a faciality machine: this is because it exploits the affective-embodiment of the user (rather than reconnecting the user to his/her affective-embodied self as in Hansen’s DFI) and turns it into a compulsive ritual (the “selfie performativity,” with its “poses” and “countenance,” in the words of Levinas), which enables surveillance-oriented non-human algorithmic procedures aligned with a postmodern type of faciality machine. The body is in the circuit only as input and output, but not in-between, where everything is played out within the computational functioning of the AFI reacting to the user’s facial affective input. In the AFI, the accent is on the hidden algorithmic processes that the user’s embodied affect (literally, the face of the user) has produced. In Hansen’s DFI the accent is instead on the embodied affect itself as the medium of the interaction between the user and the DFI. Thus, if the DFI focuses on the affective input, the AFI focuses on the algorithmic manipulation of the affective input.

If both DFI and AFI asks the embodied human to complete affectively the functioning of the interface, the AFI seems to exploit the affective source coming from the user to produce the affective user it is interacting with. This production consists practically in the visual re-organization of the user’s facial traits – in Deleuzian terms the re-organization of the relationship between receptive surface and micro-movements – and in the parallel production of a data-selfie. In the case of the AFI, indeed, the face triggers a mutilated form of affective-bodily response instrumental to the algorithmic processes oriented towards producing this visual and data re-organization.

This is significant because in the AFI it seems that both the mutilated, embodied, affective framing function (the selfie performativity) and the disembodied algorithmic production (the real-time re-organization of the relationship between receptive surface and micro-movements as completely removed from the physical body) co-exist as necessary moments towards the formation of the AFI, testifying to the hybridization of the DFI with the faciality machine.

Moreover, if according to Hansen, “aesthetic experimentations with the DFI strike directly against late capitalist semiotic mechanisms […] that function specifically by reducing embodied singularity to facialized generality” (209), the AFI seems instead to reduce the affective embodiment of the user to a stereotypical performativity– the impoverished selfie performativity which appears as an embodied version of what Andersen and Pold have called the “aesthetic of the banal” (Andersen & Pold 271-289), necessary to activate the algorithmic processes happening behind the surface of the AFI. The AFI is thus enabled to extract data from the user’s face but also from the user’s phone – towards producing a data-selfie to be sold on the big data market. The privacy nightmare mentioned at the beginning in relation to Meitu face swap app stands as an example of this parallel visual and data production-extraction.

Algorithmic Facial Image and Regimes of Truth

It seems reasonable to say that the new technological processes of engaging with the human face trigger a new phase of the selfie aesthetic. If face-tracking technologies are based on the idea that one’s face is unique and non-replicable, the amount of entertaining face-tweaking apps available on the market seems to suggest that the face is indeed trackable, its features tweakable, and its uniqueness hackable. This is especially (and frighteningly) evident in relation to a software developed by Stanford University which enables a visual re-enactment method wherein two men’s facial expressions are motion-tracked and recorded, and then to be swapped in real-time over a screen: the man standing and not talking now talks and replicates the facial expressions of the other (Real-time Facial Re-enactment software). This is the same type of technology behind DeepFakes, with the difference that the script behind DeepFakes has been open-sourced on the Reddit DeepFake community (Romano).

Thus, the face as the privileged body part bearing the user’s ‘singularity’, becomes the playground for testing and refining tracking algorithms. The face as a peculiar site of singularity turns into the privileged site for trackability and datafication (Cukier & Mayer-Shoenberger), and its uniqueness gets challenged by the aggression of technologies. The more they function as new biometric security systems based on the singularity of one’s face, the more they transform the face into a replicable surface – as the Stanford face swapping software clearly demonstrates – undermining the very epistemological assumptions on which face-tracking security systems are based.

As a consequence, the truth value held by the face becomes un-assessable, and the selfie turns into the site where contradictory regimes of truth coexist and feed each other – becoming an aesthetic format which keeps an appearance of immediacy while hiding layers of algorithmic complexity. The political relevance of the AFI lies in the ambivalent regime of truth to which it belongs, and on the related practices of “circulationism” (Steyerl) and “datafication” (Cukier & Mayer-Schoenberger 28–40) this regime produces. At the same time, the hermeneutic confusion seems already to manifest in a number of selfies from contemporary internet culture: from Abdou Diouf’s Instagram account (Diouf) show-casing selfies of himself crossing borders from Africa to Europe – custom-made by a Spanish advertising firm to promote a photography festival (Mackintosh); to the Selfie of a young Palestinian man running away from two Israeli policemen – custom-made by Dam, hip hop trio from Ramallah (Withnall). The very idea of thinking of selfies (and of the face as their bodily reference) as an (calculated) spontaneous and truthful “reality grab” – the way it was perceived in the early 2000s – seems to have collapsed.

The contemporary selfie aesthetic seems to have already moved towards the algorithmically constructed hermeneutic ambiguity of the AFI, and prepares the ground for it. For example, the AFI taken by the car-sized rover Curiosity exploring the Gale crater on the planet Mars – and realized by combining a number of shots from which an algorithm subtracts the arm holding the camera from the composed image (Kaufman) – exposes a newly constructed yet apparently immediate regime of truth similar to the one described above.

Figure 5: Curiosity on Mars.

Something similar happens in the context of the AFI generated by Google car street view. If in the past users could pan down to the Google car camera and see the car and the 360 degree camera device from which the images were taken, a recent update manages to make the car and the recording device disappear from the image (Turnbull). Now users can only perceive the Google car from the shadow it projects on the ground – and are left with the sensation of controlling a fully virtual camera, and of seeing, once again, a newly constructed yet apparently immediate regime of truth.

Figure 6: Google Car Street View, before and after the update.

Thus, the new regime of visibility related to the AFI seems to be characterized by a paradoxical regime of truth. The specificity of this regime of truth bears important consequences with regard to the circulation and datafication of the AFI, and allows for a deeper understanding of the political implications in the post-truth era we are currently navigating.

Regimes of Truth and Datafication

The AFI turns the face into a site where contradictory regimes of truth coexist in a form which keeps an appearance of immediacy while hiding layers of algorithmic complexity. From a hermeneutic perspective the art of circulation and data extraction of the AFI refers to the inherent liveness of the Internet: “live” and “immediate” AFI are virally shared through social network platforms and datafied through algorithms implementing extraction practices behind the AFI surface. The AFI value derives from its circulation – itself derived from the appearance of immediacy the AFI preserves during the algorithmic processing – oriented towards what we might call first degree datafication or biodata extraction (facial features), and second degree datafication or infodata extraction (contacts, GPS, etc.). In this sense, while engaging with the user’s face, in parallel to a visual selfie, the AFI manages to produce a data-selfie of the user, which exists in the production of an abstract affective subject to be sold to companies for targeted ads. This is how the AFI produces the affective subject it is interacting with, exploiting the user’s embodied affective input (selfie performativity) as a means to gather user data and generate an algorithmic self, one that is disembodied yet affectively programmed to intervene in the user’s online and offline interactions and promote certain (affective) behaviours over others. Moreover, advertisers have a keen interest in these behaviours as part of a bigger system of data built around users which can help them understand how to target their ads better.

Interestingly, the shrinking of the distance between ‘fiction’ and reality – what I have addressed as the hermeneutic confusion inherent to the regime of truth of the AFI – is indeed matched by the shrinking between an embodied affective ‘singularity’ (in the form of the user’s face) and a surveillance-oriented disembodied algorithmic agency. This produces an algorithmic data selfie retro-acting on the user by investing the user with the affective charge the AFI has built by combining biodata and infodata towards generating an ‘abstract’ affective subject to be applied back on the ‘concrete’ user.

If the apparent immediate nature of the AFI is the reason behind its viral circulation, its algorithmic nature is instead the reason behind the AFI’s ability to extract data, and it works as an opaque mechanism behind the apparently transparent (immediate) and fast circulation of the AFI. If Hansen considers affectivity to be the genetic element of the DFI (218), we might refer to an algorithmically constructed affectivity as the opaque genetic element of the AFI. Even better, we might refer to the algorithms designing the AFI as the genetic elements, and to the algorithmically constructed affectivity as the outcome of processes of circulation (based on the AFI hermeneutic ambiguity) and datafication (based on biodata and infodata extraction). These processes begin right after the first embodied affective contact between the user and the AFI interface – namely right after the user’s selfie performativity with its “poses” and “countenances” that activate the functioning of the AFI. The AFI mediates the transformation of an analog affective input into an algorithmic affective output, and prepares the further re-embodiment of the affective output into the analog affective flow of the user. In this sense, the functioning of the AFI is similar to the functioning of Deleuze and Guattari’s faciality machine, which “overcodes the body on the face” (Hansen 208), however, with the difference that it overcodes it at the level of the algorithm – and not at the level of the framing. The AFI extracts a data selfie from the facial affective input coming from the user, which is turned into an affective output compatible (or better, specifically designed) to match (or better, re-direct) the affective flow of the user – thus conditioning the user’s behavior, online and offline.

Finally, the faciality machine of Deleuze and Guattari seems still able to provide a useful conceptual tool to encompass both the functioning of the AFI and DFI. The DFI and AFI remain material instantiations of the abstract faciality machine, and the differences between them can be read as variations. The different role of affects between faciality machine, DFI and the AFI proves the extreme flexibility of the facial machine – unsurprisingly capable of holding instantiations with very different political implications, as expected from a machine embedded in the semiotic fluxes of late capitalism.