Imagine all the people deepfaked

A person removing a mask

Via Kottke.

This ‘deepfake’ video of lots of current and former world leaders and other famous people is interesting and provokes all sorts of questions. Some suggest legislation against them, which is what the US seems to be pursuing, but that of course asks further questions about how to ‘police’ them and who has agency. There are, perhaps, some interesting resonances with the increasing use of performance holograms to re-animate dead performers – but there, of course, the legal issues are different. Nevertheless, all sorts of ideas, aesthetic, ethical and otherwise, about ‘authenticity’ crop up (e.g. this from New Scientist, or this on trust in ‘evidence’ re ‘deepfakes’), which we will increasingly be provoked into discussing.

It is interesting, I think, that while those of us in what we call ‘critical’ social sciences or humanities have been developing fairly nuanced articulations of identity and subjectivity, arguing they are not necessarily essential and acknowledging how they are performed (for example), contemporary digital/ social media, and our uses of them, have forged new norms of ‘authenticity’ in relation to identity. Facebook wants ‘true’ names, for instance. “Finsta” (‘f’ denoting ‘fake’), the phenomenon of setting up hidden, often pseudonymous, Instagram accounts – only for selected friends (as opposed to your curated “rinsta” account (‘r’ denoting ‘real’)) shows how these two understandings of the performative nature of identity and the construction of a normative insistence on ‘authenticity’ collide. We might reasonably ask, for instance, why the ‘finsta’/’rinsta’ labels don’t actually mean the reverse if the more public of the two accounts is heavily curated and the ‘secret’ one is in some senses then more ‘authentic’.

‘Deepfakes’ are, amongst other things, a sensory ‘trick’, an attempt to somehow fool the conscious and sub-conscious habitual discernment of what feels whatever it is we mean by ‘authentic’, ‘genuine’ or ‘real’. In some senses, ‘deepfakes’ reveal back to us the extent to which digital media may have shifted how we pay attention and how we feel (about ourselves, others and the world around us) with and through them. Digital media cultivate attention in different ways, many of them perhaps oriented towards a capitalist imperative, also, perhaps, with them we cultivate forms of paying attention. If this is the case then, as was argued in terms of the potency of TV advertising, we may begin to ‘see through’ the ‘tricks’ precisely because we are bombarded with them (for example, the Putin bits in the video above are not very convincing to my eye). Or, to be pessimistic, we may simply begin to assume nothing can be trusted, that all media is created in bad faith, which of course prompts discussions of a crisis of democracy because how can a population make informed decisions without ‘trustworthy’ sources.

“AI will displace 40 percent of world’s jobs in as soon as 15 years” – Kai-Fu Lee

Industrial factory robot arms

In a widely-trailed CBS ’60 minutes’ interview, the A.I-pioneer-cum-venture-capitalist Kai-Fu Lee makes the sorts of heady predictions about job replacement/displacement that the media like to lap up. The automative imagination of ‘automation as progress’ in full swagger…

We should perhaps see this in the context of, amongst other things, geopolitical machinations (i.e. China-USA) around trade and intellectual property; a recently published book; a wider trend for claims about robotic process automation (especially in relation to ‘offshoring‘); and a large investment fund predicated upon ‘disruption’.

“Merger” by Keiichi Matsuda – automation, work and ‘replacement’

A still from the 360-degree video "Merger" by Keiichi Matsuda
“With automation disrupting centuries-old industries, the professional must reshape and expand their service to add value. Failure is a mindset. It is those who empower themselves with technology who will thrive.
“Merger is a new film about the future of work, from cult director/designer Keiichi Matsuda (HYPER-REALITY). Set against the backdrop of AI-run corporations, a tele-operator finds herself caught between virtual and physical reality, human and machine. As she fights for her economic survival, she finds herself immersed in the cult of productivity, in search of the ultimate interface. This short film documents her last 4 minutes on earth.”

I came across the most recent film by Keichii Matsuda which concerns a possible future of work, with the protagonist embedded in an (aesthetically Microsoft-style) augmented reality of screen-surfaces, and in which the narrative denouement is a sort of trans-human ‘uploading’ moment.

I like Matsuda’s work. i think he skilfully and playfully provokes particular sorts of conversations, mostly about what we used to call ‘immersion’ and the nature of mediation. This has, predictably happened in terms of human vs. AI vs. eschatology (etc etc.) sorts of narratives in various outlets (e.g. the Verge). The first time I encountered his work was at a Passenger Films event at which Rob Kitchin talked about theorisations of mediation in relation to both Matsuda’s work and the (original) Disney film ‘Tron‘.

What is perhaps (briefly) interesting here are two things:

  1. The narrative is a provocative short story that asks us to reflect upon how our world of work and technological development get us from now (the status quo) to an apparent future state of affairs, which carries with it certain kinds of ethical, normative and political contentions. So, this is a story that piggybacks the growing narrative of ‘post-work’ or widespread automation of work by apparently ‘inhuman’ technologies (i.e. A.I) that provokes debate about the roles of ‘technology’ and ‘work’ and what it means to be ‘human’. Interestingly, this (arguably) places “Merger” in the genre of ‘fantasy’ rather than ‘science fiction’ – it is, after all, an eschatological story (I don’t see this final point as a negative). I suppose it could also be seen as a fictional suicide note but I’d rather not dwell on that…
  2. The depiction of the interface and the interaction with the technology-world of the protagonist– and indeed the depiction of these within a 360-degree video –are as important as the story to what the video is signifying. By which I mean – like the videos I called ‘vision videos’ back in 2009/10 (and (in some cases) might be called ‘design fiction’ or ‘diagetic prototypes’) – this video is also trying to show you and perhaps sell you the idea of a technology (Matsuda recently worked for Leap Motion). As I and others have argued – the more familiar audiences are with prospective/speculative technologies the more likely we are (perhaps) to sympathise with their funding/ production/ marketing and ultimately to adopt them.

Call for papers: Geography of/with A.I

Still from the video for All is Love by Bjork

I very much welcome any submissions to this call for papers for the proposed session for the RGS-IBG annual conference (in London in late-August) outlined below. I also welcome anyone getting in touch to talk about possible papers or ideas for other sorts of interventions – please do get in touch.

Call for papers:

We are variously being invited to believe that (mostly Global North, Western) societies are in the cusp, or early stages, of another industrial revolution led by “Artificial Intelligence” – as many popular books (e.g. Brynjolfsson and McAfee 2014) and reports from governments and management consultancies alike will attest (e.g. PWC 2018, UK POST 2016). The goal of this session is to bring together a discussion explicitly focusing on the ways in which geographers already study (with) ‘Artificial Intelligence’ and to, perhaps, outline ways in which we might contribute to wider debates concerning ‘AI’. 

There is widespread, inter-disciplinary analysis of ‘AI’ from a variety of perspective, from embedded systematic bias (Eubanks 2017, Noble 2018) to the kinds of under-examined rationales and work through which such systems emerge (e.g. Adam 1998, Collins 1993) and further to the sorts of ethical-moral frameworks that we should apply to such technologies (Gunkel 2012, Vallor 2016). In similar, if somewhat divergent ways, geographers have variously been interested in the kinds of (apparently) autonomous algorithms or sociotechnical systems are integrated into decision-making processes (e.g. Amoore 2013, Kwan 2016); encounters with apparently autonomous ‘bots’ (e.g. Cockayne et al. 2017); the integration of AI techniques into spatial analysis (e.g. Openshaw & Openshaw 1997); and the processing of ‘big’ data in order to discern things about, or control, people (e.g. Leszczynski 2015). These conversations appear, in conference proceedings and academic outputs, to rarely converge, nevertheless there are many ways in which geographical research does and can continue to contribute to these contemporary concerns.

The invitation of this session is to contribute papers that make explicit the ways in which geographers are (already) contributing to research on and with ‘AI’, to identify research questions that are (perhaps) uniquely geographical in relation to AI, and to thereby advance wider inter-disciplinary debates concerning ‘AI’.

Examples of topics might include (but are certainly not limited to):

  • A.I and governance
  • A.I and intimacy
  • Artificially intelligent mobilities
  • Autonomy, agency and the ethics of A.I
  • Autonomous weapons systems
  • Boosterism and ‘A.I’
  • Feminist and intersectional interventions in/with A.I
  • Gender, race and A.I
  • Labour, work and A.I
  • Machine learning and cognitive work
  • Playful A.I
  • Science fiction, spatial imaginations and A.I
  • Surveillance and A.I

Please send submissions (titles, abstracts (250 words) and author details) to: Sam Kinsley by 31st January 2019.

Sophia – show robots and deception

Hanson Robotics' "Sophia"

…when first we practice to deceive…

Walter Scott

Prof Noel Sharkey has written a thoughtful, informative and entertaining piece for Forbes (so, for a general audience) that does some unpacking of ‘Sophia’ with reference to the history of ‘show robots’ (such as the Westinghouse show robots of the the mid-C20, like Elektro, and of course Honda’s Asimo). It’s worth reading the piece in full but here’s a couple of choice clips:

Sophia is not the first show robot to attain celebrity status. Yet accusations of hype and deception have proliferated about the misrepresentation of AI to public and policymakers alike. In an AI-hungry world where decisions about the application of the technologies will impact significantly on our lives, Sophia’s creators may have crossed a line. What might the negative consequences be? To get answers, we need to place Sophia in the context of earlier show robots.


The tradition extends back to the automata precursors of robots in antiquity. Moving statues were used in the temples of ancient Egypt and Greece to create the illusion of a manifestation of the gods. Hidden puppeteers pulled ropes and spoke with powerful booming voices emitted from hidden tubes. This is not so different from how show robots like Sophia operate today to create the illusion of a manifestation of AI.

For me, the biggest problem with the hype surrounding Sophia is that we have entered a critical moment in the history of AI where informed decisions need to be made. AI is sweeping through the business world and being delegated decisions that impact significantly on peoples lives from mortgage and loan applications to job interviews, to prison sentences and bail guidance, to transport and delivery services to medicine and care.


It is vitally important that our governments and policymakers are strongly grounded in the reality of AI at this time and are not misled by hype, speculation, and fantasy. It is not clear how much the Hanson Robotics team are aware of the dangers that they are creating by appearing on international platforms with government ministers and policymakers in the audience.

AI as organ -on|-ology

Kitt the 'intelligent' car from the TV show Knight Rider

Let’s begin with a postulate: there is either no “AI” – artificial intelligence – or every intelligence is, in fact, in some way artificial (following a recent talk by Bernard Stiegler). In doing so we commence from an observation that intelligence is not peculiar to one body, it is shared. A corollary is that there is no (‘human’) intelligence without artifice, insofar as we need to exteriorise thought (or what Stiegler refers to as ‘exosomatisation’) for that intelligence to function – as language, in writing and through tools – and that this is inherently collective. Further, we might argue that there is no AI. Taking that suggestion forward, we can say that there are, rather, forms of artificial (or functional) stupidity, following Alvesson & Spicer (2012: p. 1199), insofar as it inculcates forms of lack of capacity: “characterised by an unwillingness or inability to mobilize three aspects of cognitive capacity: reflexivity, justification, and substantive reasoning”. Following Alvesson & Spicer [& Stiegler] we might argue that such forms of stupidity are necessary passage points through our sense-making in/of the world, thus are not morally ‘wrong’ or ‘negative’. Instead, the forms of functional stupidity derive from technology/techniques are a form of pharmakon – both enabling and disabling in various registers of life.

Given such a postulate, we might categorise “AI” in particular ways. We might identify ‘AI’ not as ‘other’ to the ‘human’ but rather a part of our extended (exosomatic) capacities of reasoning and sense. This would be to think of AI ‘organologically’ (again following Stiegler) – as part of our widening, collective, ‘organs’ of action in the world. We might also identify ‘AI’ as an organising rationale in and of itself – a kind of ‘organon’ (following Aristotle). In this sense “AI” (the discipline, institutions and the outcome of their work [‘an AI’]) is/are an organisational framework for certain kinds of action, through particular forms of reasoning.

It would be tempting (in geographyland and across particular bits of the social sciences) to frame all of this stemming from, or in terms of, an individual figure: ‘the subject’. In such an account, technology (AI) is a supplement that ‘the human subject’ precedes. ‘The subject’, in such an account, is the entity to which things get done by AI, but also the entity ultimately capable of action. Likewise, such an account might figure ‘the subject’ and it’s ‘other’ (AI) in terms of moral agency/patiency. However, for this postulate such a framing would be unhelpful (I would also add that thinking in terms of ‘affect’, especially through neuro-talk would be just as unhelpful). If we think about AI organologically then we are prompted to think about the relation between what is figured as ‘the human’ and ‘AI’ (and the various other things that might be of concern in such a story) as ‘parasitic’ (in Derrida’s sense) – its a reciprocal (and, in Stiegler’s terms, ‘pharmacological’) relation with no a priori preceding entity. ‘Intelligence’ (and ‘stupidity’ too, of course) in such a formulation proceeds from various capacities for action/inaction.

If we don’t/shouldn’t think about Artificial Intelligence through the lens of the (‘sovereign’) individual ‘subject’ then we might look for other frames of reference. I think there are three recent articles/blogposts that may be instructive.

First, here’s David Runciman in the LRB:

Corporations are another form of artificial thinking machine, in that they are designed to be capable of taking decisions for themselves. Information goes in and decisions come out that cannot be reduced to the input of individual human beings. The corporation speaks and acts for itself. Many of the fears that people now have about the coming age of intelligent robots are the same ones they have had about corporations for hundreds of years.

Second, here’s Jonnie Penn riffing on Runciman in The Economist:

To reckon with this legacy of violence, the politics of corporate and computational agency must contend with profound questions arising from scholarship on race, gender, sexuality and colonialism, among other areas of identity.
A central promise of AI is that it enables large-scale automated categorisation. Machine learning, for instance, can be used to tell a cancerous mole from a benign one. This “promise” becomes a menace when directed at the complexities of everyday life. Careless labels can oppress and do harm when they assert false authority. 

Finally, here’s (the outstanding) Lucy Suchman discussing the ways in which figuring complex systems of ‘AI’-based categorisations as somehow exceeding our understanding does particular forms of political work that need questioning and resisting:

The invocation of Project Maven in this context is symptomatic of a wider problem, in other words. Raising alarm over the advent of machine superintelligence serves the self-serving purpose of reasserting AI’s promise, while redirecting the debate away from closer examination of more immediate and mundane problems in automation and algorithmic decision systems. The spectacle of the superintelligentsia at war with each other distracts us from the increasing entrenchment of digital infrastructures built out of unaccountable practices of classification, categorization, and profiling. The greatest danger (albeit one differentially distributed across populations) is that of injurious prejudice, intensified by ongoing processes of automation. Not worrying about superintelligence, in other words, doesn’t mean that there’s nothing about which we need to worry.
As critiques of the reliance on data analytics in military operations are joined by revelations of data bias in domestic systems, it is clear that present dangers call for immediate intervention in the governance of current technologies, rather than further debate over speculative futures. The admission by AI developers that so-called machine learning algorithms evade human understanding is taken to suggest the advent of forms of intelligence superior to the human. But an alternative explanation is that these are elaborations of pattern analysis based not on significance in the human sense, but on computationally-detectable correlations that, however meaningless, eventually produce results that are again legible to humans. From training data to the assessment of results, it is humans who inform the input and evaluate the output of the black box’s operations. And it is humans who must take the responsibility for the cultural assumptions, and political and economic interests, on which those operations are based and for the life-and-death consequences that already follow.

All of these quotes more-or-less exhibit my version of what an ‘organological’ take on AI might look like. Likewise, they illustrate the ways in which we might bring to bear a form of analysis that seeks to understand ‘intelligence’ as having ‘supidity’as a necessary component (it’s a pharmkon, see?), which in turn can be functional (following Alvesson & Spicer). In this sense, the framing of ‘the corporation’ from Runciman and Penn is instructive – AI qua corporation (as a thing, as a collective endeavour [a ‘discipline’]) has ‘parasitical’ organising principles through which play out the pharmacological tendencies of intelligence-stupidity.

I suspect this would also resonate strongly with Feminist Technology Studies approaches (following Judy Wajcman in particular) to thinking about contemporary technologies. An organological approach situates the knowledges that go towards and result from such an understanding of intelligence-stupidity. Likewise, to resist figuring ‘intelligence’ foremost in terms of the sovereign and universal ‘subject’ also resists the elision of difference. An organological approach as put forward here can (perhaps should[?]) also be intersectional.

That’s as far as I’ve got in my thinking-aloud, I welcome any responses/suggestions and may return to this again.

If you’d like to read more on how this sort of argument might play out in terms of ‘agency’ I blogged a little while ago.

ADD. If this sounds a little like the ‘extended mind‘ (of Clark & Chalmers) or various forms of ‘extended self’ theory then it sort of is. What’s different is the starting assumptions: here, we’re not assuming a given (a priori) ‘mind’ or ‘self’. In Stiegler’s formulation the ‘internal’ isn’t realised til the external is apprehended: mental interior is only recognised as such with the advent of the technical exterior. This is the aporia of origin of ‘the human’ that Stiegler and Derrida diagnose, and that gives rise to the theory of ‘originary technics’. The interior and exterior, and with them the contemporary understanding of the experience of being ‘human’ and what we understand to be technology, are mutually co-constituted – and continue to be so [more here]. I choose to render this in a more-or-less epistemological, rather than ontological manner – I am not so much interested in the categorisation of ‘what there is’, rather in ‘how we know’.

Talk – Plymouth, 17 Oct: ‘New geographies of automation?’

Rachael in the film Blade Runner

I am looking forward to visiting Plymouth (tomorrow) the 17th October to give a Geography department research seminar. It’s been nearly twenty years (argh!) since I began my first degree, in digital art, at Plymouth so I’m looking forward to returning. I’ll be talking about a couple of aspects of ‘The Automative Imagination’ under a slightly different title – ‘New geographies of automation?’ The talk will take in archival BBC and newspaper automation anxieties, management consultant magical thinking (and the ‘Fourth Industrial Revolution’), gendered imaginings of domesticity (with the Jetsons amongst others) and some slightly under-cooked (at the moment) thoughts about how ‘agency’ (what kinds of ‘beings’ or ‘things’ can do what kinds of action).

Do come along if you’re free and happen to be in the glorious gateway to the South West that is Plymouth.

(More) Gendered imaginings of automata

My Cayla Doll

A few more bits on how automation gets gendered in particular kinds of contexts and settings. In particular, the identification of ‘home’ or certain sorts of intimacy with certain kinds of domestic or caring work that then gets gendered is something that has been increasingly discussed.

Two PhD researchers I am lucky enough to be working with, Paula Crutchlow (Exeter) and Kate Byron (Bristol), have approached some of these issues from different directions. Paula has had to wrangle with this in a number of ways in relation to the Museum of Contemporary Commodities but it was most visible in the shape of Mikayla, the hacked ‘My Friend Cayla Doll’. Kate is doing some deep dives on the sorts of assumptions that are embedded into the doing of AI/machine learning through the practices of designing, programming and so on. They are not, of course, alone. Excellent work by folks like Kate Crawford, Kate Devlin and Gina Neff (below) inform all of our conversations and work.

Here’s a collection of things that may provoke thought… I welcome any further suggestions or comments 🙂

Alexa, does AI have gender?


Alexa is female. Why? As children and adults enthusiastically shout instructions, questions and demands at Alexa, what messages are being reinforced? Professor Neff wonders if this is how we would secretly like to treat women: ‘We are inadvertently reproducing stereotypical behaviour that we wouldn’t want to see,’ she says.

Prof Gina Neff in conversation with Ruth Abrahams, OII.

Predatory Data: Gender Bias in Artificial Intelligence

it has been reported that female-sounding assistive chatbots regularly receive sexually charged messages. It was recently cited that five percent of all interactions with Robin Labs, whose bot platform helps commercial drivers with routes and logistics, is sexually explicit. The fact that the earliest female chatbots were designed to respond to these suggestions
deferentially or with sass was problematic as it normalised sexual harassment.

Vidisha Mishra and Madhulika Srikumar – Predatory Data: Gender Bias in Artificial Intelligence

The Gender of Artificial Intelligence

Chart showing that the gender of artificial intelligence (AI) is not neutral
The gendering, or not, of chatbots, digital assistants and AI movie characters – Tyler Schnoebelen

Consistently representing digital assistants as femalehard-codes a connection between a woman’s voice and subservience.

Stop Giving Digital Assistants Female Voices – Jessica Nordell, The New Republic