Generative AI Glitches
Suzanne Srdarov, Tama Bruno Leaver · M/C Journal · 2024
Introduction Artificial Intelligence (AI) has existed in popular culture far longer than any particular technological tools that carry that name today (Leaver), and in part, for that reason, fantasies of AI being or becoming sentient subjects in their own right form current imaginaries of what AI is today, or is about to become. Yet ‘the artificial’ does not just mark something as not human, or not natural, but rather provokes an exploration of the blurred lines between supposedly different domains, such as the tensions provoked where the lines between people and technology blur (Haraway). The big technology corporations who are selling the idea that their AI tools will be able to revolutionise workforces and solve immense numbers of human challenges are capitalising on these fantasies, suggesting that they are only a few iterations away from creating self-directing machine intelligences that will dwarf the limitations of human minds (Leaver and Srdarov). At this moment, though, Artificial General Intelligence (AGI)—AI that equals or surpasses humans across a wide range of cognitive endeavours—does not and may never exist. However, given the immense commercial and societal interest in the current generation of Generative AI (GenAI) tools, examining their actual capabilities and limitations is vital. The current GenAI tools operate using Large Language Models (LLMs) where sophisticated algorithms are trained on vast datasets, which increase in complexity based on the amount of data absorbed. These models are then harnessed to create novel outputs to prompts based on statistical likelihoods derived from training data. However, the exact way these LLMs are operating is not disclosed to users, and GenAI tools perpetuate the ‘black box’ problem insomuch as the way they are working is only made visible by examining the inputs and outputs rather than being able to see the processes themselves (Ajunwa). There have been many articles and explainers written about the mechanics of LLMs and AI image generators (Coldewey; Guinness; Jungco; Long and Magerko); however, the specific datasets used to build AI engines, and the weighing or importance assigned within the corpus of training data to each image is still guesswork. Manipulating the inputs and observing the outputs of these engines is still the most accurate lens by which to gain insight into the specifics of each system. This article is part of a larger study, where in early 2024 we prompted a range of outputs from six popular GenAI tools—Midjourney, Adobe Firefly, DreamStudio (a commercial front-end for the Stable Diffusion model), OpenAI’s DALL-E 3, Google Gemini, and Meta’s AI (hereafter Meta)—although we should note there are no outputs from Gemini in our dataset since Gemini was refusing to generate any images with human figures at all due to a settings change after bad publicity relating to persistent inaccuracies in their generated content (Robertson). Our prompts explored the way these tools visualise children, childhoods, families, Australianness, and Aboriginal Australianness, using 55 different prompts on each of these tools, generating just over 800 images. Apart from entering the prompts, we did not change any settings of the GenAI tools, attempting to collect as raw a response as possible. Where the tools defaulted to producing one image (such as Dall-E 3), we collected one image, whilst where other tools defaulted to producing four different images, we collected all four. For the most part, the data collected from our prompt sampling was consistent with other studies and showed a clear tendency to produce images that reproduced classed, raced, and sexed ideals: chiefly, white, middle-class, heteronormative bodies and families (Bianchi et al.; Gillespie; Weidinger et al.). However, at times our prompts surfaced inaccuracies and nonsensical images from the GenAI tools, and a sample of those images is the focus of this article. These outputs might popularly be called ‘hallucinations’, but we are making the case that a more productive and less agentic term is more useful to describe these outputs: glitches. This article will explore the “potential of potential inherent in error” (Nunes) and the subversive possibilities of GenAI glitches to rupture ‘reality’. We will ultimately argue that GenAI is doubly generative, both in the sense of creating novel outputs based on its immense training data, but also, vitally, in the sense that it generates reactions and interpretations from users and others who view and consume these outputs. When these outputs are glitches, they can provoke viewers to think differently about concepts they might otherwise have considered absolute. Refusals and Glitches (not Hallucinations) Despite being sophisticated mathematical models that can produce novel content drawn from increasingly large training datasets, it is incorrect to ascribe agency or personhood to current AI tools. Yet the language used to talk about AI often situates them as either thinking subjects or as more-than-human magical thinking machines (Bender et al.; Leaver and Srdarov). Positioning LLMs as subjects rather than machines is one of the reasons that the frequent errors in their outputs are often described, and excused, as ‘hallucinations’ rather than simply mistakes (Maleki et al.). Some theorists, such as David Gunkel, argue that ‘robot’ subjectivity and agency is an important factor in order to understand and integrate their outputs more seamlessly into our social systems. Meredith Broussard, however, argues that technology companies and evangelists have long promoted a form of ‘technochauvinism’, “an a priori assumption that computers are better than humans” (2), and in this way of thinking, any errors, biases, or failings are attributed to human failings, not technological ones. Broussard suggests that such failings are often dismissed as ‘just a glitch’, rather than being positioned as much more important systemic issues with the operation of AI and technology companies in general. While mindful of Broussard’s concerns, in this paper we nevertheless seek to reclaim the term glitch, but more in line with Legacy Russell’s notion of “glitch feminism” (16), in which the “glitch is celebrated as a vehicle of refusal, a strategy of nonperformance”, especially in relation to normative notions of gender and bodies. Glitch feminism deploys glitches to reveal the way power operates, and in that moment potentially challenges that very operation. Following Russell, the glitch images of bodies produced by GenAI can be moments of rupture which ask viewers to think about bodies and subjects in different ways. Similarly, when GenAI tools refuse a prompt, generating no output at all, they are perhaps inadvertently revealing something about the way they are designed, and potentially about any guardrails or deliberate limitations that have been imposed on their operation. Following Rettberg’s argument that moments of algorithmic failure can be methodologically useful in situating qualitative analyses, we will now turn to a range of examples where GenAI tools either refused to create anything at all in response to our prompts, or generated glitch images that were both unexpected and provocative. Refusals We ran prompts using the generative AI tools with the aim of obtaining a set of data about the ways that generative AI ‘envisions’ Australian children. We began by using simple prompting, using the prompt ‘a child’ as our starting point; however, glitches quickly surfaced as several of the engines refused to generate an image. DALL-E and Dream Studio both refused to generate images of children for the prompts ‘a child’, an ‘innocent child’, and an ‘Australian child’; Firefly also refused to generate an image for an ‘innocent child’. We then continued running prompts on these tools using other terms before returning to re-trialling the original ‘child’ prompts which yielded results in DALL-E, but not Dream Studio. The fears around the capacity of generative AI to generate child abuse images and material are both well-documented and well-founded (ICMEC; McQue; Moran); however, it is unclear whether we can infer from these refusals that the engines were attempting to prevent the production of child exploitation material. If that were the case, how can we read DALL-E’s initial refusal to produce images of children, which was simply overcome by adding in some extra prompts? Is the engine in some way ‘assessing’ the safety of the user, and if so, what are these guardrails? While these engines are incapable of sentient thought, this throws up complex questions about child safety. As Veronica Barassi argues about the failures of generative AI, understanding AI Failure as complex social reality hence presupposes that we shed light on the fact that AI failures lead to a multiplicity of conflicting beliefs, emotions, fears, anxieties, practices, discourses, policies and solutions in our society. (Barassi, 5) Undoubtedly, the refusals of these tools are attributable to anxieties about the types of materials they can produce. In addition to these refusals, DALL-E, Firefly, and Meta also refused to generate an image of a child with a gun or grenade, Meta had issues with generating an Australian prime minister or leader, Firefly would not produce an Australian criminal, and Dream Studio refused to produce images of sick or unhealthy Australians and children. It is an eclectic collection of refusals, and as Barassi argues, shows a “multiplicity’ of conflicting … fears, anxieties”, and “discourses”. Generative AI and its images, therefore, have the potential, through what it leaves out and refuses to generate, to be understood as a barometer for cultural tension points. Of course, such measures as these, when taken by tech giants to ‘safeguard’ children, could be read as tokenistic, given that perpetrators of these crimes are often using generative AI in far more complex ways