Stochastic parrot

Stochastic parrot is a metaphor used in the field of artificial intelligence to describe large language models (LLMs) as systems that can combine linguistic forms in statistically plausible ways, but lack any genuine understanding of their meaning.

The term was introduced in March 2021 in the research paper On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?, published at the FAccT conference. The authors were Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell.

Definition and Concept

According to the paper's authors, a stochastic parrot is "a system for haphazardly stitching together sequences of linguistic forms observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning".

The term consists of two parts:

• Stochastic — from the Ancient Greek στοχαστικός ("based on conjecture"), which in modern mathematics refers to a process determined by a random probability distribution.
• Parrot — an allusion to the ability of parrots to mimic human speech without understanding its meaning.

The concept posits that LLMs, trained to predict the next word in a sequence, are essentially sophisticated autocomplete systems that manipulate symbols without access to their meaning.

Key Arguments of "On the Dangers of Stochastic Parrots"

The paper highlights four main categories of risks associated with developing excessively large language models.

1. Unknowable Training Data

LLMs are trained on vast, unannotated datasets collected from the internet (e.g., Common Crawl). Such datasets inevitably contain biases, toxic language, and hegemonic viewpoints that harm vulnerable groups. For example, internet content disproportionately represents white men from developed countries (67% of Reddit users in the US are male).

2. Lack of Genuine Language Understanding

The authors argue that LLMs do not possess a genuine understanding of language. They refer to the theory that language is a system of signs where the form (the word) is inextricably linked to the meaning (the concept). The training data for LLMs contains only form, denying the model access to meaning. Therefore, LLMs only mimic meaningful speech.

3. Synthetic Text and Potential Harm

Since LLMs generate grammatically correct and convincing text, people are inclined to attribute meaning to it and trust it. This creates a risk of spreading disinformation, hate speech, and fraud. The more perfect the imitation, the higher the risk that people will overestimate the AI's capabilities and entrust it with critical decisions.

Academic Impact and Publication Controversy

The paper became the center of a major scandal at Google, where co-authors Timnit Gebru and Margaret Mitchell were employed at the time. In late 2020, during an internal review, Google management demanded that the authors either retract the paper or remove the names of Google employees from it.

Timnit Gebru, a leading AI ethics researcher, refused to comply with this demand, which led to her dismissal from Google in December 2020. In February 2021, Margaret Mitchell, who had spoken out in support of Gebru, was also fired. These events caused a widespread public outcry. More than 2,200 Google employees and thousands of members of the academic community signed a protest letter, accusing the company of academic censorship and suppressing research that could affect its commercial interests. Ultimately, the paper was published in March 2021 at the FAccT conference.

Reception, Debates, and Evolving Views

The "stochastic parrot" metaphor quickly spread and became a central point in the debate about the nature of artificial intelligence. The American Dialect Society (ADS) chose "stochastic parrot" as its AI-related Word of the Year for 2023, where it surpassed even "ChatGPT" and "LLM".

Criticism of the Concept and Evidence of Understanding

The concept has been challenged by many leading researchers.

• Geoffrey Hinton, one of the "godfathers" of deep learning, argued that "to accurately predict the next word, you need to understand the sentence". In 2023, after leaving Google, he stated that large models already "understand" what they are taught and can draw their own conclusions.
• Emergent abilities: Studies have shown that upon reaching a certain scale, LLMs exhibit a sudden emergence of new abilities, such as solving arithmetic problems, that were not explicitly programmed into them.
• Internal world models: A 2022 study showed that a model trained to play Othello based on textual records of moves spontaneously formed an internal representation of the game board, suggesting the development of an abstract model of the world it describes.
• Performance on benchmarks: Modern models, such as GPT-4, achieve human-level (or higher) performance on complex professional exams, which some argue is impossible without understanding.

Ironic Usage and Public Discourse

The term became so popular that it was even used ironically by OpenAI CEO Sam Altman, who tweeted: "i am a stochastic parrot, and so r u". With this, he implied that human speech is also largely a probabilistic prediction of the next word, playing on the criticism leveled at AI.

Impact on Scientific Discourse

The "stochastic parrot" metaphor remains central to debates about the capabilities and limitations of LLMs. It has helped articulate the problem of language models' lack of genuine understanding and has drawn attention to the risks associated with their development. At the same time, rapid progress in the LLM field compels a constant reassessment of this metaphor, as the latest models exhibit increasingly complex behavior that does not fit the image of a "mindless parrot". The term continues to influence scientific discourse, emphasizing the importance of critically analyzing the capabilities of AI systems and their social consequences.

Literature

• Bender, E. M.; Gebru, T.; McMillan-Major, A.; Mitchell, M. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. FAccT 2021.
• Floridi, L.; Chiriatti, M. (2020). GPT-3: Its Nature, Scope, Limits, and Consequences. Minds & Machines, 30(4), 681-694.
• Bommasani, R. et al. (2021). On the Opportunities and Risks of Foundation Models. arXiv:2108.07258.
• Weidinger, L. et al. (2021). Ethical and Social Risks of Harm from Language Models. arXiv:2112.04359.
• Kaplan, J. et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361.
• Wei, J. et al. (2022). Emergent Abilities of Large Language Models. arXiv:2206.07682.
• Perez, E. et al. (2022). Red Teaming Language Models with Language Models. EMNLP 2022.
• Bai, Y. et al. (2022). Constitutional AI: Harmlessness from AI Feedback. arXiv:2212.08073.
• Du, Z. et al. (2024). Understanding Emergent Abilities of Language Models from the Loss Perspective. arXiv:2403.15796.
• Gerstgrasser, M. et al. (2024). Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data. arXiv:2404.01413.