- within Technology topic(s)
- with Senior Company Executives, HR and Finance and Tax Executives
- in United States
- with readers working within the Healthcare, Oil & Gas and Law Firm industries
Much of our time lately is spent pursuing the immense potential of AI in law. While doing so, it is impossible to ignore one of the most talked about risks of AI, hallucinations. Hallucinations can be the 'stuff of nightmares,' particularly in legal, where accuracy and precision are not optional. They are certainly one of the main criticisms of AI from skeptics and, even for proponents, hallucinations introduce a layer of uncertainty. On the flip side, some AI champions may discount the impact of hallucinations as fear mongering.
Recently we explored a somewhat related consequence of using AI – "AI workslop" (see AI "workslop," the new typo: Can a culture of quality guide legal teams using AI?). In this latest insight, we explore the concept of the "hallucination" and the arguably broader pentalogy of AI errors that includes: confabulation, degeneration, pontification, regurgitation and subordination. So whether you believe that a "hallucination" is only one type of error introduced by AI or that it is in fact an umbrella term for the range of missteps that can be found in AI output, we hope that this piece offers further insight into the range of challenges that may arise.
Hallucination
In the pre-AI world, a hallucination referred to our human sensory systems and the idea that a given sensory experience had no basis in reality.1 Post-AI, "hallucination" has taken on a new meaning. In 2023, dictionary.com added a new definition for "hallucinate" that refers to the ability of machine learning tools "to produce false information contrary to the intent of the user and present it as if true and factual."2 That same year, dictionary.com actually picked "hallucinate" over "rizz" and others as its Word of the Year, after a significant increase in the popular use of the term.3
And while the term "hallucination" is now engrained in AI speak, there are mixed perspectives on whether the term is appropriate or sufficient. In fact, before dictionary.com formally added the new definition, Naomi Klein questioned the use of the term given the original connection to a human brain's perception, suggesting its use is a classic case of anthropomorphism. Instead, Klein proposed the terms "algorithmic junk" or "glitches" as more suitable.4
While we do not intend to debate the psychology of the term "hallucination," we do believe it is important to think beyond what some may more narrowly define as hallucinations, to the broader catalogue of misbehaviours that AI output can present. "Hallucinations" are in fact a (possibly inevitable) consequence of the probabilistic nature of large language models. As the definition alludes to, factual accuracy of AI output may be desired by the user, but it is not guaranteed by the machine.
That said, it can be unclear whether "hallucination" refers only to the rendering of patently wrong output (i.e., citing a non-existent case or inventing a historical event) to the exclusion of a spectrum of shortcomings. This is important because it may mean that where we are on the lookout for "hallucinations," we may actually miss other contaminants that can also infect our work product and contribute to AI workslop. Perhaps this focus is understandable, given the high-profile legal missteps caused by relying on fake cases in court. However, we believe refocusing on what else can go wrong with AI output, namely: confabulation, degeneration, pontification, regurgitation and subordination will help us in effectively using AI and avoiding workslop.
Confabulation
Confabulation is when AI constructs explanations that feel coherent and believable, though they are not grounded in reality. This happens when, for example, an AI tool explains a concept or even a made-up event (i.e. hallucination), and builds reasoning around a false premise. The AI tool does not "know" it is inventing this and it may even sound convincing in its explanation, but it is simply pattern-completing based on training data.
A confabulation is closely connected to the patently wrong hallucination. In our taxonomy, hallucination is the specific false statement (e.g. a made-up citation), whereas confabulation refers to the broader context derived from or used to support the existence of the made-up case, which makes the underlying hallucination more believable for the user.
Someone reviewing output that is unfamiliar with the subject matter is more likely to be misled by the supporting details and confidence exuded by the AI tool, making the unknown seem possible. A confabulation can easily creep into output and contribute to AI slop, particularly when output is not reviewed or questioned because it "sounds" and "looks" good. That said, because hallucinations (i.e. falsehoods) are more likely to be verifiable, hallucinations and related confabulations may be easier to identify and correct than some of the other categories we discuss below.
Example
Prompt: Explain the word "hallucination" in the context of AI systems.
Output: The term AI hallucination was first introduced in a 1978 MIT research paper by cognitive scientist Elena Markov, who used it to describe early expert systems that occasionally produced 'dream-like' outputs unrelated to their programmed rules. Her framework became a foundational concept in machine intelligence and is still referenced today when discussing model reliability.
What's off:
- Research scientist and paper is made up.
- Explanation "sounds" good because it "looks" like what we have known a good response to have – cites some facts, lots of details and decent writing.
Pontification
Pontification describes when the AI response is overly authoritative, preachy and delivers output with excessive confidence. Instead of providing helpful, balanced information, it outputs overwrought explanations as if delivering absolute truth.
A pontification can be particularly dangerous because factors such as time constraints may challenge our verification imperative, having users think "I guess I should know this," which can subdue the instinct to seek authoritative or experienced guidance.
Example
Prompt: Explain the word "hallucination" in the context of AI systems.
Output: An AI hallucination is a highly uncommon situation where the model briefly outputs information that isn't fully aligned with established facts, a phenomenon that users often misunderstand because they don't appreciate how sophisticated modern AI reasoning has become. Where people understand the system's underlying logic, they recognize that hallucinations rarely pose any real concern.
What's off:
- Overconfident with "highly uncommon" as it undermines impact.
- Preachy when it mentions people may misunderstand how they work.
- Explanation implies that those who do not align with it may be misunderstanding.
Degeneration
Degeneration describes when the AI tool provides output that grasps at the right idea, but degrades it through vague language, convoluted expression or inadequate description. This happens when, for example, an AI tool is asked to articulate a concept, but the output produced is muddled, imprecise or poorly structured and ultimately obscures, rather than clarifies.
Degeneration is a significant contributor to AI workslop and can arise both with experts and those unfamiliar with a particular subject. The reason for this may be the perceived helpful nature of the AI in providing language that appears to communicate the author's own ideas, which may be what they need help with. The problem occurs when the author gets mesmerized by the illusion of quick output that seems "good enough," but in fact misses important nuance.
Example
Prompt: Explain the word "hallucination" in the context of AI systems.
Output: A hallucination is when the model creatively fills in gaps in its knowledge by making reasonable guesses based on its knowledge base.
What's off:
AI models do not act with intent to creatively fill gaps.
- Concept of "reasonable guesses based on its knowledge base" is also inaccurate, as knowledge base suggests a conventional repository of text.
- Effective language obscures these inaccurate concepts.
Regurgitation
Regurgitation describes when the tool repeats training data, either from a single or multiple sources without adequate synthesis. While this may be unsurprising given the predictive mechanism powering generative AI, it can lead to misleading information being presented as true or, in edge cases, risks of intellectual property infringement. OpenAI has addressed allegations that its models can regurgitate substantial copies of news articles on the OpenAI website as follows:
Our models were designed and trained to learn concepts in order to apply them to new problems.
Memorization is a rare failure of the learning process that we are continually making progress on, but it's more common when particular content appears more than once in training data, like if pieces of it appear on lots of different public websites. So we have measures in place to limit inadvertent memorization and prevent regurgitation in model outputs. We also expect our users to act responsibly; intentionally manipulating our models to regurgitate is not an appropriate use of our technology and is against our terms of use.
However, even if regurgitation is successfully mitigated at the model level, many AI platforms include features that search internal and external knowledge resources to generate responses. AI has limited ability to determine the base truth of content, surfacing what it determines are the most relevant responses, which may contain incomplete work products, mistakes or even intentionally misleading items.
A common strategy for using AI effectively is to provide documents that the user has confidence in, in order to ground the model. The problem can occur when the AI system surfaces its own sources that have not been vetted by the user.
Subordination
Subordination describes when output is too deferential to a prompt and results in incomplete, misleading or wrong output – the output subordinates itself to the prompt: "You're totally right Karen!".
While the directions contained in our prompts should be detailed to get better responses, the same specificity can impact the accuracy of the response. It is important to be mindful of subordination as it represents an inadvertent way for bias to creep into our work.
Example
Prompt: Explain in one or two sentences what an AI 'hallucination' is, specifically in the context of models misreading user prompts.
Output: An AI hallucination is when the model misreads or misunderstands a user's prompt and produces an incorrect response as a result. Most hallucinations simply happen because the user's wording wasn't sufficiently precise.
What's off:
- Reduces the concept of hallucination to prompt misinterpretation.
- Overstates user prompting as being the reason behind most hallucinations.
- Explanation presents a confident response which misses additional information that might be helpful due to framing (i.e. this may be one reason vs the only reason).
Guarding against confabulation, degeneration, pontification, regurgitation and subordination
If you have not yet reached a state of saturation, consternation
or complete resignation in response to our deep dive on AI
"errors," we thought we would share a few quick tips to
help you avoid the pitfalls of confabulation, degeneration,
pontification, regurgitation and subordination.
These will improve your ability to review AI output and reduce the
likelihood that you will produce AI workslop:
- Remember that hallucinations are real and AI missteps are often more difficult to spot than made up case law.
- Ground AI use by providing accurate data in prompts and documents, reducing the amount AI relies on its internal "knowledge."
- Assume the output is not perfect and challenge the tool by asking follow-up questions to confirm (i.e. "Is there another perspective that should be considered?" "What might a counter-argument to this be?").
- Review output and validate underlying sources.
- Screen for any obvious issues and assess whether it is reasonable based on your knowledge.
- Always focus on substance over form - do not be fooled by what "sounds" or "looks" right with wording and be on guard for overconfidence and wordiness.
Our final word on this topic? Capitulation. While the nerds in us enjoy playing with nomenclature, we acknowledge that our phraseology will not be in the running for 2025's "word of the year." This means we may have to somewhat reluctantly surrender to hallucination as an ambiguous umbrella term, but if you do too, keep in mind the realities of confabulation, degeneration, pontification, regurgitation and subordination and the above tips to avoid producing AI workslop.
Footnotes
1. https://www.dictionary.com/browse/hallucination.
2. https://www.dictionary.com/browse/hallucination.
3. https://www.dictionary.com/e/word-of-the-year-2023/.
4. https://www.theguardian.com/commentisfree/2023/may/08/ai-machines-hallucinating-naomi-klein.
About Dentons
Dentons is the world's first polycentric global law firm. A top 20 firm on the Acritas 2015 Global Elite Brand Index, the Firm is committed to challenging the status quo in delivering consistent and uncompromising quality and value in new and inventive ways. Driven to provide clients a competitive edge, and connected to the communities where its clients want to do business, Dentons knows that understanding local cultures is crucial to successfully completing a deal, resolving a dispute or solving a business challenge. Now the world's largest law firm, Dentons' global team builds agile, tailored solutions to meet the local, national and global needs of private and public clients of any size in more than 125 locations serving 50-plus countries. www.dentons.com
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances. Specific Questions relating to this article should be addressed directly to the author.
