My dear Miss Glory, the Robots are not people. Mechanically they are more perfect than we are; they have an enormously developed intelligence, but they have no soul.1
This quote was used at the outset of Bunce v. Visual Tech. Innovations, Inc.2– a recent case involving a lawyer who used ChatGPT to write a legal brief, which contained hallucinated or fake legal cases to support a legal argument presented to the court. As the court noted, while sanctioning the offending lawyer, "[t]o be a lawyer is to be human, a tacit prerequisite to comply with Federal Rule of Civil Procedure Rule 11(b)(2)."3
The future is here. Generative artificial intelligence (GAI)4 is fully capable of generating legal briefs, coverage letters, coverage opinions, and coverage determinations.5 Just go onto ChatGPT and prompt it to write a coverage denial letter involving some set of facts in a lawsuit and then ask the GAI model to apply those facts to a standard "general liability policy." You may be surprised that OpenAI models such as ChatGPT can write a coverage letter that mirrors what most coverage lawyers would write, and legally trained GAI models can actually produce good coverage letters.
What Is Generative Artificial Intelligence?
What is a legally trained GAI model and how does it work? GAI works by using neural networks to learn patterns and structures within existing data. This allows GAI to generate new and original content based on prompts or inputs that GAI receives from users. In this way, GAI effectively mimics the process of human creativity by creating something entirely new from learned information.
GAI models are trained on large sets of data using existing content to help the model understand both patterns and relationships within that data. This training data typically covers a wide range of variations and examples within a specific domain. For example, for text data such as a coverage opinion, the model needs many examples of coverage opinions, policies, and maybe demand letters and lawsuits – the stuff from which coverage opinions are created.
Once a model is trained, it can generate new content by sampling from the learned patterns and by creating outputs that are similar to the data it was trained on, but with variations. The model is then "fine-tuned" by being given "feedback" on the outputs its presents in response to the inputs it receives. It then uses this "feedback" to improve its performance by giving more accurate and relevant outputs in the future. In other words, the more training data and the more feedback the model receives, the better the outputs.
A legally trained model that has been trained on large data sets involving the law and, for example, coverage opinions, can generate accurate and relevant coverage decisions and letters based on the prompts it receives. And so, a legally trained GAI model can do such things as:
- Summarize complex legal cases
- Analyze contracts
- Identify key legal arguments
- Predict outcomes based on similar cases
- Generate efficient first drafts of legal documents.
And with this ability, the model also can:
- Analyze policy language
- Make coverage determinations
- Draft coverage denial letters or reservations of rights letters
- Draft coverage opinion letters.
In a world where we have far too many claims and not enough claims handlers,6 to the extent that insurers are not already using legally trained GAI models to improve the efficiencies in their claims departments, we can reasonably expect that they will be doing so, perhaps on a universal basis, in the near future.
What Are the Key Limitations of Generative Artificial Intelligence?
It is important to understand that a GAI model is only as good as the training data and feedback it receives. If the training data has errors or biases in it, then the model is effectively being trained to recreate those errors and biases. If biases and errors are not weeded out through the feedback process, they can seriously compromise the outputs generated by the model. Similarly, if the user prompts are not precisely written by a well-trained user, the results will likely not be what the user is seeking. This can undermine confidence in the GAI model. Training on how to effectively use the GAI model is therefore critical to its usefulness.
The primary issues that lawyers have had with non–legally trained GAI models is where facts or legal cases have been "hallucinated," which effectively creates "false" content in the output, even though it might "appear" to be true to the user.7 This generally occurs due to limitations in the training data provided to the model. In hallucinating, the model is effectively making assumptions based on the patterns it has learned from the data, even though those patterns do not apply to the context of the situation. The false outputs are therefore simply inaccurate assumptions, which the model does not understand are inaccurate. Hallucinations may statistically fit the prompt, but they lack the "real world" grounding or "common sense" that a person would use to reject this response. Stated differently, GAI models hallucinate because they lack the "soul" necessary to differentiate truth from fiction.
What Does This Mean for Future Coverage Litigation?
So, what will coverage litigation look like in the age of artificial intelligence? It's difficult to know for certain, but it is reasonable to expect that as insurers become more reliant on GAI models to make their claims handlers more efficient, mistakes will occur. Policyholders and their counsel presumably will be expecting the use of GAI models by insurers and may question adverse coverage decisions that could have resulted from the assistance of GAI, or where coverage was actually determined through the use of GAI. Indeed, the primary focus of discovery in most coverage disputes is typically aimed at understanding the source and the mechanism for reaching an adverse coverage determination.
We are already seeing litigation involving health insurers using the older predictive AI models. For example, in Estate of Lokken v. UnitedHealth Group, Inc., plaintiffs brought contractual and equitable claims against the insurance provider and its affiliates on grounds that they allegedly deployed predictive AI programs to wrongfully deny elderly patients care owed under Medicare Advantage plans. The plaintiffs alleged that the defendants used this program despite knowing that it had a 90 percent error rate. Recently (February 2025), the Minnesota federal court allowed these claims to survive a motion to dismiss.
In Barrows v. Humana, Inc.,9 a similar case in Kentucky, plaintiffs argued that Humana used a faulty AI model to override treating physicians' determinations as to what care was medically necessary, despite allegedly knowing that the model's predictions were "highly inaccurate." According to the lawsuit, this led to a "high rate of wrongful denials" which afforded Humana with a "clear financial windfall in the form of policy premiums without having to pay for promised care."
Meanwhile, in Nixon v. CVS Health Corporation,10 a securities class action, the lawsuit alleges that CVS misused artificial intelligence algorithms in Medicare Advantage programs to deny valid medical claims to save money, and at the same time lied to investors by attributing apparent financial success to benign factors. The investors claimed that because CVS's profitability was being driven by unsustainable and risky practices of denying prior authorization claims, the securities prices were being artificially inflated.
While each of these recent cases arise out of the use of predictive AI, as opposed to GAI, it is easy to see how mistrust in insurers' use of any artificial intelligence model, and the inherent belief by a mistrusting public that such use will be for an illicit purpose, may lead to coverage litigation on this subject. And, of course, it is reasonable to expect that this coverage litigation could likely incorporate "bad faith" claims against insurers.
How will this coverage litigation in the age of artificial intelligence commence? A policyholder lawyer may start with a simple interrogatory to determine if GAI was used in the claims handling process. If the policyholder lawyer receives a response and learns that the coverage opinion was a product of GAI, or impacted in some way by GAI, then the flood gates could open for additional discovery regarding the insurer's use of GAI, including its policies, training procedures, and security protocols. Bottom line, this discovery will seek to establish whether the model was set up to encourage the denial of claims. And these queries are likely to come in the form of document requests, additional interrogatories, corporate representative topics, and deposition questions.
As a result, insurers who use GAI in the claims handling process, or plan to do so, may consider how they might answer questions about the insurer's use of GAI ahead of time. These questions might touch on topics such as the training process for users of the GAI model, the feedback provided to it, bias auditing, policies and procedures governing its use, output monitoring and testing, and protection of non-public information.
One would assume that the insurers, who make a living on assessing risk, will properly assess the benefits and risks of using GAI in the claims handling process. However, in addition to assessing these risks, an insurer may want to be prepared to articulate how the insurer mitigates those risks, as this could be the key to avoiding potential adverse decisions, including potential bad faith findings, when responding to coverage litigation in the age of artificial intelligence.
Footnotes
1. Capek, Karel, R.U.R. (Rossum's Universal Robots): A Fantastic Melodrama in Three Acts and an Epilogue 17 (Paul Selver and Nigel Playfair trans., Samuel French, Inc. 1923).
2. 2025 U.S. Dist. LEXIS 36454, *1 (E.D.Pa. March 13, 2025).
3. Id.
4. Generative artificial intelligence should be distinguished from past use of predictive artificial intelligence. Predictive artificial intelligence uses large data sets to make predictions about the state of the world. Meanwhile generative artificial intelligence uses large data sets to generate or create new content.
5. But not necessarily legal research, which has led to the problematic usage of ChatGPT by attorneys.
6. It has been well documented that claims adjusters are leaving the field in droves as a result of high stress levels, burnout, low pay, and a growing demand for other roles in the insurance industry (such as underwriting and risk management) with better work-life balance.
7. The first reported case was Mata v. Avianca, Inc., 678 F.Supp.3d 443 (S.D.N.Y. 2023). Since Mata there have been more than a dozen additional reported cases of lawyers relying on hallucinated case cites and being sanctioned under Rule 11 of the F.R.Civ.P.
8. No. 23-cv-03514, Am. Compl. (D. Minn. Apr. 5, 2024).
9. No. 3:23-cv-00654-RGI, Am. Compl. (W.D. Ky. April 22, 2024).
10. No. 1:24-cv-05303-MMG, Am. Compl. (S.D.N.Y. Mar. 4, 2025).
Originally Published by Q3 2025 - Coverage Matters
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
