Emerging AI Risks: The Importance Of AI-crisis Response Plans

The rapid adoption of generative Artificial Intelligence (“AI”) tools across the corporate landscape is giving rise to new categories of challenges and compliance risks. Organisations are facing AI-generated grievances and tip-offs, hallucinated content in corporate records and regulatory submissions, and AI-enabled fraud schemes involving deepfakes and fabricated documentation. It is important that organisation understand these challenges and have response plans in place to guide decision-making and actions.

AI-drafted grievances and whistleblower reports

One of the most immediate challenges is the proliferation of employee grievances and whistleblower reports drafted or substantially augmented using AI tools such as ChatGPT. Although these tools can be used to positive effect, including by empowering good faith whistleblowers, we have noted a rise in problematic trends emerging from the deployment of this technology, including:

• Factual embellishment. Large Language models are designed to produce persuasive, coherent text and show a tendency to elevate the apparent severity of underlying facts. A genuine but relatively minor workplace concern can, once processed through a generative AI tool, read as though it describes systemic misconduct. The resulting narrative is then more dramatic and the consequences are framed as far more grave.
• Volume and verbosity. AI-drafted reports are often substantially longer and more detailed than complaints drafted without AI. What might previously have been a one-page grievance or tip-off becomes a lengthy submission containing legal terminology, structured arguments with references to legislation, and even recommendations regarding investigation structuring and remedial action to be taken. For compliance teams operating under time pressure, the additional burden of distilling genuine issues from AI-generated verbosity is considerable.

The challenge is not determining whether AI was used, but whether factual allegations can be independently verified. Complaints cannot simply be disregarded where AI-enhancement is suspected. The underlying issues may still have merit even where they are padded with significant embellishment. In our experience, the most effective approach requires a careful assessment of the essence of the complaint, and a structured and defensible investigation methodology designed to test the veracity of the issues raised. Organisations should move quickly to preserve any potentially relevant evidence prior to proceeding to the investigation phase, In combatting the challenges associated with large datasets, incorporating advanced analytics and generative AI into the investigation methodology which ensures that large volumes of data can be processed efficiently as part of the fact-finding process.

Click here to read our intelligENS team’s insights on defensibly accelerating investigations with generative AI.

Click here to read our Employment team’s insights on AI-generated grievances in the workplace and the challenges they present for employers and human resource practitioners.

The risk of hallucinated content in corporate documents and public filings

There have been many high-profile public examples of AI hallucination in case law and despite the now familiar cautionary tale about the importance of human-oversight and review, these issues continue to be on the rise. As organisations increasingly adopt generative AI to increase efficiency, hallucinated content will inevitably make its way into regulatory submissions, investor communications and other corporate documents. Hallucinations can be especially difficult to detect where the misstatement is not so egregious as to be outright false, but where it strays into the realm of embellishment. Even in the latter instance, the misstated facts can still be misleading and expose the organisation to regulatory risk.

If such hallucinations are detected, a critical step in this response will be identifying what occurred, who was responsible, what actions have been taken to fix the error and what controls have been put in place to ensure this doesn’t happen again. Answering each of these questions will be crucial to providing an informed response to external stakeholders. Investigating these incidents requires a particular combination of skills: capability to gather relevant evidence in a forensically-sound manner, and analytical infrastructure to review large document sets.

AI as an instrument of fraud and corruption

Beyond its role as an inadvertent source of error, AI is increasingly being deployed as a deliberate tool of corporate fraud and corruption. Several emerging typologies are becoming apparent:

• Deepfakes and impersonation. Voice cloning and video deepfakes enable threat actors to impersonate senior executives, authorising fraudulent payments or sensitive disclosures in ways that circumvent traditional verification procedures.
• Fabricated documentation. False invoices, fabricated contracts, fictitious audit confirmations and invented supporting correspondence can now be produced at scale using AI.
• Fictitious audit trails. Generative AI can produce entire ecosystems of false documentation including shell company records, fictitious third-party communications, fabricated due diligence reports and so on.

Investigating AI-enabled fraud demands tools and methodologies capable of operating at scale. There is a degree of irony in that the solution to these AI-facilitated issues may lie in the responsible use of AI. Manual document review is insufficient when the volume of potentially fabricated material is large and its quality is high.

What should organisations do?

The challenges outlined above are manifesting now, across sectors. Organisations should consider whether their compliance, investigation and governance frameworks are equipped for an environment in which AI-generated content is increasingly common given that when these issues arise, they can exacerbate adverse outcomes and waste resources on ineffective investigations. Organisations should consider the following:

• reviewing existing compliance programmes and investigation protocols to assess whether they adequately address AI-related risks;
• ensuring teams have received proportionate and effective training on these programmes and protocols;
• ensuring investigation teams have access to advanced analytics and eDiscovery platforms capable of processing AI-generated content at scale; and
• updating whistleblower and grievance handling procedures to account for the likelihood that complaints may be AI-drafted or AI-augmented.

Organisations that prepare for this reality now will be far better positioned to manage any regulatory, legal and reputational risk in the future.