At the Summer 2022 National Meeting of the National Association of Insurance Commissioners ("NAIC"), the Innovation, Cybersecurity, and Technology (H) Committee and its Big Data and Artificial Intelligence (H) Working Group held their first Collaboration Forum session on the topic of algorithmic bias. The Collaboration Forum was established at the Spring National Meeting as a platform for multiple NAIC committees to work together to identify and address foundational issues and develop a common framework that can inform the specific workstreams in each group.

Rather than being a single event, the Collaboration Forum is intended to promote ongoing discussion among insurance industry stakeholders during regularly hosted events and presentations. The Collaboration Forum on Algorithmic Bias was designed to cover issues such as what kinds of algorithms raise concerns for insurance regulators, how bias might arise in algorithms, which tools might be effective in minimizing bias and detecting bias, and what are potential regulatory frameworks for addressing algorithmic bias.

The presentations made during the Collaboration Forum at the Summer 2022 National Meeting covered the following topics:

  • Perspectives on Artificial Intelligence ("AI") Risk Management and Governance
  • Bias Detection Methods and Tools
  • Ethical and Responsible Use of Data and Predictive Models
  • Today's Approaches to Algorithmic Bias
  • The Risk of Biased AI

Some of the key themes explored during these presentations were the following:

  • Risk Management Approach to AI: Several presenters discussed that, in the absence of more specific guidance from insurance regulators on the use of AI/machine learning ("ML"), the industry should treat its use of AI/ML as part of regular risk management. That is, a comprehensive AI/ML risk management and governance framework should include the following components: development and communication of written policies and procedures (including assignment of responsibility and accountability with respect to such policies and procedures), training and monitoring with regard to the policies and procedures, and taking corrective action (and documenting that action) when the policies and procedures are not followed.

  • Ethical Use of Data and Predictive Models: Several presenters discussed the principles that they believe should guide the industry's use of AI/ML, including fairness, safety, transparency and accountability. There was significant discussion of how the industry, guided by these principles, could avoid bias in all stages of AI/ML model development, including during the pre-design, design and development, testing and evaluation, and deployment stages.

  • The Need for Testing: Several presenters emphasized the need for testing as a critical tool for identifying unintended discrimination. There are several forms of testing available that could be used to identify bias, including the Control Variable Test, the Interaction Test, the Nonparametric Matching (Matched Pairs) Test, and the Double Lift Chart. According to the presenters, the appropriate test for any particular model will vary based on the model type, the intended use, the output, the volume of data available, and the granularity of protected class data available.

  • Access to Protected Class Data: The issue that insurers currently do not have systematic data about policyholders' membership in protected classes was raised several times during the discussion. The lack of this data could make testing for bias more difficult.

  • The Need for Diversity: Several presenters highlighted the importance of diversity in combating algorithmic bias. They explained that, to prevent bias in the development stage, models should be established with diverse users in mind, and a diverse and inclusive workforce is critical for the oversight or monitoring of AI/ML use because diverse perspectives can help identify bias.

  • Model Explainability: Several presenters emphasized the importance of transparency and model explainability. In furtherance of this guiding principle, a proposal was made to develop model cards, which would present certain basic information about an AI/ML model (e.g., a description of the model goals, limitations of the model, trade-offs with respect to the use of the model and performance of the model). This proposal was described as being the equivalent of nutrition labels for AI/ML models.

The insights shared at the Collaboration Forum will be used by the Innovation, Cybersecurity, and Technology (H) Committee and its Big Data and Artificial Intelligence (H) Working Group to evaluate existing regulatory frameworks for overseeing and monitoring the use of big data, algorithms, and machine learning—including AI/ML in underwriting, rating, claims and marketing practices of insurers—potentially leading to the development of or modifications to model laws, regulations, handbooks and regulatory guidance.

Visit us at mayerbrown.com

Mayer Brown is a global legal services provider comprising legal practices that are separate entities (the "Mayer Brown Practices"). The Mayer Brown Practices are: Mayer Brown LLP and Mayer Brown Europe - Brussels LLP, both limited liability partnerships established in Illinois USA; Mayer Brown International LLP, a limited liability partnership incorporated in England and Wales (authorized and regulated by the Solicitors Regulation Authority and registered in England and Wales number OC 303359); Mayer Brown, a SELAS established in France; Mayer Brown JSM, a Hong Kong partnership and its associated entities in Asia; and Tauil & Chequer Advogados, a Brazilian law partnership with which Mayer Brown is associated. "Mayer Brown" and the Mayer Brown logo are the trademarks of the Mayer Brown Practices in their respective jurisdictions.

© Copyright 2020. The Mayer Brown Practices. All rights reserved.

This Mayer Brown article provides information and comments on legal issues and developments of interest. The foregoing is not a comprehensive treatment of the subject matter covered and is not intended to provide legal advice. Readers should seek specific legal advice before taking any action with respect to the matters discussed herein.