OSFI Announces Proposed Revisions To Guideline E-23 On Model Risk Management

On May 20, 2022, the Office of the Superintendent of Financial Institutions ("OSFI") released a letter "Proposed Revisions to Guideline E-23 on Model Risk Management (the "Letter") to all federally regulated financial institutions ("FRFIs") and federally regulated pension plans ("FRPPs") outlining OSFI's proposed revisions to Guideline E-23, Enterprise-wide Model Risk Managementfor Deposit-Taking Institutions ("Guideline E-23"). With the proposed revisions, OSFI intends to provide clarity for deposit-taking institutions ("DTIs") in the areas of enterprise-wide model risk management guidance, their scope when applied to the guideline, and the application of the proportionality principle towards smaller institutions.

While the original Guideline E-23 issued in September 2017 applied only to DTIs, significantly OSFI indicated in the letter that the guidance to be provided in the revised Guideline E-23 should also extend to federally regulated insurance companies ("FRICs") and FRPPs, as this will provide consistent guidance across all FRFIs (including both DTIs and FRICs) and FRPPs in the area of risk management of models.

What is Model Risk Management?

Model risk management is used by FRFIs and FRPPs to make business decisions. OSFI has recognized that these models leverage significant quantities and types of data and use very complex techniques. When these complex data model techniques are combined with more advanced analytics such as digitization or the use of artificial intelligence and machine learning ("AI/ML"), there is likely an increase of model risk.¹ With the proposed revision of Guideline E-23, OSFI has set the expectation that all FRFIs and FRPPs need to appropriately assess and manage model risk at the enterprise level - which OSFI states will be specifically tailored using a balanced proportionality approach that will be applied to each model risk management framework that enterprises operate upon.

Proposed Revisions to Guideline E-23

As part of OSFI's mandate to monitor prudential risk and solvency of the entities that it supervises, OSFI wants to preserve FRFIs' and FRPPs' abilities to continue to innovate and stay agile while also reinforcing the importance of responsible model risk management. To accomplish this, OSFI has set out three new principles of Soundness, Accountability and Explainability that will apply to all in-scope models - with the degree of compliance depending on the models' materiality, complexity and use.

Soundness

In order to improve model soundness, OSFI has identified five key issues for FRFIs and FRPPs to focus on:

Data: OSFI suggests stronger coverage of controls and governance through data lineage to address the ever-increasing amounts and variety of data and the speed at which data is being leveraged in model development.
Model Development, Validation and Implementation: OSFI intends to strengthen the accuracy employed by model owners, users and validators by ensuring more robust models that are more reliable, and have appropriate oversight of implementation and contingencies in place for model failures.
Monitoring: Timely and effective monitoring involving the appropriate frequency and intensity is required for AI/ML models and early warning against model failures ensures proper model performance.
Bias: This manifests in models in many different forms, which can lead to concerns around fairness in the AI/ML space. Model bias is an emerging topic in this area as it can pose a reputational risk to enterprises.
Documentation: Provide a more agile approach to documentation that acknowledges model risk while taking into consideration current industry trends towards more flexible model development that balances frequent recalibration without always introducing new models. OSFI suggests that this will provide the opportunity for enterprises to leverage platforms within the model lifecycle and recognizes the varied collective contribution towards model documentation.

Accountability

OSFI intends to enhance the scope of Guideline E-23 using a risk-based approach that incorporates more models than capital calculation and risk management. This suggestion stems not only from the increasing reliance on AI/ML models in this space, but also from the number and variety of stakeholders that continue to increase over the model lifecycle. OSFI has identified four key areas of focus to further enhance accountability:

Legal and Compliance: Ensure multidisciplinary model risk management includes control functions.
Interrelationships: A focus on communication between various models and data that allows data lineage to be transparent and effective.
Technology: Advances in AI/ML and other technologies have led to evolving model risks.
Transparency: The opacity of models and third party dependencies have a direct effect on model outcomes and results.

Explainability

OSFI suggests a renewed focus on the explanation of model outputs, as it reinforces the principles of soundness and accountability, which will further enhance the ability for enterprises to mitigate risks or unintended outcomes while using models. However, they do acknowledge that the degree of explainability will vary and they have attributed two specific factors to determine the level of explainability required:

The intended model use across business areas of the organization; and
The different types of model stakeholders, which includes taking into consideration the varying goals of stakeholders and how that impacts the scope of explainability required.²

In addition, OSFI recognizes that these two levels may not always encompass all facets of explainability and encourages robust discussion on the intuitiveness of the model drivers and the roles involved with the model explainability assessment. OSFI recommends this discussion include, but not be limited to, the dynamic nature of AI/ML models, the limited capacity to explain third party products and the monitoring requirements of model explainability.

International Developments in Responsible AI Standards and Regulation

The OSFI publication of the Letter comes at a time that has been particularly active as regards the development of standards and regulation for the responsible deployment of AI. For example, in April 2021, the European Commission published a proposal for a regulatory framework for AI that contains dedicated requirements for AI trustworthiness and AI risk management. The requirements will be supported by harmonised standards developed by European Standardisation Organisations related to risk management and a unified approach to trustworthiness. Moreover, in March 2022, the U.S. National Institute of Standards and Technology ("NIST") released the first draft of an AI Risk Management Framework.

Members of the McCarthy Tétrault team have been active participants in identifying and developing core principles for the responsible deployment of AI, such as accountability, fairness, non-discrimination and explainability. Please see our posts which address a set of analogous responsible data governance principles: Responsible AI: A Global Policy Framework 2021 Edition | McCarthy Tétrault (which contains a "responsible AI impact assessment" template that may well be of practical assistance); and International Technology Law Association releases Responsible AI Policy Framework | McCarthy Tétrault

What's Next?

OSFI is planning a consultation process for March 2023 on Guideline E-23, with a plan for the final guidance to be published by the end of 2023, with an implementation target by June 2024. All FRFIs and FRPPs are encouraged to provide their input on Guideline E-23 where additional detail or clarity would be beneficial. Any comments or input should be submitted to [email protected] by June 30, 2022.

Footnotes

1 See OSFI's discussion paper, Developing Financial Sector Resilience in a Digital World.

2 Examples of stakeholders include senior management, model owners, customers, auditors and regulators.

To view the original article click here.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.