Where there is a model, there is model risk. Any company which is deploying "next-level" capabilities should ask itself, do we also have 'next-level' management techniques to support our new capabilities?
New technologies, market complexities, and new demands on the business are driving a heavy reliance on models. The ongoing disruption is urging organisations to invest in a wide array of new models, analytics, and data tools. By applying sophisticated modelling techniques and new tools to emerging data sets, one would be transitioning to a model-based economy which:
- provides deeper insights
- quantifies risks
- generates increased revenue
- increased operational agility
Is there anything one should do to help the transition scale without adding unmanaged risks?
Where there is a model, there is model risk. This presents a unique risk for many companies as they may not have existing model risk functions, or they may have invested more in modelling capabilities than in model literacy and model risk management capabilities.
Any company which is deploying "next-level" capabilities should ask itself, "do we also have 'next-level' management techniques to support our new capabilities?" For context, model risk arises from actions taken and decisions made based on incorrect, misspecified, or misapplied models.
As an example, legacy models may date back years or decades, relying on tribal knowledge and declining skills to maintain a critical business tool, thus exposing companies to risk and scale problems. Other external events such as changes in reporting requirements or adoption of regulatory changes may also cause model drift or make models inadequate to support new business needs.
In the absence of adequate model governance frameworks, model management processes, and effective control structures, the residual risk may expose companies to significant losses and strategic missteps. In addition, it will be difficult to capture the benefits of significant data and analytics spend when the models don't work.
What should companies be doing right now to ensure they are matching digital capabilities with model management capabilities?
Companies should first look to establish a robust model management framework for its combined digital and modelling efforts. This starts with a meaningful conversation to understand the current balance between modelling ambition, model risk, and model literacy. Then, companies should develop model management frameworks, operating models, and an infrastructure to help modelling ambitions scale without creating runaway model risk.
This effort should consider the six aspects of a healthy model management framework:
- Inventory: Identify functional capabilities delivered to the organisation via models, including vendor models, internal builds, UECs, and internally developed apps used in data analytics.
- Organisation: Identify the people developing, implementing and governing the models, as well as the skills, roles, responsibilities and support activities for each model and where they reside.
- Delivery: Develop standards for model development and usage, including workflows for model builds, implementation, documentation, testing, validation, and ongoing monitoring.
- Technology: Specify the application, infrastructure, and operational components required to support model technology and functionality.
- Data: Define information requirements, master data strategy, data quality and remediation processes, data quality KPIs, monitoring and data aggregation methods which enable model delivery.
- Governance: Document governance principles and practices for model development, risk management, and model literacy, with a specific eye towards managing operational and financial risks.
What benefits can companies expect to gain from taking a robust approach to model management?
Effective model management can help unleash this value while mitigating risks from model error and misuse. Models do not create value by themselves— it is the organisation's model literacy and data fluency which creates value from the models it uses to make decisions and enhance operations.
New advances in technology and available data modelling techniques promise to provide companies with significant improvements in enterprise value, customer experience, and employee experience.
We see a combination of "hard" and "soft" benefits to such an approach.
- Hard benefits should result from enhanced business operations and improved management decisions. By developing and deploying models in a managed and scalable way, companies should be able to accrue the anticipated benefits of their model builds while building an organisational trust in the ability of models to operate as advertised.
- Soft benefits should result from reduced operational risk and improved data quality. Enhanced governance and control of modelling environment should enable early detection of operating risk events preventing losses. Data quality improvements and business unit partnerships can also help to reduce data churn and enable resource efficiency and scalability.
In short, model literacy helps an organisation understand, trust, and use its improved modelling capabilities, while model management helps an organisation develop a coordinated, scalable, and trusted modelling environment.
Originally published March 2021
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
