Model risk management: Vital to regulatory and business sustainability

By Sridhar Sourirajan, Director, Risk Product Portfolio and Strategy, SAS

Analytical models are the lifeblood of modern financial institutions. Throughout their life cycles, they affect the needs of many different stakeholders across the organization. Adverse consequences result if business decisions are made as a result of misinformed or incorrect model use – especially during times of stress.

Download a free GARP-SAS survey on model risk management

Accordingly, financial institutions increasingly recognize the need to marshal their resources to address a variety of analytics and governance challenges underlying current and emerging business and regulatory requirements. As they do, model risk management becomes a pivotal competency. Historically, models were developed in silos without enterprise-level governance, leading to inconsistencies in data quality, quantitative methodology, model usage and validation processes. It is these shortcomings that model risk management is specifically designed to address.

The rise of model risk management

Following the 2008 crisis, regulators significantly increased their focus on model risk management. The US Federal Reserve’s regulation SR 11-7 mandates bank holding companies (BHCs) manage and control risk associated with models by using rigorous design, implementation and validation techniques. Model risk management became a critical component of stress testing programs, such as Comprehensive Capital Analysis and Review (CCAR) in the US and the European Banking Authority (EBA) EU-wide stress test. In these cases, banks must submit documented evidence of compliance with model governance policies and procedures, model approvals by management, and evidence of effective challenges and their resolutions.

Beyond BHCs, financial institutions across the board – from asset managers to insurance providers – are recognizing that they must do a better job managing analytical models at the core of their investment, capital allocation, business performance, risk management and compliance processes. 

Relying on legacy systems and processes: What can possibly go wrong?

There are various things that can go wrong in the design, development, testing, implementation and usage of models – which in turn leads to adverse consequences from decisions made using incorrect outputs.

Relying on incorrectly functioning models or decisions based on wrong or misused model outputs results in model risk. Model risk can lead to failure to gain regulatory approval for capital plans, financial loss, damage to a bank's reputation and loss of shareholder value.  

There are two fundamental reasons a model can be wrong or suffer performance degradation (loss of uplift):

1. A model is fundamentally wrong due to incorrect data, the wrong design, the wrong application of theory, an error in the mathematical calculations, or wrong assumptions. Any of these errors will produce incorrect estimates, which when applied against the business objectives, will inevitably result in decisions leading to potentially adverse impacts.
2. The model design and development is fundamentally correct, but the model is misused or misapplied. A model designed for a specific situation can pose high model risk when used in a different environment with different assumptions. It is very important to define the limitations and scope for a model.

We can trace these challenges back to a lack of control over model governance and workflow, as well as a reliance on legacy systems. For example, a recent GARP-SAS survey on model risk management showed that many firms still rely heavily on traditional tools such as SharePoint. Such tools can’t support the demands of model governance going forward. Legacy systems suffer from a number of challenges, including:

• Database technology that’s no longer supported by current enterprise information systems, making it nearly impossible to update and scale to meet new requirements.
• Lack of data integrity from inadequate system controls.
• No audit trail, necessary governance or records to document the management of changes across disparate model inventory databases and other storage locations, such as Microsoft SharePoint.
• No integration between data sources, making it difficult – if not impossible – to track dependencies and correlation risk.
• A high risk of error due to the manual effort required to compile/consolidate data from many sources.
• Very limited reporting capabilities or increased reporting demands that are handled manually.

Best practices for model risk management

To improve model risk management, you can establish controls and guidelines to measure and address model risk at every stage of the life cycle. Examples of controls and measures include:

• Conducting a conceptual soundness assessment during the design phase.
• Conducting peer reviews during development.
• Establishing limits on model use.
• Conducting ongoing monitoring and maintenance reviews frequently after a model is implemented.
   

To implement best practices for model risk management at your bank, you’ll need the following:

• High data integrity and a single source of the truth throughout the entire model life cycle and all related artifacts.
• Elimination of manual reporting and implementation of executive dashboards.
• Scheduling and monitoring built into the system.
• End-user capability to respond to changes in data needs, workflow and reporting.
• Coordination between model development and validation teams – including peer review.
• Establishment of an enterprise-level model inventory.

Model risk management is key to making your business sustainable

Many financial institutions now realize that model risk management is about more than just regulatory compliance – it’s a crucial business function if your business is to be sustainable.

Model risk management becomes a pivotal competency involving a change in culture and the adoption of best practices to measure and mitigate risk associated with using models. With a disciplined and aligned model development and implementation process, effective management of the validation cycle, and a well-defined model usage process supported by strong governance policies, controls and management structure, you are well on your way to a strong model risk management framework that fits into the broader business, risk management and compliance goals of your organization.