Banks need to pilot their way through the model maze

By Simon Goldsmith, Head of Risk Solutions, SAS UK & Ireland

Imagine the situation: The chief risk officer (CRO) of a global bank is in front of a government review committee facing questions about why the bank has just made significant losses. The reason: The computer had said “Yes” to a number of decisions when it should have been saying “No.” Regulators have recently issued very clear guidelines on how they expect banks to manage this so-called “model risk” in critical decision models.

The CRO of this scenario is probably struggling to demonstrate the bank has even identified all its critical models (which are likely to be in the hundreds), never mind showing that these models are being controlled and governed in accordance with the guidelines.

Complex models are increasingly being used to support very significant business decisions across the organization. For example, questions arise such as:

Are we holding sufficient capital to support our business should we have a major economic downturn?

Are we raising sufficient capital to support our aggressive five-year growth plans?

Could our credit approval and pricing models be significantly underestimating the risks in new business?

In addition to the increased regulatory pressure, senior managers are now much more appreciative of the risk there is in the models used by their banks.

Model risk is inevitable

The trouble is, models by definition are simplifications of reality. As the British mathematician George E.P. Box pointed out, “Essentially, all models are wrong, but some are useful.”

The risks posed by model-supported decisions are broken down into two areas:

Is the model giving inaccurate outputs?

Is the model being misused?

This could be misunderstanding of outputs or use for things outside the scope of design.

This risk of an incorrect decision being made is exacerbated by the fact that the analytic model developers/builders are often separate from the decision makers. This results in a “computer said yes” approach, where the person using the results from the model does not necessarily understand the model or why it arrived at its answer.

The consequences of “poor” model guidance on significant business decisions are:

The bank could have insufficient capital (or cash) and fail (or require a government bailout).

The bank could suffer significant losses. Even if these are not large enough to cause capital problems, these will damage the bank’s value and reputation.

A wake-up call for regulators and senior management

Regulators now have a heightened awareness of model risk and are tackling the issue. The US Federal Reserve led the way in 2011 by issuing SR 11-7 Guidance on Model Risk Management. The European Banking Authority has followed US regulators and in December 2014 incorporated specific model risk management assessment directions in the latest Supervisory Review and Evaluation Process (SREP) Guidance. This document is issued to all EU regulators to direct how they conduct their periodic SREP reviews of each bank (typically annually) and is effective Jan. 1, 2016.

In addition to the increased regulatory pressure, senior managers are now much more appreciative of the risk there is in the models used by their banks. It means that banks are now building a new approach to enterprisewide governance and control of model risk – and, crucially, mechanisms to provide evidence to the regulators that this has been done.

Addressing model risk

The fundamental approach taken to managing model risk is to have regular, independent checks of models. A rigorous exercise needs to be undertaken when the model is first created or undergoes significant change (model validation). Then there should be periodic checks on the model to confirm that it’s still giving results within tolerances (model reviews).

The challenge for a large bank (with perhaps 2,000 significant models over many teams and countries) is ensuring that the entire model portfolio is getting the appropriate level and quality of model validation and review. The central risk team can issue model risk policy – setting out what should be done. But how does senior management enforce this policy?

Discover Financial Services uses the new SAS® Model Management solution with excellent results. Before using SAS, it took Discover four to five weeks to collect and prepare its data, documents and reports for a CCAR (US regulatory) review. With the SAS solution, it took less than a week, and model interdependency and linkage were readily available.

But the real value is in providing systematic help that comes from centralizing model information management. Discover has six major units and a few dozen smaller groups heavily involved in model development, deployment and usage. With this approach, model risk management can be centralized, but not the actual development, testing and implementation of the models, which are left to the business units.

As a result, Discover’s CRO can feel comfortable fielding questions from regulators about the organization’s models and how it monitors and controls risk – plus provide the evidence that it’s happening.

Simon Goldsmith develops risk solutions for banks and insurers as Head of Risk Solutions, SAS UK & Ireland. He has over 25 years’ experience in business across a broad spectrum of industries and business functions. He is a Chartered Accountant, Chartered Marketer and has a Natural Sciences degree from Cambridge University.