Manage model risk across model lifecycle

Can you centrally manage the risk involved in building, deploying and using models while improving efficiency across the model life cycle? That was the goal of Discover Financial Services – a central repository that could help the institution keep tabs on models while facilitating autonomy and special needs of individual business units.

It was a critical need. Like most financial institutions, Discover depends on statistical and financial models for daily and strategic decision making. Virtually everything from determining marketing offers to gauging investment risks is tied to developing and executing complex models built with different tools.

It’s also critical from a regulatory standpoint. Federal regulators are increasing scrutiny on model risk management programs within financial institutions like Discover. The expectations cover risk management across all the elements of model life cycle, such as model development, implementation, use, validation, ongoing monitoring and retirement.

We want to build a sustainable and scalable solution for managing models at Discover. The new SAS system will help us reduce model risk and increase business value.

Abhinav Anand
Chief Model Risk Officer

For Chief Model Risk Officer Abhinav Anand, managing the data around the entire model life cycle was increasingly labor-intensive and time-consuming. Processes for generating, maintaining and analyzing the data related to models were not sustainable in the current form. Though he and his department initially considered hiring more staff, they decided to take a different approach and adopted SAS® Model Risk Management.

“Early results have been impressive,” says Anand. “There has been a significant reduction in the manual effort required to gather and process the data for risk management purposes. Additionally, preparation time for Federal Reserve Comprehensive Capital Analysis and Reviews (CCAR) dramatically dropped, and outdated models have been discovered and retired,” he elaborates. “This helped us shift our staff’s focus to larger-scope analytical projects rather than just doing data aggregation and administration.”

“Using the new system, we immediately started identifying and indexing the legacy uses of models in the production system,” says Badrish Prakash, Senior Manager of Model Risk. “Earlier, such an effort would have taken several meetings and reconciliation, but now all of that can be streamlined through SAS Model Risk Management. Knowing the exact usage of models is critical so that model risks can be identified, measured, monitored, mitigated and reported effectively.”

Knowing the exact usage of models is critical so that model risks can be identified, measured, monitored, mitigated and reported effectively.

Badrish Prakash
Senior Manager of Model Risk

Getting a handle on model risk management

Modeling has proliferated in the financial services world along with an explosion in data. Where developing models was once a labor-intensive process that involved a small number of employees in one or two business units, model development is now common throughout financial services organizations. As models deployed in marketing, underwriting, planning or operations can affect the overall risk profile of the firm, understanding what models are used where and how is more critical than ever. Model validation is essential for accurately projecting a bank’s balance sheet under stressed conditions, as well as ensuring the right customer decisions are made.

Just as important is the ability to distinguish a model from a nonmodel. “We had some spreadsheet calculations that were described as calculation tools but they were indeed models, and then some that weren’t,” Prakash says. But the bigger problem was the proliferation of models. “We didn’t have a centralized place to store model-related information that we could quickly scale,” says Prakash. There was an ad hoc workflow for model risk management. Information gathering around the models was done via chains of emails and shared spreadsheets on network drives. “You could ask, via email, if a business is using a particular model, but verifying all the details was challenging because of staff turnover or multiple versions of the models.” With the self-service interface of SAS Model Risk Management, different business units log their models and usage into a single model inventory for the entire enterprise.

A big efficiency win

The first major success for Discover was a dramatic decrease in the time it took to put together data for a CCAR review. Before using SAS it took four to five weeks to collect and prepare the data, documents and reports. With the new solution, it took less than a week, and model interdependency and linkage were readily available.

But the real value is in providing systematic help that comes from centralizing model information management. Discover has six major units and a few dozen smaller groups heavily involved in model development, deployment and usage. With this approach model risk management can be centralized, but not the actual development, testing and implementation of the models, which are left to the business units. The risk management group can see what is happening across units and provide recommendations on issues such as when a model should be retired. Individual business units can make informed decisions about the value of borrowing an existing model (instead of building their own) because they can “peek under the hood” at what the model is and what it can or cannot do.

Catalog and index more than 10,000 documents related to the development, validation, deployment and monitoring of models.

Create more than 12,000 automatically generated linkages among nearly 700 models that help stakeholders understand how a model relates to other models, model validation reviews, documents, related usages, findings and remediation plans.

Establish clear ownership of models and their usage by tracking documents related to approvals, legal reviews and change requests.

While many Discover units use SAS to build models, another key value of SAS Model Risk Management is that it can catalog and manage models built in any tool. “And people who are not very familiar with the platform can access the information using the dashboard,” Prakash says. Discover uses the platform not only to manage risk, but also to manage model usage and centralize model performance monitoring. “We needed to learn where the bottlenecks were in building and deploying models,” Anand adds.

With the solution in place, Anand and Prakash are looking more closely at model lifecycle processes and have comprehensive information to evaluate model risks. Discover will be adopting the visualization elements of the solution so higher-level executives can view and interact with a dashboard providing information on the aggregated model risk, and drill down to the exact source of risk to monitor and mitigate it. Additionally, model information is more accessible to potential users, encouraging appropriate model reuse. “We want to build a sustainable and scalable solution for managing models at Discover. The new SAS system will help us reduce model risk and increase business value,” Anand says.

Solution

Benefits

Ensures the institution’s model risk management program is audit-ready.

The results illustrated in this article are specific to the particular situations, business models, data input, and computing environments described herein. Each SAS customer’s experience is unique based on business and technical variables and all statements must be considered non-typical. Actual savings, results, and performance characteristics will vary depending on individual customer configurations and conditions. SAS does not guarantee or represent that every customer will achieve similar results. The only warranties for SAS products and services are those that are set forth in the express warranty statements in the written agreement for such products and services. Nothing herein should be construed as constituting an additional warranty. Customers have shared their successes with SAS as part of an agreed-upon contractual exchange or project success summarization following a successful implementation of SAS software. Brand and product names are trademarks of their respective companies.