Which IRR vendor models are appropriate < from the 2012 FAQ

For years bankers have been asking (pleading with?) their regulators to just tell them what model to use. And for years regulators have answered something to the effect of, “the type and size of the model you need should match the size and complexity of your bank.” Somewhat vague I agree, but I understand where the regulators are coming from. They don’t want to let the banker off the hook, they don’t want to suggest a tool that doesn’t appropriately capture the bank’s risk exposure, and they certainly don’t want to be in the business of “suggesting” one vendor over another. In other words, it’s up to the bank to determine what’s appropriate or not.

Interestingly enough the now non-existent OTS used to take the very opposite position on this by running their NPV model for every thrift. This has now created somewhat of a problem for the OCC since they are now the primary regulator for thrifts. Many thrifts have never run an IRR model themselves (in-house or out-sourced), and many lack the expertise to do so.

Question from the FAQ: 1) How should financial institutions determine which IRR vendor models are appropriate?

Answer from the FAQ: Models can vary significantly depending on complexity, data management, and cost. Achieving the proper balance among risk positions, risk measurement processes, and cost is critical to a successful model risk management program. When creating an IRR model or evaluating third-party models, institution management should thoroughly assess the model’s ability to reasonably capture risks in the institution. Additionally, management should reevaluate the model’s appropriateness as risk positions, strategies, and activities change. When reviewing modeling options, management should at a minimum consider the following:

The ability to reasonably model the institution’s current and planned on- and off-balance-sheet product types (on both income and capital valuation bases). Material positions in highly structured instruments or institution-specific products should be key considerations. The model should support the level of data aggregation and stratification** necessary to properly measure these types of products.

The extent to which the model uses automated processes compared with manual procedures. Management should consider whether the model has automated interfaces with institution source systems. Management should also consider the cost, hardware and software requirements, staff resources, and expertise needed to run the model and integrate any separate (manual) add-ons (also see question #2).

The level of model transparency and the adequacy and comprehensiveness of vendor model validations and internal control reviews (also see question #9).

The level of vendor implementation and ongoing support received, including available training from the vendor.

To better control third-party model risk, financial regulators expect financial institutions to have sufficient in-house knowledge in case vendors or financial institutions terminate contracts for any reason, or if vendors are no longer in business. Financial institutions should maintain contingency plans for addressing how management should respond to such lapses in vendor support.

The bottom line is that the bank should be well aware of the capabilities and drawbacks of using a certain model. If the model is too simplistic it is probably going to be missing some risks (the most commonly overlooked risk is option risk.) But, you don’t want to over buy either, that is implement a model that is too sophisticated, or requires too much time and other resources to update. There are just as many good and bad in-house models as there are good and bad out-sourced models. Beware that you don’t fall into the trap of “the worst of both worlds.”

Believe it or not many banker’s don’t know if or how their model handles:

Just to name a few. You should recognize that there are many poorly implemented and updated models (both in-house and out-sourced) that don’t adequately address these behaviors. Just using a “reputable” vendor’s model is no guarantee.

** “data aggregation and stratification”…couldn’t they just have said “collect and organize data”? It’s wording like this throughout the FAQ that makes it a bit hard to read.