NASSIM TALEB: The Fed Is Looking At The Banking System All Wrong

No one is ever happy with the results of the stress tests that the Federal Reserve and the European Union conduct periodically to evaluate the health of big banks and what could happen to them in the event of adverse economic and financial shocks.

Nassim Taleb has long been a critic of traditional forecasting methods like the ones underlying these stress tests. He even coined a now oft-repeated term to capture his criticism – "black swan" – which became a huge New York Times bestselling book.

Now, he warns that "fragility is especially high for the banks with the worst outcomes" according to a new metric he's developed to better analyze the risks facing the banks.

In a new white paper with researchers at the IMF, Taleb explains the reason why all of the stress tests conducted by central banks and international financial institutions like the Federal Reserve, the ECB, and the IMF come up short:

First, many stress tests focus on the point estimates of very few scenarios, and often pay little attention to how the impact would change in case of different scenarios, e.g., a slightly more severe one. Second, if stress tests do not take into account the possibility of model and parameter error, it can be misleading to rely only on the point estimates of even well-designed stress tests. Without considering the potential for these errors, one could miss the convexities/non-linearities that can lead to serious financial fragilities.

A better approach, according to Taleb and his IMF co-authors Elie Canetti, Tidiane Kinda, Elena Loukoianova, and Christian Schmeider, is to measure the difference between outcomes arising from different scenarios instead of focusing on the estimates of potential losses themselves.

According to Taleb, this is the real way to measure the "fragility" of a bank or a country in the event of a negative economic shock. Because point estimates are so prone to errors from faulty model assumptions, measuring the distance between them to detect how quickly losses pile up as the economic shock gets larger becomes a vastly more reliable measure of risk.

In other words, it's not the size of the losses themselves that is important. Instead, it's the rate of change of potential losses as the economic situation deteriorates that determines how fragile a bank is, by Taleb's standards.

This makes intuitive sense. What regulators should really want to get a handle on is how quickly things could spiral out of control in a bad situation.

And looking at the risks banks and countries face this way produces some surprising results that clash with the results of the official stress tests.

Taleb and the IMF researchers looked at the results of the original stress tests and compared them to tests they re-ran with their improved metric. Results from the paper show that the banks in the U.S. that face the biggest risks from a bad economic environment are different from those identified by official stress tests.

The table below ranks the riskiness of the big U.S. banks according to the Fed's stress tests and compares that ranking to the outcome of their test. "Bank 3," "Bank 7," and "Bank 12" are all judged to be considerably riskier by Taleb's estimates than the Fed stress tests show:

The same is true for countries facing heavy debt loads. Taleb writes in this case that "the results illustrate that large negative growth shocks have a disproportionately higher impact on net debt compared to smaller or positive growth shocks, indicating a non-linearity," and that "due to the non-linearity, there is a disproportionately higher cost to underestimating the growth shock than overestimating it."

Taleb hopes this metric of fragility will improve future stress tests. In conclusion, he and the IMF researchers write:

Such results may have important policy implications. For example, if a country finds the structure of its public finances make it particularly fragile to growth shocks, it may conclude it has less room for countercyclical deficits. By the same token, a stress test could help a bank or a banking supervisor find otherwise hidden fragilities coming from, say, large illiquid positions subject to fire sales in some conditions, derivatives exposures with nonlinear payoffs, or feedback loops between losses and funding costs.