Everybody loves rankings, but always think of context and bias

Whether you are Buzzfeed, the United Nations, or an investor, everyone likes rankings. It is how investors invest, and how Buzzfeed generates clicks. Whether rankings are entirely conjecture and fiction (Buzzfeed) or a quantitative algorithm finding optimal weights for various factors of an equity, they inform decision-making. It could be wasting an hour of your time on Buzzfeed or a $100mm investment in an equity.

However, it is always important to remember that rankings can be severely biased. One of those biases is obvious, which is that rankings are subject to availability of data. If data is unavailable an equity, country or item cannot be considered for ranking. That, to some extent makes the item invisible. A great example of this is Forbes wealthiest list. There are at least as many billionaires that appear on the list that are unknown and remain fiercely private. This is a significant bias in the assessment of economies and countries because the amount of research on a country is directly proportional to the number of academics studying the country. There are countries in Asia and Africa that almost nobody studies and they lack their own institutions to do so. This really makes it difficult to measure progress in much of the world’s countries.

Gizmodo has a great list of bias here. We think understanding this list and checking your own biases is critical to investing successfully and assessing investor reports and assessments. There’s been a ton of research into how equity research analysts are biased in their assessment of companies, but there is comparatively limited study on the economic rankings that many of the world’s institutes and NGOs produce. The primary reason for this, in our assessment, is that these rankings are very relevant for developing countries but less relevant for developed countries. We do not really care about the difference from a government stability perspective between the US, Canada, Australia, and the UK. It really is the same for us, regardless of Trump, May, or Rudd.

This post was inspired by the World Bank. The World Bank came out to the Wall Street Journal about 10 days ago, indicating that competitiveness rankings had been materially doctored over the last dozen years. This particularly had a detrimental impact on Chile, which was likely due to Augusto Lopez-Carlos, the Chilean director in charge of the competitiveness report. Augusto strongly denies any doctoring, but the truth is in the results. He is known to have a strong bias against socialists, and Chile has been off-and-on run by Michelle Bachelet, who is a strong socialist. Chile did not materially fall in any existing measures but did particularly poorly in new measures that were added during this time. As new measures were added, numerous countries had significant changes in their respective rankings. These changes happened to sync with the biases of World Bank staff.

What is perhaps less clear is that Chile was not the only country impacted by this admission. As the World Bank recalculates its rankings, it is very likely that other countries will be similarly impacted. We know that the World Bank for all intents and purposes is an American organization. Its bias is the same as those of the United States, but what is disconcerting is that Chile remained throughout the time that results were being doctored, a strong ally of the United States. We covered how to invest in Chile here, and we remain long-term bullish on the country.

We have done several pieces on the World Economic Forum’s Global Competitiveness Report, including this one, on why it is useful. What is interesting, is that in 2017, WEF’s report ranked Chile 33rd and World Bank ranked it 55th. This is without any legitimate or reasonable criticism of Chile.

The other angle to this store is that World Bank Economist, Paul Romer is the one who came out with these findings. He remains the economist and he was complicit for a time, in the way that the World Bank was ranking countries. Why he chose to come out now, is unclear. However, in principle regardless of motive, we do not like to criticize whistleblowers and prefer to look at the content of their allegations rather than their motives.

It is worth noting that even the World Economic Forum, which is Swiss run has its own agenda, that of increasing capitalism in the world and shifting the onus from government onto business. It is funded by numerous companies that certain people find unethical, such as HSBC, Johnson & Johnson, JP Morgan, and Saudi Aramco. We do not share this view and do think that businesses are in the best position to assess the ease of doing business and competitiveness rank of various countries.

There are even issues with GDP, a critical component of measuring an economy. We will be the first to admit that we have a heavy reliance on GDP, but it is not without error. The Economist provides an excellent overview of the problems associated with GDP here. A high-level summary is that GDP does not measure things that are not tangible well, it also fails to adequately consider services and financial services in particular. In addition to this, GDP does not adequately consider the value of government services either. All of this suggests that GDP should be taken with a grain of salt. This is something that ought to be kept in mind when one considers a measure that is dependent on GDP, such as Debt-to-GDP or GDP per capita. A country may be more or less indebted than expected because the GDP number is inappropriate for the structure of a country’s economy. This also creates opportunity, if you identify such countries that are measured incorrectly and may be better than measures suggest.