Three days ago, the Heritage Foundation published the 2018 edition of its Economic Freedom Index, which ranks the world’s nations according to their level of economic freedom. For methodology, see here. In many countries, the local media have reported on how the nation’s ranking has moved compared to last year. For instance, the Dhaka Tribune reports that Bangladesh’s ranking remains unchanged, while the media in Kazakhstan are evident proud of the country’s “great strides” in the rankings. In the US, the media has noticed that the alleged level of economic freedom has increased during President Genital-Grabber Trump’s year in the White House. According to the Heritage Foundation’s press release, “The United States managed to halt its recent slide, recording a score of 75.7, more than half a point above its 2017 score (its lowest score in Index history).” Freedom from sexual harassment in the workplace is not included in this overall index.

In previous years, the publication of this index generates a fair bit of discussion in the economics blogosphere about methodological issues. This year, the recent scandal about the World Bank’s Ease of Doing Business country rankings (see here and here) has prompted even more people than normal to reflect about the Heritage Economic Freedom Index. As many readers of this blog are likely to know, two think-tanks, the conservative Heritage Foundation in the US and the neoliberal Fraser Institute in Canada have long sought to measure economic freedom in nations with indices that based on 12 and 5 components, respectively. Some of the components in both indices relate to institutional quality a measure things like rule of law, the protection of property, and the inflation rate. More controversially, the indices also penalize countries for have large welfare states, which many academics regard as odd since generous social safety nets can actually build support for free enterprise. As Jan Ott of Erasmus University has pointed out, “levels of government spending, consumption, and transfers and subsidies appear to correlate positively with the other indicators related to institutional quality, while this correlation is close to zero for the level of taxation as a percentage of GDP.”

In the eyes of many academics, these two economic freedom indices are highly suspect. I suppose the dominant view here in the ivory tower is that far from being objective and social-scientific, these indices simply reflect the personal opinions of the individuals who created them and the plutocrats who pay for the whole operation. In other words, these measures are highly crypto-normative.

Some people might therefore conclude that it would be impossible to create an economic freedom index that would command the respect of academic social scientists or even non-academics who aren’t libertarians. However, I don’t think that we should throw the baby out with the bath water and conclude that it would be impossible to create a reputable economic freedom index. It would be possible to create a more convincing index of economic freedom, provided we followed the ground rules that I outline below.

1) Recognize that indices of this type reflect the values and policy preferences of the individuals who create them. Indices of economic freedom should therefore be designed by teams that include people of different political persuasions as well as both genders, different ages, nationalities, a variety of income levels, and different cognitive styles. When I say include different nationalities on the team, don’t just select people from different Western countries. Involve smart social scientists from every continent.

2) Think carefully about how many individuals should be part of the team that designs the methodology for the economic freedom index. I don’t know what the ideal number is. It could be five. It could be fifty. However, I suspect that there is some social-scientific research that can inform our thinking about the optimum number of team members. Research can also inform the selection of team members and the design of the team’s internal processes.

3) Producing the world’s first high-quality index of economic freedom will cost money. You should therefore provide a budget that is large enough that all team members can come together to discuss why they think particular components should be included or excluded from the index or should be given a particular weighting. For instance, the team might include people whose initial preference is to include the rate of inflation in the index and others who disagree with this view. Forcing both sides to spend some time together to argue about the issue will result in a better product.

4) If disputes within the team of experts about what to include in the index can’t be resolved by debate, put the issue to a vote. Consider using a more sophisticated voting system than simple majority to resolve contentious issues related to the inclusion or exclusion of variables.

5) Borrow ideas from the research transparency movement and share lots of information about the process by which the methodology was developed. Ideally, videos of the deliberations of the team should be put on YouTube and their email correspondence should also be placed in a data repository. Share biographical details about the team members. Also tell readers what the budget was and who paid for it all. Maybe considering involving BITSS, the Berkeley Initiative for Transparency in the Social Sciences. They will help you to be more transparent, and thus more credible with sceptics like me.

6) Finally, have your team develop the methodology over a period of time. Don’t ask the team to develop the methodology in one go, in one weekend say. Marking student essays has a subjective element, but conscientious professors have techniques for limiting the arbitrariness of the marking. One common practice is to spread the marking of essays over a few days: read the essays and write up comments on them but refrain from assigning the numerical grades until a few days have gone by. In other words, you sleep on your decision rather than arbitrarily scribbling down a mark that may be informed by hunger pains or something unrelated to the quality of the essay.

Some readers may be wondering why I think that the team should diverse in terms of age as well as by nationality, gender, ideology, etc. Here’s why I think that a certain degree of age diversity is essential: some government-imposed restrictions on freedom on contract impinge more or certain ages than others. Freedom of contract is not an absolute principle and I am glad that governments in most countries try to enforce minimum ages of purchase alcohol. However, my subjective, gut-reaction view is that US drinking age of 21 is absurdly high. Indeed, university students in the UK mock the US drinking age and when they talk about it, they almost invariable mention, ironically, that the US is the “land of the free.” Now when I was 20, and a Canadian who sometimes went to the United States, I thought a great deal about the US drinking age and had strong feelings about it. Now that I am older, I feel less strongly about that issue, even though I still disagree with the US drinking age. One of the things I’ve noticed about the Heritage and Fraser Indices is that they don’t take restrictions on alcohol and marijuana sales into account. That serious omission likely reflects the age of the individuals who created the indices and it may also reflect their personality type and preferences. (I’m guessing that the guys who developed these indices never tried pot when they were in university, which was probably a long time ago).

Now I don’t know exactly how what sort of weighting one ought to give to strict alcohol and drug laws in an economic freedom index. Clearly that is something a team is better placed to decide, since in a diverse team personal biases can cancel each other out. I do know that Saudi Arabia’s total ban on all forms of alcohol clearly ought to be taken into consideration in developing the index, as should the relatively high drinking age in the US. However, there are some grey areas. Canada will soon have legal recreational marijuana, while the UK continues to ban this drug. So looking just at marijuana policy, I would give Canada a few more points for economic freedom. However, the UK has a much more liberal policy regarding alcohol sales than most Canadian provinces, especially those where alcohol sales are still a state monopoly. Judged strictly on alcohol policy, the UK has more economic freedom. But which should have heavier weighting in the index, marijuana or alcohol policy? I don’t know. But I do know that in answering this question, two heads are better than one.