The Pritchett Test for Evaluating Policy Programs

There is a growing practice of conducting intense policy impact assessments in OECD countries, one which developing countries are trying to emulate. For example, in the EU, every department that wants to propose a policy has to conduct a policy Impact Analysis (IA). This usually involves the setting up of committees, boards, consultations between departments, evaluations of the IA, etc – a complex bureaucratic process. International organizations like the World Bank are notorious for their IA tomes. These are thorough and meticulous documents covering all possible micro aspects of the policy. However, in the pursuit of the tiny details, sometimes the macro picture can get lost. The danger of this is that micro policies that can be easily evaluated tend to get preferred over other policy initiatives that might pose bigger challenges for evaluation.

Lant Pritchett, Senior Fellow at Center for Global Development and ex-World Bank man, had written a blog post in June 2014 on whether these detailed impact evaluations are asking the right questions for economic growth and development. He proceeds to develop a simple test containing four simple criteria for impact assessment. These are the questions analysts must ask even before undertaking an IA. If the policy proposal meets all or most of these criteria, the nuances and complications can be addressed later. While this test might not entirely apply to issues such as institutions, human development, approaches to trade, intellectual property, etc, it is extremely relevant to most issues relating to economic growth.

To judge whether a policy/program (let us call it X) is an important determinant of development or economic growth, it must respond in the affirmative to these questions:

Question 1: Is X done more in developed countries than developing countries?

Countries differ in their level of development by an order of magnitude. Countries that are developed should have more of X than countries that are developing. If UK and France don’t have more of X than Mali or Eritrea, X needs to be revisited.

Question 2: Is X done more today in developed countries than it was before?

The magnitude of development changes significantly with time. So, developed countries will be significantly more developed than they were a hundred years ago. Thus, there should be more of thing X now than a hundred years ago. If Germany and Japan didn’t have more of thing X now than they did in 1915, X needs to be reassessed.

Question 3: Is X done more in rapidly growing country than stagnant ones?

Apart from magnitude, countries also differ in their pace of growth. Naturally, X should be present more in the rapid growth countries than development failures. If Korea and Singapore don’t have more of X than Haiti and Nigeria, X should probably be dumped.

Question 4: Does a country’s growth accelerate/decelerate when a country does more/less of X?

Countries don’t grow at a uniform pace all the time. They differ, sometimes dramatically, in their rate of development and growth from time to time. Therefore, X should be present in the country during a rapid growth phase rather than a slow/stagnant phase. If more of X is not present in China after 1978 than before or in India after 1991 than before, then X needs to be revaluated. If we look for countries that switch from a regime of slow economic development to a regime of rapid development, do we see a parallel shift in the rate of growth of change in X?

Urbanization as the policy Variable (X)

Paul Romer, noted American economist, took Urbanization as the policy variable (X) and put it through the Pritchett test with fascinatingly clear results. Using cross-sectional and time series data for urban share of population and per capita GDP growth (levels and growth rates), Romer concludes that Urbanization as a macro policy variable passes criteria number one, three and four of the Pritchett test. He does not examine the second criteria reasoning that it requires a separate treatment.

In brief, urbanization is positively correlated with GDP per-capita. Those countries with higher GDP per capita also have a larger share of their population living in urban areas. This answers the question of whether X is done more in developed countries than developing ones.

Romer then shows that a 1% increase in the urban population share is associated with a 2.7% increase in GDP per capita. He then goes on to show that countries that have a bigger increase in GDP per capita also tend to have a bigger increase in the urban population share. This is an affirmation to the second question of whether X is done more in rapidly growing country than in stagnant ones.

Finally, to answer Pritchett’s fourth question of whether a country’s growth accelerate/decelerate when a country does more/less of X, Romer uses China as the example country and 1980 as the break point between low and high growth. Before 1980, the urban share increased at the rate of 0.2% per year and GDP per capita grew at 1.0% per year. The corresponding values for after 1980 are 2.5% and 6% per year respectively.

After undertaking a primary analysis such as this, other complexities can be studied and policy conclusion drawn. Romer’s study of urbanization and GDP per capita is a brilliant application of Pritchett’s simple yet powerful tool of analysis. It is a breath of fresh air from the complex and verbose policy impact assessments coming from bureaucrats.

Anupam Manur is a policy analyst at Takshashila Institution and tweets @anupammanur

No comments yet.

This is a community blog by the public policy students, alumni, scholars and staff of the Takshashila Institution. The opinions are those of the respective authors and do not represent the position of the editors or that of the Takshashila Institution.