In what seems to me an unfortunate conflation, the literature on women’s empowerment frequentlyrelies on the same characterization as pornography: “you have trouble defining it but you know it when you see it.” If “empowerment” is hard to define, it is even harder to measure. This is a problem for researchers trying to establish a clear causal relationship between microfinance interventions and better outcomes for women.

In theory, microcredit could empower women through a number of different channels. For example, giving loans to women could increase their bargaining power within the family, and afford them greater control over household resources and decisions. The peer monitoring component of group-lending could provide protection against abuse, and deter domestic violence. Empirically, however, the picture is quite mixed.

This post looks at two studies, one from 1996 and one from 2010, which I suggest are representative of their times in terms of scope, ambition and rigor. Both find positive effects of a microfinance intervention on women’s empowerment, but differ in intervention studied, study design, and approach to measurement.

Hashemi Schuler and Riley (1996) evaluate the effect of Grameen and BRAC microfinance programs on women’s empowerment in rural Bangladesh. Much more than just handing out loans, these programs require participants to form groups, open savings accounts, and recite daily pledges that reach far beyond the financial sphere—for example, the women promise to limit the number of children they have, refuse to take or give dowries, disavow injustice, and reject child marriage. The methodology is ethnographic research, including participant observation and informal interviews, over a three-year time frame.

Moving 14 years forward, Ashraf, Karlan and Yin (2010) use a more rigorous study design to establish a possible causal impact of offering commitment savings accounts to people in the Philippines. People who accept these accounts agree not to withdraw money until a certain savings goal that they articulate in advance is reached, but there are no other components or requirements—the intervention here is much narrower in scope. The authors employed a randomized control trial, and conducted a detailed follow-up survey after one year.

I get the sense that Hashemi et al first thought deeply about the concept of empowerment and how it might manifest itself in rural Bangladesh, and then how those manifestations of power might be measured. The indicators the authors chose include a wide range of human behavior: contribution to family income, mobility, the ability to make small and large purchases, involvement in major decisions, relative freedom from domination by the family, political and legal awareness, and participation in public protests and political campaigning.

In the second study, female empowerment is defined much more narrowly. Rather than trying to capture the myriad, subtle ways empowerment might manifest itself among new savers, the authors choose a single indicator that could be precisely, quantitatively measured: decision-making power in the household. They do this first by asking account holders how buying decisions are made in the household, and second, by looking at the share of “female-oriented” durable consumer goods purchased (identified by researchers in the Philippines and focus groups as: washing machines, sewing machines, electric irons, kitchen appliances, air-conditioning units, fans and stoves).

Hashemi et al find positive, significant effects along all eight indicators of empowerment. Ashraf et al find positive, significant effects on female-driven buying decisions and the purchase of female-oriented consumer goods for one group of people in the study: married women who had below-median decision-making power in the household before the study. So moving from 1996 to 2010, we’ve gone from a very comprehensive program that finds sweeping effects using less rigorous research methods, to a very targeted intervention that finds precise effects on a sub-sample of the study population using a randomized control trial.

While these are just two of the many studies in this field, perhaps they can be taken as somewhat representative of their eras, in their ambition and rigor. I also can’t help but think of them as adding to the case for the Stainless Steel Law of Evaluation: “The better designed the impact assessment of a social program, the more likely is the resulting estimate of net impact to be zero.”

The first study does not allow us to eliminate the possibility that the women who participated in the loan programs were somehow different (more ambitious, more outgoing, or more “empowered” in the first place) than other women (as the authors admit)—or that the changes occurred for some other reason. It also doesn’t help us see precisely what elements of the very broad intervention caused the changes that the researchers found—maybe it was the loans, or maybe it was the calisthenics, or maybe it was reciting the promise to “fight injustice” every morning, or maybe it was participating in a research study, or all of the above. Who knows? In any case, the study suggests that something quite fundamental, something that reaches into many aspects of daily life, has shifted for this group of women as a result of their involvement with the loan program.

None of this confusion is present in the second study. If there is a flaw, though, to my mind, it is that in reading such a well-crafted piece of research we may gloss over the fact that the study essentially equates the purchase of a very specific kind of consumer good (a sewing machine, an air conditioning unit) with “empowerment,” a broad concept that - while notoriously hard to define - touches on what it means to have agency, exercise fundamental rights, and be human.