Heterogeneity Matters: When the average hides what is true for most - Guest post by Jessica Hoel

Development economists know that how husbands and wives make decisions affects many outcomes: savings, time use, and investment in education, health, agriculture, and business. While the literature has recognized that households may have different incomes, different preferences, and make different choices, tests between models of how those choices are determined have implicitly assumed that all households make decisions in the same way. With rare exception, tests of models of household decision-making have examined only average statistics. Why? Heterogeneity is understudied because estimating the counterfactual for an individual is difficult. In field experiments, we assign each respondent to only one treatment condition, so we can’t identify the treatment effect for any individual. It is then challenging to discuss the distribution of behavior without making strong assumptions. Studies that use observational data have analogous limitations, so we most often discuss average treatment effects. However, it is possible that a minority of households could be so strongly different as to generate a misleading average statistic. In tests of models of the household, the average may point to one model, while most of the sample is better fit by another. In my job market paper, I study heterogeneity in household decision-making explicitly, using a within-subject lab-in-the-field experiment with married couples in Kenya to ask: how many people behave inefficiently? How many people respond to asymmetric information?

First, a brief primer on models of the household. Currently, there are two competing theories: the Collective model and non-cooperative models. The Collective model assumes efficiency, which is essentially that spouses must get along well enough to at least not leave money on the table. In a one-time opportunity (like a laboratory game), efficient spouses will maximize household income and reallocate between themselves later. Non-cooperative models cover everything else, and inefficiencies are usually engineered with limited contracting (maybe I can’t enforce our agreements) or some kind of information problem (maybe I’m able to hide my choices from my spouse or maybe I don’t know his preferences). It’s important that we know which model fits most people best because different models of the household suggest different optimal policies (e.g. should cash transfers be given in person, when my spouse is standing right next to me, or by mobile phone, so it’s a bit more private? See Aker et al. 2012).

Methodology: Lab-in-the-Field

Laboratory games offer a distinct advantage for researchers studying heterogeneity. I can play several games with each respondent, measuring their behavior in both a treatment and a control situation, so I can say for certain what the treatment effect is for each individual. A within-subject design allows me to detail the entire distribution of behavior. Other researchers have used within-subject games to test the effect of information on giving between friends (Leider et al. 2009) and neighbors (Ligon and Schechter 2012), but they do not focus on the heterogeneity in treatment effects. Previous laboratory studies of the effect of information on giving between spouses have used between-subject designs (e.g. Mani 2010, Iversen et al. 2011), so they discuss only the average effect of information.

For my dissertation work, my team and I interviewed 392 couples in rural Western Kenya, in Siaya County. Couples arrived at the survey site together, were split for their individual interviews, and played a series of games with a trained field assistant following a script in their native language. (You can find the script and my surveys here.) Each person made several decisions, and one was selected at random to come true. Couples returned to the study site together exactly one week later to collect their winnings. The maximum payout from one game was 750 Kenyan shillings ($7.50; about 50% of average male income over the previous week or 112% of female weekly income).

Respondents played dictator games against their spouse. The field assistant handed the respondent several bottle caps and asked him to divide the tokens between himself and his spouse. (I’ll use male pronouns here, but both husbands and wives played the games.) Tokens that the respondent gave to his spouse were worth 30 KSH while tokens he kept for himself were worth 20 KSH, so the efficient decision is to give all tokens to the spouse. One game was played in public, in which the field assistant told the respondent that the study team would tell his spouse what he chose. The second game was played in secret, in which the field assistant told the respondent his choice would remain private. The difference between behavior in the public and secret games is a test for information problems between spouses. To obfuscate behavior in the secret game, every respondent played two additional games that paid off anonymously, including a standard dictator game played in secret against a stranger.

Results

In the standard Secret Stranger dictator game, people gave just less than half of the money to the stranger on average. This generous behavior has been observed in communities around the world (Henrich et al. 2004). We expect people to give more to their spouse than they give to strangers, and they do. In my study, people gave an average of 9 percentage points more to their spouse than to a stranger. However, if households were efficient, we would expect them to maximize household income by giving all the tokens to their spouse. Almost no one does this.

97% of choices in the Public Spouse game were inefficient. Respondents gave up an average of 16% of potential income in order to maintain control over about half the pot. This behavior defies the predictions of the Collective model of household behavior, thus a non-cooperative model would better fit this sample. But what kind of non-cooperative model? Could information problems be causing the inefficiency?

This is where acknowledging heterogeneity becomes essential. On average,people gave 10 percentage points more in the Public game than in the Secret game. That is, on average, people were more generous in public and more selfish in secret. The difference is the same for men and women, and is highly statistically significant no matter what specification or controls I use. If I looked only at averages, I would conclude that households are inefficient and it’s likely because of information problems.

However, most people do the same thing in public and secret. Only 37% of people give more in the public game. 50% of people don’t respond to asymmetric information, but they behave inefficiently anyway. This leads to an entirely different conclusion than is suggested by the average statistics. When I allow for heterogeneity, I see that information problems cannot be the only or even primary cause of inefficiency in my sample.

Conclusion

In my job market paper, I demonstrate that allowing for heterogeneity across households in a test of household decision-making model assumptions matters a lot: the average statistic suggests one conclusion, while most people are better fit by a competing model. Where does this leave us? First, acknowledging heterogeneity is important. Empiricists can test for heterogeneity by examining sub-groups and using techniques like quantile regression and matching, and should be cautious in the interpretation of average effects. Laboratory experimentalists can use within-subject designs to explore heterogeneity more explicitly. Field experimentalists can use lab games as diagnostic tools. Finally, theorists should explore non-cooperative models of the household that explain heterogeneous responses to asymmetric information and use limited contracting to generate inefficiencies.

Jessica Hoel is an Economics PhD Candidate at the University of Michigan.

Hi Anonymous,
I take your point. I went back and forth between first presenting behavior in the secret games or the public games, and perhaps should have presented the secret games first.
Ultimately, my point is that average statistics imply that information improves efficiency while for most people, information doesn't matter. I think the point that allowing for heterogeneity changes the conclusion stands.
Jessica

What are the guidelines related to informed consent and volunterism for non-medical reserach? Are they the same as for any reserach involving human subjects? Do parrticipants in the control group have to be informed about the benefits that may be denied to them? How can I find these guidelines? I noticed that the 3ie site contains only the following mention in its guidelines, which seems vague, only singling out "do no harm", which is only one of the usual principles:
"Finally, 3ie adheres to strict ethical standards in data collection and expects grantees to follow do no harm principles in all contexts. Investigators will be required to obtain ethical clearance with the appropriate institutions in conducting their studies."
Also not specified are what the "appropriate institutions" are. Where can I find more information and specific guidance?
thanks!
MB

http://blogs.worldbank.org/impactevaluations/node/594
http://blogs.worldbank.org/impactevaluations/node/598
http://blogs.worldbank.org/impactevaluations/node/609
If you search the blogs, you'll find that there are more...

Hi Marguerite,
Ethics in research involving human subjects is important no matter the topic, and is especially important when studying vulnerable populations like poor people in the developing world. When I designed my project, I was careful to think of potential risks for my subjects and took every reasonable precaution to protect them and their privacy. My research was supervised by Institutional Review Boards (IRBs) in Michigan and in Kenya. You can find more information about what my Michigan IRB required here: http://www.irb.umich.edu/.
To answer some of your specific questions, all of my subjects were volunteers recruited by their village elder. They gave their verbal consent after being informed about the nature, benefits, and risks of my study. You can find a copy of my consent statement at the beginning of my baseline survey here: http://www-personal.umich.edu/~jesshoel/Hoel_MCDM_Baseline.pdf. My field team and I worked together to ensure that the language used in my consent statement could be easily understood, and translated well into the local language. The consent statement also states that the respondent may receive a monetary payout, but everyone would take home a small appreciation gift (powdered soap).
For people working out of academic institutions, the "appropriate institutions" the 3ie document mentions are that university's IRB. Some countries also require approval from a local IRB for all research (I think Malawi and Uganda require local IRB for all research), and some require different IRBs for different kinds of research (Kenya requires a KEMRI IRB approval for medical research and any social science research that collects bio-specimens). Many countries (including Kenya) require a research permit issued from the national government (mine is from the Ministry for Higher Education, Science, and Technology).
It would be difficult for one person to know all the regulations for all countries, so if you ever do research outside the US, you should consult an organization or researcher with lots of experience in that country. I am very grateful to the management team at Innovations for Poverty Action - Kenya for guidance.
Jessica