Site Search Navigation

Site Navigation

Site Mobile Navigation

The Questioner Matters, Not Just the Question

By Catherine Rampell July 27, 2010 5:30 pmJuly 27, 2010 5:30 pm

Photograph by Kevin Van AelstWhen asking people about their views, framing matters. And that includes who’s doing the framing.

Asking people what they think about policy is tricky. Frame the question just so, and you can get just about any answer you want. Different framing can help explain why Americans often seem to simultaneously hold contradictory views, such as on therelative priorities of job growth versus deficit reduction.

While imperfect question wording can distort survey results, at least that wording is usually disclosed to outsiders trying to judge the worth of those survey results. There is, however, another potentially distorting, and less transparent, variable when it comes to surveys: who’s asking the questions.

Consider this fascinating new study, based on surveys in Morocco, which found that the gender of the interviewer and how that interviewer was dressed had a big impact on how respondents answered questions about their views on social policy. From the survey’s abstract:

[T]his paper asks whether and how two observable interviewer characteristics, gender and gendered religious dress (hijab), affect survey responses to gender and non-gender-related questions. Drawing upon data from a nationally-representative survey of 800 Moroccans conducted in 2007, the study finds strong evidence of interviewer response effects for both gender-related items, as well as those related to support for democracy and personal religiosity … Interviewer gender and dress affected responses to survey questions pertaining to gender, including support for women in politics and the role of Shari’a in family law, and the effects sometimes depended on the gender of the respondent. For support for gender equality in the public sphere, both male and female respondents reported less progressive attitudes to female interviewers wearing hijab than to other interviewer groups. For support for international standards of gender equality in family law, male respondents reported more liberal views to female interviewers who do not wear hijab, while female respondents reported more liberal views to female respondents, irrespective of dress.

To be sure, Moroccan culture is different from American culture, and so the gender (or race or dress or age) of interviewers may affect respondents from different cultures differently. But previous research has found that observable interviewer characteristics — like the gender of the voice on the phone — probably make a difference here, too.

Since polling organizations don’t typically give out demographic data on the people conducting the polls, those of us who follow surveys just have to hope that all the noise created by interviewer characteristics cancels itself out.

The Affordable Care Act imposes economic burdens that are the equivalent of taxes, an economist writes. Read more…

About

Economics doesn't have to be complicated. It is the study of our lives — our jobs, our homes, our families and the little decisions we face every day. Here at Economix, journalists and economists analyze the news and use economics as a framework for thinking about the world. We welcome feedback, at economix@nytimes.com.