regnerus’ three mistakes

I was having a conversation a little while ago about scandals in sociology and the topic of of Regnerus’s article came up. That was the article in 2012 that claimed that children of same sex parents had bad life outcomes. My conversation partner suggested that Regnerus’ mistake was in asking the question. I don’t think so. Asking if there are differences by family type is legitimate and we should be ready to accept less than flattering answers. I also don’t think it is inherently wrong to have biases in research. If he has a favorite answer, then so be it. Long as he doesn’t tinker with the data to get his favorite answer. Nor is problematic to have research sponsors. People with political goals are not barred from research.

What were Regnerus’ mistakes, in my view? He made three:

Making big claims with delicate data. Instead of investing the time to properly locate same sex families and make a good sample, he instead relied on what respondents remembered about family life. Furthermore, scholars who have analyzed the Regnerus data have found that the results may be due to mis-coding cases and data handling errors. In other words, family structures are hard to measure and code properly. And with a small number of cases, small errors can have big effects on the final. This is not the kind of data on which you make big, bold claims.

Tactically, by rushing the paper through peer review with a friendly editor, it has the appearance of being a hack job. If the paper had been vetted at conferences and undergone a more traditional peer review, then it would have improved.

The issue is that Regnerus was doing something rather ambitious. He was trying to overturn an established research finding, and one with major policy implications. While I don’t agree with his politics, he certainly has a right to do this. At the same time, you don’t take on such a task without seriously thinking things all the way through. That is probably the most basic lesson of all. You should strive toward virtue in your research, especially if you are swimming against the tide.