About Jason Thibeault

Hi! I'm a tech guy, skeptic, feminist, gamer and atheist, and love OSS and science of all stripes. I enjoy a good bit of whargarbl now and again, and will occasionally even seek it out. I am also apparently responsible for the death of common sense on the internet. My bad.

I have opinions. So do you. You want to share them with me. I would like to do likewise. Please don't expect a platform for proselytizing that will go unchecked and unchallenged, though. Contact me via the clicky thingies under my banner.

The commenting rules are simple: don't piss me off. This rule has worked for me for a decade; I have never found a need for any other rule, because any other rules leads to rules-lawyering. Just remember -- this is my property, not yours.

In case you haven’t heard of this ongoing debacle, Dr. Satoshi Kanazawa recently published a rather controversial article claiming that black women are objectively less attractive. This study was published in Intelligence, a journal well-known for its persistent use of IQ as a valid measure of intelligence despite the academic dissent that IQ does not measure any one single thing and therefore can’t be used as a metric to study anything but the weak causal relationship IQ scores have with actual intelligence. When Dr. Kanazawa was presented with a good deal of dissent about the methods by which he produced the study, he went on to blog on Psychology Today about the study’s validity, claiming some interesting just-so hypotheses to explain why his results were correct, rather than engaging with the criticisms. The blog post was almost immediately retracted and Psychology Today apologized for the distress it caused.

This touched off a firestorm, mostly in that Kanazawa evidently has a history of not engaging with critiques of his papers contemporaneously. A number of scientists rallied to his defense, claiming that Kanazawa was “Sinned Against, Not Sinning”. There’s just one problem with the defense rallied: the defenders claimed that any critiques must needs be made in the journals themselves, and once past peer review, the paper is beyond reproach.

There are legitimate discussions to be had on the role of peer-review feedback in shaping the final published product. However, having that discussion and recasting a complaint about Kanazawa’s resistance to incorporating feedback are two very different things. Also, given what the criticism of Kanazawa actually was (that he doesn’t interact with feedback prior to publication) it seems a little odd to note that he incorporates feedback into later work. If the criticism is important enough to be dealt with, wouldn’t he produce stronger papers by dealing with it up front?

But back to the letter. There are a few short paragraphs providing information about two times Kanazawa later responded to criticism, followed by this closing:

Finally, we believe that the proper place to make criticisms of academic papers is in the journals in which they were published, not in letters to the press where they cannot be adequately answered.

Sorry, Stephanie, I have to interject to say: are you fucking kidding?

Okay, go ahead.

This–this!–is what makes this letter so entertaining. Even forgetting that Kanazawa brought himself and his work into the general public eye by writing a blog post about his “findings,” this is the richest vein of irony I’ve mined in some time. You see, while the idea that scientific ideas and their validity should be hashed out in journals is relatively common among scientists, it’s pretty rare among the signatories to this letter.

She goes on to list an easy pickings set of links that show times when each signatory to the defense letter actually blogged about science in public, in direct contrast with their professed beliefs. I personally see no harm in blogging about science, engaging with your audience (and in many cases, with audiences that aren’t actually normally “yours” to begin with). It gives you perspective you might not otherwise be exposed to, and can oftentimes provide a baffle against the temptation to insulate yourself into an echo chamber. What I DO see harm in, is in ignoring valid criticisms outright, especially when they’re coming from people with as good of credentials (or better). Simply ignoring criticisms and carrying on as though your work is totally valid and the points they’ve made so utterly incompetent as to not merit consideration is galling. It’s the type of thing you see when someone has an unfalsifiable belief and they move the goal posts right in front of you when you provide them with evidence that they’re wrong.

Engaging with your critics and surmounting their criticisms is a fundamental part of the scientific process, and I can’t help but think that your science would come out all the better for it if people point out the flaws and you amend your work to compensate. You know, amend your CURRENT work. Not simply “incorporating the dissent” into future works. Especially when those future works are also apologetic to a cause you’re evidently trying to advance, despite precious little valid data to back you up.

Stephanie’s list of links also has a bit of a secondary trend, which I’m sure is not accidental. Each of the blog posts she links to seems to have a fairly controversial bent, regarding all manner of things from eugenics to speeches in front of White Nationalist conventions to the “perils of diversity” to defense of sweatshops. The common theme to all of them appears to be a generalized defense of racism. Considering Kanazawa’s paper, considering Kanazawa’s already controversial history, and considering the vast criticism leveled against his academic practices, the defense paper’s purpose is all too transparent: protect one of your own.

One question that Stephanie raised piqued my interest: “Someone for whom impact factor is a big deal will have to do the research on whether the letter writers are correct [in asserting Kanazawa’s been published by many high-impact journals], but I would love to see the results.” As she and I both point out, Intelligence is fairly high-impact, but also high-controversy — it caters almost exclusively to people who believe IQ is actually worth something. It will therefore be cited very heavily by scientists who believe likewise. This may or may not be a self-feeding subculture of scientists, who may or may not be engaging in an amount of cherry-picking, bias, or other scientific fallacies that depend on people desperately wanting to be right even at the cost of parsimony with reality. It is akin to scientists in the Creation Science field, wherein people presume Goddidit and the science must flow from that initial premise or it is out of orthodoxy with their subculture.

I’m working on finding impact studies for each of these journals in which Kanazawa was published. I found an Excel spreadsheet of journals from 2007 with their Thomson Reuters impact factors, but his papers span from 1992 through 2011, and it would be unfair to provide a snapshot view of the impact of these journals in only 2007.

If I can’t find anything more recent (e.g., if nobody provides me with a login for the current Thomson Reuters Journal Citation Reports tool), I’ll put together a follow-up blog post with the numbers from 2007, with a scale as to where they fall in the “impact factor” for that trade. I might also have to eliminate some of the top journals in the field, as a number of them appear to act as aggregators and get disproportionately high journal impact which would skew the point I intend to make: that the journals Kanazawa is published in, are not in fact “high-impact” by any reasonable standard as implied by his defenders.

Like this:

Related

About the author

Hi! I'm a tech guy, skeptic, feminist, gamer and atheist, and love OSS and science of all stripes. I enjoy a good bit of whargarbl now and again, and will occasionally even seek it out. I am also apparently responsible for the death of common sense on the internet. My bad.

I have opinions. So do you. You want to share them with me. I would like to do likewise. Please don't expect a platform for proselytizing that will go unchecked and unchallenged, though. Contact me via the clicky thingies under my banner.

The commenting rules are simple: don't piss me off. This rule has worked for me for a decade; I have never found a need for any other rule, because any other rules leads to rules-lawyering. Just remember -- this is my property, not yours.

If you want my reasons for supporting Kanazawa, they’re here. In short, I reckoned the prior letter in The Times pretty awful: whatever critiques folks have of Kanazawa’s academic work, some of which are more reasonable than others, academics piling on in the midst of a populist campaign for Kanazawa’s ouster based on outrage over a blog post was not on. It was opportunistic, crass, and did harm to academic freedom. Even if you accept every one of the (initial) letter writers’ concerns, publishing that letter while folks were getting their rage on over a blog post was malicious. Every academic who also blogs ought be worried if this is the new norm.

I’ll also vehemently disagree that my defense of sweatshops is at all racist; rather, I argue in the post that sweatshops are a less bad option than others available to poor people in poor countries. You can disagree with my analysis, but I’ll thank you for perhaps calling me stupid rather than racist if you wish to go ad hominem in critique.

Thank you for your comment, Eric. I feel quite strongly that in your blog post — which, by the way, undercuts your outrage that these papers should be discussed in the blogosphere! — you’ve mischaracterized the opinions that run counter to Kanazawa’s research. While there are a good many of us who recognize racism lending to selection bias, the actual critiques of his data collection, methodology, and lack of engagement with the critiques as pointed out by Stephanie Zvan (linked in my post) and others stand. The “populist campaign” merely served to amplify the outcry against his data collection, methodology and lack of engagement with the critiques. It gave these critiques more light than they would have received otherwise.

Do you disagree that these critiques were sound, and if so, why? If not, you likely wouldn’t have signed your name to that letter, would you? Either that, or you’ve got some massive cognitive dissonance going off, I could well imagine.

As for whether or not you’re a racist, do you in fact believe that there is such a thing as “race” that transcends mere cultural transmission and enters the realm of genetics beyond simple pigmentation or other such minor cosmetic differences? And do you in fact believe there is an objective standard for “attractiveness” that is not also culturally transmitted? If you believe in either of these things, then you believe that race is a valid societal construct, and therefore are (in a loose and non-stigmatized sense) a racist.

Meanwhile, an ad hominem is in fact when one uses an insult as a premise for the argument, not as part of the conclusion. My argument in the particular case of your defending sweatshops is not that defending sweatshops is inherently racist, despite the prevalence of sweatshops in underprivileged countries to the monetary advantage of a “first world” country, but that when viewed with the prominent actions of each of your co-signatories it’s quite easy to view that defense of sweatshops as part of a larger trend. It is, therefore, quite easy to get the impression whether correct or not that you are racist, in the more pejorative sense, in that you ignore the methodological issues with Kanazawa’s works (to the point where you admire him) ostensibly because you agree with his conclusions.

I have a few questions for Kanazawa and any of his apologists. What do you think of the significant occurrence of fibroids resultant from low testosterone and high estrogen in black women? And how do you figure one can control for own-race facial preference in determining “attractiveness” objectively?

Beyond that, how exactly does one map either allele frequency or attractiveness onto a scalar value?

I’ve no problem with papers being discussed in the blogosphere. I do have a problem with that popular outrage against a blog post was latched onto by folks who didn’t like Kanazawa’s published academic work for a hit piece in the Times Higher Ed that seemed aimed at getting Kanazawa fired.

Whatever flaws there might be in Kanazawa’s papers, and I’ve not the free time now in the middle of earthquake-land here in Christchurch to run a more thorough review of his work than I posted earlier, did not merit the treatment he received.

As for whether I agree with Kanazawa’s conclusions, that depends on the paper. I agree with him on the Savannah Principle (his 2004 Managerial and Decision Econ piece); his De Gustibus Est Disputandum piece is thought provoking; his “Teaching may be hazardous to your marriage” piece is great fun and probably right. On relative racial attractiveness, I’ve pretty flat priors and would go where the data takes it. If the critics who said incorporating fourth wave data eliminates the effect are right that’s fine. I can’t believe malice on Kanazawa’s part in using only the first three waves’ data; that’s what would have been on his hard drive from prior work and it’s not worth updating hard to get datasets for a blog post. I’d read Kanazawa’s testosterone speculations as just being a candidate explanation for an odd pattern in the data. It could be wrong; I have no clue about relative testosterone levels. But that’s the kind of thing you get to put in blog posts – the “Hey, maybe there’s something here!” bits that might or might not lead anywhere.

As for whether objective non-culturally specific beauty exists, I’ll propose a bet. Take two pictures to one of the very few paleolithic tribes still living isolated in Brazil or the Pacific. One picture is of Halle Barry; the other is of Roseanne Barr. Both at age 30. I’m pretty happy to bet that the median rater among that isolated tribe would prefer Barry. If beauty is purely cultural, you’d have to reckon even odds across pictures. So I’ll be generous and give you 2:1 odds.

Is Roseanne vs Halle a fair test? What about picking two people with approximately the same BMI and waist-to-hip ratio, varying only on the facial features or skin pigmentation? You’re introducing a number of variables we already know skew general attractiveness cross-culturally, both of these values vary culturally (e.g., in China the average preferred waist-hip is 0.6, in Africa 0.9), and there are in fact societies both historical and present-day that prefer plumper women. In at least one anecdotal case, I know in Russia larger women with wide hips were more likely to weather harsh winters and bear children.

And you are avoiding the question of actually mapping attractiveness and race to a scalar and objective value. That’s pretty important as a prior before we can actually interpret the data with any degree of usefulness, regardless of the fact that the data collection methods themselves were specious at best.

The problem, Eric, is that the data doesn’t take us anywhere all by itself. If it did, we wouldn’t need science. We’d just look at the world around us and understand everything.

You keep talking as though the letter-writers had a problem with Kanazawa’s results. They were quite clear that it was his methodology they found lacking, had found lacking for years, had complained about for years.

That’s important, because methodology is what actually leads us wherever it is we end up. Not data.

And methodology is the problem with that idiotic blog post, as has been pointed out more than once. Using only three waves might be fine. Making claims about women from data on adolescent girls is not. Using single-reviewer ratings of attractiveness is decidedly iffy, even for a blog post. Using them as indicators of anything “objective” is unconscionable.

Pointing out that the blog post is part of a larger history of methodological problems is both reasonable and important. It is not as though the letter-writers have been silent up to this point or have ever declined to put names behind their accusations.

Could all this get Kanazawa fired? I don’t know. If it happens because of accusations of poor research, and those accusations are substantiated, I’m okay with that. That research, after all, is what we use to interpret the world around us, to make sense of those facts. If someone indulges is research malpractice, after years of ignoring criticism, there should be some consequences to that.

@jthibeault: If there is no culture in the world in which the average person would find Roseanne Barr more attractive than Halle Barry, that’s at least consistent with that there are some fundamentals of beauty that are consistent across cultures and are probably evolutionarily determined. Like left-right body symmetry as signal of genetic fitness. And what lit I’ve seen on waist-hip ratio suggests pretty common agreement on somewhere around 0.7 being best. Mapping attractiveness onto a scalar just doesn’t seem that hard, at least if the scalar figures just represent an ordinal ranking. Again, I’d be willing to bet that if you took a set of ten people’s pictures where there were reasonable across-rater within-culture agreement on rank-order beauty (don’t go picking a set of ten fashion models where variance in true beauty is low enough that rater idiosyncracies dominate), you’d also get reasonably high agreement on that same rank order among raters from another culture. And you have to know that contamination of race data in the US by healthy racial mixing just attenuates results, right? So all the “how do you measure race” stuff: that’s almost certainly attenuation bias. I can imagine ways of generating bias rather than just increased variance by such data contamination, but they don’t seem all that likely.

@Stephanie: Y’all are holding Kanazawa’s work to a higher standard than that to which other social scientists are commonly held, and I think it’s because his results aren’t popular. Gelman’s published critique, for example, is statistically correct that you need to correct standard errors for the number of potential ways of categorizing data. But the thing is that NOBODY in the social sciences actually does this. I’d be surprised to find even one in fifty doing it in Econ papers where such correction would be warranted.

So long as beauty in adolescence correlates with beauty in adulthood, using the former measure where the latter isn’t quickly available, at least in a blog post, is fine; your confidence intervals for claims about adult differences just increase. Sure, there’s noise between ordinal rankings of beauty from adolescence to adulthood. Again, think of it in terms of a bet. What do you reckon the odds are that the folks who were most beautiful in your middle-school classes would also be found so now, on average, if a similar peer group were asked to rank-order them? I’d be betting on no systematic rank order changes. You’d need to make the claim that there’s racial difference in the path of beauty over the life cycle (racial differences in the proportion of ugly ducklings who turn into beautiful swans for example) for that to seriously screw up his results. Otherwise, it’s just noise. Single reviewer ratings? If that’s all you’ve got, it’s better than no-reviewer ratings, but I’d thought he’d run factor analysis across multiple raters; I’m rather sure what he meant by use of “objective” was that his averaging across rater ratings through factor scores reduced reviewer-level idiosyncracies.

I suspect we could argue about this for rather longer than the opportunity costs of my time allow. But a parting thought. How many of the folks who wrote letters against Kanazawa would have written an identical letter against Stephen J. Gould, whose work on cranial capacity is now found to have been fabrication? Would you be leading a charge to get Gould fired for using worse method in his big published book than Kanazawa used in a blog post?

Y’all are holding Kanazawa’s work to a higher standard than that to which other social scientists are commonly held

Citation needed.

The rest of this is an obvious “I’m right” parting shot. Is there something that needs specific answering, ladies and gentlemen of my actual audience? Perhaps Greg Laden might like to take up the Gould / Morton debacle, seeing as he’s discussed brain capacity and racism in the past?

Ah, hello Dr. Bryan Pesta. Long time no see. How’s that fixation with fighting with bloggers who suspect your colleagues of misdeeds toward the end of racism working out for you?

I’ve long suspected that there are smaller journals that depend on gaming the citation and impact factor system to gain legitimacy for their otherwise unscientific hypotheses, like the specific example that you’re trying to generalize to “experts vs bloggers”. That said, a conspiracy theory is less likely than is simply a group of people who, coincidentally, happen to have the same biases.

I don’t believe people should be fired for their methodology being wrong. I believe they probably should have been exposed for shoddy methodology before they got tenure. But if one or two slip through the cracks, I absolutely believe there should be a process by which their tenure can be called into question. It would suck for those people that gain tenure for evolutionary biology and have it called into question by creationists, but without such a mechanism, racists trying to prove their racism has a scientific basis should certainly not escape scrutiny by virtue of their tenure alone. The fact that they’re espousing a societal bete noir theory is bad enough, but the fact that they’re hiding behind the fact that they’ve been good enough to gain tenure does not compensate for the fact that they’re espousing ideas that are not supported by the evidence (no matter how many times you folks try, and no matter how hard you skew the data).

I suspect all journals game the impact factor to their advantage (what gets measured gets attention). I don’t think self-citations in a specialty journal are always motivated by raising the journal’s impact factor, though.

“Intelligence” for example is around 3.2– very high for a journal in social science. Part of that could be they sometimes publish controversial research. But, look at their 10 most cited papers:

Only one relates to race and IQ. Most of it’s mainstream, reputable / good science (Sternberg has one of the top 10. Why would he associate with a journal where known racists can get their junk science published?).

As for Kanazawa, why not measure his impact directly by looking at how many times his stuff’s been cited?

You can say the citations are inflated because people are attacking his crappy work. Perhaps, but most junk science is never published, and if so, it’s ignored by journal articles (how many top evolution journals publish papers debunking creationism?).

One criterion for good scientific theories is that they be fruitful (generate lots of activity in the peer reviewed literature). Crappy scientific theories aren’t crappy because they’re wrong, or controversial, but because they are ignored.

Additionally, Elsevier doesn’t list Intelligence as a social sciences journal, but as a multidisciplinary. That impact factor is less impressive in that context.

Kanazawa, as Stephanie and I both pointed out in our original posts, has published a good deal of more mainstream and uncontroversial scientific studies. They give cover to the articles where he uses specious methodology to reach specious conclusions that just coincidentally (I’m sure) happen to support a racist viewpoint. And when other scholars critique him, he does not improve the paper in question but rather only addresses those criticisms in later papers.

Regardless of anything else, it truly does appear what parts of his scholarship are unobjectionable, are there simply to help him make his controversial and specious claims. If he’s doing this on purpose, his tenure should absolutely be called into question. Especially since his crappy scientific theories are being published in journals that appear to have a known bias toward the results he found, and everyone who’s screaming about them throughout academia are doing so not because they are unaware that the proper protocol is that you’re supposed to ignore crappy science (which I strongly doubt), but rather are screaming because they can’t believe this crappy science is being published, repeatedly.

It’s not bait. He’s a published scientist. He thinks Intelligence is biased. I said in my article and in these comments that I suspect as much myself, what with all the race proponents getting published in it. We can call their being published my evidence on the reputation of the journal. That and the fact that many published, intelligent, influential scientists think as James Flynn does: “IQ tests do not measure intelligence but rather correlate with a weak causal link to intelligence.”

Everything else built around IQ to justify people mistreating certain classes of individuals is pseudoscience and wishful thinking, if you ask me.

The journal printed this, which was written by one of their editors and signed by a number of others, including the editor in chief. Then there’s the large overlap between their editorial board and recipients of “research” funds from the Pioneer Fund.