I still think the bad apple analogy has some merit. By not only tolerating but promoting bad apples it suggests a problem with the culture at Cornell. I’ve only attended two different universities so maybe I’m not in the best position to judge this, but I immediately noticed a huge difference in culture between Berkeley and UVA. It’s not hard to imagine that if I enrolled in a program at Cornell I would also detect a difference in culture. You could say I might smell what they are cooking.

P.S. In case you were wondering, UVA is where science goes to die.

]]>By: AnonAnonhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-465676
Fri, 14 Apr 2017 18:16:40 +0000http://andrewgelman.com/?p=33459#comment-465676Jordan: the problem with the bad apple analogy is that it uses our disgust response to obfuscate that the mechanism by which apples rot doesn’t clearly analogize to how “bad actors” taint the whole group.

That is, Wansink’s research malpractice is quite disgusting. But for the bad apple analogy to hold, there must be a corruptive influence that Wansink has. And I’d suggest that at least within Cornell it would be limited to the Food lab. And perhaps more broadly if you accept the argument that Wansink’s work has central theme of mindlessness when it comes to eating. But I wouldn’t label

Bem, on the other hand, has had I think a much wider corruptive influence. From his “Writing the Empirical Journal Article,” which I was required to read in my first year of graduate school.

“Analyzing Data. Once upon a time, psychologists observed behavior directly, often for sustained periods of time. No
longer. Now, the higher the investigator goes up the tenure ladder, the more remote he or she typically becomes from the grounding observations of our science. If you are already a successful research psychologist, then you probably haven’t seen a participant for some time. Your graduate assistant assigns the running of a study to a bright undergraduate who writes the computer program that collects the data automatically. And like the modern dentist, the modern psychologist rarely even sees the data until they have been cleaned by human or computer hygienists.

To compensate for this remoteness from our participants, let us at least become intimately familiar with the record of their behavior: the data. Examine them from every angle. Analyze the sexes separately. Make up new composite indexes. If a datum suggests a new hypothesis, try to find additional evidence for it elsewhere in the data. If you see dim traces of interesting patterns, try to reorganize the data to bring them into bolder relief. If there are participants you don’t like, or trials, observers, or interviewers who gave you anomalous results, drop them (temporarily). Go on a fishing expedition for something—anything—interesting.

No, this is not immoral. The rules of scientific and statistical inference that we overlearn in graduate school apply to the “Context of Justification.” They tell us what we can conclude in the articles we write for public consumption, and they give our readers criteria for deciding whether or not to believe us. But in the “Context of Discovery,” there are no formal rules, only heuristics or strategies. How does one discover a new phenomenon? Smell a good idea? Have a brilliant insight into behavior? Create a new theory? In the confining context of an empirical study, there is only one strategy for discovery: exploring the data.

Yes, there is a danger. Spurious findings can emerge by chance, and we need to be cautious about anything we discover in this way. In limited cases, there are statistical techniques that correct for this danger. But there are no statistical correctives for overlooking an important discovery because we were insufficiently attentive to the data. Let us err on the side of discovery.”

Bern, D. (2003). Writing the empirical journal. The compleat academic: A Practical Guide for the Beginning Social Scientist, 2nd Edition.

]]>By: Jordan Anayahttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-465661
Fri, 14 Apr 2017 17:21:01 +0000http://andrewgelman.com/?p=33459#comment-465661Cardinal Sins: I think the United comparison is interesting. Yes, being dragged off your flight for no reason is a rare occurrence, and doesn’t reflect on the employees, but when it happens it raises questions about why and how often it occurs. Since the event stats have come out showing United overbooks flights more often than its competitors and based on the CEO’s response it was clear that the employees were following standard operating procedures. As a result, I think it’s safe to say that if you flew United you were at a higher risk.

Similarly, I’m sure a Wansink is a rare event, but given Cornell’s response I have no reason to believe they view Wansinks as a problem and I have no reason to believe Cornell isn’t enriched in Wansinks. In fact, on this very blog Gelman talks about Daryl Bem and Thomas Gilovic, who are both from Cornell. Based on all the evidence it’s not hard to imagine that Cornell specifically recruits these types of “scientists”. Perhaps this is only limited to one or two departments, but until I have reason to believe otherwise any time I see the affiliation of Cornell on a paper I’m going to view it as a huge red flag.

Extending the bad apple analogy, imagine you went to the store and found a rotten apple in clear view of all the employees. They clearly see it but you mention it to them anyways. They tell you they don’t see anything wrong with the apple. You contact their manager and he/she says there’s nothing wrong with the apple. The CEO of the store chain releases a statement saying they investigated the case found no problems with the apple. Are you really going to keep shopping at a store that pawns off rotten apples as fresh fruit? Do you really want to stick around to see the quality of the other food?

]]>By: Anonymous2http://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-465618
Fri, 14 Apr 2017 15:00:31 +0000http://andrewgelman.com/?p=33459#comment-465618Cardinal sins: +1000
]]>By: Anonymoushttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-465617
Fri, 14 Apr 2017 14:59:48 +0000http://andrewgelman.com/?p=33459#comment-465617Jordan Anaya: Berkeley is an outlier in terms of activism, even today, although it’s a far cry from what it was in the
60s (Mario Savio, etc.). You cannot expect other schools to operate the same way.
]]>By: Cardinal sinshttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-465586
Fri, 14 Apr 2017 13:38:10 +0000http://andrewgelman.com/?p=33459#comment-465586I’m a tenured scientist at Cornell. I work in a different department from Wansink, in a different college, on a different part of campus. In the decade plus that I’ve been here, I’ve never met the guy. AEM is only on my radar because it has some excellent development economists, and because the last (and really only) act of a now-deceased president was to merge AEM with two other units into a new undergraduate business college, which got a lot of press in the campus newspaper.

I’m not proud that Wansink is at Cornell, but that doesn’t make me responsible for his shoddy research, nor does it give me special powers to do anything about it. I also don’t think that all United Airlines employees are imbeciles who are responsible for the decisions of the gate agents of flight 3411 or for CEO and PR department’s tone-deaf response to the “overbooking” incident.

What’s interesting is that, in terms of quality of work, Daryl Bem in his ESP research is just about as bad an “apple” as Brian Wansink. But Bem doesn’t bother people as much, perhaps because he’s viewed as more of a lovable eccentric, whereas Wansink is going on government panels and garnering millions of dollars of government grants.

Regarding the reaction, or non-reaction, of Cornell students and faculty: I think they just have better things to do. Here I am at Columbia University, where our most famous faculty member is the notorious Dr. Oz. I’m not happy that Dr. Oz is at Columbia, but I’m not distributing petitions to get him fired. I’m just ignoring him. If the Columbia public information office started promoting Dr. Oz, this wouldn’t make me happy, but I don’t know that there’s much I could really do about it.

]]>By: Jordan Anayahttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-464883
Wed, 12 Apr 2017 22:45:22 +0000http://andrewgelman.com/?p=33459#comment-464883Anonymous: I didn’t mean to downplay anyone’s contributions. The video was meant to be from my perspective, I’m sure if Nick or Tim made a video the story would be slightly different.

Tim has made very important contributions to this investigation. Because my talk was limited to an hour with discussions I could not cover everything. Maybe I should have made a presentation with no time limit…

I actually do think it is somewhat valid to cast doubt on the “scientists” at Cornell because of one bad apple, or at least the people in Wansink’s department. Wansink is not some obscure Cornell faculty member, he’s featured prominently on Cornell’s websites and he teaches courses and gives lectures at Cornell. No one at Cornell ever raised concerns about his obvious p-hacking? No one at Cornell ever noticed his numbers don’t add up?

And I’m sure students and professors at Cornell are aware of the current developments, so why haven’t they done anything? If I was a faculty member and saw that a fellow faculty member committed clear academic misconduct I would start a petition to have him disciplined. All the real scientists (if they exist) at Cornell should be deeply embarrassed. And if I was a student in one of Wansink’s courses I would have been extremely disruptive, peppering him with questions throughout the lecture. He would never dare set foot in that class again.

I have to imagine that if misconduct at this level happened at my alma mater of Berkeley the students and faculty would have done something. I would have.

“If they [Cornell University] told us they were launching a full investigation into Wansink’s work when we first contacted them”

“paints Cornell as a university which probably shouldn’t be taken seriously by the scientific community”

You are naive in the extreme to think that Cornell would do much of anything when first contacted by three graduate students. It just doesn’t work that way.

Why should one bad apple cast doubt on the research of the many fine scientists and academic programs at Cornell?

]]>By: Anonymoushttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-464800
Wed, 12 Apr 2017 18:26:41 +0000http://andrewgelman.com/?p=33459#comment-464800Jordan Anaya: Very interesting video. You would make a good teacher. But why is there no mention of Tim van der Zee? You make it sound as though you and Nick Brown did all the work. I’ve seen van der Zee’s summary (dossier, he calls it) and it’s clear that he has contributed a lot of boring, painstaking work to this project.
]]>By: Jordan Anayahttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-464771
Wed, 12 Apr 2017 17:26:28 +0000http://andrewgelman.com/?p=33459#comment-464771Dalton: You make a lot of good points.

Yeah, of the three of us (Nick, Tim, and I) I’m the loose cannon of the bunch with the least amount of patience. At some point during this investigation Nick Brown also got fed up. I think Tim van der Zee is getting there.

I gave a similar talk recently at UPenn and I think it was well received. I believe we followed pretty good practices in our investigation, but Wansink’s and Cornell’s responses have led me to use a little bit of explosive rhetoric here and there.

]]>By: Dalton Hancehttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-464766
Wed, 12 Apr 2017 17:06:38 +0000http://andrewgelman.com/?p=33459#comment-464766I’m very hesitant to paint with such a broad brush, the perils of aggregation and all that. For one, I think it’s a little unfair to not distinguish between Fox News broadly and Bill O’Reilly/Jesse Watters narrowly. Fox News has some good journalists and some good entertainers and provecateurs. Just as NBC as some good journalists and some good comedians. From a media relations perspective you wouldn’t give the same respone to Lester Holt and Jimmy Kimmel. Watters is in the latter category of Fox News and has repeatedly demonstrated himself so. From a media relations perspective I would come into any engagement with him with a very strong prior of “This man is a troll.”

For some of these same reasons, I strongly disagree with the statement that “Cornell as a university which probably shouldn’t be taken seriously by the scientific community.” Cornell’s Media Relations probably shouldn’t be taken seriously by the scientific community. But I think that is more so because it is an object of the class “PR Department” than an object of the class “Cornell Organ”. The PR departments of many universities shouldn’t be taken seriously by the scientific community. I agree that Brian Wansink should definitely not be taken seriously by the scientific community. Exact in so far as he paints himself as a scientist and it is the duty of other scientists to tear that masquerade apart. The response so far by Cornell does make it seem like somebody higher up within the food chain shares some of the blame, but it’s a matter of how high we take that. Probably the rot doesn’t extend much beyond the School of Applied Economics and Management which is the next higher level of organization for the Food and Brand Lab. This itself is part of the College of Agriculture and Life Sciences which also contains the (very fine) Ecology and Evolutionary Biology and the Biological Statistics and Computation Biology Departments. Brian Wansink’s sins do not make me doubt the work being done in those other areas. What I want to see is a response by Brian Wansink’s boss. Not some poor PR peon tasked with responding to the entire ecosystems of outside media.

For some of these same reasons, I’m not surprised that Cornell didn’t immediately launch an investigation into Wansink. We live in a very information dense, partisan and sometimes hostile scientific environment. Unfortunately it’s perhaps difficult to sort out valid criticism (yours) from less valid criticism (e.g. Watters World or the criticisms directed at Penn State’s Michael Mann). So I think your last paragraph with the Kobe Bryant analogy is perhaps apt here, but that there is another side to the coin. If I were a climate scientist or otherwise working on some topic with policy implications (e.g. endangered species, environmental contamination and human health, etc.) I would surely hope that my employer has my back and presumes my innocence unless proven otherwise.

That said, you and Andrew and others are doing good work to point out these flaws in the research. It’s also not a surprise your encountering resistance. Inertia is a bitch. But I think presuming ill-will on part of the larger body that employs a scientists with questionable publications may work to undercut your own case by allowing room for a counterargument that you yourself are a troll. (You’re not.) Implicitly defending true trolls also may not be a good rhetorical strategy. I think you will have to be persistent, but it doesn’t hurt to also be polite.

]]>By: Jordan Anayahttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-464749
Wed, 12 Apr 2017 15:57:53 +0000http://andrewgelman.com/?p=33459#comment-464749I agree that Fox news is trolling, however I found Cornell’s response very interesting since you could argue that is exactly how they have responded to pizzagate, which I think we can all agree is a legitimate story.

When dealing with a troll typically you first start by trying to address their concerns. When they then continue to troll regardless of what explanations you provide is when you should start ignoring them. I’m sure the Fox footage was heavily edited, but given my experience with Cornell it would not surprise me if Cornell never provided any meaningful answers to Fox news.

Cornell’s policy seems to be to treat everyone as if they are trolls and don’t deserve legitimate responses, but when someone first contacts you how do you know for a fact they are a troll? Perhaps Fox news didn’t deserve a response in this case, but I don’t think Fox was looking for a response from the administration in the first place, they were simply interviewing students. Cornell presumably thought that those interviews would hurt their reputation, so they kicked Fox news off the campus, which I guess they have the right to do and maybe it was warranted, but I don’t really see how the news crew was causing the students any harm (who doesn’t want to be on TV?).

Admittedly, the goal of Fox news was probably to make Cornell look bad, but Cornell did them a favor with the manner in which the media relations department reacted, and made it easy for Fox news to make them look even worse. Maybe the United airlines media relations department and Cornell’s follow the same handbook or something.

You could argue that how Cornell has responded to pizzagate has also made the story much worse. If they told us they were launching a full investigation into Wansink’s work when we first contacted them then maybe we wouldn’t have had to launch our own investigation, which has led to one retraction and counting. And their continuing portrayal of massive amounts of impossible numbers and duplicate publications as minor criticisms that don’t amount to academic misconduct paints Cornell as a university which probably shouldn’t be taken seriously by the scientific community.

On the one hand I do think their response has some merit. When the Lakers made Kobe Bryant the highest paid player in the league despite being injured and old some called it a terrible contract. But in doing so the Lakers showed that they were loyal to their superstars. As a result, other superstars may view the Lakers favorably and want to play for them. Cornell’s response basically shows that no matter what happens they will not discipline tenured faculty, which is great if you are a tenured faculty member. As a result, other “superstar” faculty may see Cornell’s response and think “hey, that’s a place I’d want to work since I know the university has my back, ride or die!”. The problem is the “superstars” this type of thing attracts could team up to form the Mighty Ducks; quack quack.

Cornell is perfectly within their rights to not engage with trolls. This is no different from asking a film crew for the Daily Show or one of the late night programs to not film. It is not shutting down journalism, it is about not being a foil for an entertainer to make a buck of you.

Furthermore the entire premise of the piece is bogus. The alleged injustice is that 96% of political donations from the faculty went to liberal/progressive/democratic (whatever word you want to use) candidates. While this is perhaps evidence of the political leanings of faculty, it is not evidence of a deliberate discrimination in hiring based on political viewpoints. If Cornell were to ask candidates what political causes they donate to and then make hiring decision based on that information, then there would be a story here. But it seems that is exactly what Bill O’Reilly and his buddy are suggesting they do to “remedy” this situation.

We don’t criticize scientists without at least reading the entire paper. Let’s not laud a journalist and demonize a reporting subject without at least looking a bit into the context.

]]>By: Anonymous2http://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-463644
Mon, 10 Apr 2017 21:42:53 +0000http://andrewgelman.com/?p=33459#comment-463644Retraction Watch reported the retraction today.
]]>By: Nickhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-463612
Mon, 10 Apr 2017 20:38:32 +0000http://andrewgelman.com/?p=33459#comment-463612There is now a retraction notice: http://onlinelibrary.wiley.com/doi/10.1111/joss.12259/full
]]>By: Anonymoushttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-463509
Mon, 10 Apr 2017 17:36:16 +0000http://andrewgelman.com/?p=33459#comment-463509There is now an explanatory note (“Retraction Statement”) attached to the retraction at JoSS. The article is retracted due to “major overlap with a previously published article.” There is no indication that Wansink agreed to the retraction.
]]>By: Jordan Anayahttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-463304
Mon, 10 Apr 2017 10:53:36 +0000http://andrewgelman.com/?p=33459#comment-463304Neurocopiae did an analysis of the data:https://neurocopiae.wordpress.com/2017/04/10/when-you-handle-trash-do-you-still-have-to-handle-it-with-statistical-care/

A lot of the survey questions in the data set are extremely similar to each other, so I’m not quite sure how to interpret neurocopiae’s analysis at the moment.

]]>By: Nickhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-462755
Sun, 09 Apr 2017 14:24:53 +0000http://andrewgelman.com/?p=33459#comment-462755(Anonymous)
>>I wonder if Nick would share with us the letter to the editor
>>of JoSS that resulted in the retraction?

I didn’t contact the journal. In fact this particular article only appeared on my radar quite recently (I blogged about it on March 20), so the journal has acted quite rapidly. As I noted in my blog post, the 2002 article (which has two authors, Wansink and Park) acknowledges the contribution of two other authors to “the original version of this project”, by which Wansink and Park appear to mean the 2000 version of the article, which had four authors. Indeed, the 2000 version is cited on p. 484 of the 2002 version: “In extending past research (Wansink et al. 2000)”.

]]>By: Jordan Anayahttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-462561
Sun, 09 Apr 2017 09:24:14 +0000http://andrewgelman.com/?p=33459#comment-462561I emailed the journal. I wasn’t informed that the paper was being retracted, so I don’t know if my email led to the retraction or not.
]]>By: Daniel Lakelandhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-462286
Sun, 09 Apr 2017 00:30:15 +0000http://andrewgelman.com/?p=33459#comment-462286Duh, for the cash mo-nay

]]>By: Anonymoushttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-462242
Sat, 08 Apr 2017 23:04:11 +0000http://andrewgelman.com/?p=33459#comment-462242I see that the article by Wansink and Park (2002, Journal of Sensory Studies) has been retracted now, although there is no explanatory note attached. Presumably the retraction occurred because the article duplicated the earlier article by Wansink, Park, Sanka, and Morganosky (2000, International Food and Agribusiness Management Review), as detailed by Nick on his steamtraen blog.

Kudos.

Interesting name: Se-Bum Park.

I wonder if Nick would share with us the letter to the editor of JoSS that resulted in the retraction?

How could Cornell University not consider a duplicate publication to be research misconduct? Maybe it hasn’t gotten to anything but the four “pizzagate” papers yet?

The four pizza papers were published in 2014 and 2015. It is possible that the person who now handles editing journal submissions (Lindsey Brill) was not the person who handled these four papers.

]]>By: Smut Clydehttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-462120
Sat, 08 Apr 2017 19:27:33 +0000http://andrewgelman.com/?p=33459#comment-462120It was a pity that the person who was trusted to finish and submit papers could not be trusted to reply to PubPeer critiques.
]]>By: Martha (Smith)http://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-462106
Sat, 08 Apr 2017 19:13:17 +0000http://andrewgelman.com/?p=33459#comment-462106The lifehacker link sounds like a good way to deal with the situation — sounds like they are being very upfront.
]]>By: Andrewhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-462050
Sat, 08 Apr 2017 17:42:27 +0000http://andrewgelman.com/?p=33459#comment-462050Harvard has those “the replication rate in psychology is quite high—indeed, it is statistically indistinguishable from 100%” guys. Princeton had that guy who gave Hillary Clinton a 98% chance of winning. And, here at Columbia, we’ve Dr. Oz!
]]>By: Jonathanhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-462027
Sat, 08 Apr 2017 16:41:02 +0000http://andrewgelman.com/?p=33459#comment-462027Lifehacker even posted a friendly PSA about all the posts that now might be wrong:http://lifehacker.com/some-of-our-best-food-hacks-may-be-wrong-1794125552

I wonder if this is the correct way of handling the situation. Not sure how a major publication would (i.e. NyTimes or WSJ).

]]>By: Dan F.http://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-461928
Sat, 08 Apr 2017 12:12:19 +0000http://andrewgelman.com/?p=33459#comment-461928Cornell has no particular claim on bullshit, even in the Ivy League setting.
]]>By: Ron G.http://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-461911
Sat, 08 Apr 2017 11:38:22 +0000http://andrewgelman.com/?p=33459#comment-461911Is it safe to say that Cornell University should no longer be considered Ivy League? Because c’mon, protecting this kind amount of misconduct tells all about an organization that you need to know.
]]>By: Anonymoushttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-461871
Sat, 08 Apr 2017 10:20:20 +0000http://andrewgelman.com/?p=33459#comment-461871“After all, if he really didn’t collect the data, or he really didn’t remember how the data were collected, and if he really didn’t write the paper, or he really didn’t remember what the paper said, how can he be so sure that what he or his colleagues did was correct? Especially given the zillions of errors that have already been revealed in his work.”

Are you thinking of the Pubpeer comment, where Wansink explained the false description of authors’ contributions as being the fault of the secretary (or whoever) who finished and submitted the paper? Who was following a standard template which in this case was not correct?

I’m aware of but not particularly following the Wansink business; however, that sentence got my attention. In the context of the letter it’s taken from the takeaway is “With respect to Wansink’s Stata code, the researcher could not distinguish between ‘runs to completion’ and ‘works’ if their life depended on it.” If you’re not going to attempt to establish whether the result is worth damn then why bother? Your time would be just as well spent picking your nose and drinking beer.

]]>By: Lilyhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-461524
Fri, 07 Apr 2017 21:42:28 +0000http://andrewgelman.com/?p=33459#comment-461524This is like watching the ending scenes in the movie BRAVEHEART.
]]>By: Daniel Lakelandhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-461475
Fri, 07 Apr 2017 19:30:50 +0000http://andrewgelman.com/?p=33459#comment-461475@Bob, yes a feedback control system that applies forces on the mirrors so that as the cavity expands due to thermal changes, the mirrors don’t get farther apart. Well, how do you know that the changes are due to thermal expansion? The fact is the feedback control system is sensitive to *some* changes in cavity length (such as those caused by a slow trend of expansion caused by heating), and insensitive to other changes (such as those caused by gravitational waves involving an oscillating change with frequencies within some band)

So, the output signal that is being analyzed is not a pure measurement of “what happened to the cavity” it’s a measurement of “what happened to the cavity given that a sophisticated pre-model was used to control the length of the cavity in a certain way”.

which is in many ways as good as “we measured the length of the cavity, and then subtracted out the signal caused by a known cause such as thermal expansion” at least, at the point of the Bayesian analysis of the data you’re assuming p(Signal | AllTheKnownEngineeringInTheDetector, SomeAstroPhysicalParameters) and Signal = ActualChange – CompensatedChange and CompensatedChange has a very precisely known form thanks to your AllTheKnownEngineering…

According to a reference I found, they don’t filter out the variations in length from the data, rather they move the mirrors to cancel out variations in the length of the cavity. The move the mirrors by shining a one watt laser on them. Photon recoil does the rest. Seehttp://ligo.org/science/Publication-GW150914Calibration/index.php

A quote:
The Advanced LIGO detectors sense this induced length variation of the interferometer arms. These length variations are tiny—at the same scale as those of the passing gravitational waves, about 1×10−19 m. It is remarkable that the force caused by the photons from a small, one-watt laser beam is able to move a 40 kg (88 pound) mirror by an amount that is easily measurable by the LIGO detectors! – See more at: http://ligo.org/science/Publication-GW150914Calibration/index.php#sthash.Qa7xFDnf.dpuf

Say the media catches a CEO with his pants down or drunken driving or something. The first response is to deny or downplay the scandal or waffle. Most times these defense tactics work and the issue dies down and everyone’s happy.

But sometimes you cannot suppress the disaster, and then such initial defenses can look silly or worse. In those cases in hindsight the PR team wishes they had not tried the ducking n weaving.

I’m not justifying Wansink’s conduct; just saying that this seems typical PR strategy.

]]>By: Martha (Smith)http://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-461367
Fri, 07 Apr 2017 16:21:14 +0000http://andrewgelman.com/?p=33459#comment-461367The Stark commentary seems good as a brief “cautionary guide” for laypersons. I haven’t had a chance to read the entire Shalizi commentary yet, but the introduction sounds good — I get the impression that he is focusing more on teaching than addressing his comments to laypersons.
]]>By: Andrewhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-461340
Fri, 07 Apr 2017 16:08:17 +0000http://andrewgelman.com/?p=33459#comment-461340Nick:

Of all the evidence so far, the most striking to me are: (1) the survey where someone’s supposed to be 105 years old and everyone else is exactly 18; and (2) that set of numbers that so rarely end in 0 or 5. Of course, these could’ve been created by someone wanting to please Wansink, not Wansink himself. Indeed, coming up with data summaries sees seems below Wansink’s pay grade, as it were.

I agree with you that the carrot story and the elementary schools story are horrible. The most charitable interpretation I can come up with is that Wansink has no idea what is going on in these studies, that he’s not closely involved in design of the studies, nor in data collection, nor in data analysis, nor in writing the papers. (He’s already on record saying he’s not the one who submits the papers to the journals.) So he starts bullshitting about food fights and the quarter-plate method because he himself has no idea what’s going on. And the claim that Elmo appeals to 8-year-olds is coming from some third party who was writing up the paper by putting together paragraphs of previously-written material (which, as we know, is one of Wansink’s habits).

After all, if he really didn’t collect the data, or he really didn’t remember how the data were collected, and if he really didn’t write the paper, or he really didn’t remember what the paper said, how can he be so sure that what he or his colleagues did was correct? Especially given the zillions of errors that have already been revealed in his work.

]]>By: Nickhttp://andrewgelman.com/2017/04/06/dear-cornell-university-public-relations-office/#comment-461221
Fri, 07 Apr 2017 15:26:09 +0000http://andrewgelman.com/?p=33459#comment-461221One of the things that finally convinced me that this whole affair involves more than just sloppiness was when Dr. Wansink, in his second interview with Retraction Watch, not only invoked the quarter plate method (which, to be fair, could have been “in development” when the carrot research was done”; but the article doesn’t mention it, and the draft of the “carrots” article that we found online explicitly states that the uneaten carrots were *weighed*), but also claimed that the discrepancy between “taken”, “eaten”, and “uneaten” was due to some of the carrots being thrown in a food fight or dropped.

This sounds like the kind of amusing anecdote you might say to an interviewer, who you hope will chuckle and move on, but it falls apart under any sort of scrutiny, because /a/ the article explicitly states that they calculated Eaten = Taken – Uneaten, and /a/ unless they had some kind of carrot-o-meter attached to each kid’s oesophagus, they *could not possibly* have had any measure of “eaten” that did *not* include carrots that were dropped, or thrown, or beamed up by aliens. Or it might be something that someone with no knowledge of the experiment might say if you stopped them in the street, gave them a 30-second explanation of the procedure, and asked then why the numbers didn’t add up (“I dunno, maybe some of the carrots got lost or something”). But it seems absolutely inconceivable that the principal investigator and lead author would construct an explanation like this, given that the reality of the study was so simple.

A similar flashing light and loud siren went off for me when I read the comment about “preliterate children”. I cannot imagine under what circumstances one would write up a study that was conducted in seven elementary schools with children aged 8-11 and then describe those 208 children as “preliterate”. I simply don’t see how that word could accidentally emerge from the author’s thought processes. The choice of Elmo as a character to appeal to this age group also appears less than obvious. It is tempting to imagine that the discussion section of this (brief) article was written for a completely different (and, apparently, as yet unpublished study) and then “repurposed”. (And that is about the most charitable interpretation I can come up with.)

The problem is, this is a tiny fraction of what p values are used for. In the case where they’re used for inference on a parameter… the thing that’s bad about them is that they don’t mean what you think they do (you really want the bayesian p(Parameter | Data), and they are often used to make a decision (even though you can’t make any kind of good decision without considering the consequences of your actions, and p values don’t inform you about consequences). So, there’s lots wrong with p values.

The thing is, if you’re not doing the filtering thing described in that link, then… you’re not using p values for what they actually mean which is “this is a strange data point if I assume some particular substantive ‘null’ model”

Yes, it’s possible there are data for some studies and not others. Or that in some papers, the data were slightly massaged in order to get the reported numbers, whereas for other papers, the numbers in the tables were entirely made up. And of course there are all sorts of intermediate possibilities.

I definitely agree in general, and clearly something is up with those examples you mention. But in this case with the data, the code for the corrected analyses, and his descriptions of what caused the 150 errors, it should be easy to replicate those errors. Assuming that can be done, I would be convinced that this is the real data, since it would be really sophisticated to fake data that can match the original and corrected analyses.

Of course, this only addresses one aspect of these 4 papers, and even if the 150 errors were all honest mistakes and sloppy data analysis, there still would be the greater issue of generating hypotheses based on the data, and all the issues in his other papers.