Skepticism

EVENTS

The incredible self-destructing psychologist

Holy crap. A Dutch social scientist’s career has just crashed flamingly. He apparently had a tremendous reputation.

“Somebody used the word ‘wunderkind’,” says Miles Hewstone, a social psychologist at the University of Oxford, UK. “He was one of the bright thrusting young stars of Dutch social psychology — highly published, highly cited, prize-winning, worked with lots of people, and very well thought of in the field.”

But maybe someone should have been made a bit suspicious by this behavior:

Many of Stapel’s students graduated without having ever run an experiment, the report says. Stapel told them that their time was better spent analyzing data and writing. The commission writes that Stapel was “lord of the data” in his collaborations. It says colleagues or students who asked to see raw data were given excuses or even threatened and insulted.

Graduate students who were not doing experiments, a PI who was, graduate students doing all the analysis and writing, a PI who wasn’t? This was an obvious inversion of the natural order. Human sacrifice, dogs and cats living together… mass hysteria! Such peculiar behavior should have alerted someone early on — data are primary.

And now, the denouement:

Diederik Stapel was suspended from his position at Tilburg University in the Netherlands in September after three junior researchers reported that they suspected scientific misconduct in his work. Soon after being confronted with the accusations, Stapel reportedly told university officials that some of his papers contained falsified data. The university launched an investigation, as did the University of Groningen and the University of Amsterdam, where Stapel had worked previously. The Tilburg commission today released an interim report (in Dutch), which includes preliminary results from all three investigations. The investigators found “several dozens of publications” in which fictitious data has been used. Fourteen of the 21 Ph.D. theses Stapel supervised are also tainted, the committee concluded.

Stapel initially cooperated with the investi­gation by identifying fraudulent publications, but stopped because he said he was not physically or emotionally able to continue, says Levelt. In a statement, translated from Dutch, that is appended to the report, Stapel says: “I have made mistakes, but I was and am honestly concerned with the field of social psychology. I therefore regret the pain that I have caused others.” Nature was unable to contact Stapel for comment.

No, he was not honestly concerned with the field of social psychology. If he actually cared about what the evidence told him about the world, he wouldn’t have made it up. He was honestly concerned with the selfish goal of making a name for himself, nothing else.

He’ll never be trusted in any field of science ever again. He’s going to have to look for a new career…maybe theology, where making up data is a way of life.

Comments

Yes, it’s a sad day for social scientists, since it’s inevitable that someone will use this incident as further evidence as to why “serious” “real” scientists shouldn’t take “soft” sciences like psychology seriously.

Piltdown! Nebraska man! Frauds have occurred in evolutionary science, ergo it’s nonsense. (Sure, some religious frauds have occurred, but not in True Religion.)

Uh, yeah, when humans are involved with something, fraud is possible. Of course the IDiots at UD are spinning this as yet another “science isn’t trustworthy” yawp, without any irony due to their own ongoing fraudulent claims.

IMO this smells like a true believer (vice a con artist), and this fraud sounds like a(nother) crime of arrogance, like Jan Schon’s. IMO Stapel believed (like Schon) that future experiments would vindicate his claims. He was just so convinced he was right, he didn’t feel the need to test his claims.

My evidence is pretty flimsy; I don’t think an outright fraud/con artist would have cooperated in identifying papers where he used made up data. That sounds like the act of someone who truly believes his own bullsh*t. But, that’s the way I’m calling it. :)

Oh wow. I want to look up the false data papers to see if what I learned in social psych was based on the fictitious things this man wrote. And if I was one of the 14 PhDs who had my thesis falsified…I think I would consider some type of legal action. That’s a huge chunk of life and work wasted.

Is there a list yet of the papers that used falsified data? It sounds as if any list will be incomplete as yet if he has clammed up. He can do yet more damage if he refuses to identify the data that aren’t trustworthy…. for one thing, people may be forced to throw out everything that he contributed to.

And of course his refusal to show the data hit them not only when they were in a subordinate position, but also at an age where they would take an unreasonable no for an answer. In some situations, ‘No’ means escalate to a higher authority or check with someone else whether the answer was reasonable. In my experience, it took a few incidents of costly acquiescence to refusal before I learned to Question Authority.

I apologize if this sounds like it’s validating “No means try harder” but it’s not about whims of the heart here.

Here’s a really ghastly example. Remember that pig farmer in B.C. who was murdering women and burying them on his farm, but because they weren’t virginal teenagers their friends couldn’t get the police to take the missing women cases seriously? It has just come out that one of the “highly respected and popular” police officers who was being pressed to investigate one women’s case actually told her friends that she was in rehab and didn’t want to talk to them–just to keep them from bothering him to do his job! Officer lied about whereabouts, witness says. The friends are now wishing they’d pushed it more and perhaps gotten the murderer caught earlier.

it’s inevitable that someone will use this incident as further evidence as to why “serious” “real” scientists shouldn’t take “soft” sciences like psychology seriously.

No doubt. Especially coming on the heels of the M*rc H*user debacle (about which the full truth will probably never be known, at least publicly).
(Of course, if the shoe fits…)

I feel sorry for the people who have worked with/for him.
Their work might be solid but this will taint them.

Their work might or might not be solid. The fact that they were happy to publish without collecting or even seeing the data, however, ought to taint them.

will the students get their degrees for learning how to do research?

But they didn’t (necessarily) learn how to do research. As far as I can tell (I don’t read Dutch), they didn’t learn how to formulate a question, design a valid study, and collect meaningful data.

Those poor grad students!

Yeah. Too bad it didn’t occur to them at the time to insist on designing and carrying out a piece of original research, the basic criterion for awarding the Ph.D.

Is this kind of shit common in the Social Sciences? Not fraud, but are people routinely getting Ph.D.s for analyzing experiments they neither designed nor ran? Because that’s bullshit on both scientific and educational levels.

This is yet another reason that all primary data that any publication is based upon should be published at the time of or prior to that paper’s publication, under a non-restrictive license. It’s much harder to falsify oodles of data, together with all the relevant meta-data than it is to just invent some graphs and regression lines, and it’s potentially much easier to check that the raw data looks believable and that there’s a reason to trust that it was collected as it claims to have been.

Fairly common, yes. Sometimes people work with national datasets or multi-center studies, so they never get their hands dirty collecting data. That was always a bone of contention in my department – the dataset kids could finish PhDs two years sooner than the folks who had to design, perform, and fund their own experiments.

By the way, is P.Z. aware that there is a big banner ad at the top of his blog homepage calling for people to “Defeat Obama” and “Defund Planned Parenthood”? And there’s a google ad offering to make people Ordained Pastors (some nifty online degree, it appears).

Gee, things have sure changed since P.Z. moved his seat of operations here from ScienceBlogs.

Well, score one for self correcting anyways. This will lead to more questions, it will lead to more rigorous inspection of future research, it will lead to stronger science, but what a horrid way to get there.

Chas, Neoturtle: I imagine, like anything else in academia, the quality and rigor of social science PhD programs varies. In my graduate psych program, the only sub-discipline where one would possibly be allowed to use secondary data for a dissertation would be Statistics. Everyone else (Social Psych included) was very much expected to design and carry out an original piece of research from start to finish. We may have worked on other projects with secondary data, but that was not considered suitable for a dissertation at all.

We considered even smaller deviations from “you need to do it all yourself” to be unacceptable. It was rumored that one of the professors in School Psychology (a sub-discipline that, admittedly, all the rest of us saw as a little weak in general) had hired someone to do the statistical analyses on her own dissertation at another university. This alleged dodging-the-stats-work was discussed among the grad students as something that, while not exactly fraudulent, certainly was not entirely ethical. And it did make her PhD seem a bit suspect, not quite fully earned.

It’s relatively easy for someone to verify ‘hard science’ work by collecting data from the same site, except for the expense and effort of getting there. It might be harder, in the soft sciences, to gather a second group of testees or to verify the existence of the first group if someone is sitting on the data. Physical effort vs. people effort? And then I remember all the negotiating and collaborating it took to mount an expedition to Liang Bua, and how the chief Javanese scientist wouldn’t let anyone else look at the specimens, then returned them damaged. Sigh.

The debate became personal at the end of 2004, when emeritus professor Teuku Jacob from Indonesia’s Gadjah Mada University ‘borrowed’ the Liang Bua bones from Jakarta’s Centre for Archaeology and debunked Homo floresiensis in the media.

Australian anthropologist Mike Morwood, of the University of Wollongong in New South Wales and a partner on the Liang Bua find, cried foul on an institutional agreement regarding management of the bones. The brouhaha descended to “a level of shouting and name-calling that you do not often hear in Indonesia,” Morwood would later report. Most distressing, however, was that the bones were returned to the centre scarred, broken and clumsily repaired. A member of Jacob’s lab confessed that the damage was the result of botched efforts to cast the crumbly remains.

Don’t consider it faking data, just consider it necessary life experience. Making shit up is a valuable life skill in this political environment. He now has the work history to join the church or republican think tank. I am sure that AIG (both the ministry and insurance company) could use someone of his skill set.

Fairly common, yes. Sometimes people work with national datasets or multi-center studies, so they never get their hands dirty collecting data.

From what I understand of the article though, they weren’t allowed to see the raw data (I am confused on that point). It’s fine to work on data someone else has collected so long as its going to be worked in a different way and as long as you understand how they collected it.

In my PhD I used some data from my supervisor because it so happened his raw data had been collected in a manner compatible with my data and he had never got round to processing it (thousands of measurable artefact photographs so understandable). That is somewhat different to just working with data people have already processed.

Teuku Jacob was Javanese, so no mistake on your part.
The Javanese are the most populous ethnic group in Indonesia, which is jokingly called a Javanese kingdom. All presidents but one have been Javanese.

There’s an interesting subtext to the debate: while Teuku Jacob was angry about Western encroachment upon his territory and status (as the nation’s most eminent palaeontologist, the Florinese are a part of the country’s east, which feels often neglected and exploited (for its raw materials) by the Javanaese-dominated (domestic) West.

There was also a Catholic sub-context, as Flores is 85% Catholic and almost all higher institutions of learning there are run by the Catholic church. Indeed there was the feeling there that the Australians had stolen/scooped a Dutch (I believe) missionary’s research there.

So this debate was more about pride and sensitivities than about the scientific facts themselves.

If anybody tells me that psychology isn’t a ‘real’ science, they better take off their dancing pants cuz we’re going to wrestle.

I would not be struggling through this BS Psych/ Bio if psychology wasn’t reality based.

I, too, feel bad for the grad students. I’ve been in the research position where numbers are thrown at you. Deadlines are to be met. You cannot review every single data point if you have a paper to write by 4 PM. Do I think they should have probed a little further? Yeah. But when you are a subordinate, you really have your back against the wall.

Not seeing data at all is really weird. If he sent his students an Excel file full of data, that would be standard. (And I would be jealous!) But if he just handed his students a bunch of “p<.001 and don't ask questions!" that is a whole other realm of freakiness.

I can't imagine what that student's defense would be like, never having even done their own statistics.

Not seeing data at all is really weird. If he sent his students an Excel file full of data, that would be standard.

I think that before I decide what to think about the grads (I was initially sympathetic but then again, hope I would have smelled a rat) I would like to know what is meant by the statement:

tapel told them that their time was better spent analyzing data and writing. The commission writes that Stapel was “lord of the data” in his collaborations. It says colleagues or students who asked to see raw data were given excuses or even threatened and insulted.

As it stands this sounds contradictory. If they were performing analyses what were they performing analyses on if not raw data? It that was what was happening then I have a lot of sympathy for them because it would be hard to pick up if they were faked. If all they were doing was producing higher level syntheses of existing stats (without questioning how they were arrived at) then I have less (and as you say, serious concerns about how they passed their viva/ defence).

Many of Stapel’s students graduated without having ever run an experiment, the report says. Stapel told them that their time was better spent analyzing data and writing.

I keep coming back to this.
Why was this OK with the students? Why was this OK with their other committee members? Why was this OK with Stapel’s collaborators and colleagues and department heads and deans?

It’s just simply not OK. If you’re training scientists, then you train them to do science, not to be scribe-slaves.

As a final note (as someone only familiar with the UK system when it comes to PhDs). Why, if they weren’t working with raw data and performing the sorts of things PhD students typically do, did someone equivalent to a ‘second supervisor’ not pick this up?

I didn’t see my second supervisor that often admittedly but I talked to all sorts of people about my work. Why did it not become obvious they were being sold short if they weren’t being allowed access to primary data?

Is this kind of shit common in the Social Sciences? Not fraud, but are people routinely getting Ph.D.s for analyzing experiments they neither designed nor ran? Because that’s bullshit on both scientific and educational levels.

Does anyone know how that compares to something like people getting PhDs in physics by analyzing data from the LHC? I really don’t know if its similar at all, but that’s what it made me think of. Presumably in that case the physics students are at least seeing the raw data, though.

Does anyone know how that compares to something like people getting PhDs in physics by analyzing data from the LHC? I really don’t know if its similar at all, but that’s what it made me think of. Presumably in that case the physics students are at least seeing the raw data, though.

Yes. There are some algorithms not of your own making that you probably use to pre-process the data before you do your thing with it, but those algorithms should be transparent in what they do, and officially approved by the collaboration.

I collected all my own data and had a ridiculously large N for a clinical psych Ph.D. (around 400 subjects) – cost a fortune in money and effort and took forever. Did my own data entry and analysis, albeit with the help of a consulting statistician to make sure I wasn’t doing something horridly wrong. Most of the students in my department did their own data collection and research design.

It’s not fair to toss out all research using pre-existing data sets, though – there are a lot of longitudinal research projects where using previously collected data gives one the opportunity to examine issues that wouldn’t be possible if one was doing all the data collection. Also, there are a lot of huge psychiatric epidemiology data sets around.

Funny, isn’t it, that this guy could have become a well-known and esteemed scientist by doing the work of one, if only he had the confidence in himself.

The worst example of this was the South Korean scientist who was faking his data for stem cell research…and was so focused on making a name for himself he actually missed what actually interesting and important data.

Yes, it’s a sad day for social scientists, since it’s inevitable that someone will use this incident as further evidence as to why “serious” “real” scientists shouldn’t take “soft” sciences like psychology seriously.

In this study a questionnaire was administered via e-mail to 84 Dutch social psychologists. The authors asked to what extent respondents felt that a widely published plagiarism scandal involving a Dutch psychologist affected themselves and public opinion of their profession. As predicted, findings indicate that the impact of the scandal was dependent on the salient identity of both the perceiver (‘social psychologist’ or ‘psychologist’) and the stimulus target (‘clinical psychologist’ or ‘psychologist’). Respondents were more affected by the plagiarism scandal when the self-category that was made salient matched the category of the target of the scandal. (PsycINFO Database Record (c) 2010 APA, all rights reserved)

“Funny, isn’t it, that this guy could have become a well-known and esteemed scientist by doing the work of one, if only he had the confidence in himself.”

It is not that easy to come up with interesting publishable findings at the rate displayed by Stapel. He was publishing 5-10 studies per year, admittedly as co-author with others, mostly in substantial journals. That would be hard to achieve by actually doing the data collection involved.

Well as a Master’s student in Psychology it would be possible to not see the original data sets if you were doing something like a meta-analysis of already existing literature, but beyond that I would expect the students to be doing their own data collection and analyses. Though students also typically work with professors who share their research interests and styles, so those who worked with him may have been the type who wanted to be more micromanaged or focus on data analysis as opposed to data collection. The students who wanted to collect their own data would have chosen to work with someone else

Though it is hard to say no to a professor, especially one in charge of your thesis/dissertation. Most students try to go with the flow as much as possible because the last thing you want to do is piss off your advisor. Grad school is hard enough without having your advisor hate you. So since the students weren’t being forced to do anything terribly unethical I think most of them just figured the professor was anal and just did what he said to make the process less painful.

I do feel bad for those grad students- to know all the hours put into their thesis papers was based on tainted data would make me want to punch somebody (namely the professor). Good to know I will most likely be hearing more crap about how “psychology isn’t a real science” in the near future thanks to this jerk.

I, too, feel for the students. Even if they should have known better, they were at a stage of their development where they were supposed to learn from their supervisor, and their supervisor was widely regarded as one of the superstars of the field.

And we don’t know how many students did raise concerns only to be railroaded. The link provided by Hurin @#57 leads to this very sad statement from the report:

“Members of the [redacted] expended considerable time attempting to reproduce Respondent’s results. The Committee found that the wasted time and effort, and the onus of not being able to reproduce the work, had a severe negative impact on the graduate careers of three (3) of those students, two of whom [redacted] were asked to leave the [redacted] and one of whom decided to leave after her second year.”

I’m from the Netherlands, and it looks to me like some information was lost in translation.

His students/corroborators would take care of designing the experiments, making the questionnaires, analyzing the results etc.
Stapel claimed that he had excellent contacts with a number of (high) schools, which would allow him to quickly get the responses to the questionnaires. Often he would also try to find out what kind of response people expected. Then a few weeks later, he would hand in the data he had ‘collected’, which conveniently matched expectations. Even more convenient, these ‘schools’ refused to deal with anyone but Stapel himself.

I have not heard that he insulted or threatened people who got suspicious. I have heard that he would win them over by inviting them for dinner at his house, theaters etc.

Teuku Jakob was in fact Acehnese, not Javanese. Teuku is a minor Acehnese aristocratic title.

He was the grand old man of Indonesian archaeology, and had such prestige in retirement that he could get hold of the hobbit skeleton even when he wasn’t at all involved in the project.

Morwood has a fascinating account of all this in his book “The Discovery of the Hobbit.”

The next logical step in examining the bones is to try to extract DNA, which would presumably tell us once and for all whether the hobbits really are a new species and not microcephalic humans, as claimed by Jacob and his allies.

Morwood gave a public lecture last year where he said he was hoping that DNA could be obtained, but nothing seems to have happened since.

I’ve read Morwood’s book. And what I found disappointing about it was that he didn’t include the Florinese perspective, it was all about the Indonesian establishment v. the Australians. Flores is a remote region in Indonesia and not part of its mainstream Muslim Javanese culture. I don’t claim to know their position exactly, probably most Florinese aren’t really aware of this problem anyways. A cultural anthropologist working in the region has made some inquiries about the impact of Homo floriensis but not specifically investigated the question of academic imperialism in this context.

Wasn’t there a fairly well-publicised case like this to do with a journalist… and it got made into a movie? I think what happened with that guy is that he switched from journalism to fiction… so… it worked out for him!

Unusually for me, I’ve read down all the responses. Nearly all about whether the grads should have smelt a rat sooner and taken it further. Which if I read the report aright is what happened in the end. I don’t know yet which straw broke the camel’s back, but it seems a little ??obtuse?? to blame students who come into a department, set out a plan of study, are told by their adviser the best way to do it, and do that. They probably have no other comparison to decide whether this is or is not reasobnable; if they are unsettled, they ask other grad students, who tell them this is the way it’s done here.

The huge .. _HUGE_ .. failure is peer review. No one thought to ask for the raw data on this “wunderkind’s” latest cadillac job to just do one little statistical test? Not ten minutes to set it up and look at the happy outcome. Nothing to lose for them, apart from perhaps not getting his next paper to look at.

Concentrating on the students illustrates the final stage of the project’s progress- punishment of the innocent.

“Voodoo Science: The Road from Foolishness to Fraud” talks about the tendency of some scientists to become so convinced that they are on the right track that they start making the data that will be confirmed (they think) eventually.

In Flores, the Verhoeven, the missionary, published some evidence in the 1950s-60s that H. erectus (they thought) had been there before. In the 1990s, a Dutch-Indonesian team did some more exploration and confirmed his findings of stone tools and fossils about 700,000 years old.

In the 2000s, the Indonesian-Australian team began excavating in Liang Bua. Morwood does talk about getting the local Flores inhabitants involved. I believe that the team had to be led by Indonesians.

I guess that original research materials would be marked up test sheets. Raw data would presumably be a file containing the answers for each sheet in machine-readable form. Processed data would be totals or summaries?

Not fraud, but are people routinely getting Ph.D.s for analyzing experiments they neither designed nor ran? Because that’s bullshit on both scientific and educational levels.

It’s probably more common than we’d like to admit, considering I watched Jonathan Wells get his PhD from Molecular and Cell Biology at Berkeley, under essentially exactly those circumstances.

I recall the profs in Zoology at the time complaining loudly not only about his admission, but his thesis “topic”, which wasn’t even a topic FFS, just processing the results from the lab in general he worked in.

That said, I would stress one thing here.

Graduate students, for the most part, are IN graduate school to LEARN what doing real research is all about.

Instead, what all too often happens in bigger universities focused on pumping out publications is that this is entirely forgotten, and students are basically either left on their own to figure these things out, then given thumbs up or down depending on their ability to learn how to swim with no instruction, or they are simply used as slave labor for a PIs project, and given a degree based on not screwing up.

I can only speak from a direct sample of the handful of Unis I have worked with personally, but it’s common enough to have made me conclude over the last 20 years that some serious overhaul in the way we train future scientists needs to occur.

We work hard to TEACH undergraduates the basic history and theory behind science, but often do a terrible job teaching how to apply that in the field when they become graduate students.

and, those graduate students who end up with poor training, but still manage to survive?

Not seeing data at all is really weird. If he sent his students an Excel file full of data, that would be standard.

My interpretation is that he did give them a spreadsheet full of data; but that isn’t raw data, it’s processed. Raw data would be the actual responses to questionaires, interviews, etc. It’s easy to take a spreadsheet, populate it with a lot of phony numbers, and give it to someone claiming it’s a compilation of thousands of data points; much harder to do that with the actual data sheets themselves. To use a molecular biology comparison, the spreadsheet is like a set of DNA sequences in text form; the raw data is the chromatograms they were generated from.

I work with a lot of test data, and believe me, if you’re putting your name on an analysis you want to be make sure that no major errors creep in any step of the way. For every foolproof system you devise, there are fools out their doing everything in their power to break it.

You’d think that somewhere along the line someone, a student, a collaborator, a colleague, a supervisor, would have asked to see the response sheets or, perhaps more likely, observe the subjects when they responded, and maybe interviewed a few afterwards. And if they did, they might have found it odd that they couldn’t get access to the subjects, and the handwriting was the same on all the response sheets.

all the Indonesian archeologists on the team were outsiders, I believe.

The problem is that Florinese are also marginalised within their own country, so you wouldn’t necessarily get qualified locals to participate, which generally is a problem in developing countries.
The High Seminary is also full of outsiders, like Verhoeven, who was probably the guy who people felt was robbed. But due to the role of Catholicism on Flores that would probably come closest to a native academic institution Florinese people would identify with.

As I said, I don’t know all the facts, but I do know the High Seminary people are still pretty upset about what the foreigners have (allegedly) done to them. After Morwood allegedly stole their results, they instituted a policy that no-one is allowed to borrow any BA or MA theses from their library without the rector’s permission. I can’t say if it was a cultural misunderstanding, if Morwood did something wrong, or whatever. In academia, if you publish something, it’s considered fair game, but not necessarily in Indonesian culture.

I haven’t checked the book now, but I think I remember Morwood employed locals from Manggarai to work as assistants/guides. He also travelled with his team to a different district to meet with some people who had legends reminiscent of the hobbits. There apparently he also met someone interested in that type of work. But this person was also tied up by responsibilities at home and wouldn’t have had the opportunity to work with the team at the other location, even if they had been invited.

He probably learned his data manipulation skills from some well known warmist climate scientists. Science really has become infected with the dishonesty disease

phalaris 03:52 AM 11/2/11

Doesn’t surprise me in the least: in the so-called social sciences, the science is often very thin, the theses tendentious, and intended to bolster some left-wing agenda.

tuned 11:12 AM 11/2/11

Let’s face it. Social science and psychology are the sketchiest ‘science’. Everything about it is variable and subject to whim and downright falsehood, as is the brain itself. Medical science and DNA type researches are the real forefront with universally repeatable results.

The preliminary report (in Dutch; Google translation (note that ‘stapel’ is Dutch for ‘stack’ and is translated as such)) by the committee investigating the fraud, has some interesting details. His partners — students or colleagues — were involved in designing and preparing the experiments, right down to providing the candy or other rewards for the subjects. These were often put in the back of Stapel’s car. With that level of preparation, I can imagine it being hard to suspect something.

In one case, he seems to have intimidated a student who requested the raw data. It didn’t help that the confidential advisor was the rector magnificus himself, either.

It could also be a very common human drama. Successful person starts doing more and more of what made him successful in the first place, can not cope at some point, gets psychological problems and builds a tower of lies, until they come crashing down around him.
The http://en.wikipedia.org/wiki/Peter_Principle often creates this…
Selfishness might play a part, but not necessarily.