Thursday, 29 September 2016

“Methodological terrorism” and other myths

Most readers of my blog will be aware of Professor Susan Fiske’s
leaked letter to the APS Observer last week. In her article, Fiske delivers a blistering invective on the dangers of social media, warning that scrutiny of
psychological research by “online
vigilantes” and “self-appointed data police” is destroying careers. In actions
she describes as “methodological terrorism”, Fiske warns of “graduate students
opting out of academia, assistant professors afraid to come up for tenure,
mid-career people wondering how to protect their labs, and senior faculty
retiring early”. Her solution? Scientific critique should be restricted to
moderated groups and behind the walls of peer reviewed academic journals.

Well.

Fiske
refuses to identify the “terrorists” because apparently that’s unprofessional
(though see this
argument from Neuroskeptic as to why it is vital if such allegations are
true). This leaves her in an unenviable Trumpian territory of delivering a
broadside against an entire community based on what “people have been saying”.
Lots of people, no doubt. Good people.

The whole premise is so flimsy that it would probably have been ignored if not delivered by someone so high ranking.

At the time
the article leaked I was at a conference in Germany that Fiske was also
attending. I immediately emailed her, pointing out what I saw as some of the
problems with her argument, and I invited her to join our session the next
morning to publicly debate the issue. She declined.

Two days later, PhD student
Anne Scheelhad the courage to stand up in
front of a packed auditorium and challenge Fiske directly about her “terrorist”
charge (if you’re not following Anne, go
do that now). In response, Fiske claimed that everyone overreacted. Apparently, she
just used that term to get people’s attention, but says she would still defend
the “conceptual
basis for that wording”.

Oh irony.

Now you may well ask, what evidence is there that Fiske's terrorists even exist in sufficient numbers to warrant such a panic? That answer is, not much. Since 2015, UCL neuroscientist Sam Schwarzkopf has operated a blog inviting
those who believe they have been bullied by methodologists or the open science
movement to speak up, anonymously if they prefer. How many victims have come forward?

Not one.

So what’s
really going on here? The truth is that we are in the
midst of a power struggle, and it’s not between Fiske’s “destructo-critics” and their
victims, but between reformers who are trying desperately to improve science and a vanguard of traditionalists who, every so often, look down from their thrones to throw a log in the road. As the body count of failed
replications continues to climb, a new generation want a different kind of
science and they want it now. They're demanding greater transparency in research practices. They
want to be rewarded for doing high quality research regardless of the results. They want researchers to be accountable for the quality
of the work they produce and for the choices they make as public professionals. It's all very sensible, constructive, and in the public interest.

Placed in
this context, Fiske’s argument – as inflammatory as it is – is actually quite
boring and more than a little naive. Social media is democratizing science, and nobody, not even the
most powerful among us, can stop its inexorable rise. Does anyone really
think that Pandora’s box can be closed, returning to the halcyon days of the
1990s (or earlier) when critique of science happened behind closed doors among selected
elites?

What’s happened instead is that technology has empowered the
new generation to speak freely and publicly on scientific issues, to critique poor
quality science and practices, to bust fraud, and to break the bounds of
peer review. Twitter in particular has shaken the traditional academic
hierarchy to its core. On Twitter a PhD student with thousands of followers
suddenly has a greater voice than an Ivy League professor who might have no
social media presence at all.

If the traditionalists want to preserve their legacies (and they are worth preserving) then they would do well to remember who they truly serve. Psychology does not
exist to support the interests of researchers seeking to command petty empires and prestigious careers,
less still to protect the reputations of those who have already “won” in the
academic game of life. Psychology has but one master: the public. That’s why
psychology featured in over 400 “impact
case studies” in a recent audit of UK science, and it’s why millions of
dollars, pounds and euros are being invested in new opportunities for all
scientists, including psychologists, to do more transparent and robust
research.

We remain forever accountable to that public for the quality of research we produce. Those who are prepared to critique science – and scientists
too for their choices – are performing a public service that warrants our appreciation
not condemnation. To Fiske’s “terrorists” I therefore say thank you. And to
Fiske herself, who complains about the tone of scientific debate while calling it terrorism, I say: physician heal thyself.

Hi Chris, Interesting topic and I agree with the points made that it is not possible nor desirable to turn back the clock re. social media, and that scrutiny of psychological science data may encourage high standards, however, if you find yourself and your work being thoroughly investigated because the voluntary data police report some trivial error/plagiarism in one of your past papers you might feel that the issue warrants greater attention in terms of protection for the academic. I am aware that this has happened to a highly qualified academic due to such an organisation reporting that an ex. Ph.D. student copy/pasted part of the introductory section from a paper previously published, the senior academic was held to account in an investigation that lasted several months. As an academic myself, i found it quite concerning that one could be subjected to this type of treatment, and the resultant negative impact on a career that had a previously impeccable research record, all because a postgraduate student committed minor plagiarism that you didn't pick up on? Come on!!! To me the whole episode seemed like a witch-hunt, and i wondered who reported this to the voluntary organisation, and who are the people that sign up to these organisations (disgruntled students?). Anyway, with that scenario in mind, I feel that although Susan Fiske's speech was a bit OTT (but blistering invective???) she has perhaps done some service for academics in pointing out that it is a v. vulnerable position to find yourself in; these anonymous sources can set out to damage a career perhaps due to some personal grudge. For this reason i think i won't put my name to this -- it's an old saying now, just because you're paranoid doesnt' mean they're not out to get you! :)

Hi Chris, Interesting topic and I agree with the points made that it is not possible nor desirable to turn back the clock re. social media, and that scrutiny of psychological science data may encourage high standards, however, if you find yourself and your work being thoroughly investigated because the voluntary data police report some trivial error/plagiarism in one of your past papers you might feel that the issue warrants greater attention in terms of protection for the academic. I am aware that this has happened to a highly qualified academic due to such an organisation reporting that an ex. Ph.D. student copy/pasted part of the introductory section from a paper previously published, the senior academic was held to account in an investigation that lasted several months. As an academic myself, i found it quite concerning that one could be subjected to this type of treatment, and the resultant negative impact on a career that had a previously impeccable research record, all because a postgraduate student committed minor plagiarism that you didn't pick up on? Come on!!! To me the whole episode seemed like a witch-hunt, and i wondered who reported this to the voluntary organisation, and who are the people that sign up to these organisations (disgruntled students?). Anyway, with that scenario in mind, I feel that although Susan Fiske's speech was a bit OTT (but blistering invective???) she has perhaps done some service for academics in pointing out that it is a v. vulnerable position to find yourself in; these anonymous sources can set out to damage a career perhaps due to some personal grudge. For this reason i think i won't put my name to this -- it's an old saying now, just because you're paranoid doesnt' mean they're not out to get you! :)

Ah - if only life was so simple. Now one of my old professors courtesy of Sue Greenfield did the Royal Institution Christmas lectures a few years back. Now this guy hates philosophy, mainly because he can't do, wouldn't know Bertrand Russell from Russell Brand. Tried very hard to explain the Teleological concept to him. Not a chance, complete waste of time. However he knows and chooses to ignore his inability, like most of us, cognitive dissonance is a bitch. Anyway, does Sue Fiske have a point. No. Knowledge is not a closed field. She should welcome the distraction with open arms and a smile playing on the lips. First rule of learning - a wise man can learn from a fool, but it only works one way. CDH.

under "researchers seeking to command petty empires and prestigious careers", but it only now occurred to me that you would be among the experts to ask: one can now see the slow formation of an alternative narrative to competition, the "commons". I can see this development both in scholarship (scholarly commons https://www.force11.org/group/scholarly-commons-working-group ) and in the wider political public, e.g.: https://europeancommonsassembly.eu/#section1

On the competitive-collaborative spectrum, can individuals we reasonably classified as belonging more to one or the other end of the spectrum and if so, is there any data on which populations tend to be composed of which fraction of each class? Worded a little more neutrally: are there studies classifying different human populations according to a competitive/collaborative score and if so, what are the distributions found in these populations?

Big problem is that we train rather than educate our students. Serious thinking about difficult issues does not even glance off the edifice we have erected. Nor will it (any time soon), since those trained practitioners become next generation's mentors.

About Me

I'm a psychologist and neuroscientist at the School of Psychology, Cardiff University. I created this blog after taking part in a debate about science journalism at the Royal Institution in March 2012.
The aim of my blog is give you some insights from the trenches of science. I'll talk about a range of science-related issues and may even give up a trade secret or two.
You can follow me on Twitter: @chrisdc77