Menu

Data Management at University of East London

Month: December 2012

A belated welcome to John Murtagh. John is the full-time project officer employed for TraD – formally his job title is Research Data Management Officer. He comes from working at the University of the Arts London where he was involved in two strands of work – Kaptur (the VADS-led project looking to adapt repositories for creative arts research data) and UAL’s own RDM work. UAL (like UEL) is one of the Digital Curation Centre’s institutional engagement partners, and has also developed an institutional policy on the management of research data. We’re very pleased that John has joined us in building RDM capability in East London.

One of the work outputs on the TraD project is is to deliver a course in research data management for Psychology postgradate students at University of East London. We’ve therefore been keenly following an academic scandal directly affecting the world of social psychology involving the Dutch psychologists Diederik Stapel (falsifying data) and Dirk Smeesters (massaging data). Similar scandals and retractions in the field have also involved Lawrence Sanna (2012) Marc Hauser (fabricating data 2010) and Karen Ruggiero (2001).

Dutch investigators have released their final report into the case of Stapel from Tilburg University, entitled: Flawed science: The fraudulent research practices of social psychologist Diederik Stapel.

Its findings reveal that Stapel fabricated data in 55 articles and book chapters. So far, 31 of those published papers have been retracted — three others have expressions of concern — although more might follow. In addition, 10 dissertations by students Stapel supervised were found to contain fraudulent data and this is what should be brought to the attention of our psychology Postgrad students here at UEL.

The final report makes for sober reading (Stapel personally taught his department’s scientific ethics course, for example) and a damning assessment of the discipline itself referring to “a failure to meet normal standards of methodology. [bringing] into the spotlight a research culture in which… sloppy science, alongside out-and-out fraud, was able to remain undetected for so long.” P. 5

The report has highlighted a number of issues with regards to research data handling, standards and attitudes and which we are likely to cover in our course. More points from the report:

“The Stapel group had no protocols for, for example, the collection of data (including standards for questionnaires) or research reports. The PhD students in Mr Stapel’s group were not familiarized with fixed and clear standards. (Stapel) underscored the lack of (fixed and clear) standards; but that too was far from a local issue, according to Mr Stapel.” P. 42

We cannot be certain if as Stapel says this is not a local issue but one that crosses social psychology as a whole but how research data is collected and standards applied to that data (as well as taught) should be something that is paramount in the teaching of postgraduates. The report again:

“The doctoral examination board must form a clear impression of the way in which research data has been collected.” P. 57

The importance of research data verification and replication of findings from research data will be something we shall likely emphasize to students studying for doctorates in our course.

“Research data that underlie psychology publications must remain archived and be made available on request to other scientific practitioners. This not only applies to the dataset ultimately used for the analysis, but also the raw laboratory data and all the relevant research material, including completed questionnaires, audio and video recordings, etc. It is recommended that a system be applied whereby on completion of the experiment, the protocols and data used are stored in such a way that they can no longer be modified. It must be clear who is responsible for the storage of and access to the data. The publications must indicate where the raw data is located and how it has been made permanently accessible. It must always remain possible for the conclusions to be traced back to the original data. Journals should only accept articles if the data concerned has been made accessible in this way.” P. 58

There is not much we can add to this set of recommendations; whether they can or will be implemented is as yet unclear particularly as as far back as 2002 similar recommendations were made following the Ruggiero case. But there seems to be a desire to change and the way data is managed in this field is something that unbeknownst to us is something we are now a part of.

As Uri Simonsohn, the researcher who flagged up questionable data in studies by social psychologist Dirk Smeesters, has said: “We in psychology are actually trying to fix things… It would be ironic if that led to the perception that we are less credible than other sciences are. My hope is that five years from now, other sciences will look to psychology as an example of proper reporting of scientific research.”

We hope that the psychology course in research data management we will be running at UEL will be a part of this hoped-for progress and will be seen as an example of good practice for future psychologists.

We launched our questionnaire to researchers (both academic staff and PGR students) on 29 November. This aims to inform us of current researcher RDM practice, where the challenges are, and what they would like from a support service and infrastructure. It will also help with training needs analysis, both generally and specifically in Psychology and Computer Science.

Sarah Jones, Monica Duke and I met in May to consider what we wanted to ask in the questionnaire, and in this we were greatly helped by some existing surveys from elsewhere. We directly used questions from each of the three resources below, and adapted others, as well as contributing our own questions more tailored to the situation at UEL:

UWE’s pilot study ‘Managing Research Data: a pilot study in Health and Life Sciences’

David Shotton’s ‘Twenty questions for Research Data Management’, University of Oxford

John Murtagh reworked some of the questions in October, with his work on the Kaptur project and DCC’s institutional engagement at UAL fresh in mind. In particular, he added scenario-based questions in the final section to ground researchers’ consideration of a future RDM support service in the real world. John compiled the survey using LimeSurvey software.