Teach students about publication bias with the help of a tweeted analogy

Publication bias and the file drawer effect

I always pay attention to Claudia Hammond’s tweets (@claudiahammond) as they are a great way to make sure I don’t miss anything spec-related on her Radio 4 programme (All in the Mind). On the 3rd July, however, her tweet caught my eye for a different reason, as she was sharing a clever analogy used by Psychologist Robert Cialdini (Social Psychologist and author of Influence: The Psychology of Persuasion) who she had heard speaking at a talk at Ogilvy and Mather ad agency. According to her tweet he had compared (I am presuming the decisions made by journals…) to not publish non-significant results, to being like “explorers not putting deserts on the map”.

A2 Psychology: application of the scientific method

In Unit 4 (AQA A,A2) students are required to learn about the application of the scientific method in Psychology which includes validation processes such as peer review. When students are considering how scientific knowledge enters the public domain, and why validation is important, it seems sensible to consider wider issues with the publication of research such as publication bias and the file drawer phenomenon. Awareness of these sorts of Psychology and Science wide issues also increases the likelihood that students will recognise opportunities to apply these sorts of concepts when they are analysing the research they learn about in the “topics” aspects of the A Level course (so don’t leave it until Unit 4 if you teach this last to introduce these ideas). Consideration of this issue also engages with 2 of the How Science Works principles to be addressed through the study of Psychology as a scientific discipline (K and L, see AQAA spec page 18).

Lesson starter: analogies and metaphors

This analogy would make a great lesson starter or plenary stimulus. Share the explorer/desert analogy and ask students to explain what it actually means. You could also ask students to come up with their own analogy or metaphor for this, or for another issue that impacts the research process. I think this would be an appropriate way to introduce the issues with publication for the first time (looking out for opportunities to put the key terms on students intuitive ideas) or to assess understanding, and ensure students can transfer their learning, once they have come across these concepts in another context.

Related resources

Anyone interested in this post may also be interested in a previous post (follow this link to read more) about the policy of some journals to reject publication of replications – even if the replication fails to replicate the published finding – based on an article in the BPS publication The Psychologist.