One of the first quotes I share with my students is one that (as far as I know) is attributed to Carl Sagan…

“Science is a way of thinking much more than it is a body of knowledge.”

I think at first, the students shrug and say, “Ok, neat quote, C-P.” But I really want them to internalize this idea. Mull it over. Reorganize their understanding of what they (think they) know about science and the world they live in. Really appreciating what this means can be transformative for a student. Often, they have developed a sense that “knowing science” is simply having at their disposal a set of facts and statements about the relationships between those facts. It is difficult to convince them that “knowing science” is more about understanding how those facts came to be, to what degree we can accept or work with those facts, and how it is that science moves forward. It is not about simply gathering “more facts”. It is about how we can best maintain a balance between being skeptical and accepting ideas about the world that are put before us.

With that thought in mind, I thought I would share a recent rumination by Dan Simons on a study that was recently published in Nature (article here and a more accessible description of the study here). Simons provides a textbook example of how a scientist thinks… The study in question examined how playing a video game might affect the visual attention and multitasking capabilities of an older adult. In short, the study found evidence that the game play increased certain cognitive functions of the older adults (aged 60 to 85). The response that Simons has is what he calls a “HIBAR” (“Had I Been A Reviewer” – referring to the peer review process that scientific publication undergo before being published). Some of his concerns are methodological (e.g. tasks completed by the control conditions), some are statistical (e.g. lack of measures of variability for differences scores), some are theoretical (e.g. why the video game affected some functioning but not others). Altogether, they illustrate what productive skepticism looks like. He is not simply dismissing the study or blindly embracing it. He is trying to better understand what the results show (and don’t show) about the topic, and he is trying to work through how we should think about those results. It would be silly to think that a single study answers all the potential questions about something as complex as attention, training, and aging. However, if we take a single study, like the one in question, and then carefully assess it, we see a way ahead. That is science.

It is embarrassing that my last post was 366 days ago… and I wrote about motivation. Pretty clear how that worked out for me. I won’t offer much in the way of excuses (although I do have a long list to draw from), but instead I will refocus on using this blog as a place to think through (and share) some psychological issues that I feel deserve some consideration.

For now, I will share a quick thought about my failure to maintain my writing. I started this blog with a rather simple goal – to share with my students (and friends) ideas. I also wanted an excuse to write more – I like to write, but often find that all the other obligations in my life seem to overwhelm me and writing (especially for fun) is one of the first tasks I set aside. Last year, I lost sight of these goals. So, I thought I should share a little bit about why I might have struggled with maintaining this goal.

I’ll share a couple of ideas from Dr. Timothy Pychyl – he is a professor of psychology at Carleton University and has his own blog over at Psychology Today (Don’t Delay) where he writes about various issues related to procrastination. There is a lot there (he has authored a couple of books and many academic papers on the topic), but I’ll cherry pick a few ideas I think are worth sharing:

How you define the goal matters. When I thought about adding to this blog, I unfortunately got into the mindset that it was something I had to do and my goal shifted to “I have to avoid looking bad when my students check in on my blog” (bummer – an avoidance goal). I plan this year to consider this blog as an opportunity to think and write and share – all things I want to accomplish (voila! – an approach goal).

Maintain a reasonable number of goals. This one really got me this year – trying to maintain a full teaching load and introduce some new elements to my classes, crank out a couple of papers, start a new line of research, be an involved father and husband, run (and rehab), renovating an entire basement on my own – and I suffered. I still have a very busy life (the basement still isn’t completed a year later…), but I can recognize the multiple draws on my time and focus on a subset of goals instead of trying to do everything at once and feeling overwhelmed.

As I was making my way up the hill today, I was thinking about motivation. What motivates me to spend a gorgeous Sunday afternoon planning psychology lab experiments? What motivated me to renovate my basement on my own this summer (and fall. and winter) given my own paltry building skills? How can I better motivate my students to be more active participants in their learning? I had already decided to address the topic of motivation earlier in the semester in my Introduction to Psychology classes this semester, because I think a better understanding of the drives behind what we do will allow the students to better situate later information about social influences, the brain, development, cognition, etc.

why am I doing this?

Anyhow, as I was piecing together resources on the topic for my students, I came across an article titled “Tough Track: How—and why—an average guy became an ultramarathoner”. I don’t think there is anything terribly profound in the article about motivation, but it did get me thinking about my own running and how some of the motivations I have “cultivated” in my running carry over into my more work-a-day life. As I have written other places here, I run for a number of reasons. Sometimes when I am tired, or it is dark and raining, or other activities consume my time, I consciously think about why I run. The satisfaction I gain from simply moving through the world at my own pace. The increased strength and fitness I feel (at least when I am not exhausted from running). How I have incorporated “runner” and all that it entails into my self-concept. Endorphins. When I find myself in a tough spot at work – I am “not-quite-as-interested-in-the-topic-at-hand” as I could be, I’m worried about the potential of rejection of a paper I am writing, or it just seems like so much work to get a line of research off the ground – I think about the motivations that get me out the door on a run. It helps me re-engage at the office and work through the things I have to do. I mention that I do this consciously – however, I do think there are many times that this “adoption” of motivation occurs outside of my awareness as well. The fact that I am motivated to run, and this motivation has been a consistent aspect of my life in the past decade, makes a difference for how I deal with initiating behavior in all situations. Of course, that idea opens up a whole new crop of questions and ideas that I can’t address right now because I need to get back to those lab preparations…

Classes begin at Denison on Monday, so I am in those last minute throes of “just what are we going to talk about?” when the students fill the seats. I found a link to this short clip on Daniel Simons google+ blog – a good spot to drop by to find interesting things about psychological science and visual cognition in particular. The clip here is great because (a) it is about The Hobbit (my favorite book) and (b) it shows how filmmakers can play with some of the inherent biases of our visual system to create real neat illusions…

It seems that the past year has been a rough one for psychological science – several researchers have had their work called into question based on evidence suggesting that they had falsified data. This is a tricky issue. For science to work, the community of scientists has to trust that everyone is on the up-and-up. Honesty in reporting data is paramount. So, it is critical that individuals that are not living up to this standard are removed from the community. At the same time we are looking out for these nefarious researchers, we have to trust that everyone else is on being honest. So, there is obviously tension – I must trust, but I also must be skeptical…

The tricky part is how do we know? Well, we can feel a bit more confident because individuals like Uri Simonsohn are on the case. Here is an interview with him about how he detected fraudulent data in one high profile case this past year. This morning I saw that there was a retraction notice in the recent issue of Psychological Science, and following the story a bit found out that Simonsohn had identified another batch of suspect data. I found this situation particularly interesting because of the simplicity of his approach. Lawrence Sanna, a professor of psychology at the University of Michigan, had published a study purporting that the level of altruism an individual felt was related to his or her actual level of elevation – higher elevation = higher level of atruism. The method was interesting – it involved escalators and hot sauce, but that is a story for another post… Anyhow, Simonsohn noticed that although the means for the different conditions were very different, the standard deviations across the conditions were nearly identical. Simply put, different groups of individuals had been tested at different levels of elevation – the participants at higher elevations on average had much higher levels of altruism. The problem in the data was that although these means differed significantly, the range and distribution of responding within each group of participants was essentially identical. Simonsohn correctly noted that this is highly unlikely, so he located some other studies using similar methods and found that not only were the differences between groups far smaller, the variability within each group also differed more appropriately. Several other papers written by Sanna showed a similar lack of variability in the reported variability across the conditions, so Simonsohn contacted Sanna and some of his co-authors with his concerns during the fall of 2011. As of June 2012, Sanna has resigned from U of Michigan, retracted at least four papers, and now maintains his silence (under legal counsel).

All of this occurred because of some lowly, little standard deviations. By the way – you should use this as a prompt to review just what a standard deviation is and why your friendly research methods instructor would spend so much time making sure you understood that statistic…