Social scientists have long debated the “true” nature of sexual orientation. In the past, this debate has been characterized as another example of the classic battle of nature vs. nurture/biology vs. society. Today, most acknowledge that a simplistic this or that explanation doesn’t fly for most things and likely doesn’t rightly characterize sexual orientation, either. Over the past decade or so, psychologists have begun gathering evidence indicating that there are likely multiple pathways to sexual orientation; for instance, some may be attracted to people of the same-sex because of something in their biology, others because of their social environment.

A recent study by Diamond and Wallen in the April issue of Archives of Sexual Behavior (“Sexual Minority Women’s Sexual Motivation Around the Time of Ovulation”) found evidence supporting a multiple pathways perspective in women’s sexual orientation. Continue reading →

A recent McClatchy-Marist poll found that nearly 6 in 10 Americans think a nuclear disaster similar to what happened in March of this year in Japan could happen here. Why do so many people suddenly think that nuclear disaster is likely? Recent research in the Journal of Experimental Psychology: General might shed some light on the issue. Continue reading →

In the poor, rural Gansu province in China, 10-15% of young students need glasses but only 2% of those kids actually have glasses. To follow up on my previous post on the science of charitable giving, in this post I’ll briefly describe a recent study which found that simply giving these students glasses significantly increased their average test scores! Continue reading →

In the past, my partner and I have mostly haphazardly divvied up our good intentions to whichever charities are most easily accessible to us because of advertisements or a person standing in a grocery store parking lot. Lately, we are rethinking that (lack of) strategy. As Yale economists Dean Karlan and Jacob Appel wrote in a recent issue of the Yale Alumni Magazine, “We don’t pick stocks based on colorful stories in mailers; we shouldn’t pick charities that way, either.”

Certainly, this is good, solid advice no matter what the domain. Think before you act, make good use of your money, etc. However, the research of Karlan and and others indicates that if we don’t turn to science to guide our charitable giving, we may end up doing more harm than good.

I’m going to come out right now as a deconstructionist – I believe that everything we experience is shaped at least in part by our historical time and our social, physical, and psychological context. In other words, I don’t believe that when you and I look at the same object, we are actually seeing exactly the same thing. But don’t take my word for it. There is a lot of very interesting research coming out of the relatively young discipline of social vision which demonstrates that lots of things other than the objects or events we are looking at can impact what we actually perceive. I want to talk about a series of studies coming out of this line of research that I happen to think are very, very cool.

Whose job is it to make sure that the non-scientist consumers of science get it right?

I’ve had a few discussions with various psychologists about this lately and they frequently bring up two answers to this question:

(1) It’s the consumer’s job. I heard from a few of the psychologists I spoke to that they are frustrated that non-scientists who read science and then use it (i.e., your dad when he listens to the evening news and then tells you about it) don’t take enough time and put in enough effort to understand it accurately. This leads to misrepresentations of the results and, even worse, a misuse of the findings. The summary of this argument is that “people are too lazy/uninformed to do anything more than read headlines and then run with it.” The implication there is that non-scientists should be more motivated to go the extra mile and try to understand what scientists are doing.Continue reading →

I listen to a lot of podcasts in which various psychological articles are often discussed (e.g., stuff you should know, radiolab, etc.). As a psychologist, I am often frustrated when a podcast mentions a study’s finding (e.g., having a sister is associated with better self-esteem than having a brother) but then says something like this: “well, I’m kind of suspicious of that finding/we should take that finding with a grain of salt/I kind of question that finding because I don’t think that’s the whole story/I have a brother and my self-esteem is great.”

I get frustrated because a little bit more information about effect sizes would help turn those kind of statements from ones that undermine and disregard what are often interesting, useful findings into the kind of statements that would help people understand exactly how useful those findings are.

So, with that in mind, let’s talk about effect sizes. Usually, when you hear someone talk about a study, if they say that something is associated with something or there’s a difference between two things, that means that the study found a statistically significant effect, which most of the time means that the effect they found is different from 0 at at least 95% certainty. With regards to the sisters and self-esteem example, this means that the relationship between having a sister and self-esteem is different from 0 at at least 95% certainty. That’s interesting information, sure, but it doesn’t tell you anything about the strength or size of that relationship. Different from 0 just means that the strength of the relationship could be anywhere from just a tiny bit above 0 to a huge number.

This is where effect sizes come in. Effect sizes give you some information, like you might expect, about the size of the effect, which is much more useful than simply knowing that the effect is different from 0. When you know something about the effect size, you can understand that the effect of having a sister may not be able to explain everything about self-esteem (hence the counter-examples you can think of) but it can explain something, which makes it useful. Understanding effect sizes gives you a sense of just how useful.

I’m going to go over two commonly-used effect sizes: R-Squared and Cohen’s d.

R-Squared

Let’s stick to the sister/self-esteem example here. Let’s imagine that someone does a study of self-esteem and that all of the self-esteem data from the study is represented in the circle below.

We can also talk about this circle as showing 100% of the variance (how much people in the study differ from each other) in self-esteem; if we could explain 100% of the variance, we could predict the exact self-esteem everyone in the study has. Let me say right off that no study in psychology ever does this; if you find a study that comes even close, be very, very suspicious. Why? Because there is so much that goes into self-esteem, or any other psychological construct, for that matter, that it would be very difficult to capture it all in one study. For example, part of self-esteem could be explained by having a sister, part of it by how you happened to feel that morning, part of it by your relationship with your parents, part of it by how the experimenter looked at you when you first came in, and so on, so on.

So when a study reports a statistically significant association between having a sister and self-esteem, that means that having a sister explains more than 0% of the variance in self-esteem. That means that having a sister might explain anywhere from .001% of the variance to 99% of the variance.

What R-Squared tells you is what percentage of the total variance in self-esteem is explained by having a sister; it’s that easy. If a study reports an R-Squared of .5, that means that 50% of the total variance in self-esteem is explained by having a sister (shown in red below).

Another quick thing to note about R-Squared before I move on is that R-Squared is most commonly used as a measure of how much variance is explained by a set of predictors. For example, if a study says that when trying to explain self-esteem from having a sister, parental relationship, and body image, they found an R-Squared of .62, that means that 62% of the variance in self-esteem was explained by a combination of having a sister, parental relationship, and body image.

Cohen’s d

In order to talk about Cohen’s d, we need to think about our example in a slightly different way. Let’s say a study shows that there is a statistically significant difference in self-esteem between two groups: one that has a sister (group a) and one that does not (group b). Remember from before that knowing that there is a statistically significant difference in self-esteem between the two groups means only that the difference between the two groups is different from 0. That could mean that the self-esteem score in group a is 5 and in group b is 4.5, or that the self-esteem score in group a is 5 and in group b is 2.

Cohen’s d is pretty simple. It takes the literal difference between the two groups (groupa self-esteem – groupb self-esteem) and then divides it by the standard deviation of the data (a measure of how much the data varies). It’s easiest to explain why the difference is divided by the standard deviation using an example.

Let’s say we want to know how big the difference is between how much a garden snail weighs on day 1 and day 2 (after eating a big meal on day 1). Let’s say that difference is .02 ounces. Is that big? Is that small? The number itself is very small but does that mean that the difference is actually small? Let’s say the difference between how much an elephant weighs on day 1 and day 2 (after eating a big meal on day 1) is 3.4 lbs. Is that big? Compared to the .02 ounces for the snail, that’s huge! The problem with looking at the raw difference is that we don’t know what an “average” difference in snail weight or elephant weight actually is so we can compare it to the difference we care about. If we knew that elephants actually fluctuate in weight day-to-day even without a big meal by about 3.35 lbs, that 3.4 lbs doesn’t seem like that big of a deal. Similarly, if we know that snails actually fluctuate in weight day-to-day even without a big meal by about .001 ounces, that .02 ounces is a very large difference.

Cohen’s d allows us to understand the size of a difference, even across completely different comparisons. Using Cohen’s d, we can compare the size of the difference between a snails’ weight from one day to another to an elephant’s weight from one day to another.

Conveniently, Cohen gave us some guidelines for interpreting d. He said that .2 – .3 is a “small” effect, around .5 is a “medium” effect, and .8 and above is a “large” effect.

A Final Note on Effect Sizes

It’s important to remember that effect sizes need to be taken in context with the rest of the study. If the study is on a topic we already know a lot about, explaining a tiny bit of variance (a small R-Squared) or a small Cohen’s d is not very impressive. However, if the study is on a topic we know nothing about and is breaking new ground, small effect sizes can still be provocative and useful – they can mean that the study gives us some information as a starting point for future research! The best way to be a good consumer of science is to try to use as much information as possible before drawing your conclusions. It’s a little more difficult than just reading a headline, but science is complicated, particularly the science of human behavior, thought, and emotion.

I hope this quick tutorial on effect sizes is helpful. I’ve only gone over two here – there are many other kinds of effect sizes. A more mathematically-involved outline of effect sizes is available on Wikipedia (http://en.wikipedia.org/wiki/Effect_size), and a less mathematically-involved outline of effect sizes is available here (http://www.leeds.ac.uk/educol/documents/00002182.htm). A google search will reveal many other resources on effect sizes and their interpretation.