Is this causal claim justified?

09/20/2018

He is concentrating so much that he might not notice the coffee smell.... Photo: baranq/Shutterstock

At times, we've all been so engrossed in a task that we've lost awareness of our surroundings. Maybe you didn't hear someone calling your name when you were finishing your paper, or maybe you missed the oven timer when you were reading that mystery book. Now researchers Sophie Forester and Charles Spence have reported that concentration impacts our sense of smell. Here's how the research was described on the APS website:

They set up a room to be distinctively aromatic, hiding three small containers of coffee beans around the room overnight. Over the course of two experiments, they led 40 college students into the room one at a time to perform a tough visual-search task on a computer, finding the letter “X” or “N” in a circle of similar-looking letters (“W,” “M,” “K,” “H,” “Z,” and “V”). 40 other students completed an easier version of the same task; searching for the letter “X” or “N” among a circle of lowercase “o”s. [Students had been randomly assigned to either the difficult or easy task.]

The experimenters then took the students into another room and asked them some follow up questions that grew increasingly leading :

“Describe the room you just completed the task in. Try to describe it using all of your senses.”

“Did you notice any odors in the room, if so what?”

“Could you smell coffee in the room?”

Students assigned to the difficult search task were far less likely to report having picked up the aroma (25% of participants said they noticed a coffee smell) compared to the participants assigned to the easy task (60%-70% percent of participants). When the experimenters led the students back into the test room, all of them said they could smell it. Some of them even commented that the room smelled like a cafe.

Questions

What kind of study was this--experimental or correlational? How do you know?

What was the independent variable? What was the dependent variable?

Think about construct validity: What do you think of the way they measured their dependent variable? Is this a good measure?

Now think about statistical validity: How large does this effect seem to be? Take another look at the results and make a comment on the practical effect size.

What about external validity? To whom might these results generalize? Do you think the pattern for coffee and letter detection might generalize to other smells? To other tasks?

Now consider internal validity. The authors claim that it was concentration that caused people to not notice the smell. Can you think of any confounds in this design?

08/20/2018

What evidence would it take to convince us that it's the schools, not other factors, that are responsible for the outcomes of private school students? Photo: Image Source / Alamy Stock Photo

A large study has compared the outcomes of children who've attended private schools to those who've attended public schools. A journalist summarized the report in the Washington Post. The study provides a nice example of how multivariate regression can be used to test third variable hypotheses.

When we look simply at the educational acheivement of students in private schools vs. public schools, private school students have higher achievement scores. However, all such studies are correlational, because the two variables--Type of School and Level of Achievement--are measured.

Therefore, such studies how covariance, because the results depict a relationship. The study may even show temporal precedence, because attending school presumably precedes the measure of achievement. However, such studies are weak on internal validity. We can think of several alternative explanations for why children in private schools are scoring higher.

One major alternative explanation is socioeconomic status. Children from wealthier families are more likely to afford private schools. And in general, children from wealthier families tend to score higher on achievement tests.

The Washington Post journalist quoted one of study's authors, Robert Pianta, who summed up the study's results this way:

“You only need to control for family income and there’s no advantage,” Pianta said in an interview. “So when you first look, without controlling for anything, the kids who go to private schools are far and away outperforming the public school kids. And as soon as you control for family income and parents’ education level, that difference is eliminated completely.”

Questions

a) Draw little diagrams similar to those in Figure 8.15 (in the 3rd ed.) to depict the arguments being made in this study. What would A be? What about B? In the quote from Pianta, above, what would the C variable(s) be?

b) The researchers used type of school (private vs. public), which is a categorical variable. But in some analyses, the researchers also used "number of years in private school" as an alternative version of this variable. Is "number of years in private school" categorical, ordinal, interval, or ratio data?

c) Sketch a mock-up regression table with the criterion variable at the top and predictors below (Use Table 9.1 as a model). Which variable do you think the researchers selected as the criterion (dependent) variable in their analyses? Which variable(s) would have been the predictors?

d) Now that you know what the results were, think about how the beta associated with "number of years in private school" would change when parental SES is added and removed from the regression analyses.

a) A and B would be Type of School and Level of Acheivement. It doesn't really matter which one is A and which one is called B. C would be Family Income and/or Parental Education.

b) Ratio data (zero is meaningful in this scale because you could attend zero years of private school)

c) The criterion variable would be Achievement, and the predictors would be Number of Years of Private School, Family Income, and Parental Education.

d)When the Number of Years of Private School is on the table (in the analysis) by itself, its beta is likely to be positve and significant (more years of private school goes with higher achievement). When Family Income and Parental Education are added to the table, the beta for Number of Years of Private School should drop to zero. This pattern of results is consistent with the argument that Family Income and Parental Education are the alternative explanation for the original relationship .

07/20/2018

Is social media use responsible for depressed mood? Photo: Ian Allenden/Alamy stock

Do smartphones harm teenagers? If so, how much? In this blog, I've written before about the quasi-experimental and correlational designs used in research on screen time and well-being in teenagers. In that post you can practice identifying the different designs we can use to study this question.

Today's topic is more about the size of the effect in studies that have been published. A recent Wired story tried to put the effect size in perspective.

One side of the argument, as presented by Robbie Gonzalez in Wired, scares us into seeing social media as dangerous.

For example, first

...there were the books. Well-publicized. Scary-sounding. Several, really, but two in particular. The first, Irresistible: The Rise of Addictive Technology and the Business of Keeping Us Hooked, by NYU psychologist Adam Alter, was released March 2, 2017. The second, iGen: Why Today's Super-Connected Kids are Growing Up Less Rebellious, More Tolerant, Less Happy – and Completely Unprepared for Adulthood – and What That Means for the Rest of Us, by San Diego State University psychologist Jean Twenge, hit stores five months later.

In addition,

...Former employees and executives from companies like Facebook worried openly to the media about the monsters they helped create.

But is worry over phone use warranted? Here's what Gonzalez wrote after talking to more researchers:

When Twenge and her colleagues analyzed data from two nationally representative surveys of hundreds of thousands of kids, they calculated that social media exposure could explain 0.36 percent of the covariance for depressive symptoms in girls.

But those results didn’t hold for the boys in the dataset. What's more, that 0.36 percent means that 99.64 percent of the group’s depressive symptoms had nothing to do with social media use. Przybylski puts it another way: "I have the data set they used open in front of me, and I submit to you that, based on that same data set, eating potatoes has the exact same negative effect on depression. That the negative impact of listening to music is 13 times larger than the effect of social media."

In datasets as large as these, it's easy for weak correlational signals to emerge from the noise. And a correlation tells us nothing about whether new-media screen time actually causes sadness or depression.

There are several things to notice in the extended quote above. First let's unpack what it means to, "explain 0.36% of the covariance". Sometimes researchers will square the correlation coefficient r to create the value R2. The R2 tells you the percentage of variance explained in one variable by the other (incidentally, they usually say "percent of the variance" instead of "percent of covariance."). In this case, it tells you how much of the variance in depressive symptoms is explained by social media time (and by elimination, it tells you what percentage is attributable to something else). We can take the square root of 0.0036 (that's the percentage version of 0.36%) to get the original r between depressive symptoms and social media use. It's r = .06.

Questions

a) Based on the guidelines you learned in Chapter 8, is an r of .06 small, medium, or large?

b) Przybylski claims that the effect of social media use on depression is the same size as eating potatoes. On what data might he be basing this claim? Illustrate your answer with two well-labelled scatterplots, one for social media and the other for potatoes. Now add a third scatterplot, showing listening to music.

c) When Przybylski states that the correlation held for the girls, but not the boys, what kind of model is that? (Here are your choices: moderation, mediation, or a third variable problem?)

d) Finally, Przybylski notes that in large data sets, it's easy for weak correlation signals to appear from the noise. What statistical concepts are being applied here?

e) Chapter 8 presents another example of a large data set that found a weak (but statistically significant) correlation. What is it?

f) The discussion above between Gonzalez and Przybylski concerns which of the four big validities?

a) An r of .06 is probably going to be characterized as "small" or "very small" or even "trivial." That's what the "potatoes" point is trying to illustrate, in a more concrete way.

b) One scatterplot should be labeled with "potato eating" on the x axis and "depression symptoms" on the y axis. The second scatterplot should be labeled with "social media use" on the x axis and "depression symptoms" on the y axis. These first two plots should show a positive slope of points with the points very spread out--to indicate the weakness of the association. The spread of the first two scatterplots should be almost the same, to represent the claim the two relationships are equal in magnitude. The third scatterplot should be labeled with "listening to music" on the x axis and "depression symptoms" on the y axis, and this plot should show a much stronger, positive correlation (a tighter cloud of points).

c) It is a moderator. Gender moderates (changes) the relationship between screen use and depression.

d) Very large data sets have a lot of statistical power. Therefore, large data sets can show statistical significance for even very, very, small correlations--even correlations that are not of much practical interest. A researcher might report a "statistically significant' correlation, but it's essential to also ask about the effect size and its practical value (the potatoes argument). Note: you can see the r = .06 value in the original empirical article here, on p. 9.

e) The example in Chapter 8 is the one about meeting one's spouse online and having a happier marriage--that was a statistically significant relationship, but r was only .03. That didn't stop the media from hyping it up, however.

f) Statistical validity

g) The research on smartphones and depressive symptoms is correlational, making causal claims (and causal language) inappropriate. That means that we can't be sure if social media is leading to the (slight) increase in depressive symptoms, or if people who have more depressive symptoms end up using more social media, or if there's some third variable responsible for both social media use and depressive symptoms. As the Wired article states,

...research on the link between technology and wellbeing, attention, and addiction finds itself in need of similar initiatives. They need randomized controlled trials, to establish stronger correlations between the architecture of our interfaces and their impacts; and funding for long-term, rigorously performed research.

b) What foods might be associated with your own cultural identity (or identities?)

Here are some elements of the journalist's story. NPR reported about...

...a recent study in the Journal of Experimental Social Psychology, authored by Jay Van Bavel, social psychologist at New York University and his colleagues. The researchers found that the stronger your sense of social identity, the more you are likely to enjoy the food associated with that identity. The subjects of this study were Southerners and Canadians, two groups with proud food traditions.

The first experiment, containing 103 people, found that the more strongly someone self-identifies as Southern, the more they would expect Southern food to taste good, food like fried catfish or black-eyed peas.

c) In the study above, what are the two variables? Do they seem to be manipulated or measured?

d) Given your answer to question c) is this study really an "experiment"?

e) Can this study (above) support the causal claim that "identity impacts the food you like"? What are some alternative explanations? Hint: Think about temporal precedence and third variable explanations.

Here's the description of a second study:

In a second experiment, containing 151 people, researchers also found that when Southerners were reminded of their Southernness — primed, in psychology speak — their perception of the tastiness of Southern food was even higher. That is, the more Southern a person was feeling at that moment, the better the food tasted [compared to a group who was not primed].

e) What are the two variables in the study above? Were the variables manipulated or measured?

f) Given your answer to question e) is this study really an "experiment"?

g) Can this study support the claim that "identity impacts the food you like"?

They found a similar result when taste-testing with Canadians, finding that Canadian test subjects only preferred the taste of maple syrup over honey in trials when they were first reminded of their Canadian identity.

h) You know the drill: For the study above, what kind of study was is? What are its variables?

i) Challenge question: Can you tell if the independent variable in the Canadian study was manipulated as between groups or within groups?

In sum, it appears that two out of the three studies reviewed by this NPR article were experimental, so they're more likely to support the causal claim about "identity impacting the food you like." The journalist calls attention to this manipulation of identity in this description:

The relationship between identity and food preference is not new. However, the use of priming to induce identity makes this study different from its predecessors.

"Priming is like opening a filing drawer and bringing to your attention all the things that are in the drawer," says Paul Rozin, food psychologist at University of Pennsylvania, who was not involved in the study. "You can't really change peoples' identities in a 15-minute setting, but you can make one of their identities more salient, and that's what they've done in this study."

j) What other ways might you manipulate cultural identity in an experimental design?

Good news! The empirical journal article is open-access here. When you read it, you'll see that the journalist simplified the design of the studies for her article in NPR.

05/10/2018

I'm standing at my desk as I compose this post....could that make my writing go better? Yes, according to an editorial entitled, "Standing up at your desk could make you smarter." The editorial leads with a strong causal claim and then describes three studies, each with a different design. Here's one of the studies:

A study published last week...showed that sedentary behavior is associated with reduced thickness of the medial temporal lobe, which contains the hippocampus, a brain region that is critical to learning and memory.

The researchers asked a group of 35 healthy people, ages 45 to 70, about their activity levels and the average number of hours each day spent sitting and then scanned their brains with M.R.I. They found that the thickness of their medial temporal lobe was inversely correlated with how sedentary they were; the subjects who reported sitting for longer periods had the thinnest medial temporal lobes.

a) What were the two variables in this study? Were they manipulated or measured? Was this a correlational or experimental study?

b) The author writes that the study "showed that sedentary behavior is associated with reduced thickness of the medial temporal lobe." Did he use the correct verb? Why or why not?

Here's a second study described in the editorial:

Intriguingly, you don’t even have to move much to enhance cognition; just standing will do the trick. For example, two groups of subjects were asked to complete a test while either sitting or standing [randomly assigned]. The test — called Stroop — measures selective attention. Participants are presented with conflicting stimuli, like the word “green” printed in blue ink, and asked to name the color. Subjects thinking on their feet beat those who sat by a 32-millisecond margin.

c) What are the two variables in this study? Were they manipulated or measured? Was this a correlational or experimental study?

d) Does this study support the author's claim that "you don't have to move much to enhance cognition; just standing will do the trick"? Why or why not?

e) Bonus: What kind of experiment was being described here? (Posttest only, prettest/posttest, repeated measures, or concurrent measures?) Comment, as well, on the effect size.

It’s also yet another good argument for getting rid of sitting desks in favor of standing desks for most people. For example, one study assigned a group of 34 high school freshmen to a standing desk for 27 weeks. The researchers found significant improvement in executive function and working memory by the end of the study.

f) What are the variables in this study? Were they manipulated or measured?

g) Do you think this study can support a causal claim about standing desks improving executive function and working memory?

The author added the following statement to the third study on high school freshmen:

True, there was no control group of students using a seated desk, but it’s unlikely that this change was a result of brain maturation, given the short study period.

h) What threat to internal validity has the author identified in this statement?

i) What do you think of his evaluation of this threat?

j) Of the three studies presented, which provides the strongest evidence for the claim that "standing up at your desk could make you smarter"? What do you think? On the basis of this evidence, should I keep standing here?

How do we know that dressing up as Batman works? Let's learn more about the study behind the catchy headline. I'll be quoting from this British Psychological Society summary of it, as well as from the original journal article in the scientific journal Child Development(paywall--only available through University libraries).

The study was conducted to test a theory about self-regulation. All of us--children or adults--have to exercise self-control to make ourselves stick to important (but sometimes boring) tasks. One strategy researchers are examining is "self-distancing," in which people view a situation from a third-person perspective--one more distant and objective--rather than a self-immersed perspective, which can be more emotional and impulsive. The research tests the hypothesis that seeing oneself as "Batman" will engage kids in this self-distanced perspective.

Now for the design of the study. The team of scientists...

recruited 180 kids aged 4 to 6 years and ...asked them to complete a boring, slow but supposedly important ten-minute computer task that involved pressing the space bar whenever they saw a picture of cheese or not pressing anything when the screen showed a cat. The children were encouraged to stay on task, but they were told they could take a break whenever they wanted and go play a game on a nearby iPad.

Some of the children were assigned to a “self-immersed condition”, akin to a control group, and before and during the task were told to reflect on how they were doing, asking themselves “Am I working hard?”. Other children were asked to reflect from a third-person perspective, asking themselves “Is James [insert child’s actual name] working hard?” Finally, the rest of the kids were in the Batman condition, in which they were asked to imagine they were either Batman, Bob The Builder, Rapunzel or Dora the Explorer and to ask themselves “Is Batman [or whichever character they were] working hard?”. Children in this last condition were given a relevant prop to help, such as Batman’s cape.

Here are the results (I've focused on the 4-year olds here):

...those in the Batman condition spent the most time on task (...about 32 per cent...). The children in the self-immersed condition spent the least time on task (...just over 20 per cent...) and those in the third-person condition performed in between.

a) In this study, what is the independent variable? How many levels were in this IV, and what were the levels? Was the IV independent groups or within groups?

b) What was the dependent variable?

c) Sketch a well-labeled line or bar graph of the results.

d) Why do you think the researchers included the condition in which kids were asked to think about themselves in the third person?

e) Notice that almost all of the headlines and twitter comments about this study have focused on Batman. Even the researchers call it "The Batman Effect" Is that accurate?

f) Finally, think about the fact that in the Batman condition, kids not only got to pretend to be a character. They also got to make an important choice about their participation in the study (the choice among the four different options of Batman, Rapunzel, Bob the Builder, and Dora). The kids in the self-immersed and third-person conditions did not make any choices. What kind of problem might this be in the study? (Which one of the four big validities does it address?)

g) Can the study really support the claim that "Pretending to be Batman helps kids stay on task"? Apply the three causal criteria, paying special attention to the point raised in question f), above.

Note to Instructors: If you include the results for the 6 year olds, you can also teach this as an 2x3 IVxPV design, using age (4 vs. 6 year olds) as the participant variable. Here are the full results:

The six-year-olds spent more time on task than the four-year-olds (half the time versus about a quarter of the time). No surprise there. But across age groups, and apparently unrelated to their personal scores on mental control, memory, or empathy, those in the Batman condition spent the most time on task (about 55 per cent for the six-year-olds; about 32 per cent for the four-year-olds). The children in the self-immersed condition spent the least time on task (about 35 per cent of the time for the six-year-olds; just over 20 per cent for the four-year-olds) and those in the third-person condition performed in between.

11/20/2017

The sun sets in Amarillo, TX an hour later than it does in Huntsville, AL though they are on the same time zone. Amarillo residents get less sleep and earn more money: Is there a causal connection? Photo: Creativeedits/Wikimedia Common

Sleep is an essential human function and getting more sleep is associated with improved mood, cognitive performance, and physical performance. Therefore, it might make sense that sleep would improve people's productivity and ability to earn money. That's the topic of a Freakonomics episode on the "Economics of Sleep." You can read the transcript or listen to the 45 minute episode here. (The section I focus on starts around minute 10.)

Freakonomics' hosts interviewed a set of economists (including Matthew Gibson, Jeff Shrader, Dan Hamermesh, and Jeff Biddle) about their research on sleep, work hours, and income. The economists mentioned that, in order to establish a causal link between sleep and income:

What we need is something like an experiment for sleep. Almost as though we go out in the United States and force people to sleep different amounts and then watch what the outcome is on their wages.

While it is theoretically possible to conduct such an experiment, it is practically difficult to assign people to different sleep conditions for a long enough period of time to notice an impact on their wages. So the economists took an alternative path and used quasi-experimental data. In a creative twist, they compared wages at two ends of a single American time zone. The example they gave is Huntsville, AL and Amarillo, TX. Here's why. Gibson stated:

It turns out that ever since we’ve put time zones into place, we’ve basically been running just that sort of giant experiment on everyone in America.

The story continued. You'll see the transcript version quoted below:

Consider two places like Huntsville, Alabama — which is near the eastern edge of the Central Time Zone — and Amarillo, Texas, near the western edge of the Central zone. [...]

...even though Amarillo and Huntsville share a time zone, the sun sets about an hour later in Amarillo, according to the clock, and since the two cities are at roughly the same latitude as well, they get roughly the same amount of daylight too.

So you’ve got two cities on either end of a time zone, roughly the same size — just under 200,000 people each — where, according to the clock time, sunset is an hour apart. Now, what good is that to a pair of economists interested in sleep research?

GIBSON: It turns out that the human body, our sleep cycle responds more strongly to the sun than it does to the clock. People who live in Huntsville and experience this earlier sunset go to bed earlier.

GIBSON: If we plot the average bedtime for people as a function of how far east they are within a time zone, we see this very nice, clean nice straight line with earlier bedtime for people at the more eastern location.

But since Huntsville and Amarillo are in the same time zone, people start work at roughly the same time, which means alarm clocks go off at roughly the same time.

GIBSON: That means if you go to bed earlier in Huntsville, you sleep longer.

The economists didn't use only Huntsville and Amarillo--they also conducted multiple comparisons of cities around the U.S. that were similarly on each end of a single time zone. Using "city of residence" as their quasi-experimental operationalization of "amount of sleep", the economists were ready to report the results for wages:

So now Gibson and Shrader plugged in wage data for Huntsville vs. Amarillo and other pairs of cities that had a similar sleep gap.

GIBSON: We find that permanently increasing sleep by an hour per week for everybody in a city, increases the wages in that location by about 4.5 percent.

Four and a half percent — that’s a pretty good payout for just one extra hour of sleep per week. If you get an extra hour per night, Gibson and Shrader discovered — here, let me quote you their paper: “Our main result is that sleeping one extra hour per night on average increases wages by 16%, highlighting the importance of restedness to human productivity.”

Questions:

a) What is the independent variable in this time zone and wages study? What is the dependent variable?

b) Is the IV independent groups or within groups?

c) Which of the four quasi-experimental designs is this? Non-equivalent control group posttest only, Non-equivalent control group pretest-posttest, Interrupted time series, or Non-equivalent control group interrupted time series?

d) The economists asserted, "sleeping one extra hour per night on average increases wages by 16%" (italics added). What do you think? Can their study support this claim? Apply the three causal rules, especially taking note of internal validity issues that this study might have.

e) If you consider only one pair of cities, there are multiple alternative explanations, besides sleep, that can account for wage differences. Name two or three such threats (considering Huntsville and Amarillo as an example). Now consider, how might many of these internal validity threats be reduced by conducting the same analysis over many other city pairs?

f) This Freakonomics episode was aired in 2015, but the study (about time zones) they reviewed is not yet published. What do you think about that?

Answers to selected questions

a) The IV is "Hours of sleep" (but you could also call it "location on the time zone: East or West") and the DV is "Wages".

b) The IV is independent-groups.

c) Non-equivalent control group posttest only.

d & e) The results of the study support covariance: People in cities in the Eastern portion of time zones get more sleep and have higher wages than people in the Western portions. Temporal precedence is unclear, I think: Because the data were collected at the same time, it's not clear if the timezone came first, leading to more sleep and higher wages, or if people began to earn higher wages first, and then systematically moved Eastward. (However, the second direction certainly seems less plausible than the first.)

As for internal validity, if we consider only the city pair of Huntsville and Amarillo, we could come up with several alternative explanations. The two cities have different historical trajectories and different ethnic diversities; they are in two different states that have different fiscal policies and industry bases. Perhaps Amarillo has poorer wages in general and people are losing out on sleep there because they are working more than one job. However, these internal validity threats become less of an issue when you consider multiple pairs of cities. It is less plausible that internal validity threats that apply to one city pair would also, coincidentally, apply to all the other city pairs that are at opposite ends of a time zone.

Even though the method is fairly strong, psychologists would be unlikely to make a strong causal claim simply from quasi-experimental data like these, because the independent variable is not truly manipulated. Nevertheless, the method and results of this quasi-experiment are certainly consistent with the argument that getting more sleep may be a factor in earning higher wages.

10/20/2017

Should she have put her phone away in the next room? That depends. Photo: Suwat Sirivutcharungchit/Shutterstock

Students, if you're not familiar with the study tips on The Learning Scientists website, you should be. This page in particular sums up six evidence-based things you should be doing while you study (spoiler alert: The list does not include highlighting!).

The Learning Scientists' latest blog post sums up the results of an experimental study on where your phone should be while you engage in cognitive tasks. It's titled, "Separation from your cellphone boosts your cognitive capacity." Take a look at the description to get an overview of the research design:

They invited students to participate in an experiment where students were randomly assigned into one of three conditions.

In the "other room" condition, students were asked to leave their belongings (including their cellphones) in the lobby before coming into the room where the experiment would take place.

In other two conditions, students were asked to take their belongings with them to the experiment room, and were either told to leave the cellphone out of sight, e.g., in their bags or pockets (bag/pocket condition) or place it face down on the desk within sight (desk condition).

Then, participants worked on two cognitive tasks: One working memory task – called Automated Operation Span task (OSpan) – where people are asked to actively process information while holding other information in mind....For the other task – the Raven’s Standard Progressive Matrices (RSPM) – participants had to identify the missing piece in a matrix pattern. This test is used to assess fluid intelligence and your performance depends to a large extent on the available attentional capacities to identify the underlying rule of the pattern matrix.

a) Based on the description, what kind of experiment was this: Concurrent measures? Repeated measures? Posttest-only? or pretest/posttest?

b) What is the independent variable here? There are two dependent variables in this design. What are they? (Note: You might recognize the OSpan task from Chapter 8; it was used in a correlational study about ability to multitask.)

c) What results would you predict from this study? Take a moment to sketch your prediction in graph form. Then click over to the blog post and scroll to the graphs they've made of the results.Do they match your own prediction?

You can stop working here if you're studying Chapter 10. But if you're studying Chapter 12, keep reading, because there's more! The second part of the blog post is headed "Cellphone dependence as moderator". Get ready for a factorial design.

The researchers separated people into two new participant variable (PV) groups: Those who reported feeling dependent on their cellphone throughout the day, and those who did not. They then used this PV in combination with the IV of the original design.

d) Given the description, how would you state this design? (Use the form: __ X __ factorial.)

Here are the results:

For people who reported a strong dependence, putting the cellphone in the bag or leaving it in another room made a tremendous difference for their cognitive capacity: They performed much better in these two conditions compared to the one where the phone was on the desk.

For people who reported a weaker dependence, it made no difference where the phone was. Thus, their performance was not affected by the location of the phone.

e) Sketch a bar graph or line graph of the factorial results described above. You can do it either way, but I'd recommend putting the "cell phone condition" IV on the x-axis.

f) Do you see an interaction in the results? (You should, because the term "moderator" is a sign of an interaction)

g) Let's return to the headline, "Separation from your cellphone boosts your cognitive capacity." Does the headline seem appropriate for this study? Why or why not?

Good news: The published article on which this blog post was based is open source! You can view it here.

09/20/2017

One conspiracy theory holds that the government spreads dangerous chemicals in the exhaust from airplanes (the "chemtrails" theory). According to the studies, who is more likely to believe this theory? Credit: Jason Batterham/Shutterstock

There's a causal claim in this journalist's story about research on people who believe conspiracy theories. First, here's the causal headline:

b) Draw a diagram of this causal claim, using this form: A ---> B. (That is, which variable comes first, according to the wording the journalist used?)

Now that you've established the claim, does the research support it? The journalist reports on several studies conducted by researchers at the University of Mainz. Here's one:

The researchers first asked a sample of 238 US participants recruited via Amazon’s Mechanical Turk survey website to complete a self-reported “Need For Uniqueness” scale (they rated their agreement with items like “being distinctive is extremely important to me”) and a Conspiracy Mentality scale (e.g. “Most people do not see how much our lives are determined by plots hatched in secret.”) before indicating whether or not they believed in a list of 99 conspiracy theories circulating online....[The results showed that] participants’ self-reported Need For Uniqueness ... correlated with their stronger endorsement of the conspiracy beliefs.

c) In the study above, sketch a scatterplot of the results they described. Label your axes mindfully.

d) Is the study above correlational or experimental? Why or why not?

e) Does this study support the claim that the journalist assigned to it? That is, does it show that believing conspiracy theories causes people's need for uniqueness to be satisfied? As you explain your answer, pay careful attention to possible third variables. That is, what third variables ("C"s) might be reasonably correlated with both need for uniqueness and belief in conspiracies?

Here's another study in the series:

The second study replicated this finding with a further 465 Mechanical Turk participants based in the US, but this time half the sample read a list of the five most well known conspiracy theories and the five least known ones, whereas the other half of the group read the five most popular conspiracy theories and the five least popular. Again, self-reported Need For Uniqueness correlated with stronger agreement with the various conspiracy theories.

f) Is the second study correlational or experimental? Why? Sketch the results in a well-labeled scatterplot.

g) Does this second study rule out any of the third variables you came up with when answering question e) above?

Here's a final study:

Note, the conspiracy theory that featured in this final experiment was entirely made-up by the researchers. ...The conspiracy theory was about smoke detectors and the claim was that they produce dangerous hypersound. The researchers led half of 290 participants on Amazon’s Mechanical Turk to believe that this was a popular conspiracy theory in Germany where it was alleged to be believed by 81 per cent of Germans. The rest of the participants were led to believe that the theory was doubted by 81 per cent of Germans.

While information about the popularity of the theory didn’t affect participants overall, it did impact those who said that they tended to endorse a lot of conspiracy theories. Among these conspiracy-prone participants, their belief in the made-up smoke detector conspiracy was enhanced on average when the conspiracy was framed as a minority opinion.

h) Is the above study correlational or experimental? Why?

i) Sketch the results of the study--Hint: you might need to use a bar graph that also includes two colors of bars.

j) To what extent does this final study support the claim that believing "conspiracy theories causes people's need for uniqueness to be satisfied"?

k) If the study doesn't support the claim given in j), is there another, modified causal claim that the study can support? What is it?

In the final section of the story, the journalist concludes, "Taken together, the findings provide convincing evidence that some people are motivated to agree with conspiracy theories with an aura of exclusiveness."

l) Should this final claim be classified as causal or association? To what extent is this claim supported by the studies?

My thanks to Marianne Lloyd of Seton Hall University for sharing this story!

Selected answers

d) This is a correlational study because both need for uniqueness and belief in conspiracy theories were measured variables.

e) In this correlational study, we can't know for sure whether Need for Uniqueness --> Belief in C0nspiracy Theories, whether Belief in Conspiracy Theories --> Need for Uniqueness, or whether some third variable is associated with both. One possible third variable might be social anxiety: Perhaps socially anxious people believe more conspiracy theories and socially anxious people are also high on need for uniqueness. Being highly educated is associated with lower belief in conspiracy theories (according to this study), and if highly educated people are also low in need for uniqueness, then education is a potential third variable as well.

f) Both variables were measured again, so....correlational study!

g) You could raise the same objections here that you did in e)

h) This study is experimental because they manipulated the proportion of people who supposedly believed in the theory. There are two measured variables in this study as well; one is the tendency to believe in conspiracy theories (high or low), and the other is the extent of belief in the smoke detector theory.

j) No; in fact need for uniqueness wasn't reported by the journalist as a variable at all, so it probably can't support that claim.

k) I believe this study can support the following claim: Telling people that only a minority of Germans support a conspiracy theory causes people to believe it more, only if they already are the type of person who believes in conspiracy theories.

l) The final statement is an association claim. Because most of the studies were correlational, the studies do support it!

04/10/2017

Does giving a child a sip change his or her long-term drinking habits? Photo: Tang Ming Tung/Getty Images

It's a strong causal claim: Giving kids sips of beer turns them into teenage drunks. Did the journalist get it right? Here are some quotes from the story, posted in the food website Munchies:

Those innocent tastes of Chianti at the Thanksgiving dinner table could morph your child from a sweet, sober cherub into a bleary-eyed teenage booze-guzzling ne'er-do-well.

New research in the Journal of Studies on Alcohol and Drugs has found that children who sip alcohol as youngsters have an increased likelihood of becoming drinkers by the time they reach high school. In a long-term study by Brown University of 561 students in Rhode Island, researchers found that those who had tried even small sips were a whopping five times more likely to have tried a whole beer or cocktail by the time they reached ninth grade, and four times more likely to have gotten rip-roaring drunk.

a) What keywords in this quote indicate that the journalist is making a causal claim?

b) What were the two variables studied by the researchers? Explain whether whether you think each one was measured or manipulated.

c) What kind of study is this claim apparently based upon--correlational or experimental?

d) Given the study's design, is the causal claim appropriate? Apply the three causal criteria.

In an interview with Munchies, the lead researcher, Kristina Jackson, mentions several possible third variables for the association:

But Jackson also believes that other factors correlate with these numbers, in addition to the "early sipper" factor. Parents' drinking habits, a family history of alcoholism, and general personality and behavioral characteristics also have strong impacts on the boozy worldviews of children and teenagers.

e) Chapter 9 readers: Do you see any evidence that the researchers controlled for these potential internal validity problems in their analyses? You might have to hunt down the original journal article to find out.

f) The journalist made a dramatic point about the statistic about kids who'd sipped beer "being four times more likely to have gotten rip-roaring drunk." Which of the four big validities is this statement about?

Even though the journalist's causal claim is probably not justified, adolescent substance use is a serious issue. The journalist supplemented the story with several frequency claims. You might be interested in some of these statistics.

Roughly 30 percent of the students said that they had tasted alcohol when in sixth grade..., mostly due to exposure from their parents while at a party, on vacation, or in other special circumstances. Of that group (the "early sippers"), 26 percent reported having consumed a full alcoholic drink by ninth grade, while only 6 percent of non-early-sippers had experienced the pleasures of an ice-cold Natural Ice or homemade Screwdriver. And at that same age (roughly 14-15 years old), 9 percent of early sippers had gotten totally trashed, while only 2 percent of those with less-loose parents had.

If you’re a research methods instructor or student and would like us to consider your guest post for everydayresearchmethods.com, please contact Dr. Morling. If, as an instructor, you write your own critical thinking questions to accompany the entry, we will credit you as a guest blogger.