...research suggests that Tetris can ease us through periods of anxiety by getting us to a blissfully engrossed mental state that psychologists call "flow."

"The state of flow is one where you're completely absorbed or engaged in some kind of activity," Sweeny explains. "You lose your self-awareness, and time is just flying by."

Here's more on the detail:

Sweeny and her collaborators gathered a group of more than 300 college students and told them their peers would be evaluating how attractive they were. "I know, it's kind of cruel, but we found it's a really effective way to get people stressed out," Sweeny says. While the participants awaited their attractiveness scores, the researchers had them play Tetris.

Some played a painfully slow, easy version of the game — which bored them. Some played an extremely challenging, fast version — which frustrated them. And everyone else played the classic version, which adapts to each player's individual skill level and gets them into that state of flow.[People were randomly assigned to the three groups.]

In the end, everyone experienced a degree of worry. But the third group reported slightly higher levels of positive emotions (on average, about a quarter of a point higher on a five-point scale) and slightly lower levels of negative emotions (half a point lower on a five-point scale).

"It wasn't a huge difference, but we think it's noticeable," Sweeny says. "And over time, it can add up."

Questions:

a) In this study, they decided to manipulate the conceptual variable, "degree of flow." How did they operationalize this variable?

b) What were the dependent variables in this study? (there seem to be two DVs here)

c) What was the independent variable? What were its levels?

d) Does this seem to be an experiment or a correlational study? How do you know?

e) Sketch a graph of the results.

f) The journalist mentions details about the results (e.g., "about a quarter of a point higher on a five-point scale" and "half a point lower on a five-point scale"). Which aspect of statistical validity is being discussed here?

g) What questions would you ask to decide if this study was internally valid? Which of the internal validity threats in Table 11.1 could you rule out? Which could you ask about?

h) What about the external validity of this study? How might you see if this effect might generalize to other flow-related activities (other than Tetris)?

“We know that in humans there’s a strong correlation between cognitive health and social connections, but we don’t know if it’s having a group of friends that’s protecting people or if it’s that people with declining brain health withdraw from their human connections,” [Study researcher] Kirby said.

[The n]ew research ...found that mice housed in groups had better memories and healthier brains than animals that lived in pairs.

a) Before reading on, reflect: Why would a researcher probably need an animal model to test this question experimentally?

Here's some more detail about the experiment:

Some mice lived in pairs, which Kirby refers to as the “old-couple model.” Others were housed for three months with six other roommates, a scenario that allows for “pretty complex interactions.”

The mice were 15 months to 18 months old during the experiment – a time of significant natural memory decline in the rodent lifespan.

In tests of memory, the group-housed mice fared better.

One test challenged the mice to recognize that a toy, such as a plastic car, had moved to a new location. ...“With the pair-housed mice, they had no idea that the object had moved. The group-housed mice were much better at remembering what they’d seen before and went to the toy in a new location, ignoring another toy that had not moved,” Kirby said.

In another common maze-based memory test, mice are placed on a well-lit round table with holes, some of which lead to escape hatches. Their natural tendency is to look for the dark, unexposed and “safe” escape routes.

The “couples” mice didn’t get faster at the test when it was repeated over the course of a day.“But over the course of many days, they developed a serial-searching strategy where they checked every hole as quickly as possible. It’d be like walking as quickly as possible through each row of a parking lot to look for your car rather than trying to remember where your car actually is and walk to that spot,” Kirby said.

The group-housed mice improved with each trial, though. “They seemed to try to memorize where the escape hatches are and walk to them directly, which is the behavior we see in healthy young mice,” Kirby said. “And that tells us that they’re using the hippocampus, an area of the brain that is really important for good memory function.”

b) What was the independent variable in this study? How was it operationalized?

c) What was the dependent variable? What were the two ways it was operationalized?

d) How does this experiment help us decide which comes first--social life or better memory? (note: This is temporal precedence!)

e) Do you think the journalist is justified in generalizing this study's results from mice to older adult humans? Why or why not?

f) Chapter 3 explains how internal validity and external validity are often in a trade-off. Describe how this study with mice illustrates that trade-off.

09/20/2018

He is concentrating so much that he might not notice the coffee smell.... Photo: baranq/Shutterstock

At times, we've all been so engrossed in a task that we've lost awareness of our surroundings. Maybe you didn't hear someone calling your name when you were finishing your paper, or maybe you missed the oven timer when you were reading that mystery book. Now researchers Sophie Forester and Charles Spence have reported that concentration impacts our sense of smell. Here's how the research was described on the APS website:

They set up a room to be distinctively aromatic, hiding three small containers of coffee beans around the room overnight. Over the course of two experiments, they led 40 college students into the room one at a time to perform a tough visual-search task on a computer, finding the letter “X” or “N” in a circle of similar-looking letters (“W,” “M,” “K,” “H,” “Z,” and “V”). 40 other students completed an easier version of the same task; searching for the letter “X” or “N” among a circle of lowercase “o”s. [Students had been randomly assigned to either the difficult or easy task.]

The experimenters then took the students into another room and asked them some follow up questions that grew increasingly leading :

“Describe the room you just completed the task in. Try to describe it using all of your senses.”

“Did you notice any odors in the room, if so what?”

“Could you smell coffee in the room?”

Students assigned to the difficult search task were far less likely to report having picked up the aroma (25% of participants said they noticed a coffee smell) compared to the participants assigned to the easy task (60%-70% percent of participants). When the experimenters led the students back into the test room, all of them said they could smell it. Some of them even commented that the room smelled like a cafe.

Questions

What kind of study was this--experimental or correlational? How do you know?

What was the independent variable? What was the dependent variable?

Think about construct validity: What do you think of the way they measured their dependent variable? Is this a good measure?

Now think about statistical validity: How large does this effect seem to be? Take another look at the results and make a comment on the practical effect size.

What about external validity? To whom might these results generalize? Do you think the pattern for coffee and letter detection might generalize to other smells? To other tasks?

Now consider internal validity. The authors claim that it was concentration that caused people to not notice the smell. Can you think of any confounds in this design?

b) What foods might be associated with your own cultural identity (or identities?)

Here are some elements of the journalist's story. NPR reported about...

...a recent study in the Journal of Experimental Social Psychology, authored by Jay Van Bavel, social psychologist at New York University and his colleagues. The researchers found that the stronger your sense of social identity, the more you are likely to enjoy the food associated with that identity. The subjects of this study were Southerners and Canadians, two groups with proud food traditions.

The first experiment, containing 103 people, found that the more strongly someone self-identifies as Southern, the more they would expect Southern food to taste good, food like fried catfish or black-eyed peas.

c) In the study above, what are the two variables? Do they seem to be manipulated or measured?

d) Given your answer to question c) is this study really an "experiment"?

e) Can this study (above) support the causal claim that "identity impacts the food you like"? What are some alternative explanations? Hint: Think about temporal precedence and third variable explanations.

Here's the description of a second study:

In a second experiment, containing 151 people, researchers also found that when Southerners were reminded of their Southernness — primed, in psychology speak — their perception of the tastiness of Southern food was even higher. That is, the more Southern a person was feeling at that moment, the better the food tasted [compared to a group who was not primed].

e) What are the two variables in the study above? Were the variables manipulated or measured?

f) Given your answer to question e) is this study really an "experiment"?

g) Can this study support the claim that "identity impacts the food you like"?

They found a similar result when taste-testing with Canadians, finding that Canadian test subjects only preferred the taste of maple syrup over honey in trials when they were first reminded of their Canadian identity.

h) You know the drill: For the study above, what kind of study was is? What are its variables?

i) Challenge question: Can you tell if the independent variable in the Canadian study was manipulated as between groups or within groups?

In sum, it appears that two out of the three studies reviewed by this NPR article were experimental, so they're more likely to support the causal claim about "identity impacting the food you like." The journalist calls attention to this manipulation of identity in this description:

The relationship between identity and food preference is not new. However, the use of priming to induce identity makes this study different from its predecessors.

"Priming is like opening a filing drawer and bringing to your attention all the things that are in the drawer," says Paul Rozin, food psychologist at University of Pennsylvania, who was not involved in the study. "You can't really change peoples' identities in a 15-minute setting, but you can make one of their identities more salient, and that's what they've done in this study."

j) What other ways might you manipulate cultural identity in an experimental design?

Good news! The empirical journal article is open-access here. When you read it, you'll see that the journalist simplified the design of the studies for her article in NPR.

05/10/2018

I'm standing at my desk as I compose this post....could that make my writing go better? Yes, according to an editorial entitled, "Standing up at your desk could make you smarter." The editorial leads with a strong causal claim and then describes three studies, each with a different design. Here's one of the studies:

A study published last week...showed that sedentary behavior is associated with reduced thickness of the medial temporal lobe, which contains the hippocampus, a brain region that is critical to learning and memory.

The researchers asked a group of 35 healthy people, ages 45 to 70, about their activity levels and the average number of hours each day spent sitting and then scanned their brains with M.R.I. They found that the thickness of their medial temporal lobe was inversely correlated with how sedentary they were; the subjects who reported sitting for longer periods had the thinnest medial temporal lobes.

a) What were the two variables in this study? Were they manipulated or measured? Was this a correlational or experimental study?

b) The author writes that the study "showed that sedentary behavior is associated with reduced thickness of the medial temporal lobe." Did he use the correct verb? Why or why not?

Here's a second study described in the editorial:

Intriguingly, you don’t even have to move much to enhance cognition; just standing will do the trick. For example, two groups of subjects were asked to complete a test while either sitting or standing [randomly assigned]. The test — called Stroop — measures selective attention. Participants are presented with conflicting stimuli, like the word “green” printed in blue ink, and asked to name the color. Subjects thinking on their feet beat those who sat by a 32-millisecond margin.

c) What are the two variables in this study? Were they manipulated or measured? Was this a correlational or experimental study?

d) Does this study support the author's claim that "you don't have to move much to enhance cognition; just standing will do the trick"? Why or why not?

e) Bonus: What kind of experiment was being described here? (Posttest only, prettest/posttest, repeated measures, or concurrent measures?) Comment, as well, on the effect size.

It’s also yet another good argument for getting rid of sitting desks in favor of standing desks for most people. For example, one study assigned a group of 34 high school freshmen to a standing desk for 27 weeks. The researchers found significant improvement in executive function and working memory by the end of the study.

f) What are the variables in this study? Were they manipulated or measured?

g) Do you think this study can support a causal claim about standing desks improving executive function and working memory?

The author added the following statement to the third study on high school freshmen:

True, there was no control group of students using a seated desk, but it’s unlikely that this change was a result of brain maturation, given the short study period.

h) What threat to internal validity has the author identified in this statement?

i) What do you think of his evaluation of this threat?

j) Of the three studies presented, which provides the strongest evidence for the claim that "standing up at your desk could make you smarter"? What do you think? On the basis of this evidence, should I keep standing here?

How do we know that dressing up as Batman works? Let's learn more about the study behind the catchy headline. I'll be quoting from this British Psychological Society summary of it, as well as from the original journal article in the scientific journal Child Development(paywall--only available through University libraries).

The study was conducted to test a theory about self-regulation. All of us--children or adults--have to exercise self-control to make ourselves stick to important (but sometimes boring) tasks. One strategy researchers are examining is "self-distancing," in which people view a situation from a third-person perspective--one more distant and objective--rather than a self-immersed perspective, which can be more emotional and impulsive. The research tests the hypothesis that seeing oneself as "Batman" will engage kids in this self-distanced perspective.

Now for the design of the study. The team of scientists...

recruited 180 kids aged 4 to 6 years and ...asked them to complete a boring, slow but supposedly important ten-minute computer task that involved pressing the space bar whenever they saw a picture of cheese or not pressing anything when the screen showed a cat. The children were encouraged to stay on task, but they were told they could take a break whenever they wanted and go play a game on a nearby iPad.

Some of the children were assigned to a “self-immersed condition”, akin to a control group, and before and during the task were told to reflect on how they were doing, asking themselves “Am I working hard?”. Other children were asked to reflect from a third-person perspective, asking themselves “Is James [insert child’s actual name] working hard?” Finally, the rest of the kids were in the Batman condition, in which they were asked to imagine they were either Batman, Bob The Builder, Rapunzel or Dora the Explorer and to ask themselves “Is Batman [or whichever character they were] working hard?”. Children in this last condition were given a relevant prop to help, such as Batman’s cape.

Here are the results (I've focused on the 4-year olds here):

...those in the Batman condition spent the most time on task (...about 32 per cent...). The children in the self-immersed condition spent the least time on task (...just over 20 per cent...) and those in the third-person condition performed in between.

a) In this study, what is the independent variable? How many levels were in this IV, and what were the levels? Was the IV independent groups or within groups?

b) What was the dependent variable?

c) Sketch a well-labeled line or bar graph of the results.

d) Why do you think the researchers included the condition in which kids were asked to think about themselves in the third person?

e) Notice that almost all of the headlines and twitter comments about this study have focused on Batman. Even the researchers call it "The Batman Effect" Is that accurate?

f) Finally, think about the fact that in the Batman condition, kids not only got to pretend to be a character. They also got to make an important choice about their participation in the study (the choice among the four different options of Batman, Rapunzel, Bob the Builder, and Dora). The kids in the self-immersed and third-person conditions did not make any choices. What kind of problem might this be in the study? (Which one of the four big validities does it address?)

g) Can the study really support the claim that "Pretending to be Batman helps kids stay on task"? Apply the three causal criteria, paying special attention to the point raised in question f), above.

Note to Instructors: If you include the results for the 6 year olds, you can also teach this as an 2x3 IVxPV design, using age (4 vs. 6 year olds) as the participant variable. Here are the full results:

The six-year-olds spent more time on task than the four-year-olds (half the time versus about a quarter of the time). No surprise there. But across age groups, and apparently unrelated to their personal scores on mental control, memory, or empathy, those in the Batman condition spent the most time on task (about 55 per cent for the six-year-olds; about 32 per cent for the four-year-olds). The children in the self-immersed condition spent the least time on task (about 35 per cent of the time for the six-year-olds; just over 20 per cent for the four-year-olds) and those in the third-person condition performed in between.

11/10/2017

How do you feel about the photo to the left? As for me, I hate looking at it, so I made it as small as possible! I am not snake-phobic, but like many other humans, I'd much rather pick up a rabbit or even touch a bear than pick up a snake. But where'd I get that fear? To what extent is humans' fear of certain creatures--like snakes or spiders--present when we are born?

Several groups of researchers have been exploring this question in humans and other primates. The results of one study have been covered by National Geographic's news site. Read this description of the study by the journalist:

Forty-eight six-month-old infants were tested at the institute to analyze how they reacted to images the researchers predicted might be frightening. While sitting on their parents' laps, infants were shown images of spiders and snakes on white backgrounds for five seconds. To prevent parents from inadvertently influencing their infants' reactions, they were given opaque sunglasses during the experiment that prevented them from viewing whatever image was shown.

...When the babies saw pictures of the snakes and spiders, they consistently reacted with larger pupils than when they were shown control images of flowers and fish.

a) What seem to be the independent and dependent variables in this experiment? (By the way, why was this an experiment rather than a correlational study?)

b) Based on the description of the study, was this experiment independent groups or within groups? How do you know?

c) Which of the four basic types of experiments was this: Repeated measures? Concurrent measures? Posttest only? Pretest/posttest?

d) Sketch a graph of the results of the study.

e) The parents were given opaque glasses to wear while holding their babies. Which of the four big validities will this step help to improve?

Now, let's focus a bit on what it means when a baby's pupils dilate. Pupil dilation was the operationalization (the operational definition) of the study's dependent variable. But what construct does pupil dilation supposedly represent? Given the story's headline, you might think the construct is "fear." But it's probably more complicated than that. Read on:

.... dilated pupils are associated with activity in the noradrenergic system in the brain, the same system that processes stress. Closely measuring changes in pupil size has been used in previous studies to determine a variety of mental and emotional stress in adults.

But the journalist also noted:

...it's difficult to characterize the exact nature of the type of stress infants experienced, but dilated pupils show heightened states of arousal and mental processing. Rather than indicating fear in particular, the study says this shows an intense focus.

f) Based on this description, what are some of the candidates for the construct measured by pupil dilation? (You can also read even more correlates of pupil dilation here).

10/10/2017

Most people's advice for success in life? Try again after failure. It almost always pays off to try again, work on a new strategy, or think through things differently. But how do people acquire the motivation to keep trying after they've hit a snag? One answer might be through social modeling: watching others around us who have succeeded after retrying.

Science journalist Ed Yong describes a study in which 1-year-old babies played with an adult under two conditions. One of the adult models performed two simple tasks easily, and the other succeeded at the same tasks only after failing multiple times. A team researchers led by MIT graduate student Julia Leonard conducted the study with 103 infants who visited a children's museum:

As the babies watched, Leonard tried to retrieve a toy from a container, and detach some keys from a carabiner, narrating her efforts along the way. In front of some babies, she succeeded at each task immediately, performing each three times in the span of 30 seconds. In front of others, she spent the same period struggling, and only retrieved the toy and keys just before the time ran out.

What happened next?

“Now it’s your turn to play with a toy,” she said to the infants. She then handed them a music box that she had already activated. The box came with a large, conspicuous, and completely useless button. Pressing it did nothing, but it was the act of pressing that mattered. Leonard found that babies who had seen her struggling with her own objects prodded the button more often than those who had seen her succeed effortlessly.

First some questions about the study:

a) Is this study experimental or correlational? (and why?) What are the independent and dependent variables?

b) If you had to guess, would you say this study was between subjects or within subjects?

c) How long do you think it might have taken for the researcher who conducted this study to get over 100 babies to participate?

Now that you've considered question c above, you'll have a greater appreciation for what happened next: The graduate student, Julia Leonard, was asked by her advisor to conduct the whole study all over again! As you read this next quoted passage, look for themes introduced in Chapter 14:

Her results came in just as psychologists were starting to grapple with their reproducibility crisis—a deep concern that many of the results in published papers might be unreliable due to poorly-designed studies and sloppy practices. To weed out such results, many psychologists have said that their field should put more emphasis on replication—repeating studies to check if their findings hold up. Others believe that more experiments should be preregistered—that is, scientists should specify their research plans ahead of time. [...]

So after Leonard had spent a year studying the value of persistence, her advisor Laura Schulz told her to do the experiment again. “It was a very meta moment,” she says. She recruited another 120 infants, and she preregistered her plans. And to her delight, she got exactly the same results.

Review the information in Chapter 14 and consider these questions:

d) Why might it have been important for Leonard to conduct a replication of her original study? Give three reasons why replication is important.

e) Did Leonard conduct a direct replication, a conceptual replication, or a replication-plus-extension study?

f) Would it have been better for the replication to have been conducted by a different scientist? Why or why not?

g) Not all replication studies are preregistered, but this one was. What are two of the main benefits of preregistering a study--either a replication study or an original study?

04/20/2017

What resume cues did the researchers use to manipulate gender? Social class? Photo: Andrey Popov/Shutterstock

The Harvard Business Review described a study that tested biases among people who read internship applications. Not just any internships, either: These are internships at prestigious law firms. The HBR reports that these internships..

... open doors to even more lucrative employment in the private sector as well as prestigious judiciary and government roles. For these reasons, employment in top law firms has been called the legal profession’s 1%.

Here's a preview of the design of the study:

Now imagine four applicants, all of whom attend the same, selective second-tier law school. They all have phenomenal grade point averages, are on law review, and have identical, highly relevant work experiences. The only differences are whether they are male or female and if their extracurricular activities suggest they come from a higher-class or lower-class background. Who gets invited to interview?

a) Given this brief description, identify the design of the study. What are the independent variables? What are the levels of each IV? What is the dependent variable? Do you think the IVs were manipulated as independent groups or within groups?

Now, here's some more detail:

We uncovered this through a field experiment with the country’s largest law firms. Specifically, we used a technique — known as the resume audit method — that is widely seen as the gold standard for measuring employment discrimination. This method involves randomly assigning different items to the resumes and sending applications to real employers to see how they affect the probability of being called back for a job interview. All in all, we sent fictitious resumes to 316 offices of 147 top law firms in 14 cities, from candidates who were supposedly trying to land a summer internship position. All applicants were in the top 1% of their class and were on law review, but came from second-tier law schools.

We signaled gender by varying the applicant’s first name (James or Julia).

...to capture the economic component of class, our lower-class applicants received an award for student-athletes on financial aid. To incorporate its educational competent, they listed being a peer tutor for fellow first-generation college students. By contrast, our higher class candidate pursued traditionally upper-class hobbies and sports, such sailing, polo, and classical music, while the lower-class candidate participated in activities with lower financial barriers to entry (e.g., pick-up soccer, track and field team) and those distinctly rejected by higher-class individuals (e.g., country music). But crucially, all educational, academic, and work-related achievements were identical between our four fictitious candidates.

Now here are the results:

Even though all educational and work-related histories were the same, employers overwhelmingly favored the higher-class man. He had a callback rate more than four times of other applicants and received more invitations to interview than all other applicants in our study combined. But most strikingly, he did significantly better than the higher-class woman, whose resume was identical to his, other than the first name.

c) Interrogate the construct validity of the operationalization of social class. Do you think this manipulation was accurate and valid?

d) Using the three criteria for causation, interrogate the internal validity of the study. What controls do you notice that reduce design confounds. Is there any concern about selection effects? Why or why not?

e) What interaction in the results shows a moderator?

f) Using the data provided in the story, do you think that the study showed a main effect for gender? A main effect for social class? How about an interaction? Describe each of these effects.

g) Finally, the researchers dug deeper by conducting a follow-up study to investigate why the law firms were biased. Read this section of the article and decide, What mediating theory does the follow-up study propose?

Why did the higher-class man do so much better than the higher-class woman? To further explore this issue, we conducted a follow-up experiment with a sample of 210 practicing attorneys from around the country. We asked each attorney to evaluate one of the same resumes we used in our field experiment and...rate their candidate on factors proven to influence how favorably people view job candidates but that vary between men and women. These included perceptions of the candidate’s competence, likability, fit with an organization’s culture and clientele, and career commitment.

Just like the employers in our audit study, the attorneys we surveyed favored interviewing the higher-class man above all applicants, including the higher-class woman. This time, though, we were able to understand why. Attorneys viewed higher-class candidates of either gender as being better fits with the culture and clientele of large law firms; lower-class candidates were seen as misfits and rejected. In fact, some attorneys even steered the lower-class candidates to less prestigious and lucrative sectors of legal practice, such as government and nonprofit roles, positions that tend to be more socioeconomically diverse than jobs at top law firms.

Thanks to Dr. Kathleen Lewis of Point Park University for sending this example my way and for writing the questions!

The year 2016 provided multiple references to implicit and explicit racial biases, especially in politics. So you might be wondering, What does it mean to hold "implicit biases?" Why are people biased against some ethnic groups, and what can we do about it?

It turns out there is a strong research tradition concerned with measuring and correcting implicit bias. There's a series of short videos grouped under the title, What, me biased? Each presents a real-world situation relevant to racial bias and discusses a research study.

a) In the opening minute, TV host Heather McGhee poses a theory about how to reduce racism to the caller. What is the theory? How did the researchers use data to test the theory?

b) What was the independent (manipulated) variable in the study? What was the dependent variable?

c) How do you know the study was an experiment? Was it an independent groups or within groups design?

d) Sketch a graph of the result, labelling your axes mindfully.

e) Work through the theory-data cycle: Did the data support Ms. McGhee's theory, or not?

More Resources for Instructors:

There are seven videos in this series, providing other opportunities to practice research methods concepts. For example, students can practice an individualized version of the theory-data cycle (Chapter 1), where Dr. Dolly Chugh discusses the idea of an "audit" in the video, Check Our Bias to Wreck Our Bias.

If you’re a research methods instructor or student and would like us to consider your guest post for everydayresearchmethods.com, please contact Dr. Morling. If, as an instructor, you write your own critical thinking questions to accompany the entry, we will credit you as a guest blogger.