Chapter 04; Ethical Guidelines for Psychology Research

02/10/2017

The death of a fellow officer on duty was rated as one of the most stressful events for police officers. Credit: W. Smith/Epa/REX/Shutterstock

What happens when people are exposed to very stressful events? A study has investigated this question using a sample of police officers. The study found a correlation between exposure to stressful job events and cortisol change over time. This is a journalist's take on the study.

Here's an overview :

For most people, cortisol, the vital hormone that controls stress, increases when they wake up. It's the body's way of preparing us for the day....[Now, a] study of more than 300 members of the Buffalo Police Department suggests that police events or conditions considered highly stressful by the officers may be associated with disturbances of the normal awakening cortisol pattern. That can leave the officers vulnerable to disease, particularly cardiovascular disease, which already affects a large number of officers.

The study's two main variables were the experience of major stress and cortisol patterns. First, read how they measured stress in the sample:

For this study, participating officers assessed a variety of on-the-job stressors using a questionnaire that asks officers to rate 60 police-related events with a "stress rating." Events perceived as very stressful are assigned a higher rating.

Exposure to battered or dead children ranked as the most stressful event, followed by: killing someone in the line of duty; having a fellow officer killed on duty; a situation requiring the use of force; and being physically attacked.

Identifying the five most intense stressors police can face was significant, Violanti said. "When we talk about interventions to help prevent disease, it's tricky because these stressors are things that can't be prevented," he said. ... The survey showed that the officers experienced one of the five major stressors, on average, 2.4 times during the month before the survey was completed.

Second, read how they measured cortisol patterns. Notice how in this case, the variable is operationalized not as a single outcome, but as a pattern over four time periods:

Cortisol was measured using saliva samples taken upon waking up, and 15, 30 and 45 minutes thereafter.

Here's how the journalist described and interpreted the result:

Officers who weren't as stressed showed a steep and steady, or regular, increase in cortisol from baseline. However, officers with a moderate and high major stress index had a blunted response over time.

That's because stress affects a system in the body known as the hypothalamic pituitary adrenal axis, or HPA Axis. When you're stressed, the HPA Axis elicits cortisol, a hormone that gets the body going and activates against the stressor, Violanti explained. Under normal circumstances, the body's cortisol pattern looks like a normal bell curve: It rises when we wake up, peaks around midday and comes back down at bed time.

"If you experience chronic stress or high stress situations, the cortisol can no longer adjust normally like this. So what happens with people under a lot of stress, the cortisol flattens out. For some people it goes down and others it goes up and stays up. That's called the dysregulation of the HPA axis," said Violanti, who served with the New York State Police for 23 years before shifting into academia.

Questions to answer:

Draw a well-labeled figure depicting the study's main result. (Will it be a bar graph, line graph, or a scatterplot? What are the best labels?)

What makes this a correlational study?

Evaluate the construct validity of a) the operationalization of stress and b) the operationalization of cortisol patterns. In your opinion, how well did they measure these two variables?

Consider the external validity of this study. What are the characteristics of the Can we assume that the results will generalize to other cops? Do we know if the results generalize to other professions or people who have experienced stress? Why or why not?

Can the study support the causal claim that "exposure to stress causes cortisol dysregulation in cops?" Consider temporal precedence (the directionality problem) as well as internal validity (the third variable problem).

Question 4 is about external validity. You might be interested to read the authors' take on this question:

While the current study focused on Buffalo officers, the findings have implications for cops around the country, said paper co-author Michael Andrew, PhD, chief of the Biostatistics and Epidemiology Branch of the CDC/NIOSH Health Effects Laboratory Division in Morgantown, West Virginia.

"These findings show that exposure to major events inherent to police work may lead to a temporary reduction in the biological ability to respond to further stressful events. Since the major stressor events in this study were originally developed to reflect events that can apply to any police department, these results should generalize, more or less, to any police department in the U.S.," Andrew said, adding, "This points to the need for continued focus on supporting police officer health."

For Question 5, you already know that this study establishes covariance. However, temporal precedence is not very clear. It's possible that cops with poor cortisol regulation are more likely to be involved in future stressful events (for some reason). Internal validity is more of a problem, because, at least based on what's presented here, we don't know if they controlled for third variables such as what type of neighborhood the cops usually patrolled, or for personality characteristics such as impulsiveness, Type A personality, or other traits. For example, an impulsive personality might be associated with more stressors on the job, and might also be associated with cortisol patterns.

12/10/2015

Why might it be ethical to assign medical residents to 30 hour shifts, in the name of research? Photo: Shutterstock

How many hours in a row should we expect physicians to work in a single shift? Is a 30 hour shift too long? Although residency programs now limit shifts to 16-hours, it used to be the case that doctors (that is, medical residents) could work shifts that were up to 30 hours long. It might seem obvious that 16 hour shifts would be better for both doctors and patients. However, the evidence is mixed. In addition, some of the participants in a current, ongoing study on this topic are questioning its ethics.

This NPR report (which you can either read or listen t0) introduces the issues and a current study:

The study compares the current rules, which limit first-year residents to working no more than 16 hours without a break, with a more flexible schedule that could allow the young doctors to work up to 30 hours. Researchers will examine whether more mistakes happen on one schedule or the other and whether the residents learn more one way or the other.

Longer shifts may actually be safer than shorter shifts, he says, because shorter shifts can lead to more night shifts, which can be even more fatiguing than long shifts. And, Asch says, mistakes often occur when residents have to hand off their patients to other doctors because a rigid schedule forces them to go home. "The new doctor taking care of you never knows you as well as the doctor who was taking care of you before," Asch says. Information handoffs are "like in a relay race, where the baton gets dropped between two runners. And it's known as a critical point where medical errors are common."

Now let's look at how people are talking about the ethics of the study. Here's one comment:

The groups argue the studies put patients and residents at risk. Sleep-deprived residents are more likely to injure themselves while doing procedures such as drawing blood, inserting intravenous lines or suturing wounds, says Michael Carome, who heads the health research group. The accidents could lead to infections with viruses such as hepatitis and HIV, he says. And tired residents tend to get into more car accidents after work.

a) Which of the three Belmont Principles does the statement above mostly address: Respect for Persons? Beneficence? or Justice?

Here's a different ethical concern:

"Patients are not being informed at all, so the patients are completely unwitting subjects of this research," Carome says. "The residents are aware that they are in this trial, but they have no choice to participate unless they want to leave the residency training program."

That's the case for David Harari, a first-year medical resident at the University of Washington Medical Center in Seattle. He knew his hours would be grueling but says he was shocked when he discovered he may have to work 30 hours straight. "I can't see how anyone could work optimally with such little sleep," Harari says. "It's extremely difficult to stay awake, stay alert and function optimally at that level." Harari was offended by the fact he had no choice in the matter. "Being asked to partake in a study in which I never provided informed consent felt extremely unethical and really uncomfortable," Harari says.

b) Which of the three Belmont Principles does the above statement mostly address: Respect for Persons? Beneficence? or Justice?

Now, read how some ethicists are responding to these concerns. The following statements address each of the concerns described above:

Asch also maintains that it isn't practical to get consent from all the doctors and patients in the study. Some independent bioethicists agree.

"They're asking important questions that we need to answer in order to create competent doctors, and I think they're doing it in an ethically sound manner," says Mildred Solomon, who heads the Hastings Center, a bioethics think tank.

Solomon's point is a good example of how we weigh both the costs to participants and the benefits to society, when we are evaluating the beneficence of a study. What do you think? Read the full NPR report here.

09/20/2015

Replication is the act of conducting a study again. If the study gets the same result twice (or more), we say that the study's findings are replicable. Chapter 14 in the book describes three types of replications: Direct replication, conceptual replication, and replication-plus-extension. This blog post is about direct replications.

In the last few years, psychologists have been focused a lot on the replicability of the field's major findings. The effort came to some fruition last month, when the results of a large-scale project, "The Reproducibility Project" were published in the journal Science. Here's how Smithsonian Magazine introduced the topic:

Academic journals and the press regularly serve up fresh helpings of fascinating psychological research findings. But how many of those experiments would produce the same results a second time around?

According to work presented today in Science, fewer than half of 100 studies published in 2008 in three top psychology journals could be replicated successfully. The international effort included 270 scientists who re-ran other people's studies as part of The Reproducibility Project: Psychology, led by Brian Nosek of the University of Virginia.

That's the first, majorpart of the story--the outcome that 60% of the 100 studies could not replicate when an independent group of researchers conducted direct replications of them.

The next part of the story is about what this outcome means. Stop for a moment and consider what it means if the majority of these 100 studies cannot be replicated. What explanations can you come up with?

One interpretation, given that some social scientists have been in the news for fabricating results (such as in this example or this example, is thatthe non-replicable findings were fabricated (did you assume this?). However, most scientists believe that fabrication is rare, and they stress that if a finding was not replicated, that does not mean it was fraudulent.

Another interpretation of this outcome is that psychology is "broken"--that it is in crisis. One version of that argument says that people will not trust certain findings or subfields any more, and that psychologists who do the unreplicable studies will have trouble getting jobs. For one version of this argument, see this story.

Finally, though, most scientists who've reflected on the finding that 60% of the studies did not replicate, have reached the conclusion that it's a typical stage in the scientific process. Here's how the Smithsonian story described it:

The eye-opening results don't necessarily mean that those original findings were incorrect or that the scientific process is flawed. When one study finds an effect that a second study can't replicate, there are several possible reasons, says co-author Cody Christopherson of Southern Oregon University. Study A's result may be false, or Study B's results may be false—or there may be some subtle differences in the way the two studies were conducted that impacted the results.

In fact, the idea that there are subtle differences between the studies was also elaborated on by Dr. Lisa Feldman Barrett in an editorial:

...the failure to replicate is not a cause for alarm; in fact, it is a normal part of how science works. [Sometimes] study A produces the predicted phenomenon, whereas Study B does not. We have a failure to replicate. Does this mean that the phenomenon in question is necessarily illusory? Absolutely not. If the studies were well designed and executed, it is more likely that the phenomenon from Study A is true only under certain conditions. The scientist’s job now is to figure out what those conditions are, in order to form new and better hypotheses to test.

Later in her editorial, Feldman-Barrett explains the importance of context in scientific findings:

Much of science still assumes that phenomena can be explained with universal laws and therefore context should not matter. But this is not how the world works. Even a simple statement like “the sky is blue” is true only at particular times of day, depending on the mix of molecules in the air as they reflect and scatter light, and on the viewer’s experience of color.

Why is the reproducibility project considered news at all, you might ask? Don't psychologists routinely make sure their results are replicable by conducting direct replications of every result? Here's an answer to that question, by author Cody Christopherson:

Unfortunately there are disincentives to pursuing this kind of research, he says: “To get hired and promoted in academia, you must publish original research, so direct replications are rarer. I hope going forward that the universities and funding agencies responsible for incentivizing this research—and the media outlets covering them—will realize that they've been part of the problem, and that devaluing replication in this way has created a less stable literature than we'd like.”

a. What kinds of policies and incentives might work to make direct replications more frequent and common?

b. Where do moderators come in here? Re-read Dr. Feldman Barrett's argument above, and see if you can detect the concept of moderators in her reasoning.

06/18/2015

Yet another case of fabricated data, this time from political science, is being analyzed in the news lately. Last December, Science published an article showing that certain types of canvassing techniques could dramatically change people's attitudes toward marriage equality. However, the data for the study were apparently fabricated by one of its key authors. Here are two good pieces that cover the timeline of the article, its discovery, and its retraction:

07/20/2014

July is turning out to be Ethics Month at EverydayResearchMethods blog. Here's a great piece about the damage that was done by a case of data fabrication.

In 1998, a physician named Dr. Andrew Wakefield published a study on 12 sample of patients whose autism symptoms, he claimed, had appeared right after receiving the vaccine for measles mumps, and rubella (MMR). At the time, the study was criticized because it had some methodological problems:

Researchers subsequently sought to rigorously test the study's main, disturbing claim: that vaccines cause autism. And every study that tested this claim with rigorous methods and large samples found absolutely no connection between the two.

Even after the journal Lancet retracted Wakefield's paper, saying that Wakefield had fabricated the results of his original study, and even after all other studies showed a null result, a small percentage parents still refuse to vaccinate and have fears of vaccination. When parents refuse to vaccinate their healthy children, it makes it more likely that diseases will spread in a community, potentially harming people who are not healthy enough to receive vaccinations.

a) One lesson from this story is about how people come to believe what they believe. Chapter 2 explains that some people base their beliefs on intuition, authority, or experience. But good research is the only basis for belief that is truly data-driven, and free of bias.

Reflect here on the impact that both good and bad science seems to have on public opinion. Why do you think Wakefield's original study was so influential to some parents? Why are some parents so readily convinced that vaccines are dangerous? And why do you think the subsequent studies, all of which had null results, seem to have had so little influence on these same parents?

b) This story also illustrates why data fabrication is so harmful. It not only harms the progress of science and harms people's trust in science. In this case, fabricated data also led thousands of parents to put their children, and other people's children, in harm's way by not vaccinating them. What might scientists and physicians do to repair the harm caused by this falsified study?

07/10/2014

Recently, Internet commentators have been discussing the ethics of an experiment conducted on Facebook. The paper was published this month in the prestigious journal Proceedings of the National Academy of Sciences (PNAS). You can read one account of the ethics of this study in Slate, here, and read other versions all over the Internet, including here, here, and here. You can read the abstract of the study, which is open to public comment, here.

What was the study about? In this experiment, the researchers used a sample of people who use Facebook. They randomly assigned the participants to one of three conditions. In one condition, the researchers arranged for a certain percentage of posts that contained positive emotional content to be prevented from appearing on the person's News Feed. In a second condition, they arranged for posts that contained negative emotional content to be prevented from appearing on the News Feed. And in a control condition, they arranged for a random selection of posts (that is, regardless of emotional content) to be prevented from appearing on the News Feed.

The dependent measure was the positivity and negativity of words that participants posted on their Status Updates. Specifically, the researchers used a computer program to count the positive and negative words (such as "happy" or "angry") that participants used after their News Feed was manipulated. The results showed that after having positive content removed from their news feeds, participants used more negative words and fewer positive words on their Status Updates compared to the control group. And after having negative content removed from their News Feeds, participants used more positive words and fewer negative words on their Status updates.

What ethical problems are people raising about this study? One issue that people have raised is informed consent. The study used 689,003 people who had a Facebook account in English. The researchers did not ask the participants to read or sign an informed consent form. According to Slate's article:

“If you are exposing people to something that causes changes in psychological status, that’s experimentation,” says James Grimmelmann, a professor of technology and the law at the University of Maryland. “This is the kind of thing that would require informed consent.”

Ah, informed consent. Here is the only mention of “informed consent” in the paper: The research “was consistent with Facebook’s Data Use Policy, to which all users agree prior to creating an account on Facebook, constituting informed consent for this research.”

a) What does informed consent typically involve in research? Do you think that this statement in the PNAS article allows people full informed consent?

Another issue in the Facebook experiment seems to fall under the possible harm done to the participants. According to the results, people who were randomly assigned to have positive emotion words removed from their news feeds used fewer positive emotions in their status updates--as Slate put it,

Facebook intentionally made thousands upon thousands of people sad.

Other commentators, however, correctly noted that the effect size of the result was very small: While the results were statistically significant, the largest effect size they reported was d = 0.02.

b) What do you think? What's your evaluation of the beneficence of this research? Was true harm done to participants?

Finally, other commentators, such as this one, have commented on the fact that Facebook manipulates people's content all the time. For example, Facebook shows people different advertisements depending on their interests, and shows people content from some of their friends more than others. This writer noted:

So the folks who are outraged about Facebook’s complicity in this experiment seem to basically be arguing that it’s okay when Facebook manipulates their emotions to get them to click on stuff more, or for the sake of in-house experiments about how to make content “more engaging” (that is, to find out how to get them to click on stuff more), but not when that manipulation is done in service of a psychological experiment.

c) What's your view? When, if ever, is it ethical for a commercial site like Facebook to manipulate content and measure your (clicking) behavior, and when, if ever, is it not ethical?

07/13/2012

Two more reports of research fraud have surfaced in the subfield of social psychology. You can read about two new suspected cases in this news piece by the journal Nature.

Read the article and think about these questions:

a) Are the cases described here examples of data falisification or data fabrication? (Is this possible to determine from the report?)

b) How did the UPenn researcher, Dr. Uri Simonsohn, use statistical techniques to detect suspicious data?

c) What does this mean for the reputation of social psychology ? Will the public come to mistrust psychological research? What are the costs and benefits to Dr. Simonsohn's investigation of others' data?

a.) Data falsification and fabrication are two slightly different problems. How are they different? (see Chapter 4) Which one is discussed in this story?

b.) Reflect on why falsifying and fabricating data count as ethical violations. Who is harmed by these actions?

c.) The article discusses how the falsification case hurts psychology's public reputation as a science. What do you think? Do you think that stories like these will damage psychology's public reputation?

d.) Data falsification also damages the process of science itself. Reflect on how falsifying or fabricating data affects the theory-data cycle, the most important one in science.

e.) After you read this article, think of changes that the field could make to the peer review or writing process that would make it less likely that psychologists would fabricate or falsify data.

If you’re a research methods instructor or student and would like us to consider your guest post for everydayresearchmethods.com, please contact Dr. Morling. If, as an instructor, you write your own critical thinking questions to accompany the entry, we will credit you as a guest blogger.