Ethics

12/10/2015

Why might it be ethical to assign medical residents to 30 hour shifts, in the name of research? Photo: Shutterstock

How many hours in a row should we expect physicians to work in a single shift? Is a 30 hour shift too long? Although residency programs now limit shifts to 16-hours, it used to be the case that doctors (that is, medical residents) could work shifts that were up to 30 hours long. It might seem obvious that 16 hour shifts would be better for both doctors and patients. However, the evidence is mixed. In addition, some of the participants in a current, ongoing study on this topic are questioning its ethics.

This NPR report (which you can either read or listen t0) introduces the issues and a current study:

The study compares the current rules, which limit first-year residents to working no more than 16 hours without a break, with a more flexible schedule that could allow the young doctors to work up to 30 hours. Researchers will examine whether more mistakes happen on one schedule or the other and whether the residents learn more one way or the other.

Longer shifts may actually be safer than shorter shifts, he says, because shorter shifts can lead to more night shifts, which can be even more fatiguing than long shifts. And, Asch says, mistakes often occur when residents have to hand off their patients to other doctors because a rigid schedule forces them to go home. "The new doctor taking care of you never knows you as well as the doctor who was taking care of you before," Asch says. Information handoffs are "like in a relay race, where the baton gets dropped between two runners. And it's known as a critical point where medical errors are common."

Now let's look at how people are talking about the ethics of the study. Here's one comment:

The groups argue the studies put patients and residents at risk. Sleep-deprived residents are more likely to injure themselves while doing procedures such as drawing blood, inserting intravenous lines or suturing wounds, says Michael Carome, who heads the health research group. The accidents could lead to infections with viruses such as hepatitis and HIV, he says. And tired residents tend to get into more car accidents after work.

a) Which of the three Belmont Principles does the statement above mostly address: Respect for Persons? Beneficence? or Justice?

Here's a different ethical concern:

"Patients are not being informed at all, so the patients are completely unwitting subjects of this research," Carome says. "The residents are aware that they are in this trial, but they have no choice to participate unless they want to leave the residency training program."

That's the case for David Harari, a first-year medical resident at the University of Washington Medical Center in Seattle. He knew his hours would be grueling but says he was shocked when he discovered he may have to work 30 hours straight. "I can't see how anyone could work optimally with such little sleep," Harari says. "It's extremely difficult to stay awake, stay alert and function optimally at that level." Harari was offended by the fact he had no choice in the matter. "Being asked to partake in a study in which I never provided informed consent felt extremely unethical and really uncomfortable," Harari says.

b) Which of the three Belmont Principles does the above statement mostly address: Respect for Persons? Beneficence? or Justice?

Now, read how some ethicists are responding to these concerns. The following statements address each of the concerns described above:

Asch also maintains that it isn't practical to get consent from all the doctors and patients in the study. Some independent bioethicists agree.

"They're asking important questions that we need to answer in order to create competent doctors, and I think they're doing it in an ethically sound manner," says Mildred Solomon, who heads the Hastings Center, a bioethics think tank.

Solomon's point is a good example of how we weigh both the costs to participants and the benefits to society, when we are evaluating the beneficence of a study. What do you think? Read the full NPR report here.

09/20/2015

Replication is the act of conducting a study again. If the study gets the same result twice (or more), we say that the study's findings are replicable. Chapter 14 in the book describes three types of replications: Direct replication, conceptual replication, and replication-plus-extension. This blog post is about direct replications.

In the last few years, psychologists have been focused a lot on the replicability of the field's major findings. The effort came to some fruition last month, when the results of a large-scale project, "The Reproducibility Project" were published in the journal Science. Here's how Smithsonian Magazine introduced the topic:

Academic journals and the press regularly serve up fresh helpings of fascinating psychological research findings. But how many of those experiments would produce the same results a second time around?

According to work presented today in Science, fewer than half of 100 studies published in 2008 in three top psychology journals could be replicated successfully. The international effort included 270 scientists who re-ran other people's studies as part of The Reproducibility Project: Psychology, led by Brian Nosek of the University of Virginia.

That's the first, majorpart of the story--the outcome that 60% of the 100 studies could not replicate when an independent group of researchers conducted direct replications of them.

The next part of the story is about what this outcome means. Stop for a moment and consider what it means if the majority of these 100 studies cannot be replicated. What explanations can you come up with?

One interpretation, given that some social scientists have been in the news for fabricating results (such as in this example or this example, is thatthe non-replicable findings were fabricated (did you assume this?). However, most scientists believe that fabrication is rare, and they stress that if a finding was not replicated, that does not mean it was fraudulent.

Another interpretation of this outcome is that psychology is "broken"--that it is in crisis. One version of that argument says that people will not trust certain findings or subfields any more, and that psychologists who do the unreplicable studies will have trouble getting jobs. For one version of this argument, see this story.

Finally, though, most scientists who've reflected on the finding that 60% of the studies did not replicate, have reached the conclusion that it's a typical stage in the scientific process. Here's how the Smithsonian story described it:

The eye-opening results don't necessarily mean that those original findings were incorrect or that the scientific process is flawed. When one study finds an effect that a second study can't replicate, there are several possible reasons, says co-author Cody Christopherson of Southern Oregon University. Study A's result may be false, or Study B's results may be false—or there may be some subtle differences in the way the two studies were conducted that impacted the results.

In fact, the idea that there are subtle differences between the studies was also elaborated on by Dr. Lisa Feldman Barrett in an editorial:

...the failure to replicate is not a cause for alarm; in fact, it is a normal part of how science works. [Sometimes] study A produces the predicted phenomenon, whereas Study B does not. We have a failure to replicate. Does this mean that the phenomenon in question is necessarily illusory? Absolutely not. If the studies were well designed and executed, it is more likely that the phenomenon from Study A is true only under certain conditions. The scientist’s job now is to figure out what those conditions are, in order to form new and better hypotheses to test.

Later in her editorial, Feldman-Barrett explains the importance of context in scientific findings:

Much of science still assumes that phenomena can be explained with universal laws and therefore context should not matter. But this is not how the world works. Even a simple statement like “the sky is blue” is true only at particular times of day, depending on the mix of molecules in the air as they reflect and scatter light, and on the viewer’s experience of color.

Why is the reproducibility project considered news at all, you might ask? Don't psychologists routinely make sure their results are replicable by conducting direct replications of every result? Here's an answer to that question, by author Cody Christopherson:

Unfortunately there are disincentives to pursuing this kind of research, he says: “To get hired and promoted in academia, you must publish original research, so direct replications are rarer. I hope going forward that the universities and funding agencies responsible for incentivizing this research—and the media outlets covering them—will realize that they've been part of the problem, and that devaluing replication in this way has created a less stable literature than we'd like.”

a. What kinds of policies and incentives might work to make direct replications more frequent and common?

b. Where do moderators come in here? Re-read Dr. Feldman Barrett's argument above, and see if you can detect the concept of moderators in her reasoning.

05/20/2015

These Bangladeshi girls received aid from BRAC, the same program that was investigated in this study. Photo: Brent Stirton/Getty Images

Do international aid programs work? Many of us send money to an organization like Heifer International, Mercy Corps, or Oxfam. Does the money these programs invest, either through direct aid, microloans, gifts of livestock, or other investment, improve the financial and psychological well-being of the recipients?

One problem was that no one was actually testing global aid programs — methodically — to see if they really changed people's lives permanently. "They haven't been taking the scientific method to problems of poverty,"

Dean Karlan of Yale University was a lead author on a study on these aid programs. The NPR story describes, first, the problem:

Take, for instance, a charity that gives a family a cow. The charity might check on the family a year later and say, "Wow! The family is doing so much better with this cow. Cows must be the reason."

But maybe it wasn't the cow that improved the family's life. Maybe it had a bumper crop that year or property values went up in the neighborhood.

a) What problem, introduced in Chapter 2, is being described in the passage quoted above?

Next, the NPR story explains Karlan's solution:

So [Karlan] and a bunch of his colleagues had a radical idea: Test aid with the same method doctors use to test drugs (that is, randomized control trials).

The idea is quite simple. Give some families aid but others nothing. Then follow both groups, and see if the aid actually made a difference in the long run.

An anti-poverty program in Bangladesh, called BRAC, looked like it was successful. It seemed to help nearly 400,000 families who were living off less than $1.25 each day. So Karlan and his colleagues wanted to test the program and see if it could work in other countries.

They teamed up with a network of researchers and nonprofits in six developing countries. They went to thousands of communities and found the poorest families.

Then they divided the families into two groups. They gave half the families nothing. And the other half a whole smorgasbord of aid for one to two years.

The focus of the aid was a gift of livestock, which could be used to make money. It also included home visits to provide training for the livestock, a savings account, mental health services, other health services, and food or cash (which prevented them from eating their livestock investment).

What about the results?

The strategy worked pretty well in five of the six countries they tried it in. Families who got the aid started making a little more money, and they had more food to eat.

But here's what sets this study apart from the rest: Families continued to make a bit more money even a year after the aid stopped.

b) This was an experiment. What was the IV in the study?

c) There were more than one DV in this study. What were at least two of the DVs?

Here's another comment about the study:

The effect of the aid was actually quite small, she says. Families' incomes and food consumption together went up by only a small amount — about 5 percent, on average, when compared with the control group.

d) What's the above comment addressing? Which validity?

Here are some comments by scientists about the outcome of the study, and its importance. As you read this, reflect on some of the lessons from Chapter 14 about importance, generalizability, and importance:

"Moving poverty is hard," Baird says. "The fact that they [Karlan and colleagues] were able to move it, and it was sustainable after a year, I think is important."

The findings are a leap forward, she says, because it shows charities and governments a basic strategy that often works.

And even a little bit of extra money can make a huge difference in these peoples' lives, she says. It can help them send their kids to school. Or even just give them a little more hope.

What do you think about the ethics of this approach? Is it fair to withhold aid from families like this? The main author described it like this:

One issue is that some families go home empty-handed, with no aid. So the idea seems unethical. But Karlan disagrees. "The whole point of this is to help more people," he says. "If we find out what works and what doesn't, in five years we can have a much bigger impact."

The above statement reminds us that one foundation of ethics--beneficence--estimates two things. First, it estimates the costs or benefits to a study's participants. But second, it estimates the costs or benefits to our society from the knowledge in the study. Karlan is emphasizing the benefit of this scientific knowledge in his comment.

Suggested answers

a) This is about confounds. When we observe experiences, lots of things change at the same time. In this case, we can't be sure that it's the charity that caused the improvement. Another name for this problem is internal validity.

b) The IV was receiving the package of aid, or not. The NPR story doesn't mention if the participants were randomly assigned to the aid group or control group. But you could look up the answer in the original journal article, linked here in the journal Science, to find out if they were assigned at random or not.

c) One DV appears to be family income. Other DVs apparently include family food input, female power, and mental health and happiness.

d) Statistical validity--specifically, effect size.

Many thanks to my student, Madeline Halkowski, for telling me about this story!

07/20/2014

July is turning out to be Ethics Month at EverydayResearchMethods blog. Here's a great piece about the damage that was done by a case of data fabrication.

In 1998, a physician named Dr. Andrew Wakefield published a study on 12 sample of patients whose autism symptoms, he claimed, had appeared right after receiving the vaccine for measles mumps, and rubella (MMR). At the time, the study was criticized because it had some methodological problems:

Researchers subsequently sought to rigorously test the study's main, disturbing claim: that vaccines cause autism. And every study that tested this claim with rigorous methods and large samples found absolutely no connection between the two.

Even after the journal Lancet retracted Wakefield's paper, saying that Wakefield had fabricated the results of his original study, and even after all other studies showed a null result, a small percentage parents still refuse to vaccinate and have fears of vaccination. When parents refuse to vaccinate their healthy children, it makes it more likely that diseases will spread in a community, potentially harming people who are not healthy enough to receive vaccinations.

a) One lesson from this story is about how people come to believe what they believe. Chapter 2 explains that some people base their beliefs on intuition, authority, or experience. But good research is the only basis for belief that is truly data-driven, and free of bias.

Reflect here on the impact that both good and bad science seems to have on public opinion. Why do you think Wakefield's original study was so influential to some parents? Why are some parents so readily convinced that vaccines are dangerous? And why do you think the subsequent studies, all of which had null results, seem to have had so little influence on these same parents?

b) This story also illustrates why data fabrication is so harmful. It not only harms the progress of science and harms people's trust in science. In this case, fabricated data also led thousands of parents to put their children, and other people's children, in harm's way by not vaccinating them. What might scientists and physicians do to repair the harm caused by this falsified study?

07/10/2014

Recently, Internet commentators have been discussing the ethics of an experiment conducted on Facebook. The paper was published this month in the prestigious journal Proceedings of the National Academy of Sciences (PNAS). You can read one account of the ethics of this study in Slate, here, and read other versions all over the Internet, including here, here, and here. You can read the abstract of the study, which is open to public comment, here.

What was the study about? In this experiment, the researchers used a sample of people who use Facebook. They randomly assigned the participants to one of three conditions. In one condition, the researchers arranged for a certain percentage of posts that contained positive emotional content to be prevented from appearing on the person's News Feed. In a second condition, they arranged for posts that contained negative emotional content to be prevented from appearing on the News Feed. And in a control condition, they arranged for a random selection of posts (that is, regardless of emotional content) to be prevented from appearing on the News Feed.

The dependent measure was the positivity and negativity of words that participants posted on their Status Updates. Specifically, the researchers used a computer program to count the positive and negative words (such as "happy" or "angry") that participants used after their News Feed was manipulated. The results showed that after having positive content removed from their news feeds, participants used more negative words and fewer positive words on their Status Updates compared to the control group. And after having negative content removed from their News Feeds, participants used more positive words and fewer negative words on their Status updates.

What ethical problems are people raising about this study? One issue that people have raised is informed consent. The study used 689,003 people who had a Facebook account in English. The researchers did not ask the participants to read or sign an informed consent form. According to Slate's article:

“If you are exposing people to something that causes changes in psychological status, that’s experimentation,” says James Grimmelmann, a professor of technology and the law at the University of Maryland. “This is the kind of thing that would require informed consent.”

Ah, informed consent. Here is the only mention of “informed consent” in the paper: The research “was consistent with Facebook’s Data Use Policy, to which all users agree prior to creating an account on Facebook, constituting informed consent for this research.”

a) What does informed consent typically involve in research? Do you think that this statement in the PNAS article allows people full informed consent?

Another issue in the Facebook experiment seems to fall under the possible harm done to the participants. According to the results, people who were randomly assigned to have positive emotion words removed from their news feeds used fewer positive emotions in their status updates--as Slate put it,

Facebook intentionally made thousands upon thousands of people sad.

Other commentators, however, correctly noted that the effect size of the result was very small: While the results were statistically significant, the largest effect size they reported was d = 0.02.

b) What do you think? What's your evaluation of the beneficence of this research? Was true harm done to participants?

Finally, other commentators, such as this one, have commented on the fact that Facebook manipulates people's content all the time. For example, Facebook shows people different advertisements depending on their interests, and shows people content from some of their friends more than others. This writer noted:

So the folks who are outraged about Facebook’s complicity in this experiment seem to basically be arguing that it’s okay when Facebook manipulates their emotions to get them to click on stuff more, or for the sake of in-house experiments about how to make content “more engaging” (that is, to find out how to get them to click on stuff more), but not when that manipulation is done in service of a psychological experiment.

c) What's your view? When, if ever, is it ethical for a commercial site like Facebook to manipulate content and measure your (clicking) behavior, and when, if ever, is it not ethical?

07/13/2012

Two more reports of research fraud have surfaced in the subfield of social psychology. You can read about two new suspected cases in this news piece by the journal Nature.

Read the article and think about these questions:

a) Are the cases described here examples of data falisification or data fabrication? (Is this possible to determine from the report?)

b) How did the UPenn researcher, Dr. Uri Simonsohn, use statistical techniques to detect suspicious data?

c) What does this mean for the reputation of social psychology ? Will the public come to mistrust psychological research? What are the costs and benefits to Dr. Simonsohn's investigation of others' data?

a.) Data falsification and fabrication are two slightly different problems. How are they different? (see Chapter 4) Which one is discussed in this story?

b.) Reflect on why falsifying and fabricating data count as ethical violations. Who is harmed by these actions?

c.) The article discusses how the falsification case hurts psychology's public reputation as a science. What do you think? Do you think that stories like these will damage psychology's public reputation?

d.) Data falsification also damages the process of science itself. Reflect on how falsifying or fabricating data affects the theory-data cycle, the most important one in science.

e.) After you read this article, think of changes that the field could make to the peer review or writing process that would make it less likely that psychologists would fabricate or falsify data.

09/29/2011

Here's Ben Goldacre in a 20-minute TED video talking about his favorite topic, Bad Science. He addresses the journal-to-journalism cycle, the flaws of authority, the placebo effect, industry-sponsored trials, publication bias, correlation/causation, ethics, and more. The overall message is the importance of publishing all data--both for and against a drug, therapy, or intervention.

If you’re a research methods instructor or student and would like us to consider your guest post for everydayresearchmethods.com, please contact Dr. Morling. If, as an instructor, you write your own critical thinking questions to accompany the entry, we will credit you as a guest blogger.