Two Good Reasons To Always Read the Methods Section of a Scientific Paper

From the Boing Boing Shop

Popular Posts

Follow Us

Sure, you could skip straight to "Conclusions" and get your soundbite. But if you make a habit of avoiding "Methods and Materials" you will miss out on some classic moments of science humor---both intentional and otherwise.

Do you know how hard it is to find Flickr shots of people reading journal articles? We'll settle for cute, instead. From KOMUnews via CC

In my summer reading, I came across two excellent examples...

report this ad

1) Chicken Soup for the Scientist's Soul (And Lunch)The Paper: "Chicken Soup Inhibits Neutrophil Chemotaxis In Vitro", in the October 2000 issue of the journal Chest. Authors: Barbara O. Rennard, BA, Ronald F. Ertl, BS, Gail L. Gossman, BS, Richard A. Robbins, MD, FCCP, and Stephen I. Rennard, MD, FCCP.
The Discovery: My friend Jeff, Ph.D., who works for a friendly Environmental Sciences department somewhere in the South, sent me a link to this paper, which attempts to establish a medical explanation for why your Grammy was always feeding you chicken soup at the slightest hint of a sniffle. The authors theorize that chicken soup may have some mild anti-inflammatory effects, which could account for its popularity as folk medicine. In fact, according to the paper, the idea of treating respiratory illness with chicken soup dates at least back to the 12th century C.E., when it was recommended by the Egyptian Jewish physician and philosopher Maimonides.
Which is kind of nifty.
And the team does find some evidence supporting their theory, at least in the lab, where they exposed white blood cells called neutrophils to varying concentrations of chicken soup. Neutrophils play an important role in inflammation and higher concentrations of soup seemed to block them from doing this. You can read the whole paper online.

The funny part? They included the actual chicken soup recipe used in the study under "Methods and Materials". Actual quote from the paper:

Traditional chicken soup was prepared according to a family recipe, which will be referred to as "Grandma's soup" (C. Fleischer; personal communication; 1970).

I haven't tried it yet, but I'm planning on making a batch sometime this fall. The paper says the recipe is "very highly regarded locally" (a claim they back up with a citation).

2) Weekend at Bernie'sThe Paper: "Sexual conflict over the duration of copulation in Drosophila
montana: why is longer better?", in the June 12, 2009 issue of BMC Evolutionary Biology. Authors: Dominique Mazzi, Jenni Kesäniemi, Anneli Hoikkala and Kirsten Klappert.
The Discovery: I wrote about this research for the National Geographic News Web site. The basic gist: Male D. montana flies get more of an evolutionary benefit from longer sex. Females, meanwhile, do better with a shorter session. The research proved that the female flies had adapted in ways that allowed them to counter males' ability to hold them still. Turns out, the ladies had more control over the length of sex than anyone previously thought.
In my initial interviews, author Kirsten Klappert explained that the team had "incapacitated" the female flies to see how long sex lasted when lady flies had no control. Naturally, my editor and I were curious as to just how this worked.

Apparently, it works best if you kill the female flies.

Rather than engaging in insect bondage, the Methods section clarified that the team had gassed female flies to death, propped them up so they appeared alive and interested in sex, and tricked the male flies into necrophilia.

More than 800 American energy and Earth science researchers have signed a letter to Donald Trump outlining six steps they’re urging him to take to address human-caused climate change to protect “America’s economy, national security, and public health and safety.” The letter is accompanied by a public change.org petition to “Tell Trump To #ActOnClimate.” Here […]

James Delingpole is an invective-hurling anti-climate science columnist who has candidly admitted that he doesn’t bother to read scientific papers, calling himself a “an interpreter of interpretations.”

Kratom (previously) is a widely used herb that has been very effective in treating opioid withdrawal and other chronic, hard-to-treat conditions — it also became very controversial this year because the DEA decided, without evidence, to class it as a dangerous drug, and then changed its mind (unprecedented!) after a mass-scale petition that included interventions […]

The Black Friday Mac Bundle 2.0 is one of the Boing Boing Store’s best-selling Mac bundles yet, and it’s about to come to an end. If you don’t get your copy now, here’s what you’ll be missing:This bundle comes packing 9 top-rated Mac apps in one package, at the hugely discounted price of just $23.99. […]

The Boing Boing Store’s Gift Guide is full of ideas for pretty much anyone in your life like hipster ice cub trays, Xbox controllers, Halo Boards, and even diamond necklaces. As always, all products in the Boing Boing Store come at great discounts, too. Shop by price bucket starting at under $20. Under $20:Bloxx Jumbo Ice Trays […]

Unlike traditional lighters, the SaberLight features an electronic plasma beam that’s both rechargeable and butane-free. This sleek lighter is even approved by TSA, so you’ll never be stuck buying lighters you’ll just have to throw away partially used. For some people, like me, this is a pretty big game-changer. The SaberLight’s beam is actually both hotter and cleaner […]

25

Hard to find pictures of people reading articles? You just need to know where to look. Allow me to help. The link is to a photo a a former doctoral student (now teaching at Harvard). He’s incorrectly identified as ‘staff’ in the blurb – I’ll have to talk to Classics and get them to fix it.

My favorite reason to read the methodology of a study is to understand to whom the research applies. For instance, the secondhand smoking study that many have been quoting from for years only concerns itself with people that spend at least eight hours a day in a closed environment with a smoker. Of course that hasn’t stopped people from claiming that an errant whiff of secondhand smoke while walking down the street is murderous.

I remember reading in an intro anthropology course, the researchers were living with a isolated South American tribe that raised a few chickens. They kept them for eggs, except when someone came down with a cold, they killed one for soup. Freaked the anthropologists out, till the locals remembered that they’d learned that from some missionaries a ways back.

And here I thought this post was just going to have some cautionary anecdotes about why you shouldn’t accept a paper’s conclusion without a critical look at the methodology. Insect necrophilia is so much more amusing!

@zikman, they do include the actual recipe, including credit for the matzo balls (back of the Manaschewitz box). Sweet potatoes in chicken soup? Never seen that before. I prefer adding a bunch of dill.

Nice. I have images of a cartoon captioned “Waiter, there’s a fly in my chicken soup”, Sinatra singing “Come Fly With Me”, PETA and NOW using these articles as fodder for new campaigns, and grad students laughing as they read journals in the library all racing through my mind…

Zikman, it’s in the fulltext link, but if you don’t want to click, here you go. Enjoy!
-v

Soup

Traditional chicken soup was prepared according to a family recipe, which will be referred to as â€œGrandmaâ€™s soupâ€ (C. Fleischer; personal communication; 1970). This recipe is as follows:

â€¢ 1 5- to 6-lb stewing hen or baking chicken;

â€¢ 1 package of chicken wings;

â€¢ 3 large onions;

â€¢ 1 large sweet potato;

â€¢ 3 parsnips;

â€¢ 2 turnips;

â€¢ 11 to 12 large carrots;

â€¢ 5 to 6 celery stems;

â€¢ 1 bunch of parsley; and

â€¢ salt and pepper to taste.

Clean the chicken, put it in a large pot, and cover it with cold water. Bring the water to a boil. Add the chicken wings, onions, sweet potato, parsnips, turnips, and carrots. Boil about 1.5 h. Remove fat from the surface as it accumulates. Add the parsley and celery. Cook the mixture about 45 min longer. Remove the chicken. The chicken is not used further for the soup. (The meat makes excellent chicken parmesan.) Put the vegetables in a food processor until they are chopped fine or pass through a strainer. Both were performed in the present study. Salt and pepper to taste. (Note: this soup freezes well.) Matzoh balls were prepared according to the recipe on the back of the box of matzoh meal (Manischewitz; Jersey City, NJ).

This comment is just about the picture: YEEEE-HAH! I have two jobs: I work for the Daniel Boone Regional Library (where this photo was taken) and KOMU-TV (them what took it). My two jobs have intersected on BoingBoing! In a picture that has nothing to do with the article! Yay, me!

Hmm. Interesting, but I don’t see how chicken soup is going to stop your neutrophils from chemotaxing unless you inject it intravenously. I’m sure the soup was good, but really, shooting it is going a bit far. I suggest they lay off the chicken and try cold turkey.

I particularly enjoy the methods sections from studies of infants, where they detail how many “subjects” (babies) had to be excluded and for what reasons, which typically include “fussing” or “failure to complete the task” (read: would not let go of mom’s leg).

It both keeps you honest in interpreting how much we can hope to learn from little humans and reminds you how incredible it is that we get any robust data from them at all.

The methods section is the only one I read really thoroughly. I read the abstract, skim the intro/review of lit if I’m interested in doing similar research and think I might get some good leads on other papers to read, then close-read methods, then read the tables in results, and might skim the discussion/conclusion for unfounded assertions.

The reason for all of this is that, in my field anyway, a lot of people set up their experiments wrong, get some poor statistician to bang out some tables the author doesn’t understand, make some ridiculous claims as conclusions, and then get it published in a peer-reviewed journal staffed by people as clueless as them. The methods section is where you can find out if that’s what happened, or if this person actually knows what they’re talking about.

(Applied linguistics.)

Also, as Maggie points out, this is often where you can see the authors slip some of their own personality into the paper, which I do appreciate. Research is fun; why do we have to pretend it’s not?

An additional 10 casts were made using molten zinc or aluminum. The metals were melted in a portable, charcoal-fired kiln provided with a draft from a car heater fan run on a battery. The bottom of a scuba tank served as a crucible. More details can be found in Tschinkel (2005). In contrast to the fragile plaster casts, metal casts of C. socius nests were very strong and never broke, and were effective weapons in barroom fights.