Monday, February 1, 2010

In light of my many posts on obvious and/or stupid research, quite a few of you have written in with that question.

There are several answers, but the most common one is busy work. And I am my own best example.

I am not an academic/research person. I have nothing against those who are, it's just not my thing. One of my career goals was to die unpublished. I didn't ever want my name in any journal, anywhere.

But when I was doing my fellowship, the chairman was of the opinion that it was critically important that everyone get published at some point, regardless of the quality of the research involved. So he came up with an absolutely bullshit project for me. And I was faced with the options of doing it or failing the fellowship.

So I did the project. It was remarkably stupid and pointless. it consisted of me reading through MOUNTAINS of old charts, going back several years, and making notes. For the record, a lot of BS research is done this way. Some poor sucker in training is forced to tediously analyze endless piles of old charts or videos or patient forms or something, to come up with worthless information, under the threat of failing out of their program.

Let's face it. You can get pointless data out of anything: "Our chart review found that people who saw the original release of The Wizard of Oz in 1939 were more likely to have Alzheimer's disease in 2009 then those who'd seen the original release of Star Wars in 1977. This suggests an unidentified risk factor for dementia in seeing MGM films vs. those made by 20th Century Fox."

And these studies are generally cheap to do, because you're already paying the salary of the resident or fellow involved (even cheaper for med students, since they work for brownie points).

And there's always a crappy journal out there, trying to get advertising dollars and willing to publish anything to get readers.

So I found some meaningless data, and at a weekly division meeting I presented it. There were 4 attending physicians and 2 fellows in my subspecialty at the time. 3 of the attendings, and both of the fellows (including me) agreed the paper and it's findings were meaningless drivel.

Unfortunately, the only person who disagreed was the chairman. And since he was editor at the time of some desperate medical journal, he got my paper published there.

To make matters worse, he then got me a poster spot at the annual neurology meeting that year. So I had to go to this meeting, set up a poster with my worthless data on it and then STAND BY IT wearing a badge that identified me as the author.

So for the required 2 hours I stood there, trying to smile at all the big league academics going by. Most looked at my poster and politely didn't say a word. A few gave me sympathetic looks. 3 made comments about how worthless it was (I silently agreed). Only one said something kind.

I left the poster hanging in the meeting hall. I think I was the only person who didn't take theirs home. I assume it's in a landfill by now.

My shitty article got published a few months later, and several intelligent neurologists (who I assume were reading in a hot tub) found my paper to be such absolute garbage that they felt the need to write to the journal to complain. And the journal editor, my chairman, forwarded the letters to me to write a rebuttal.

How do you defend the indefensible? Hell, I agreed with them.

But by this point I'd completed the fellowship, and was an attending physician. And I didn't care. So I just tossed his requests in the trash.

So my sole contribution to the medical literature is out there. Fortunately, as the years go by, it will continually be buried under newer (though equally worthless) data.

And that's where at least some crappy research comes from. And I suspect most of it has similar origins. Some poor sap who's under pressure to publish something, anything, regardless of how stupid or obvious it is, or people trying to pad their resume, or someone with way too much time on their hands and absolutely no life (if you're in the last category, get a dog. Or join Facebook. Or do ANYTHING to waste your time in a more useful way), and crappy journals willing to publish anything.

20 comments:

My med school has a 3 month long "scholarly pursuit" program that really lasts about a year. Students pass their "positions" with relevant faculty on to the next year's suckers after they match.

It's brilliant really. My institution gets to say that they support students doing research, and look at this unique program they have! And 100 faculty members get a med student/slave to make do their research for them (or else).

The master's program affiliated with my department does something similar for its fellows. I won't go into the particulars here, but suffice to say, nobody really cares if the fellows do any research after they leave.

Anyway, that's my rant for the morning. I actually like doing research, and hopefully not all of it will be pointless when I'm done!

I earned a master's degree in art history and my thesis sought to explain that a certain type of ancient Greek jewelry was primarily worn by upper-class prostitutes. And you know something? I actually feel like I contributed to the world's knowledge - I really did study a topic no one had worked on before, and it was kind of a controversial study, too. As pitifully as people portray art historians, you gotta give us credit for doing more original work than some doctors are apparently forced into. Sorry, Dr. Grumpy!

I seldom read the minor medical journals, for much the same reason. Most of the time, the people writing them (and apparently, the reviewers), aren't scientists at all, and don't understand scientific method, statistics, or anything else. You see lots of case studies and stuff like that, which boil down to mere anecdotes.

On the other hand, even very worthwhile science can often be difficult to publish. Reviewers tend to be very demanding. I review for a few different journals, and I'd say that overall about 20% or less of the articles I've referreed were ultimately accepted (usually, there are 3 reviewers). With that, I'd say most of the manuscripts described worthwhile studies, it's just that there were flaws in either the design or interpretation of the science.

There are of course also some journals of last resort for the more basic sciences just like there is for medical journals. Sure, you could publish almost anything is some of those, but you could never show your face in decent company again.

There remains the possibility, no matter how slim, that someone doing one of these research projects may accidentally have a profound insight. Such an insight would be ignored a silliness by the publisher of learned papers.

It is never too late to write a rebuttal of your own article. That should count as canceling out your original paper. If you really want to make a screed attacking lazy research, given your field you could target it at one of the higher tier philosophy journals like Mind or Nous.

SO, then a question. Responsible docs, it seems to me, continue to read to stay up on research pertinent to their practice. If these are crap (and you've got to know just from reading summaries) do you still read them in case there's some small nugget of something useful to you? Or are they invariable pretty useless? Inquiring minds

Ah ha! I suspected as much. I confess that when I do research, I do the same thing.

In those journals that offer a summary. :/ My hobby field's journals most frequently do NOT include summaries so potentially fascinating articles turn out to be duds with no warning. It's pretty frustrating.

What a waste -- your advisor could have you doing something real to contribute to the field.

It reminds me of my own profession in the professor world: loads of people putting together crap so that they can add another publication line to their vitas and get tenure out of it. Journals that no one reads and that exist solely so people can have that line on their vitas. One of my colleagues recently published a literary analysis of spam e-mails.

In my Master's program, we did program evaluations for our local non-profits. My group did a really good program evaluation and we could of published our findings However, it was still alone the lines of ...if someone takes a class in Topic A, they would understand More about topic A. Kind of a stupid thing to publish, but is exactly what a program evaluation is....

Generally, research projects our PharmD was assigned were presented in program-ending poster presentations, not aimed at getting anything 'published' deal, but practical, useful information. We spent enough time in the fourth year drug information rotation analysing crap studies put out by drug companies, that some of us approached anything where researchers reported company affiliation as automatically suspect.

Thanks for the honesty. There's some real stuff out there, but the take-home message is that we cannot believe everything in print, even published in supposedly reputable journals (JAMA, JPharmSci, Lancet, etc.), never mind specialty publishers. Heck, I've even seen stuff that people read like a Bible that have misspelled words, incorrect grammar, never mind the statistics.

Welcome to my whining!

This blog is entirely for entertainment purposes. All posts about patients may be fictional, or be my experience, or were submitted by a reader, or any combination of the above. Factual statements may or may not be accurate.

Singing Foo!

Have Dr. Grumpy delivered automatically to your Kindle for only 99 cents a month! Sign up here!

Dr. Grumpy is for hire! Need an article written (humorous, medical, or otherwise) or want to commission a genuine Grumpy piece for your newspaper/magazine/toilet paper roll? Contact me to discuss subjects. You can reach me at the email address below.

Note: I do not answer medical questions. If you are having a medical issue, see your own doctor. For all you know I'm really a Mongolian yak herder and have no medical training at all except in issues regarding the care and feeding of Mongolian yaks.