26 April 2017

You’re involved in a research project. You do a lot of work. And then your name appears nowhere on the manuscript or paper in the journal.

Pop quiz, hotshot!

What do you do?

While you think about that, let me talk about practices in another field: screenwriting. I’ve argued that movie credits provide a better model for contemporary science than current authorship practices. How do you determine who wrote a movie? (What follows is based on practices in Hollywood filmmaking, as far as I know. I don’t know if practices differ in, say, Bollywood.)

Like authorship of scientific papers, screenwriting credit is not simple and somewhat cryptic to outsiders. For instance:

“Screenplay by Jeffrey Boam and Jeffrey Boam & Robert Mark Kamen. Story by Jeffrey Boam.” Why is Boam in there twice? Why are the names joined with the word “and” and an ampersand? And how is that different from “Story by”? If you haven’t looked it up, it’s baffling. Research papers play similar games with things like authorship position.

Like research teams, you can have large numbers of people who work on a movie script. Over thirty writers were involved writing in The Flintsones live action movie. But only three names appeared on the screen.

And, just like scientific papers, you can have disputes over credit. And here’s where academic authorship and screenwriting diverge.

Credit for movie scripts can go to arbitration. Usually, the Writer’s Guild of America is the final arbiter. And they have rules for determining who gets credit, although there is wiggle room for interpretation, like what “substantial” means.

In a dispute over an authorship credit for a scientific paper, there is effectively nobody to turn to for help in resolving it. On Twitter, I asked people on journal editorial boards the “Pop quiz, hotshot” question at the start of this post. Someone says they were should have gotten authorship, but didn’t. Or possibly higher placement in authorship. What do you do?

So far, I’ve had more retweets than answers.

To make matters worse, there’s no widely accepted criteria for what constitutes authorship. Yes, there are the Vancouver Guidelines for paper authorship in biomedicine, but almost every time I mention them, I hear grumbling about how poor they are. Researchers either don’t known about, don’t care about, or disagree with those guidelines.

The ideal option to resolve authorship disputes, as far as I am concerned, is for the authors to talk to each other and try to resolve their differences on their own. But I suspect that once disputes raise to the point of withholding authorship, it’s going to be hard to resolve that on your own.

A trainee might try to inform the department chair of faculty involved in the project. But increasingly, there are multiple faculty and it may not be who is the relevant person overseeing the faculty. It also seems unlikely than many chairs are willing to step into an authorship dispute, or, even if they are, what they can do about it.

Some institutions might have a research compliance office. But because the standards for authorship are so vague, the question becomes, “What are you supposed to be complying with?” You can have a valid authorship dispute that involves no misconduct. Are compliance offices supposed to resolve differences of opinion about who deserves first author placement versus second author placement?

About the only logical step left is to appeal to the journals themselves. And the Committee on Publication Ethics has a procedure for adding authors (PDF). But if the authors don’t agree, the guidelines are to toss the ball back into the court of the institution, which is, as we just saw, problematic. And it isn’t clear if the recommendations for journals to add authors also apply to, say, changing author order or some other kind of authorship dispute.

While I haven’t gone through every entry at Retraction Watch, I am willing to bet that more retractions arise from omitted senior scientists than omitted trainees.

And that’s a big part of the problem. There is a huge power differential between trainees and senior scientists. There seem to be few places more ripe for abuses of that power than in doling out authorship credit.

Regardless, it seems unfair and unwise to expect journals to resolve authorship disputes. There are too few standards across the community (see discontent over Vancouver Guidelines). Journals probably have no resources to investigate the facts of a dispute thoroughly. This probably means that in most cases, they will favour the senior scientist (see power differential).

I don’t know what the solution is. But I think this is a problem that is not given enough discussion. It seems likely that in many cases, trainees in disputes will be left twisting in the wind.

The moral of the story, if you are a trainee of any sort, is: Extensively discuss your expectations for authorship at the very start of any project. Be prepared to negotiate.

24 April 2017

Many science questions emerge from a place similar to what Penn Jillette describes in this quote about people’s attitudes to video games. (Emphasis added.)

You know, when I was 15, 16, 17-years-old, I spent five hours a day juggling, and I probably spent six hours a day seriously listening to music. And if I were 16 now, I would put that time into playing video games.

The thing that old people don’t understand is – you know if you’ve never heard Bob Dylan, and someone listened to him for 15 minutes, you’re not going to get it. You are just not going to understand. You have to put in hours and hours to start to understand the form, and the same thing is true for gaming. You’re not going to just look at a first-person shooter where you are killing zombies and understand the nuances. There is this tremendous amount of arrogance and hubris, where somebody can look at something for five minutes and dismiss it. Whether you talk about gaming or 20th century classical music, you can’t do it in five minutes. You can’t listen to The Rite of Spring once and understand what Stravinsky was all about. It seems like you should at least have the grace to say you don’t know, instead of saying that what other people are doing is wrong.

The cliché of the nerdy kid who doesn’t go outside and just plays games is completely untrue. And it’s also true for the nerdy kid who studies comic books and turns into this genius, and it is also true for the nerdy kid who listens to every nerdy thing that Led Zeppelin put out. That kind of obsession in a 16-year-old is not ugly. It’s beautiful. That kind of obsession is going to lead to a sophisticated 30-year-old who has a background in that artform.

I think about this quote a lot.

It seems to me that many people who ask questions about science are working from that background of “They listened to Dylan for 15 minutes.”. They’ve been exposed to a few basic ideas. They’ve maybe had one or two lectures in high school about evolution. They get reproduction is important. They get that natural selection leads to adaptation. They get “survival of the fittest.”

But they haven’t mastered the art. So they ask why human evolution has stopped (it hasn’t) or why some trait is so obviously bad (lots of reasons). They can’t get those nuances without having spent that time on task.

Same with people who think that half an hour Googling an answer constitutes “independent research” on climate change or vaccines or what have you. Sorry, that’s the equivalent of listening to The Rite of Spring once.

It’s similar to what I talked about recently: you need time to live with ideas to understand the subtleties.

19 April 2017

Forgive me a fanboy moment as I react to the announcement of the new Legend of the Five Rings card game!

At first glance, it looked like the game I knew. Two decks, provinces, events, two main stats on the characters. Then, as I read deeper, I realized that the mechanics of the game were going to be almost entirely different. That will take getting used to.

But one of the things I always loved about L5R was the art. It was literally what made me pick up the game. And I have to say, I like the art direction. They’ve managed to keep the aesthetic, particularly for the characters and the clans.

At the Sociobiology blog, Joan Strassman tackles inequality in scientific awards. This topic has been making the round lately because of the National Science Foundation’s Waterman award, which this year went to two men. Again. It looks like the last time the award went to a woman was in 2004, to Kristi Anseth. Weirdly, it looks like the Waterman did a pretty good job of splitting between men and women in the first few years, and then it’s been all men since 2005.

Partially in response to community input, the NSF changed the eligibility criteria for the award. It is basically extending the time frame for eligibility since a person received their Ph.D. (And I will pause to take note that you are still considered a “young” scientist in your late 30s.)

I’m betting you right now that’s not going to fix the problem. And I doubt the measures Strassman suggests, like “Let’s be active in nominating women!” will do it, either. But here’s what will fix the problem. Guaranteed.

You get the award organizers to say, in public, “We’re going to give half these awards to women. Agender individuals are eligible for either.”

And that’s it. Dust off your hands. You’re done.

The NSF gave two awards this year and in 2012. Give two awards every year, one to a man and one to a woman. Or alternate years. It doesn’t matter.

Yes, I know people will jump in and say, “But merit...”, but I don’t care. This is not a job. Nobody’s livelihood is harmed because they did not receive an award. The question is, “Are you serious about fixing inequality or not? If you are, this fixes it immediately. Everything else just allows the problem to linger.”

I’ve had discussions with people about why the Oscars split their acting into two categories, one for men, one for women. But one good thing about it is that every year, women win awards.

18 April 2017

I have no idea what “Project C.R.A.W.F.I.S.H.” might be about, despite being the only researcher on this university who actually publishes research about crayfish. It is a little reminiscent of a neuroscience announcement a while back.

Maybe there are no crayfish at all in this thing, whatever it is.

Far too much effort has been put into that acronym, though. Someone is priming themselves for a role in the federal government with that, I reckon.

Last week saw the announcement of a shrimp that kills with sound and was named after Pink Floyd.

That is Synalpheus pinkfloydi. This newspaper article says the reason it was named for the band is because of a legend that the band once played so loud that it killed fish in a nearby lake. Plus, one of the authors is a rock and roll fans, previously naming a shrimp after Mick Jagger.

The article missed a damn good bit of the paper, though:

Distribution: Presently only known from the type locality on the Pacific side of Panama; likely more widespread in the tropical eastern Pacific, but unlikely to occur on the Dark Side of the Moon due to lack of suitable habitat.

16 April 2017

I saw the opening of “My Hero” exhibition at IMAS McAllen yesterday (originally curated by the Bedford Gallery in California). I enjoyed it a lot. But it continued a long simmering problem with “fine art” exploiting “comic art” without credit.

Here’s artist Russ Heath’s desciption of what it was liked to have his work ripped off by Roy Lichtenstein, for instance.

Almost nowhere in the exhibit do you see acknowledgements of some of the original creators. (One, a lovely pastiche Captain America in the style of Norman Rockwell, includes the names of Cap’s creators, Joe Simon and Jack Kirby, in the art.) But most don’t, even when a piece is directly referencing a specific classic image. For instance, one piece references this specific panel from The Amazing Spider-Man #50:

The piece in the exhibition is clever, and I liked it. The descriptive text accompanying the piece mentions the issue, but not the name of the original artist, John Romita, Sr.

Superhero iconography is treated as if it were created by artists lost to time, instead of people who are often still alive today, and struggling to make ends meet.

14 April 2017

Making the rounds today is a new story that an adult entertainment website has provided a $25,000research scholarship to Natalie Nevárez, a neurobiologist, to study monogamy. (Not mentioning the name of said website to try to prevent blog from being overridden with bots and spam.)

Congratulations to Natalie!

I would like to point out that in the last two months, adult entertainment websites have provided:

These are small, token gestures, sure. But at this point, adult entertainment websites are showing a better understanding of civic responsibility than many elected politicians are. All of these used to be services that we expected to be provided largely by governments.

Update, 15 April 2017: A couple of people on Twitter noted that these sites do have problematic aspects to them, such as stolen content. I certainly don’t want to let these businesses off the hook for their bad practices. Ultimately, issues like “fair compensation of workers” matters more than a scholarship here or there.

But that they have made some of the gestures above kind of makes me wonder if they might smarten up.

13 April 2017

Earlier this week, I argued that we could kill predatory junk journals with a single stroke if regular scientific journals would publish the text of the pre-publication reviews along with the paper. This way, junk journals couldn’t hide behind the claim that they are peer-reviewed.

I argued that junk journals wouldn’t want to take the time and effort to create reviews in any way. But a couple of people on Twitter responded that the junk journals could (and apparently sometimes do) ask for reviews, but ignore them.

This makes things interesting.

Even for a regular journal, soliciting reviews but ignoring them is not out of the question. The buck stops with editors. The editor makes the decision about what to publish, and in some cases this means overriding recommendations of one or all reviewers. We just don’t expect it to happen intentionally and systemically.

When viewed from the traditional norms of pre-publication review, consistently asking for reviews but ignoring them is a massive waste of effort. But the traditional norm is that reviews only exist in the files of the reviewers, editor, and author.

What happens under the suggested new norm, that the reviews are published along with the paper?

Suddenly, the difference between a traditional journal and a predatory journal gets very blurry, very fast.

Presumably, the scam publisher would ignore the reviews and publish
the paper immediately alongside the reviews. The paper would not get the
benefit of revision in light of the reviews. But that would put the paper at the same level of editorial vetting as a pre-print. Let’s take a second to note that many have found great value in pre-prints (though myexperience has been underwhelming). Even stodgy old biologists are using them more and more.

But let’s not forget that it is now a verifiable fact that the paper has indeed been peer-reviewed. The review is available for all to see to help form a judgement about that paper. And we can also judge how detailed the review is. In this scenario, we can think of pre-publication reviews as a rating instead of as a publication decision maker.

Essentially, by publishing the pre-publication reviews, the predatory journal has suddenly moved to a format that is very similar to what some scientists have been advocating for years: the “publish, then filter” model of publishing, rather than “filter, then publish.” If there are verifiable pre-publication peer reviews done, can we even still call it a “predatory” journal?

What the predatory journal no longer provides is any judgement of the importance of their submissions, which many readers badly want. Readers want guidance as to what is more likely to be a breakthrough. But then, the rise of open access megajournals has shown that journals can be successful without rating “importance.” Articles in megajournals can still be found and cited and used by people in the field.

If “publish review content” became standard practice across the board, predatory journals might start to serve a useful purpose instead of being the bane of science.

12 April 2017

Understanding something for the first time is often shown in comics and cartoons as a light bulb appearing over someone’s head. It’s off. Then suddenly it lights up. People use the phrase “light bulb moments” to describe insight all the time. Or even just “light bulb” alone, like Grue does in the Despicable Me movies.

There are few more rewarding moments for an educator than when you see someone having that “light bulb moment” right in front of you. It happens sometimes.

But we might expect too many of those light bulb moments: where there is a clear line between, “I don’t get it” and “Oh! I get it!”

Thinking about my own education, there are lots of concepts that I teach to students now that I remember trying to learn. For many of them, there was no light bulb moment. Instead, there was just an ever increasing familiarity, and in some cases, skill in carrying out tasks related to it.

The way I put it to people is, “I got used to the idea.”

I think at some level we know that learning can be a slow, gradual thing. You might follow an explanation while it’s given, but mostly forget it by the next day. You make mistakes about something you ostensibly “know.”

I think this is particularly important to keep in mind because so much of formal education is timed. We go by weeks, by semesters, and if students can’t learn something in the allotted time, the student is deemed not to have learned the material. I think it biases us to think that students who don’t get it in that prescribed amount of time won’t get it.

Professors get frustrated when students ask about concepts and materials that were covered in previous classes. The student should just know it, and they get badmouthed for not remembering. Instead of seeing this as indicating a bad student, we should see it as an opportunity to let the students get more used to the ideas.

One of the differences between between an entering university student
and a graduating one is not their raw intelligence, or study skills, but
the number of repeated exposures they have had to core ideas.

A light bulb moment might be a second long, but it can be years long, too.

10 April 2017

Two years ago, Eleftherios Diamandis wrote a horrible piece in Science Careers that glorified overwork. This was widely criticized. And for this, he now gets... a platform at Nature?

Yes, Diamandis just published a new piece in a glamour magazine, in which he freely confesses to being a negligent grad student mentor. He writes (my emphasis):

I remember remarking on the slow progress of one PhD student's research project at our second review meeting (typically held six months after their project launch). Three months later, I repeated my concerns, which were mainly about how slowly the student was learning essential techniques such as mass spectrometry, the workhorse of our lab. But instead of addressing those concerns, the student stopped scheduling meetings. I was too busy to notice for another six months.

I should be surprised by Diamandis’s lack of self-awareness, but he’s already amply demonstrated his obliviousness. I guess I’m surprised that when he boasted about all the time he spent away from his family and the hours and hours and hours of working, I somehow thought that he might actually care enough about his work to be competent at it.

Grad students are not loose change that you can lose in a couch cushion, for crying out loud.

When Diamandis suggests the student do a master’s degree:

I was horrified when my suggestion elicited tears. The student and I decided to give the programme another try, with the proviso that we would hold mandatory monthly meetings. I also ensured that the student could get technical support from my lab manager. After three years, the student published in a good journal, and 18 months and two research papers later, was ready to write a thesis.

This guy is surprised that a student cries after literally forgetting that the student did not meet with him. And why is technical support not available to grad students all the time?

He may publish a lot of papers (and he does), but this event marks him as an incompetent supervisor. Diamandis cares only about one person: Diamandis.

Additional: I’d forgotten than Diamandis had another Nature piece last year, pontificating about when he would retire. I think one of the more revealing moments in that piece is when he talks about how much he loves the h-index as a measure of productivity. It explains why he can take the time to write all these career opinion pieces but forget his student.

He measures his importance by his publication record, his h-index, and the Impact Factor of the journals he publishes in. Those are things he values. His trainees, not so much.

Given what he’s written, particularly the newest piece, a lot of people might suggest he move the clock on his retirement up quite a bit. Like, “You can retire any time now. Here’s your hat, what’s your hurry?”

Update, 11 April 2017. Edge for Scholars has summarized some of the reaction from social media to Diamandis’s article.

More update, 11 April 2017: I changed the title of this post. It was originally, “Grad student goes missing and supervisor doesn’t notice.” That was not a correct characterization of the situation. It is not like the student vanished, nobody knew where he was, and a missing persons report should have been filed. The student was there, just not making progress.

A couple of other issues raised by Diamandis’s post that have come up.

First, I noted that this article shows how disrespected master’s degrees are. It is seen as a failure, not an achievement. This is a bit of a slap to the many faculty and students who work hard at master’s degrees, whether they do not want, or are not able, to do doctoral work.

Second, Kevin Wright noted that this is a sign of the inefficiency of large labs. Someone making no progress would not escape notice in a small lab. A small lab could not afford to have a student doing very little for half a year.

Another obviously bogus paper accepted got accepted by a junk journal. This is hardly news; barely a year goes by without someone demonstrating that some journals will publish any old crap. This one made the round because it had a lot of Seinfeld references. Marginally wittier than usual.

People continue to be (in my mind) disproportionately upset about junk journals. The Ottawa Citizen has been paying a surprising amount of attention to them for a city newspaper. One recent article argued that Canadian universities are tacitly permitting their faculty to publish in junk journals with no consequences:

In a study published in the current issue of the Journal of Scholarly Publishing,
I find that the majority of research faculty in the business school at
one Canadian university have publications in predatory journals. Well
before the study was published, I made the dean, provost and others
aware of this result. It did create friction with the dean, who did not
appreciate my emails and other communications about the problem.
However, the truly surprising reaction was that there was absolutely no
attempt to discuss my findings, verify the problem or otherwise address
the issue. Indeed, the business school is currently preparing a
performance metric that will count publications in predatory and
legitimate journals equally.

The main reason that junk journals can fool people (even some in relatively sophisticated academic environments in an industrialized nation) is that they can claim to be peer-reviewed. There is no simple way to know if a journal is peer reviewed, because those critical pre-publication reviews are normally confidential.

My “not at all novel” solution for how we could kill off junk journals is:

Publish the reviews.

Just the content of the review, not necessarily the identity of the reviewers. I don’t want to wade into the “signed” versus “anonymous” peer reviews right now. The goal is to demonstrate that the paper received substantive review, not who did it.

Real journals have the reviews to publish. Junk journals will have no reviews they can publish. The effort spent generating plausible fake reviews seems to be far too high for a junk journal to keep up the charade for long.

With that one change, whether a journal is truly peer reviewed (or not) is easily verifiable.

There have been many other people who have called for publishing reviews to be a more normal part of the publication process. There are many reasons to do this, but possibly shooting a poison dart in the direction of junk journals would be a nice side benefit.

05 April 2017

Pretty regularly, American news media runs a story with some version of this headline:

Accepted into every Ivy League school

That’s obviously a big achievement for a high school student. But I have noticed this:

The students making the headlines are almost always minorities.

News coverage focuses on the unexpected. That news coverage so consistently focuses on minority students means that their success was not expected. Success is not normalized for them. Their success is some sort of bizarre, noteworthy exception.

Even while I appreciate the importance of visibility of success for minorities, the underlying message being sent – “We didn’t expect this kind of success from you” – is kind of crappy.