No, it’s not that obvious with credit for co-authorship….. If I can make my work better by collaborating with someone else, then science is better off if I collaborate, methinks. Yes, doing it alone may be more difficult, but I am not sure how many “bonus points” this justifies: After all, this is not some sports competition where you get points for doing it on one leg, blind folded and using xmgrace.

But of course, in SOME way one should take it into account…..

]]>By: Massimohttps://expbook.wordpress.com/2009/11/09/lets-get-things-straight/#comment-1151
Fri, 20 Nov 2009 14:45:16 +0000http://expbook.wordpress.com/?p=2566#comment-1151Agree. It is complicated, likely to be easily manipulated, and in the end I do not think it would provide much different a comparative assessment than the h-index itself.
The notion of multiple authorship is a funny one. We all know it should be taken “somehow” into account, but it is unclear how to do it fairly and objectively. For one thing, while for a theorist it is common, and relatively easy to work alone, for experimentalists it is essentially impossible these days to carry out any substantial research project in isolation. Also, why assign a greater weight to an irrelevant single-authored paper than one with multiple authors that garners a lot of attention ?

Once again: h-index is only a starting point, it is not a replacement for going through the person’s publication record and get a sense of whether someone can work independently. The typical case is that of a probationary faculty who keeps publishing only with his former PhD/PD advisor (and their groups) — this will definitely raise eyebrows with reappointment, promotion and tenure committees (at serious places, anyway), regardless of this person’s h-index.

Here, basically, a paper only counts for your h-bar-index, if it also counts for the h-index of ALL coauthors on this paper. Which puts studenst of famous people at a decided disadvantage. Also, you could gain if your co-authors other papers (without you) do NOT get cited, which might introduce some very itneresting dynamics into citation networks. Something tels me that this one is not going to catch on…..

]]>By: Massimohttps://expbook.wordpress.com/2009/11/09/lets-get-things-straight/#comment-1136
Fri, 13 Nov 2009 14:39:48 +0000http://expbook.wordpress.com/?p=2566#comment-1136Doug, I agree with everything you just wrote but, again, I am not in the least advocating going by h-index only, or mostly, or by any other quantitative measure. What I am thinking of specifically is the following situation:

Committee Member A: OK, here is Joe Glotz from BigNameU whose PhD advisor is Prof. K. Myboot, and current postdoctoral advisor is Prof. I. Mighttalktoyou and here is John Doe from WhereTheHellU whose PhD advisor is I. Couldonlygethere and is doing a postdoc with O. Notmuchbetter — seems like a no-brainer to me, we should go with Glotz, look at his credentials (read: pedigree)…

Committee Member B: Well… yeah but… funny though, Doe has a higher h-index than Glotz… should we maybe take a closer look at this one ?

That’s it. No more than this. I think it would make a big difference already, even though I agree that the relevance and applicability could vary depending on the level (but, Doug, the same can be said about any quantitative measure — should we not even try to use one, then ?)

]]>By: Doug Natelsonhttps://expbook.wordpress.com/2009/11/09/lets-get-things-straight/#comment-1135
Fri, 13 Nov 2009 14:26:50 +0000http://expbook.wordpress.com/?p=2566#comment-1135Massimo – I agree with the sentiment, but you know the unfortunate practicalities of hiring. You want a system more like graduate admissions: there is some small group of hotshot students that are in demand and get offers from Harvard, Cornell, MIT, etc., but there is a long list of other applicants. Graduate programs only get about 1/3 of their admits to come because of competition from other universities, so they offer slots to ~ 3x as many people as they need, going down the list well beyond the top few people. There end up being relatively few students who can’t find a home somewhere if they really want to go to grad school.

The practical aspects of faculty hiring get in the way of a system like that. Everyone on the hiring side (1) wants to get the best people that fit their programmatic needs; (2) can only afford to fly in a handful of people to interview; (3) would never hire anyone they didn’t interview face-to-face; (4) needs to get their decision-making done before the end of the spring, for financial reasons. Under those circumstances, it’s very hard to come up with a system where some perceived top group isn’t highly sought after.

As for h-index as an “equalizer”, I’m highly skeptical. I’ve been a professor for over 9 years, and only now am I getting to the point where I think my h-index reflects my research productivity. For young people, even up to the point of tenure, the h-index is strongly biased by the kind of graduate training you did. It’s not at all clear to me that ranking applicants by h-index would have the effect you desire, of identifying highly productive people that would otherwise be overlooked b/c they don’t come from “top” places. On the experimental side, at least, in my experience the junior people with highest h-indices come from large, productive groups where there are lots of coauthors on collaborative papers. To use an old example, most students that came out of Smalley’s nanotube group in the late 1990s have big h-indices, whether or not they were 5th author rather than 1st on a bunch of collaborative papers. It’s pretty rare to see previously unnoticed people from small places that have lots of papers and citations. If anything, those people tend to stand out.

]]>By: Massimohttps://expbook.wordpress.com/2009/11/09/lets-get-things-straight/#comment-1134
Fri, 13 Nov 2009 11:34:32 +0000http://expbook.wordpress.com/?p=2566#comment-1134The fact that “scientific achievement” is ill-defined is precisely why it is problematic for you to make sweeping statements like: “The h-index correlates reasonably well with overall scientific achievement….”

Nah, come on now, you are just arguing for the sake of argument. The notion that “evaluation committees try to assess overall scientific achievement” is commonly accepted — very few, upon hearing that contention, go “wait a minute, what is that, exactly ?”.
One thing is to say that it is ill-defined, the other is that there is no such thing or that any two opinions are irreconcilable.

There are some reasonable, commonly accepted measures of scientific achievement (publications, citations, number of invited talks, prizes). I understand you take issue with all of them but I do not see what reasonable alternative you have — I suspect none (“any criterion that does not select me is wrong”).
The h-index correlates fairly well with all of them, as shown in those papers.

]]>By: Schlupphttps://expbook.wordpress.com/2009/11/09/lets-get-things-straight/#comment-1133
Fri, 13 Nov 2009 10:09:14 +0000http://expbook.wordpress.com/?p=2566#comment-1133I think the important thing is to keep in mind that “subjective decisions”, which hiring will continue to be, must be kept apart from “arbitrary decisions”, which it should not be. Of course, one would not like the h-index to be set in stone,* but the two following statements are simply something very different:

1) “Candidate A has the higher-h index, but for the following reason, I still think that candidate B is better:…..”

2) “h-index, number of publications and citations don’t mean anything, really, I just know that B is better than A.”

The difference being that in case 1), one can then proceed to have a reasonable discussion about the reasons given and whether they are valid as well as strong enough. Perhaps there is going to be agreement, perhaps not. Maybe A will be chosen, maybe B. But the discussion will be fairer to A than statement 2 is.

*) Your statement that “nobody has ever suggested” something as stupid is overblown by the way. “Nobody with more than five neurons in working order” would be the correct formulation.

]]>By: Hopehttps://expbook.wordpress.com/2009/11/09/lets-get-things-straight/#comment-1132
Fri, 13 Nov 2009 06:19:45 +0000http://expbook.wordpress.com/?p=2566#comment-1132The fact that “scientific achievement” is ill-defined is precisely why it is problematic for you to make sweeping statements like: “The h-index correlates reasonably well with overall scientific achievement….” And when asked for evidence to support this claim, it is disingenuous of *you* to refer me to papers that explicitly sidestep this issue.

I take it that you feel that other factors, like where one got his/her degree and who is his/her PhD advisor are more objective measures of achievement ….

Not really. But for your “younger colleagues” looking for their first job, I don’t think it’s that much better.

]]>By: Massimohttps://expbook.wordpress.com/2009/11/09/lets-get-things-straight/#comment-1131
Fri, 13 Nov 2009 05:17:39 +0000http://expbook.wordpress.com/?p=2566#comment-1131The h-index is just a simple number, unambiguously defined, that anyone can look up and obtain the same result as anyone else.
I think you are being disingenuous — you claim that “scientific achievement” is not unambiguously defined, as if there was agreement on what constitutes it. Fact is, many take issue even with a Nobel prize being always an objective sign of achievement. And I am quite sure that you would take issue with any definition thereof one were to come up with (on the other hand, if you think that you have the “ultimate” definition, please submit it to the community, get it to accept it and show that the h-index does not correlate with it).

Until then, I think it is fair to go by the measures that are commonly accepted as at least generally indicative of achievement, such as number of publications, citations, invited talks, prizes etc. The h-index seems to correlate reasonably well with those indicators, and as such seems reliable, always taking into account the fact that we are talking an inherently imperfect operation in an imperfect world.

Mind you, I am sure that the h-index too is affected by the “ol’ boys” dynamics, I just think to a lesser extent. Anyway, it is just my opinion, I take it that you feel that other factors, like where one got his/her degree and who is his/her PhD advisor are more objective measures of achievement — and I am sure you are completely unbiased in this assessment ;-)

This is a cop-out and you know it. Right now on the ArXiv, there are twopapers by Hirsch on the h-index. Neither one provides a definition of “scientific achievement,” nor do they show, with data, how the h-index is an unbiased measure of this. In fact, the 2nd paper explicitly sidesteps this question. The weaker claim argued in these works is that the h-index is a better metric than other figures, such as total number of pubs, average number of cites per pub, etc. In the first paper, where Hirsch proposes the h-index, he gives examples of the (high) h-index of well-known, “successful” physicists. From this we can conclude that the h-index correlates well with the judgment of the “ol’ boys network.”

Slapping a number onto an “unmeasurable” that you can’t even define doesn’t mean that you have “introduce[d] an element of objectivity” into a subjective process – and it might result in giving people even less reason to question their gut.

I suspect that for faculty applicants, the h-index says a lot more about their grad and/or postdoc advisor than it does about them.