Friday, March 27, 2009

Reducing the grant review burden

An interesting news item came up in today's Science (p. 1657) regarding a controversial new policy being put into place by the U.K.'s Engineering and Physical Sciences Research Council (EPSRC).

"After rejecting options such as charging for proposal submissions or placing quotas on institutions, on 12 March the council announced a new policy to ban from any EPSRC funding consideration, for a year or even longer, any principal investigators who have "Three or more proposals within a 2-year period ranked in the bottom half of a funding prioritisation list or rejected before panel [without review]; AND An overall personal success rate of less than 25%." EPSRC notes that this policy should exclude about 200 to 250 people and is retroactive: Letters to those excluded go out on 1 April, and their proposals won't be considered after 1 June."

Now I have to confess that I don't know much about EPSRC and the number of grants that one might be expected to submit there in a two year period, but if NSF were to put this policy in place it would completely change the way I apply for funding. As of right now, I have two different grants under consideration at NSF and I plan to submit another in July. I submitted my first one last July, before I arrived here, and that one was not funded and did not receive very high ratings because of the lack of preliminary data. If the resubmitted version of this grant (submitted in January) and the most recent grant that just went in (two weeks ago) were to be similarly ranked, I would essentially be shut out from funding for the following two years. Rather than getting feedback on different proposals that as a new investigator, I would instead either have to change my research focus and apply to a different agency (a scenario where having preliminary data would be unlikely) or be looking at shutting down the lab and finding work elsewhere.

I can understand that certain researchers might be frustrating the panels with poorly written proposals, but there is a very high chance of new investigators getting caught in the cross-fire here, unless the "overall personal success rate of less than 25%." protects new investigators. This seems like a dangerous precedent to start.

I know of a guy who right now has 3 NIH R01s. His first grant was triaged. Then someone on the study section suggested a rewrite of the "relevance" section to place more emphasis on factor X of the proposal. That was the only change. Next submission = funded in top 2%.

Sometimes the stuff that scores low is crap. Sometimes it just needs a rewording to get the attention of the appropriate people.

Sending those grants to the sin-bin does everyone a tremendous disservice.

this is a stupid policy that would be devastating in the NIH environment. it would absolutely entrench the good-ol' boyz/girlz and hardly anyone would be able to elbow in.

even more disturbing is the fact that for some people in soft money jobs, this would be the end of their careers.

If we look at the people who would have met such criteria at times past but later got their awards and went on to do great science, well, it points to the potential for losing an awful lot of good work...

perhaps the EPSRC funded folks' careers are just really different from the US NIH/NSF funded ones...

As someone who knows the EPSRC system well, I think it is not good to make comparisons with NIH/NSF. This will only affect those scoring in the bottom 50%. With current success rates around the 10-15% mark, there will be many investigators that will not be funded, but will also not be excluded from trying again either. Also, based on myself and colleagues, 3 proposals within 2 years (with none being funded) is a lot of applications in this system. I agree however that a similar approach would not work for NIH/NSF funding, which works a bit differently as I understand it.

Anon - I agree that this policy is unlikely to be a problem for most investigators, but the possibility of junior people, who are just figuring out the system, to find themselves in a funding "time out" is a little scary. It would essentially end someone's career before it really got going. The question is whether or not the chance of unintentional casualties is an okay price to pay to deal with a few people spamming the system.

About Me

I am a faculty member in a university science department, who started this gig in August 2008. I am a teacher, lab manager, accountant, writer, reviewer, husband, father and colleague, in various order depending on the moment.
email: proflikesubstance[at]gmail.com