By now you’ve heard your fate on the preproposal. Some very generous bloggers are telling the story from the panelist perspective. I think the biggest question is whether they are treated like a four page version of a full proposal, or like a different creature, with a focus on ideas, not methods.

Jabberwocky Ecology had some encouraging news here on the big idea focus and thought it went well. Their first entry was here and in it they say that with shorter proposals more time was spent reading and thinking about them. Here’s another entry here from Jack Williams on Contemplative Mammoth. Jack gives a lot of interesting details on what they considered. I found this statement to be discouraging: “However, in general, I was impressed by how rarely the reputation of the investigators directly affected panel decisions and how consistently the review process focused on the merits of the pre-proposal in question.” I think the reputation of the researchers should influence the review process, in addition to the actual proposal at hand. After all, there are tons of studies that say this is the best predictor of excellent, published research. Of course, the two go hand in hand, but I don’t see why we can’t count previous productivity, making allowance for newcomers. Prof-Like Substance, here, also gave some first-hand insights into the process that are useful. He says big ideas are no more important than before, along with a lot of very helpful numbers.

I wonder if these preproposals were easier to evaluate than the ones I saw when I sat on a FIBR, Frontiers in Biological Research, I think, panel with preproposals years before. They were from a huge area, making us worry if us panel members were close enough to the work, given there were no ad hocs. That does not seem to have been a problem here.

I suppose you are wondering what happened to me personally, so I’ll tell you. One of our proposals got the go-ahead. The other did not. The one that was nixed was for methods in large part. It also complained that I mentioned blogs as part of my broader impacts. Go figure.

I guess overall the conclusions are still unclear. I’m hoping Jabberwocky is right and the new system will be better than the old. At least it is far more open than the NIH system.

6 Responses to Mixed reports on the NSF preproposal process

He said the proposals were treated like mini full proposals, and methods mattered. he seemed glad that the record of the PI did not matter.

I’m not sure I agree with this assessment. In fact, I think PI record played a big factor for proposals that were heavy on idea and light on methods. If the lab had a record in the methods being proposed the panel was less concerned about justification. Where people ran into issues was proposing novel methods without the resources, experience or data to back them up. This happened a lot. I’m not sure why I would be happy about record not being taken into account (or where I wrote that), since that’s not a great way to review applications.