We're now about 14 months since both DEB and IOS went to the new preproposal system. We're at the cusp of hearing about who got funded from the full proposal round so we will have a complete cycle to stand back and look at as a whole. Much has been made about the process, but so far we don't have any data to judge the entirety of the new landscape.

It's clear at this point that both IOS and DEB are sticking to their guns on the preproposal, whereas MCB is switching to a annual full proposal. The announcement from IOS is below, and a similar one can be found on the DEB page.

The National Science Foundation’s Division of Integrative Organismal Systems has issued an updated solicitation for the core programs (NSF 13-506) along with a revised set of IOS Frequently Asked Questions (NSF 13-09) which are now available on the IOS home page at: http://www.nsf.gov/div/index.jsp?div=IOS

The core programs covered under this solicitation in the Division of Integrative Organismal Systems (IOS) support research aimed at understanding why organisms are structured the way they are and function as they do. Areas of inquiry include, but are not limited to, developmental biology and the evolution of developmental processes, nervous system development, structure, and function, physiological processes, functional morphology, symbioses, interactions of organisms with biotic and abiotic environments, and animal behavior.

The Division of Integrative Organismal Systems (IOS) will also host a Webinar to discuss the outcomes of the new preliminary proposal system thus far and answer questions about the updated solicitation on Monday, December 10th. Information about how to sign up for the webinar will follow shortly and be posted to the IOS home page.

What interests me is that final paragraph. I'm not much for webinars, but I am interested in data. I'll be curious what NSF learned and whether we're looking at sticking with the same system for a certain number of years or what. We shall see. It may not even be discussed.

So, readers, how are YOU feeling about the preproposals? Better than you did in January? Worse? What are the things that you have thought up as a better alternative?

I would actually be in favor of shortening the full proposal length (maybe 8-10ish pages) and going without the preproposal. I would even go so far as to ditch the ad hocs and go with four panel reads. I think it's all about compromise and fixing the issues that have plagued the system. If that could get us back to two cycles, I think the community would whole-heartedly embrace it. Even if we can't, it's an improvement.

I think the major issue is just how uncomfortable everyone is with a single full proposal submission deadline, paired with a completely different 4 page document as the gatekeeper. Whereas I hear all the pros to the preproposal from the NSF perspective, I still really dislike the idea of writing two very different documents for two different sets of reviewers. Is it just me?

I also dislike the idea of writing two different styles of proposal as a way to get one grant. I prefer the MCB approach of 1 full proposal. One submission sucks, but the preproposal is another game that people do not have time to play.

As someone who has survived the IOS pre proposal gauntlet and is now sweating their full proposal decision, I am surprisingly ambivalent about the whole process. I liked the feedback that I received on the pre proposal, but the added stress and time suck of having to prepare both was a part of my life I'm never getting back. I think that the pre-proposal format could work, if they went to two pre-proposals in one year (therefore, two bites at the apple in a single year), but I have no idea about how to resolve the logistical nightmare that that would represent.

It's tough to think straight right now; not only am I checking Fastlane three times a day, but the election is a drag on everything.

If it makes you feel any better, I don't know of anyone who has heard from IOS or DEB on their full yet. That's not to say it hasn't happened and all the people I know aren't getting funded, but no word yet.

People talk about having to jump through these two hoops, and how it's a pain in the ass and costs time and effort compared to not having a preproposal. Ignore for the moment the issue surrounding how many submissions there are per year.

x = amount of work, time, energy to write a full proposal
n = number of proposals being submitted in a given period

so,
xn = the total amount of person hours invested in a round of proposals the old way.

Now, consider
p = proportion of effort it takes to write a preproposal relative to a full proposal
f = fraction of preproposals that are invited to submit full proposals

Under the preproposal system, the total amount of person hours invested in writing preproposals and writing full proposals is

npx + fnx = xn(p + f)

Based on stats at DEB, I believe f was on the order of 0.25. p is up for debate, but given that you do not have to deal with budgets, data management plans, mentoring plans, etc-- in addition to the shorter length--I would argue that it's also on the order of 0.25 - 0.4. As long as p+f < 1, then the preproposal process saves effort overall.

Based on my estimates, the system overall is 35-50% more efficient going the preproposal route.

There are plenty of valid complaints about this system, but arguments that it is more work only applies to those lucky few who get past the first hurdle, and ignores all of the time savings for the majority of folks who don't.

You're also ignoring the fact that most people had full proposal written that were being modified rather than written from scratch. No one had a preproposal written. So this year the effort was heavier than it will be as the process settles in.

[...] a lot of people out there waiting to hear back from their POs about full proposals. Between email, the blog and twitter I have gotten a lot of people asking about notifications. Here's what I know so [...]