Instead of only addressing the core values of the peer review system (that must be retained or enhanced), as requested in the recent RFI from the NIH, I thought I'd highlight the core values of the NIH-funded research system as a whole. This seems a good exercise particularly since many of my posts trend toward critique. It strikes me that many of my criticisms of the NIH arise from a failure of the system to live up to the ideals to a sufficient degree. This is a VeryGoodThing, much better than being in the position of criticizing intentional behavior. So I recognize that these strengths are not perfectly realized. It is, however, important that these are the ideals and goals of the system.

Investigator-initiated research: The unswerving belief that the most productive science results from the uncontrolled effort of very smart and well trained individuals is probably the top strength of the NIH system.

Democratic participation: The second biggest strength of the NIH is the fact that the identity of who can be a Principal Investigator on a NIH grant is not centrally controlled. In practice, a great diversity of individuals serve as PIs, bringing their various perspectives to bear on the science of their choice.

Peer Review: The first and most important level of decision on which proposals are funded is made by a "jury of peers", fellow PIs working in closely related fields of science. This is a critical component of support for the other strong features of the system.

Basic Research: The commitment to exploration for knowledge's sake with no clearly defined route to a health benefit, product or other tangible outcome is supported by a firm belief that such tangible results will eventual be attained. History has shown us that science is an incremental effort, ever building on prior work and that applications of particular scientific results often develop long after an initial discovery.

Public Health Mission: Keeping one eye on the prize, so to speak, runs in tandem with the commitment to basic research, keeping science from getting too far away from a broadly approved public good. After all, the taxpayer are supporting this, they should realize some tangible benefit.

Project based funding: The NIH system seeks to fund scientific projects of a specifically defined nature for a finite period of time. This means that what is prioritized is the quality of the specific idea rather than qualities of the individual researcher.

Breadth of Institutes and Centers: As in politics, too much democracy can be dangerous. A critical strength of the National Institutes of Health is that the term is plural ensuring that scientific coverage will be broad. This insulates the science mission against swings in popularity for particular types of inquiry that might result from the tyranny of the majority.

Yea, c'mon. You can do better.
Here is what I get from your Hallmark Gush: You like the investigator-initiated research, but not too much of it, which is why you also like that there are institutes & centers and project-based funding to keep the creativity under control. And I see you like basic research but not too much basic research, which is why you compliment the public health mission.
All that sounds like you're telling the story of Goldilocks and the three bears. This porridge is too hot, and this porridge is too cold, but this is just right...
The only clear point you seem to make is that you like that a wide variety of people get to shit on each other's projects. Maybe we could just post all the proposals on the net and let Digg make the funding decisions?

Like any other system, when the system is working for you, you find reasons to love the system. When the system is not working for you, you find reasons to hate the system.
DM must have got his grant funded shortly before this post originally went up. 🙂

Comrade,
You are not only my scientific hero, but a secret object of my lust. If you say I need to do better, then I'll do my very bestest ever to do better.
Love,
XXXOOOOO
Luigi
p.s. DM you look awesome today. Love the new haircut.

Yeah.
I'd say the biggest failing is that NIH relies on universities and, even worse, totally unregulated scientific journal editors, to determine who gets to be eligible to apply for funding.
So that part is not really democratic at all. In my mind this is the biggest problem. It's not true that "anyone" can apply for funding, and we're losing some (maybe all?) of the best people at the stage where they have this realization that they don't have a fair chance to propose their investigator-initiated ideas.
The rest, okay, I guess I can see how those are the ideals we all hope we are sincerely aiming for.
But given what I know now about only a small part, it probably also applies to the rest. And I probably won't get a chance to find out firsthand whether I'm right or wrong about that.

I'd say the biggest failing is that NIH relies on universities and, even worse, totally unregulated scientific journal editors, to determine who gets to be eligible to apply for funding.

??? This isn't true at all. Not even close. Not in any permutation or interpretation that I can even think of. You don't have to be a university employee to be eligible to apply for NIH grants. Many grantees are not. And journals have absolutely nothing to do with eligibility.

DM - can you explain to me why "Project Based Funding" is good? Haven't all of the real breakthroughs in science been by being surprised? The one consistent complaint I have ever heard about study section is that it doesn't trust the researcher to set off in new directions.
Of course, as we all know, study sections do take the researcher into account because feasibility is treated as an acceptable issue (as in "this lab can't do this project because they don't have the experience"), so it's not "prioritized by the quality of the specific idea rather than qualities of the individual researcher" anyway.
Wouldn't it make more sense to make this actually about the quality of the specific researcher? As in "this is a level X scientist and a level X scientist should get $Y to play with". Consistent success gets you continued funding. Lack of success pushes you back into the "have to apply for project-based funding" pile.
It seems to me that all "project based" does is allow study section to nitpick about the project itself. So why is "project based" good?

Not intentionally.
But they're perfectly happy to take credit for funding them when they do happen. And that mismatch is what most of us spend our time complaining about. I think that's fixable. (I mean the system. It's easy to fix the complaining. 🙂

Yea, I'm with whimple. Consider three applications:
Application #1 is from a rock band that wants to distribute a song that they say cures cancer in anyone who listens to it.
Application #2 is from a physicist who needs thirty million dollars to develop a radiation focusing system for zapping brain tumors.
Application #3 is from a researcher in Big Name cancer institute who wants to study tumor growth in rat pups, because lots of kids get cancer and no one has really described tumor growth in rat pups before.
Which breakthrough would be most exciting? Which is most likely to succeed? Imagine yourself in the shoes of a reviewer who has to rank these proposals. How would you rank them? Given that only the top-ranked proposal will get funded, who's most likely to get funded? Who's least likely to get funded?