In the latest round of "Do it to Julia!" we have the attention turning to the Intramural Research programs of the various ICs of the NIH. A comment over at RockTalk:

It is only fair that we apply the same rules to our intramural colleagues and administrators with large numbers of support staff/ budget. Are they being scrutinized by their peers for efficiency and non-overlap?

The impression in the extramural community is that a number of boondoggles exist in Bethesda and beyond.

As you know, part of the mission of the intramural program is to conduct, in government laboratories, creative and innovative biomedical research that cannot easily be performed in academic research settings. Over the past 60 years, a rigorous, multi-level external peer review process has evolved to ensure that this intramural mission is fulfilled and it has been demonstrated repeatedly that this approach to review is competitive in assuring the highest quality research and training.

This triggered some skepticism in the comments including from Jeremy Berg who commented, in part:

Some extremely high-quality research is performed within the intramural program, but the notion that much of this research “is not being done or could not be done elsewhere” seems hard to support. In addition, intramural research budgets are, in general, quite generous.

And therein lies our discussion point for the day, people. For your fields of study, 1) do you know any Intramural players and 2) do you think their contributions represent any unique studies that "cannot easily" be done in the extramural setting? If you do see unique science being done, any specificity as to why it couldn't be done extramurally is welcome.

As far as the cost goes, well, Jeremy Berg has a suggestion:

Information about the investigators in the intramural program is available through NIH Reporter (via Project Numbers containing “ZIA”).

"Unique" is an absurd standard. No one's research is "unique", and if you didn't do it, someone else would have eventually.

The relevant question is what the productivity is per intramural dollar compared to the productivity per extramural dollar. And this is an exceedingly difficult question answer, given the vast differences in how extramural research expenses are allocated and computed and how intramural research expenses are allocated and computed.

Comparing dollar amounts on NIH Reporter between extramural research project grants and intramural budgets is nearly useless for this purpose.

For the NIAID folks I know, they don't do unique research so much as provide a unique nexus of researchers. In other words, there are some awesome collaborations that have emerged that I don't think would have if everyone had been at an academic institution.

There are other things that would have come along sooner or later, like RDTs for malaria, that probably got implemented quicker or more effectively because of NIAID work.

What I'm most interested in, as a practical matter, is whether some of their vaccine research stuff could have been done in academia or industry. I'm inclined to think there might be some unique contributions there, because vaccine trials are not trival to conduct, and there's good administrative infrastructure (yeah, people complain about bureaucracies, but vaccine trials seem to be a pain, or so I've heard from people involved in conducting them).

From what I've seen (though n is very small), I don't know if the research is unique. Researchers, however, do seem freer to explore potentially interesting avenues of research than those worried about renewing the R01 (i.e., in a very limited sense, there's a bit of an HHMI-like effect).

I suppose though, lots of smart, talented people could do well under those circumstances.

The intramural players in my field appear to have greater leeway to carry out experiments that pursue basic science questions, with no immediate "expected outcome" or health benefit. The claim that extramural researchers cannot do this as easily is partly true because they have to answer to study section and programmatic review. In other words, in the extramural world, they don't give you money to just "play around in the lab." Look over intramural researchers and you will find people publishing research only vaguely connected to the stated mission of the institute they are affiliated with. Whether this is a good or bad thing remains unclear, at least to me.

Of course, the extramural folks can take the money already awarded to them and do whatever experiments they wish--in which case the difference between intra- and extramural approaches becomes negligible. Both are subjected to relatively gentle post-hoc reviews in terms of whether the precise deliverable was what got delivered. At the end of the day, good publications are the metric for success, with not too much focus on whether those publications actually report what had been proposed.

Another difference is that study sections are very "hypothesis-driven," while some facets of the intramural program are explicitly "discovery-driven" --consortiums for genomics, proteomics, lipidomics, mastrubatomics, etc. Whether we need more or less of these is a whole 'nuther can-o-worms.

The question is not whether certain types of science can or cannot "be easily done" better with one approach or the other. Berg is correct to argue there is little evidence that intramural researchers bring anything unique to the table. The more relevant question is which approach yields more bang for the buck.

One of the leading scientists in my field is at NIH. If that means he spends less time writing grants and has more freedom to choose his research direction, then I'm in favor of it. I don't see why proven scientists at universities shouldn't get the same deal but perhaps this is easier to do in an institutional framework?

no unique work in my area from the intramural labs, but I like the idea of giving people freedom to do science with the hope some of it will be translated in real life soon. I wish we could do this to university system as well. The large number of support administrator are supported in the name of university functioning with reasonable salaries and relative job securities, whereas the people who actually do the science are underpaid and live under constant fear of loosing their job. This is extremely unfair system.

From what I hear every time an intramural lab places an order (with sigma, with vwr, what have you), it costs a straight $50. I assume this is in place of an indirect cost? perhaps that helps answer rs's point; the admins are still getting theirs intramurally, just more blatantly...

The intramural labs in my field are big names that 15 years ago were doing cutting edge research. Right now, their work is ordinary at best and publications from theae labs are fewer and further between.

Having sat on an NIH intramural review panel, here are a few details....

- The budgets are large, but not humongous. Big PI will run a 8-10 person lab on $600-700k per year, so basically 3 RO1s.

- Review progress is focused very heavily on previous performance. Much more so than RO1 type grants. Typical cycle is 5 years, so your ideas for the next 5 are determined by how many papers you published in the last 5.

- The proposal is submitted just like a grant, read by primary and secondary reviewers, but also then the PI gets to present a summary to the panel (15-20 folks), and the panel asks questions directly. It's fairly easy for the PI to guess who the 2 reviewers are (theyre the ones asking all the questions). The PI then leaves the room and the panel discusses. Then a few days later the 2 reviewers submit a final report and score based on the discussion.

- A big emphasis on training. Extra brownie points in review for someone who had a lot of med/grad students in their lab, especially if they went on to greater things.

- Use of the resources of the NIH clinical center (aka doing translational stuff) also scores highly.

On the last point, it's nice to see them at least "walking the walk" in so far as getting their own people to do more translational work as well as telling those of us out in academia to do it. Finally, although (like study section/council), funding decisions are not made at the review panel level, we were told going in that in this particular cycle, anything other than a perfect score would be cut at least 10%, and anything less than half-way scores (equivalent to say a 4 or 5 at study section) would run a serious risk of that person's program being axed).

"Review progress is focused very heavily on previous performance. Much more so than RO1 type grants. Typical cycle is 5 years, so your ideas for the next 5 are determined by how many papers you published in the last 5."

You mean it's possible to fund science this way, in a big way? And have the research programs be viable? And end up with PIs who don't have to spend 50%+ of their time writing grants? WHO WOULDA THUNK IT?

I am a current postdoc in the intramural program at the NIH, at one of the higher-funded institutes. I've been here almost 4 years. I can honestly say that yes, there is amazing research that goes on here, that absolutely cannot be easily done elsewhere (most of this falls under the translational research umbrella), but that most of the PIs I know (and by know, meaning hearing mostly about them/their research from their postdocs) would not make the grade in the world of academia. Now, I'm generalizing, but for the most part, they don't thoroughly plan out studies; they get an idea, and just start with it because they can, and then years go by and papers never get published. They know every October they'll get another wad of dough, and that they can probably go overbudget because the cash will come from somewhere. Also, there is no pressure to publish. I know a lab with a project that has been going on for well over 6 year, sent to 3 different journal, and is still not being published - this is due to a lack of proper planning/experimental design, and the PI just adds more experiments, and doesn't think to address the root of the problem. If they have long studies like this, as long as they are being submitted soon, or currently under review (even it is the 3rd journal!), it helps them get through the 5-year BSC review. Sometime non-planned-out studies are a benefit - the Undiagnosed Disease Program is an example of that - but success in those studies are really "Black Swan" events and not the norm.

When I first came to NIH I thought it was the best place ever - so much $$, core facilities, etc. But the longer I am here, the more I want to go back to academia, even if it means writing grants and groveling for money. From what I've seen here, writing grants makes you a better scientist, and I want to be a better scientist.

[...] Giant Insects? Vaccinations of US children declined after publication of now-refuted autism risk Is NIH Intramural research “unique”….and at what cost? Even our electricity is vulnerable to climate change Use it or lose [...]

Based on publications, the research is pretty ordinary, in my little subfield.

I have a lot of friends in the intramural program at NIH, mostly at the post-doc level, and some at the faculty level. From talking to them (and they seem to love to talk about it), at least in our little subfield, there are several factors that counterbalance the benefits of free money. Like, difficulty getting graduate students, the zero-sum nature of funding there, the need to justify oneself, the political games.

In the end, it seems, from what I hear, that there is more pressure to spend large amounts of money on high-risk high-payoff (=science/nature papers) work. And that leads to non-productive post-docs. There are a few glamour child post-doc superstars, but there are a lot of people who get trapped in intramural purgatory.