Broader Impacts, Revisited

It’s been a tumultuous 18 months for the Broader Impacts Criterion (BIC), the NSF merit review standard that gets a lot of grant-seeking scientists and engineers thinking about including plans for education, outreach, and diversity. First, Congress asked the NSF to explain exactly what it means by "broader impacts," and how it monitors compliance. Then, the National Science Board stepped in, announcing it would review both the BIC and the IMC (the companion criterion for "intellectual merit.") Then, in June 2011, the NSB issued a set of draft guidelines for the BIC that listed explicit national goals like economic competitiveness and national security while seemingly downplaying the agency's historical commitment to diversity, education, and science literacy. You may remember my July 2011 post criticizing that draft for ripping open a “BIC loophole” by suggesting that broader impacts could be achieved “through the research itself.”

AAAS and a number of other science organizations and individuals protested en masse. (See “Beyond the Data,” by Jeffrey Mervis in Science, vol. 334, pp. 169-171). In a great affirmation of dialogue and deliberation, the NSB scrapped that draft and began work on another. We have yet to see it. Although NSF Director Subra Suresh announced in a March 2012 "Important Notice" (in132.pdf) that in April the agency would file a public Federal Register Notice containing “information highlighting specific changes that are necessary to implement the Board’s recommendations regarding the Foundation’s review criteria,” to be followed by another period of public comment - this notice has yet to appear. Nevertheless, a few structural elements have begun to trickle over the dam.

A Google search, for instance, netted a May 2012 slide presentation delivered to an EPSCoR PD/PA crowd by Jean Feldman, policy head of the NSF’s Office of Budget, Finance, and Award Management. In it, Feldman described a new review framework that would apply the standard five critical elements to BOTH merit criteria, rather than just to the IMC (Intellectual Merit Criterion), as is current practice. This means that a PI's plans for addressing the Broader Impacts Criterion will also be scrutinized for (1) their potential to advance knowledge and societal goals; (2) the extent to which they explore creative, innovative, and original concepts; (3) their rationale, organization, and assessment strategies; (4) the qualifications of the team charged with pursuing them; and (5) the adequacy of resources available to the PI – either at the home institution or through collaborations – to carry out the proposed activity.

This is exactly what some of us in the education world have been calling for. Let’s strengthen the role of the BIC by applying to it the same rigorous set of guiding review principles we currently apply to the IMC. Let's give both the scientific and the broader impacts efforts intellectually-sound, evidence-based foundations. And, let’s put an end to this double standard where review panels wink at the BIC, and researchers address it only as an afterthought. For those researchers who worry that diversity, education, outreach and societal impacts are not within their areas of expertise, the message is to do it “through collaborations.”