What big data projects often lack: a return

Nov. 17, 2013 - 12:16PM
|

Big data-crunching projects can’t just be cool, they also need to prove themselves by showing a measurable return on investment.

“Even for mature analytics programs, this has been a struggle,” says a new report by the Partnership for Public Service and the IBM Center for The Business of Government.

If senior leaders are going to support data investments, they “need to see and understand the results and how they apply to achieving the agency mission,” the authors say. One way to do that is to package analytical findings in charts, graphs and other formats easy for top officials to understand, the report adds.

A Medicare fraud prevention system, for example, can tout an estimated $32 million in reduced fiscal 2012 spending because it prompted claims denials and other steps to avert bogus payments.

At the Centers for Disease Control and Prevention, a program known as PulseNet saves close to $300 million in national health care spending annually by tracking outbreaks of food-borne illnesses and striving to reduce their spread, one academic researcher has found.

The program originated almost two decades ago after tainted beef served at Jack in the Box fast food restaurants killed four children and sickened more than 730 other people.

In response, CDC and the Association of Public Health Laboratories created a database that now contains the DNA fingerprints of more than 500,000 strains of bacteria that can be used to determine whether an outbreak has occurred.

After beginning with just four participating labs, the program now has 87, with at least one in every state.

“Start small,” advised Dr. Chris Braden, head of the CDC’s division of foodborne, waterborne and environmental diseases, at a Partnership forum on the report. “Don’t try to collect everything about that topic.”

The report’s release came the same week that the Obama administration highlighted its investments in Big Data projects under an initiative announced last year. They include a National Institutes of Health partnership with universities to make better use of large data sets in disease research and an Energy Department program that — among other advances — has led to improved accuracy in forecasting seasonal hurricanes originating off of Africa, according to the White House.

But with money for new ventures growing more scarce, such programs face intense pressure to prove their worth.

Despite spending $3 billion between 2007 and 2012 on the collection of fingerprints and other biometric data from non-U.S. citizens in Iraq and Afghanistan, the Defense Department has yet to estimate a return on investment.

“Military guys view ROI as some sort of MBA thing that doesn’t apply,” said John Boyd, who directs biometrics and forensics programs for the Pentagon, and who is quoted in the report. “What resonates better, at least within DoD, is more of a risk-assessment standpoint, in other words ... more of an outcome metric.”

Without ROI yardsticks, biometrics is at a disadvantage in the battle for funding as the Defense Department’s budget shrinks.

While the Pentagon says that it uncovered 3,000 enemy combatants among the 1.1 million people in Afghanistan from whom biometric information was collected in 2012, the report says, “it’s difficult to say whether this is a terrific outcome or merely adequate, or whether it could have been achieved more effectively by other means.”