Tue October 22, 2013

Charity Watchdog Shakes Up Ratings To Focus On Results

Dennis Chestnut stands next to a stretch of the Anacostia River in Washington, D.C., on Oct. 2. Chestnut, who has been working to clean up the Anacostia for decades, says it can take a long time for a nonprofit to see an end result.

A worker from Doctors Without Borders speaks with a sick child in Gao, in the north of Mali, on Feb. 4.

Sia KambouAFP/Getty Images

There's one area of the economy that's growing faster than business or government.

According to the Urban Institute, in the 10 years between 2001 and 2011, the number of nonprofits increased 25 percent. But most of them aren't very good at measuring their effectiveness — at least, that's the conclusion of the nonprofit watchdog Charity Navigator, which rates thousands of nonprofits to help donors make decisions on their giving.

Now, Charity Navigator is planning to change its ratings system. President and CEO Ken Berger says donors deserve to know if the money they're giving is going to programs that work.

"Think about the fact that [in] the largest nonprofit sector in the history of the world, we do not know whether or not we're having meaningful results and to what extent," Berger says. "It's not to say that they're not having results, but they often just don't know what the heck they are."

So Berger is shaking up Charity Navigator's ratings system. At the moment, Charity Navigator compares how much money a nonprofit spends on its programs with how much it spends on overhead. But beginning in 2016, the rating will also factor in results.

"How clearly do you identify the problem that you're trying to solve and how well do you have measures to know that you're on the road to solving that problem?" Berger says.

Give People A New Outlook, Then Measure It

But what if the problem you're trying to solve is in the middle of a war zone? Doctors Without Borders Executive Director Sophie Delaunay says she's leery about a system that would grade a nonprofit based on its results.

"I mean, it really depends on how they're going to use their results and what is their own understanding of what they're trying to analyze," she says.

Charity Navigator has suggested that one way nonprofits can evaluate their effectiveness is to ask the people they serve how they're doing. But Delaunay says that's "totally unrealistic" for doctors performing surgery in a war zone: "There is no way we're going to send a questionnaire to our patients, who are displaced and in a dramatic state, about whether they are satisfied with our care."

For other nonprofits, surveys are no big deal. Surale Phillips, a consultant to arts groups around the country, says many theaters and dance companies already ask their audiences for feedback.

"You probably get a questionnaire about what you thought of the show and the questions will be about the artistic process," she says.

Still, lots of nonprofit arts groups say their goal is to use theater or music to bring people together, or to make people think differently about the world. Those results are hard for arts organizations to measure, Phillips says.

"In the medical industry or environmental industry, or things that are not the arts, there are often standards that are across the board," she says. "We don't have those kinds of standards [in] the arts. We don't have, you know, CO2 levels to measure change."

But Dennis Chestnut does. He's executive director of the environmental nonprofit Groundwork Anacostia River DC, which measures change in the Anacostia River in Washington, D.C. Their mission is both restoration and education: Chestnut says they're trying to get people to care more about the environmental health of their community, learn how to improve it and then make it part of their daily lives.

"It might take ... a period of years to measure the impact," he says, "for us to, you know, actually see that end result, that outcome."

The Price Of Surveying Results

Another obstacle many nonprofits could face is how to pay for evaluations. Charity Navigator is asking for a fairly sophisticated process. With nonprofit jargon like "causal logic" and "pre-defined outputs," you need a glossary to get through the description. Phillips, the arts consultant, says it takes training to do the kind of evaluation they're asking for.

"It's not necessarily about sending one staff person to a workshop," she says. "It's a really intense process."

But Charity Navigator's Berger says nonprofits complained about the ratings formula way back when they first started doing them, 11 years ago.

"The outcry from the sector was: 'You're not measuring what matters most,' 'You need to evaluate us on our results.' That's what we were told until 2011," he says. "Now that we've got this, now we're being told, 'No, wait. It's too hard; it's too complicated; it's too expensive.' It's this, that and the other thing."

If nonprofits are going to ask people for money, Berger says, they should be able to show them their results.

Copyright 2013 NPR. To see more, visit http://www.npr.org/.

Transcript

DAVID GREENE, HOST:

Let's hear about an area the economy that is growing quickly. According to the Urban Institute, the nonprofit sector is growing faster than both business and government. In just 10 years, the number of nonprofits has increased 25 percent. And they face a problem: How to measure success. With a for-profit company, you can look to the bottom line. With a nonprofit, donors and others want to know if they're effective at accomplishing some mission in the world. Now, the nonprofit watchdog Charity Navigator is updating the way it rates charities on what they do.

KEN BERGER: Think about the fact that the largest non-profit sector in the history of the world, we do not know whether or not we're having meaningful results and to what extent. It's not to say that they're not having results, but they often just don't know what the heck they are.

BLAIR: So Ken Berger is shaking up Charity Navigator's ratings system. At the moment, Charity Navigator compares how much money a nonprofit spends on its programs with how much it spends on overhead. But beginning in 2016, the rating will also factor in results.

BERGER: How clearly do you identify the problem that you're trying to solve and how well do you have measures to know that you're on the road to solving that problem?

BLAIR: But what if the problem you're trying to solve is in the middle of a war zone?

UNIDENTIFIED MAN: Nurses and doctors trying to rebuild the withered bodies of dozens of children each day.

BLAIR: Doctors Without Borders' executive director Sophie Delaunay says she's leery about a system that would grade a nonprofit based on its results.

SOPHIE DELAUNAY: I mean it really depends on how they're going to use their results and what is their own understanding of what they're trying to analyze.

BLAIR: Charity Navigator has suggested that one way nonprofits can evaluate their effectiveness is to ask the people they serve how they're doing. Delaunay says that is totally unrealistic for doctors performing surgery in a war zone.

DELAUNAY: There is no way we're going to send a questionnaire to our patients who are displaced and in a dramatic state about whether they are satisfied with our care.

BLAIR: For other nonprofits, surveys are no big deal. Surale Phillips is a consultant to arts groups around the country. She says many theaters and dance companies already ask their audiences for feedback.

SURALE PHILLIPS: You probably get a questionnaire about what you thought of the show and the questions will be about the artistic process.

BLAIR: But lots of nonprofit arts groups say their goal is to use theater or music to bring people together or to make people think differently about the world. Surale Phillips says those results are hard for arts organizations to measure.

PHILLIPS: In the medical industry or environmental industry, or things that are not the arts, there are often standards that are across the board. We don't have those kind of standards at the arts. We don't have, you know, CO2 levels to measure change.

BLAIR: Dennis Chestnut measures change on the Anacostia River in Washington, D.C.

DENNIS CHESTNUT: Check the nitrogen, oxygen, just continue to collect all kinds of data.

BLAIR: Chestnut is head of the environmental nonprofit Groundwork Anacostia. Their mission is both restoration and education. Chestnut says they're trying to get people to care more about the environmental health of their community, learn how to improve it, and then make it part of their daily lives.

CHESTNUT: It might take over a period of years to measure the impact for us to, you know, actually see that end result, that outcome.

BLAIR: Another obstacle many nonprofits could face: How will they pay for evaluations? Charity Navigator is asking for a fairly sophisticated process. With nonprofit jargon like causal logic and pre-defined outputs, you need a glossary to get through the description. Arts consultant Surale Phillips says it takes training to do the kind of evaluation they're asking for.

PHILLIPS: It's not necessarily about sending one staff person to a workshop. It's a really intense process.

BERGER: I think that there's a lot of baloney in that sandwich.

BLAIR: Charity Navigator's Ken Berger says nonprofits complained about their ratings formula way back when they first started doing them 11 years ago.

BERGER: The outcry from the sector was, you're not measuring what matters most, you need to evaluate us on our results. That's what we were told till 2011. Now that we've got this, now we're being told, no, wait. It's too hard, it's too complicated, it's too expensive. It's this, that and the other thing.

BLAIR: If nonprofits are going to ask people for money, says Berger, they should be able to show them their results. Elizabeth Blair, NPR News, Washington. Transcript provided by NPR, Copyright NPR.