Nonprofit Data: The Case of NTEN

Nonprofit Technology Network (NTEN)

Portland, OR

12 Staff

An organization learns to identify outcomes, define indicators, and gather and report data to track its own effectiveness in the community.

NTEN’s IT Director Karl Hedstrom has been working with the organization’s dashboard since he first joined as an AmeriCorps Volunteer six years ago. Since then, the dashboard’s role has gained prominence in NTEN decision-making, and currently the organization is working to enhance it further.

“Now is an interesting time,” he said. “Up to now, we’ve run into an issue where we can track a lot of stuff but we can’t answer some questions like how much of an effect we’re having in the community at large, so we’re going through a process with the board and staff to try to figure out how to dig into those types of questions.”

To that end, the organization identified five different outcomes to represent NTEN strategies, paraphrased below:

Technology is an essential part of a nonprofit organization’s operations.

There is an increased number of technology champions within nonprofit organizations.

Those nonprofit technology champions are equipped to make well-informed decisions to help their organization’s abilities to fulfill their missions.

Conditions exist that support helping nonprofits using technology to fulfill their missions.

“Those were the five high-level things we wanted to make sure NTEN was doing,” Karl said. “We broke those down further to figure out what would indicate we were moving toward each of those outcomes.” By way of example, he said, for outcome number two, they asked such questions as:

“Does participation of nonprofit staffers in the NTEN community grow over time? Are the number of staff members representing individual organizations growing? Are they representative of diverse organizational roles and abilities? Are nonprofit staffers taking part in an increasing number of NTEN services? What data do we have internally that would point toward this outcome happening?”

The next piece of the process, Karl said, was to take each indicator and try to define the specific data points that would speak to it.

“We had an outside consultant walk us through figuring out those five outcomes,” he said. “Once we had the outcomes, a senior staff team sat down to determine the indicators and the specific data points and methods for collecting them. This is still an ongoing process. We’re trying to transition to new high-level strategic data points that can directly connect us to the outcomes we’re interested in.”

For some indicators it was more obvious than others, he said.

“Looking at the participation of nonprofits in the NTEN community and whether it grows over time, for example, we had to figure out how to show that,” Karl said. “What does participation mean? How do we measure that? Up until now, partly because of what data is available to us, the definition of participation was relatively narrow—like, people who had attended NTEN events. But that’s just a part of it. There’s also community discussion lists, report downloads, speaking engagements, and others. There’s a lot more we want to include in that definition to show overall participation instead of a narrow definition.”

Tracking that data allows NTEN staff to effectively measure its work. Karl said that one thing he learned through the process was that during the outcomes-identification stage it’s less important to worry about what you know you can track than it is to define what you want to know under ideal circumstances.

“It’s hard to not start digging into the details—how are we ever going to know that?–-and set all that aside to think about what you want to know,” he said. “Once you have those outcomes you can dig a bit deeper, and once you have those indicators you can start digging into what data you have.”

Karl cautions other organizations that they will inevitably find that some of the indicators can’t be measured.

“Once we started thinking about what we could collect, what we could do, what was available to us, and started to connect the indicators to the data points, we had some that we just had no way to do that,” he said. For NTEN, the process is ongoing. Staff is still figuring out how to collect data points for each of the indicators and outcomes, including changing or configuring systems or establishing procedures to gather data, and finding the best way to present the data in reports and on the dashboard.

“Once we have the data on the dashboard, it continues to be part of the strategic planning,” Karl said “During our monthly strategic meetings and our weekly tactic meetings, we look at the dashboards and use that information that’s there. That’s why we collect it. It needs to be actionable.”

This case study is part of the research project in 2012 conducted by NTEN with the help of Idealware. See the State of Nonprofit Data report for more information about how nonprofits are--and aren't--making data part of their decision-making processes, and the key challenges that affect an organization's ability to be more effectively "data-driven."