Nonprofits: Master “Medium Data” Before Tackling Big Data

Every day humanity adds approximately 2.5 quintillion bytes of data to our collective store of knowledge. Looking over this treasure trove, scientists, financiers, and business leaders are justifiably giddy about the potential of Big Data. For the nonprofit community, Big Data also offers immense potential. But with our mere billions of data points we’re not quite ready for it. Instead, we need to get “medium data” right first.

Big Data is the search for meaning in the haystacks of massive databases of transactions, sensor readings, and records. For nonprofits, medium data is a humbler but essential prerequisite: structured information about who you are, what you’re trying to do, and what’s happening.

This may seem like a low bar but nonprofits face legitimate challenges in gathering, organizing, and using even basic data. First, most nonprofits are simply too cash-strapped to invest in cutting-edge information systems to track their activities, engage with their stakeholders and understand their context. Second, the diversity of organizations makes comparison difficult: how could we possibly compare the work of the University of Chicago to a homeless shelter in Albuquerque or to Greenpeace? Third, it isn’t easy to know which are the most effective programs for battling climate change or child slavery or homelessness. Finally, the unique economics of the nonprofit sector — the buyer (donor) is frequently a different person from the user (beneficiary) — interrupt the direct feedback loop that often drives innovation in business.

These are not just abstract problems; they have very real consequences. Nonprofits need to be able to answer urgent questions like, “Who else is working on homelessness in my town?” or “Has anyone else ever tried this approach for reducing teen pregnancy?” or “What do the people in our job training program think of our work?” The lack of data makes day-to-day tactical decisions hard and long-term planning practically impossible. And who pays the price? The people, communities, and ecosystems that nonprofits serve.

But years of effort to address these challenges have begun to reap rewards. According to research by the Hewlett Foundation (which I led when I worked there), there are now 371 platforms for gathering data about the nonprofit world: social indicators, capital flows, research about what works, and more. My own organization, GuideStar, now has about 1.4 billion individual pieces of data about the nonprofit world in the United States.

There’s more good news. There’s a rising culture of transparency and accountability in the nonprofit community. Nonprofit leaders are feeling the pressure to give up blatantly non-transparent practices. And new tools from big players like Salesforce, boutique firms like Social Solutions, and open source platforms like Fluxx all make it easier and cheaper to run data-driven organizations.

If nonprofits can embrace this new culture, build on these new platforms, and use these tools, medium data could transform the work of nonprofits. Organizations could radically increase their ability learn quickly. Better data could enable more and better collaboration among nonprofits and with other sectors of the economy. And there is reason to believe that the highest-performing organizations will be rewarded with additional funding. Philanthropy will always be emotional and personal, but recent research shows that the right information systems can help direct billions of dollars to proven high-performers — and away from organizations unable or unwilling to prove their effectiveness.

Of course, nonprofits cannot hope to magically reap the potential of medium data without some hard work. Here’s what nonprofit leaders need to keep in mind:

Don’t freak out. Nonprofits often panic about data because they worry about revealing weaknesses or compromising their funding. Remember that data are meant to complement intuition and stories, not replace them. Information should inform, not decide.

Focus on what nonprofits have in common. Nonprofits need to agree on — and then adopt — basic data standards. Every nonprofit may well be its own unique snowflake, but if we focus only on what makes us different, we’ll never reap the rewards of medium data. We all must be willing to share our stories in similar language, through shared formats, and on common platforms. As part of this, nonprofits need to support the central players that are building the core information infrastructure — whether the Foundation Center (data about foundations), the Global Impact Investing Network (data on impact investing), or my own organization, GuideStar (data about individual nonprofits).

Default to openness. Medium data only works if we share. Most nonprofits are simply too small to have a critical mass of data on their own. But together, we have enough data to reap immense insight and impact. There will always be data an organization should not share — but we need to switch our default from opacity to openness. Instead of opting-in to transparency when nonprofits feel like it, we should opt-out only when necessary.

It’s time for every nonprofit leader to step up to the challenge of data. Fear not: It doesn’t require terabytes of data and supercomputers. Medium data is simply organized storytelling — and if there’s one thing nonprofits do well, it’s tell stories about the need in our communities. Now is the time for us to tell an honest, open, shared story about ourselves.

This post is part of an online debate about How Big Data Can Have a Social Impact, which we’re hosting in partnership with the Skoll World Forum on Social Entrepreneurship. You can view the entire debate here.