We Need to Rethink How we Measure ‘Improved Sanitation’

We Need to Rethink How we Measure ‘Improved Sanitation’

Problem: Our current measures of sanitation progress are flawed, which could be leading us astray.

Barrier to Progress: Reported figures are taken too much on faith, and the standard “sanitation ladder” focuses on technology over outcomes.

Solution: More rigorous analysis of the data and mobile reporting from system users will generate more accurate information, and moving from infrastructure to outcomes in our measurements promises to keep us on target to achieving safe, sustainable sanitation for all.

Almost 15 years ago, a private contractor wrote some orbital navigation software for a Mars orbiter using American measurements, instead of the metric system NASA uses. The $125 million craft missed its orbit and became another hunk of space junk, currently orbiting the sun. As the UN discusses sanitation aims post-MDG, I’m afraid we’re making a similar error: we’re measuring the wrong thing, and then coming up with unreliable numbers. We may miss our real target too—and it’s not improved sanitation.

The UN measures progress along the “sanitation ladder.” Open defecation is at the bottom and each step up is defined by access to better facilities. The top rung is “improved sanitation,” which protects users from contact with the waste. This can be anything from a ventilated & slabbed pit latrine to a flush toilet. According to the UN’s own estimates, 2.5 billion people lack access to improved sanitation, including over a billion who practice open defecation.

Yet those numbers, bad as they are, may understate the problem. When researchers took a look at the UN Joint Monitoring Programme’s reported sanitation gains in East Africa, they found that the results just weren’t plausible: too much political upheaval, too little funding available, too much growth in crowded, impoverished, hard-to-serve urban areas. Other sources may be no more reliable. One sanitation access veteran recently wrote about the problem of “wildly, embarrassingly, inflated” numbers used by governments and funders alike, under pressure for measurable success now. He described it as the “elephant under the radar” at the recently concluded World Water Week.

Worse, “improved sanitation” may offer relatively little improvement beyond the individual level. A recent study by the Water Institute looked at improved sanitation systems to find out what happens to the waste itself. They found that sixty percent of the world’s population—4.1 billion people—use sanitation systems that simply dump the untreated waste back into the environment. If we’re serious about improving infant mortality, reducing fecal-borne illnesses, and using our water resources wisely, we can’t stop with improved sanitation.

The first and easiest step to improve things: Interrogate the data. The researchers who studied the JMP reports recommend cross-checking the numbers with other, more reliable figures such as population and income stats. All of us—NGOs, funders, governments, private sector partners—must insist on good data and rigorous analysis. Otherwise, we have no way of knowing what’s actually working.

Then, streamline the data-gathering. We’ve learned that community-based involvement creates sanitation systems that are better accepted and more functional than those that result from top-down planning. In the same way, community monitoring can bypass organizational filters, creating more accurate data and greater transparency. Apps, integrated GPS, camera phones, and SMS surveys put reporting power directly in the hands of users.

In 2010, Water for People and AKVO rolled out FLOW, the Field Level Operations Watch. This online system gathers and analyzes user data on tens of thousands of water points in 17 countries, and those numbers are reviewed and used by over thirty funders and NGOs. John Sauer of Water For People told me, “FLOW has been instrumental for us in adjusting our programming to meet real needs as they develop.”

We need a monitoring system like this for sanitation. The 2012 Sanitation Hackathon (for which Toilet Hackers was an implementing partner) brought us a few steps closer to that goal, producing useful innovations such as Taarifa and mWater’s sanitation-specific app. Last year’s SanHack was just our first volley of innovation in ICT and mobile technology. There is still a great need for breakthrough tools in technology (in both software and hardware), market approaches, and behavior change—all of which requires a fearless commitment to collaboration, to improving existing tactics, and experimenting with new ones.

Finally, donors, funders, and implementers need to integrate waste treatment needs into our sanitation goals and measurements. Public support is growing. In discussing a sustainable development agenda post-2015, citizens, scholars, government workers, and business people alike “reconsidered the use, reuse, value and even meaning of ‘waste’ water. … Rewards could go beyond resilience to boost growth, jobs and business certainty.” And the Joint Monitoring Programme’s own proposed targets (PDF at link) for ongoing sanitation development now include sewage treatment.

Most promising researchers are developing new methods of measuring our progress, moving away from specific technologies toward desired outcomes, including sanitation treatment. This allows for local flexibility, spurs innovation, and helps us all focus less on process and more on results. Truly improving sanitation requires working within, responding to, and shaping complicated social, technological, and environmental systems. That’s not a linear process, and a ladder isn’t going to get us to our ultimate goal.

Yes, improved sanitation has improved the lives of billions of people all over the world, and it’s important to note that success. But we need to make some course corrections. The stakes are just too high. Losing a multi-million-dollar spacecraft is a shame. Leaving billions of people to live with “improved sanitation” that includes untreated sewage—that’s an economic, environmental, and humanitarian catastrophe.

Kimbowa Richard

Agreed. Monitoring sanitation should be able to add value to any future campaign to reduce the glaring gap of billions with no access to sanitation

Mary Abraham

Can’t even imagine of the terrible consequences… improved sanitation is a must for development..

Let’s keep this going …

Why not join our growing community of social innovators? You’ll get exclusive content and opportunities delivered straight to your inbox and all the latest details from the Skoll World Forum.