Initiative for Open Ag Funding : News & Multimedia

Initiative for Open Ag Funding

News & Multimedia

Highlighted News

Guest Blog: The Good, the Bad, And the Ugly of Using IATI Results Data

By Taryn Davis, Senior Associate at Development Gateway

It didn’t surprise me when I learned that -- when Ministry of Finance officials conduct trainings on the Aid Management Platform for Village Chiefs, CSOs and citizens throughout the districts of Malawi -- officials are almost immediately asked:

“What were the results of these projects? What were the outcomes?”

It didn’t just matter what development organizations said they would do -- it also mattered what they actually did.

We’ve heard the same question echoed by a number of agriculture practitioners interviewed as part of the Initiative for Open Ag Funding. When asked what information they need to make better decisions about where and how to implement their own projects, many replied:

“We want to know -- if [others] were successful -- what did they do? If they weren’t successful, what shouldn’t we do?”

This interest in understanding what went right (or wrong) came not from wanting to point fingers, but from genuine desire to learn. In considering how to publish and share data, the importance of -- and interest in! -- learning cannot be understated.

At MERL Tech DC earlier this month, we decided to explore the International Aid Transparency Initiative (IATI) format, currently being used by organizations and governments globally for publishing aid and results data. For this hands-on exercise, we printed different types of projects from the D-Portal website, including any evaluation documents included in the publication. We then asked participants to answer the following questions about each project:

1. What were the successes of the project?

2. What could be replicated?

3. What are the pitfalls to be avoided?

4. Where did it fail?

Taryn Davis leading participants through using IATI results data at MERLTech DC We then discussed whether participants were (or were not) able to answer these questions with the data provided.

Here is the Good, the Bad, and the Ugly of what participants shared:

The Good

1. Many were impressed that this data -- particularly the evaluation documents -- were even shared and made public, not hidden behind closed doors.

2. For those analyzing evaluation documents, the narrative was helpful for answering our four questions, versus having just the indicators without any context.

3. One attendee noted that this data would be helpful in planning project designs for business development purposes.

The Bad

1. There were challenges with data quality -- for example, some data were missing units, making it difficult to identify -- was the number “50” a percent, a dollar amount, or another unit?

2. Some found the organizations’ evaluation formats easier to understand than what was displayed on D-portal. Others were given evaluations with a more complex format, making it difficult to identify key takeaways. Overall, readability varied, and format matters. Sometimes less columns is more ( readable). There is a fine line between not enough information (missing units), and a fire hose of information (gigantic documents).

3. Since the attachments included more content in narrative format, they were more helpful in answering our four questions than just the indicators that were entered in the IATI standard.

4. There were no visualizations for a quick takeaway on project success. A visual aid would help understand “successes” and “failures” quicker without having spend as much time digging and comparing, and could then spend more time looking at specific cases and focusing on the narrative.

5. Some data was missing time periods, making it hard to know how relevant it would be for those interested in using the data.

2. The data and documents weren’t typically forthcoming about challenges and lessons learned.

3. Participants weren’t able to discern any real, tangible learning that could be practically applied to other projects.

Fortunately, the “Bad” elements can be relatively easily addressed. We’ve spent time reviewing results data for organizations published in IATI, providing feedback to improve data quality, and to make the data cleaner and easier to understand.

However, the “ugly” elements are really key for organizations that want to share their results data. To move beyond a “transparency gold star,” and achieve shared learning and better development, organizations need to ask themselves:

“Are we publishing the right information, and are we publishing it in a usable format?”

As we noted earlier, it’s not just the indicators that data users are interested in, but how projects achieved (or didn’t achieve) those targets. Users want to engage in the “L” in Monitoring, Evaluation, and Learning (MERL). For organizations, this might be as simple as reporting “Citizens weren’t interested in adding quinoa to their diet so they didn’t sell as much as expected,” or “The Village Chief was well respected and supported the project, which really helped citizens gain trust and attend our trainings.”

This learning is important both for organizations internally, enabling them to understand and learn from the data; it’s also important for the wider development community. In hindsight, what do you wish you had known about implementing an irrigation project in rural Tanzania before you started? That’s what we should be sharing.

In order to do this, we must update our data publishing formats (and mindsets) so that we can answer questions like, “How did this project succeed? What can be replicated? What are the pitfalls to avoid? Where did it fail?” Answering these kinds of questions -- and enabling actual learning -- should be a key goal for all project and programs; and it should not feel like an SAT exam every time we do so.

You work at a foundation, government agency, or nonprofit committed to reducing poverty and hunger. Recognizing the importance of agriculture for achieving this goal, you've decided to focus on improving the lives of smallholder farmers, who represent a significant portion of those living on less than $2 a day. You know which regions you want to work in, and now you're trying to determine which value chains you should invest in to create the greatest impact. As part of the Initiative for Open Ag Funding, Foundation Center has two new tools to help you answer that question.

First, an acknowledgment: such a decision requires an analysis of many, many data points. Among the factors to consider are: Which crops are produced by smallholder farmers? Which of those crops have the most potential to increase farmers' income? What does the market for these crops look like? What is the potential for significant productivity gains? Is there the infrastructure needed to get these goods to market? Who else is investing in these particular value chains?

The Initiative for Open Ag Funding focuses on this last question: Who is doing what, where, with whom, and to what effect? And rather than reinvent the wheel, the initiative uses the International Aid Transparency Initiative (IATI) data standard as its starting point. IATI aims to improve the transparency of international development and humanitarian resources and activities and has been widely adopted by bilateral and multilateral donors as well as many other organizations. To date, two of Foundation Center's major contributions have been: 1) filling a gap in IATI data; and 2) developing a tool to enrich that data so it better meets the needs of the agriculture sector.

Shedding Light on Foundation Funding for Agriculture

Foundation Center has been collecting and sharing data on foundations' grantmaking for decades. This data has been used to ground philanthropy research, inform grant prospecting, and foster collaboration. Given our comprehensive data on foundation grants and the fact that few foundations have published their data to IATI, we have opened our data on funding for international agriculture and food security activities. This data represents $4.3 billion worth of grants from nearly 1,900 funders to more than 3,000 organizations around the world. In addition to posting the data on the IATI Registry,* we've also made it accessible through a new and publicly available Open Agriculture Data map.

Making IATI Data More Relevant for Agriculture

At the moment, most data published to IATI is coded with OECD DAC purpose codes or the organization's own subject taxonomy. Early conversations with agricultural practitioners revealed, however, that these categories are not granular enough. In response, we developed an open source agriculture autocoder for the Food and Agriculture Organization's (FAO) AGROVOC thesaurus. Enter a project title, description, or any other text and, using machine learning, the OpenAgClassifier will return codes for terms such as rice or bananas or goats. (You can learn more about our approach to open source in this blog post by my colleague, Dave Hollander.) As a result, what would have been a time-consuming and probably manual process of identifying who is working in, say, the rice value chain is now much faster and easier.

Foundation Center and the Open Ag Funding team know that data and tools alone won't lead to smarter investments or more collaboration. Our goal is simply to give organizations a better starting point for making decisions about where and how to direct their resources. Given the progress of the open data movement, a lack of data or good tools shouldn't be a major reason why organizations duplicate efforts, why Organization A didn't know to go to Organization B to learn more about their approach, or why an organization really making a difference is invisible to those that have the means to support it. Our hope is that by putting the right data and tools at their disposal, we can make it easier for organizations to focus on the harder things about getting development right.

(*Note: To avoid duplication of data on the IATI Registry, we have removed funders already publishing to IATI from our IATI data.)

Org-id.guide is the easiest, quickest and most accurate way to locate organization identifiers.

The project allows open data publishers to uniquely and unambiguously identify any organization in the world by maintaining a register of organization lists. Those lists, in turn, provide unique identifiers for companies, charities, government agencies and other kinds of organizations. These unique identifiers help data users to find partners, improve collaboration and gain a better understanding of who is doing what, where and to what effect. Until now, no such list has existed, making the identification of organizations difficult and often inaccurate.

With guidance and funding from InterAction through Initiative for Open Ag Funding, the multi-partner project brings the open data community one step closer to the comprehensive data needed to improve the effectiveness of international development aid.

“Org-id.guide is a really clever way to avoid adding to an ever growing list of standards and code lists by building on what already exists and making unique organization identifications easier to find and use. That said, it immediately inherits all the shortcomings of the component lists and systems, hence the importance of the community governance structure and advocating for more open, better managed government registers and other third-party sources." – Reid Porter, Director of Open Data and Transparency

Related Blogs

The last week of June was a great week for open aid data. From June 28-30, Open Ag partners, software developers, donors and civil society representatives met in London (again!) to talk about tools for better data.

InterAction Project Staff

From Twitter

InterAction works to be a leader in the global quest to eliminate extreme poverty and vulnerability, strengthen human rights and citizen participation, safeguard a sustainable planet, promote peace, and ensure dignity for all people.