Certification

Audit Results

Trickle Up delivered a high-quality program to 7,195 vulnerable people living in extreme poverty in 2015 that increases income, food consumption, and savings, and continues reaching vulnerable groups at the approximate marginal cost of $340.

Impact Assessment

Trickle Up improves the lives of women and other vulnerable groups living in ultra-poverty, increasing income, food consumption, and savings.

This conclusion is reached by mapping lessons from similar programs implemented in similar settings for similar groups to the program Trickle Up runs.

Outcomes and cost

The evidence for Trickle Up’s impact, is based on rigorous, counterfactual evidence of the impact of similar programs. This evidence has the drawback of being from a similar program, and not from Trickle Up’s specific work, which does not yet have high-quality counterfactual evidence available. However, the evaluations of these other programs are methodologically strong and similar enough to Trickle Up’s program (in terms of intervention design, intervention fidelity, setting, and population reached) to conclude Trickle Up’s program very likely has large positive impact for women and other vulnerable groups living in ultra-poverty.

Operations Assessment

Trickle Up has directly delivered its program to 7,195 vulnerable people last year at high quality and is a learning organization and a transparent organization.

This conclusion is based on Trickle Up’s internal monitoring data, monitoring systems, and quality assurance protocols, which are credible and strong.

Activities and outputs: Quantity

The evidence for quantity is based on internal processed data.

Since 1979: over 200,000 participants directly

Since 2013: 6,600 households through technical assistance

Activities and outputs: Quality

The evidence for quality is based on internal protocols and program/ monitoring documents.

Internal monitoring staff: Yes

Routine staff training: Yes

Strong partner supervision: Yes

Responsive to data: Yes

Quality improvement: Yes

Learning organization

This is based on information obtained from interviews with Trickle Up’s president and senior staff.

Transparent organization

This is based on the breadth and depth of Trickle Up’s published reports, activity data, and organization data.

Four-Part Assessment

Quality of Impact Evidence:

Internal Evaluation: Samasource’s pre-post evidence shows substantial increases in earnings for participants after program completion. However, this evidence has a weak counterfactual and high risk of bias due to survey attrition.

Independent validation: Samasource is currently conducting a randomized controlled trial of its program with Innovations for Poverty Action. The study is in the planning stages but we anticipate it will be of high quality.

Evidence from elsewhere: There is high-quality evidence from a randomized controlled trial of a program that is moderately similar to Samasource.

Cost of Impact

Impact

From Samasource data: $1,150 in additional earnings after three years.

÷

Cost

From Samasource data: $1,800 in total cost per participant in the program.

=

Cost of Impact

$0.64 in additional earnings per $1 spent.

Predicted impact is based on ImpactMatters’ analysis of Samasource baseline and endline survey data from India and Kenya, 2011-2015. The impact of Samasource’s program may be short-term if other firms would eventually hire and train young underemployed workers, thus solving the information problem. There is no long-term follow up data.

Quality of Monitoring Systems:

Samasource has strong systems for tracking activities, targeting, engagement, feedback and outcomes, and routinely uses monitoring data to make decisions and validate assumptions. There are only minor areas for improvement.

Learning and Iteration

Samasource has a strong track record of learning from iteration testing. Changes are made systematically, periodically and based on high-quality data. Samasource has wound down programs that did not contribute to the mission.

Impact Audit

To develop and disseminate research-proven educational programs to ensure that all students, from all backgrounds, achieve at the highest academic levels.

Problem

Many children do not read or perform math at grade level. Many schools lack support or capacity to implement research-proven solutions to improve learning outcomes.

Intervention

SFAF implements a whole-school reform program, with a new cooperative-learning curriculum as well as staff training and regular monitoring of outcomes and reforms.

Process Metric

Schools implementing program

Outcome Metric

Child literacy

Geography

SFAF works primarily in the United States

Program Stage

Scaling

Size and Age

About 1,000 schools (2014) Founded in 1987

Certification

Audit Results

Success for All Foundation (SFAF) delivered a high-quality program to around 1,000 schools and 500,000 children last year that increases literacy as much as one grade equivalent, and reaches additional children at an approximate marginal cost of $119.

Impact Assessment

SFAF improves literacy for children, from all backgrounds, and very likely improves math achievement as well.

This conclusion is based on eight randomized and quasirandomized studies conducted on SFAF’s program over the past 22 years.

Outcomes and cost

Evidence for SFAF’s impact is based on two large, well-conducted cluster-randomized trials conducted on SFAF’s program, reported in 2007 and 2015, as well as six quasi-randomized trials conducted on SFAF’s program since 1993. Both randomized trials found economically and statistically significant improvement in child literacy. SFAF improves child literacy skills.

Operations Assessment

SFAF delivered its program at a high level of quality to about 1,000 schools teaching 500,000 children last year and is a learning and transparent organization.

This conclusion is based on SFAF’s internal monitoring data, monitoring systems, and quality assurance protocols, which are credible and strong.

Activities and outputs: Quantity

The evidence for quantity is based on internal processed data.

Last year: About 1,000 schools

Last year: About 500K students

Since 1987: About 2,000 schools

Since 1987: About 3 million children

Activities and outputs: Quality

The evidence for quality is based on protocols and other program and monitoring documents.

Internal monitoring staff: Yes

Routine staff training: Yes

Management control: Yes

Responsive to data: Yes

Quality improvement: Yes

Learning organization

This is based on SFAF’s history (28 years) of randomized and quasi-randomized trials as well as staff interviews.

Transparent organization

This is based on the breadth and depth of SFAF’s published reports, activity data, and organization data.

Learning and Iteration

Impact Audit

To move women in drought-prone pastoral communities in arid regions in Africa out of ultra-poverty and to increase their resilience to droughts.

Problem

Many women in Africa’s arid regions are living in ultra-poverty, at severe risk from droughts and suffering multiple, reinforcing deprivations, such as illiteracy and discrimination.

Intervention

BOMA puts ultra-poor women through two years of sequenced interventions (“graduation style”) with a cash transfer to start a business, training, mentoring, and group savings.

Process Metric

Business survival

Outcome Metric

Food security, income sources, savings

Geography

BOMA operates in rural, arid regions of Kenya.

Program Stage

Scaling

Size and Age

Operating since 2005

Certification

Audit Results

BOMA delivered a high-quality program to 9,432 women that increases the resilience of households through increased food security, new income sources and savings. BOMA continues to reach women with the program at an approximate marginal cost of $300.

Impact Assessment

BOMA improves the lives of ultra-poor women, increasing increases the resilience of households through increased food security, new income sources and savings.

This conclusion is from a combination of internal study results and evaluations of similar programs.

Outcomes and cost

The evidence for BOMA’s impact is from internal data and conclusions from similar programs, both of which have merits and drawbacks. However, together they add to a compelling argument. BOMA conducted a randomized trial on their own several years ago. The study found important changes in outcomes though there were some methodological weaknesses (sample size, measurement). Evaluations of similar programs are methodologically stronger and find large impact for ultra-poor women, but require judgment to compare those results appropriately apply to BOMA. They compare (in terms of intervention design, intervention fidelity, setting, and population reached) closely enough to conclude BOMA has large impact.

Operations Assessment

BOMA delivered a high-quality program to 9,432 women since 2005 and is a learning organization and transparent organization.

This conclusion is based on internal monitoring data, monitoring systems, and quality assurance protocols, which are credible and strong.

Activities and outputs: Quantity

The evidence for quantity is based on internal processed data.

Since 2005: 9,432 women directly reached

Since 2005: ~47,005 children reached

Since 2005: 2,651 businesses created

Since 2005: 452 savings groups formed

Activities and outputs: Quality

The evidence for quality is based on internal protocols and program/ monitoring documents.

Internal monitoring staff: Yes

Routine staff training: Yes

Strong partner supervision: Yes

Responsive to data: Yes

Quality improvement: Yes

Learning organization

This is based on BOMA’s 2013 randomized trial, as well as interviews with BOMA’s CEO and senior staff.

Transparent organization

This is based on the breadth and depth of BOMA’s published reports, including activity data and organization data.

Impact Assessment

Possible improves health outcomes among people who otherwise lack high quality care, and improves health care access and health outcomes for the marginalized.

This conclusion based on the quality of, and evidence-base behind, Possible’s health care, and the lack of alternative high-quality health care in the area.

Outcomes and cost

The evidence for Possible’s impact can substantiated Possible’s impact through the following logic: (A) Possible delivers health care services at high quality given the context, (B) many of those services have been shown, through a broad base of rigorous counterfactual evidence, to reduce patient morbidity and mortality when delivered at similar quality, and (C) others would not have provided similar care if Possible did not exist. Based on review of internal and external data, protocols, and other sources, all three steps in the logic chain are strongly supported, substantiating Possible’s impact.

Operations Assessment

Possible delivered high-quality health care services to 296,485 patients since 2008 and is a learning organization and a transparent organization.

This conclusion from Possible’s internal monitoring data, monitoring systems, and quality assurance protocols, which are credible and strong.

Activities and outputs: Quantity

The evidence for quantity is based on internal processed data.

Since 2008: 296,485 patients

FY 2015: 69,505 patients

FY 2015: 5,528 surgeries

FY 2015: 597 babies delivered

Activities and outputs: Quality

The evidence for quality is based on internal protocols and program/ monitoring documents.

Internal monitoring staff: Yes

Routine staff training: Yes

Management control: Yes

Strong partner supervision: Yes

Responsive to data: Yes

Quality improvement: Yes

Learning organization

This is based on the current quasi-randomized step-wedge study, and interviews with senior management.

Transparent organization

This is based on the breadth and depth of Possible’s published reports, activity data, and organization data.