Monitoring and Evaluation

Mar 13, 2015

At Room to Read we are passionate about testing our assumptions. We do not want to take for granted that what we’re doing is valuable and effective, we want to be able to prove it and improve it. When it comes to our school libraries, it can be easily supposed that our libraries are enhancing children’s reading habits, but we wanted to test this assumption and find out more.

Does having a library help make a child more likely to read for pleasure at school? And what about at home? Does having a library impact a child’s attitude towards reading? These are some of the questions we wanted to answer.

Measuring the impact of thousands of libraries across multiple countries is quite a formidable undertaking, but with support from the Bill and Melinda Gates Foundation, we commissioned an independent, external research team to examine our libraries in Laos, Nepal, India, Sri Lanka, Zambia, and South Africa. To establish a baseline, we began evaluations at the schools before a Room to Read library was established and tracked progress in reading habits over the course of two years. Across all six countries, the evaluator visited more than 2,400 schools and interviewed more than 20,000 children.

The most exciting takeaway from the study is that we have been able to confirm empirically that our libraries are helping children become independent readers. In India, Laos, Nepal and Sri Lanka, 51 percent of children from schools with a Room to Read library were reading for enjoyment at school — compared to only 29 percent of children from comparison schools without a Room to Read library.

And in India, Laos, South Africa and Sri Lanka, 51 percent of children from project schools were reading for enjoyment at home — compared to 33 percent of children from comparison schools.

The study revealed more nuanced insights as well. For example, in India, examining results on the state level revealed that Chhattisgarh libraries were noticeably outperforming the libraries in Rajasthan. A closer look revealed that training staff within the schools to manage the library instead of training external library managers proved to be far more impactful. Therefore, Room to Read now exclusively uses this more successful method. This finding also highlights that beyond good program design, the specifics of the implementation of that program design can make a big difference in results.

With affirmation and insights like those revealed in this study, we can continue to expand and grow our library programs with the confidence that we are helping millions of children become independent readers.

Aug 04, 2014

Watch this video to see how listening to data is improving the lives of students in Sri Lanka.

With so many challenges facing children trying to learn to read around the world, we want to provide more than our best intentions and best efforts. We want to provide the best solutions and the best results as well. That's why we go to such great lengths to research, monitor and evaluate our programs. We put some of the world's best global education researchers, like Matthew Jukes (who you will meet in the video) on the job, and many more trained data collectors on the ground so that we can test and objectively measure the efficacy of our programs. The data not only helps shape our program design, it also helps provide "live" feedback and guidance to the educators in the field. So they can make the adjustments necessary for their students to become better readers.

Jul 25, 2014

As a data driven organization, we love to let the numbers tell our story of impact. For this year's annual report we did just that. Ever wondered how Room to Read manages to scale quality education so efficiently around the globe? The 2013 Annual Report uses beautiful infographics in an easy-to-explore online interface to illustrate how we constantly strive to be smarter about our processes, tools, people, and impact. That's why this year's theme is "Working Smart." We hope you enjoy exploring all these stories of measured positive impact around the world.

Apr 10, 2014

Sustainability. It’s a word that is now being used to talk about everything from food to clothes and businesses and about everything in between. Sustainability has become a well-established buzzword, perhaps because there just are not very many good synonyms for the idea or perhaps because it helps sell coffee, but mostly because sustainability matters. At Room to Read, when we talk about sustainability in education, we’re talking about respecting the dignity and agency of all people, and empowering and equipping communities to take hold of their futures. Sustainability also matters in a world where there are far more things to do than time to do them. We need to know that what we’re doing is leading us to a better tomorrow, and that the investments we make can last. This was the precisely question in the air at a recent ceremony handing over 43 school libraries supported by Room to Read in Zambia to the local districts, Chilanga and Kafue.

Chilanga representative ceremonially opening the library

Samantha Msoni, who has been investing in these libraries and training teachers and librarians for the past three years there, attended the ceremonial passing of the torch to the Zambian Library Service. Even though Samantha knew that this transfer of responsibility was necessary for sustainability, and that it would enable them to start new projects in order to reach more communities in need, she couldn’t help but worry. Like a caring parent watching her child graduate, she says, “I sat listening attentively to the speeches with mixed feelings and worried about a number of things.” “Of course my first reaction was that I would miss working in these schools, the Head teachers and the Teacher Librarians in whom I had invested a lot.” She also worried about the improvements she still wanted to make in some of the schools and wanted to know that the new caretaker would help support the libraries and communities. Her fears were put to rest as she listened to the Zambian Library Service representative outline their mandate. “Listening to his speech felt like he was reading our own strategy.” She recalls sharing a knowing glance with our Zambian country director as they both felt a wave of relief knowing these 43 school libraries would be in good hands. “Now we can safely direct our efforts to other districts knowing it is well with Chilanga and Kafue!”

Room to Read Director hands over the completion certificates to Chilanga DE

At Room to Read we are very proud of the fact that we have been able to reach over 8,000,000 children, and have built more than 16,000 libraries. While we love building and establishing libraries and watching our progress numbers go up, we are much more anxious that these libraries continue to function at their highest capacity because a library that thrives helps children to thrive.

Therefore, after enough time had passed, we revisited a large sample of libraries years after they had completed our three year project support cycle to see if they were still functioning. We were eager to know whether or not our work was achieving sustainability. So how had they fared? Answer: unbelievably well. An amazing 97% of these libraries were still functioning with many having even made further quality improvements. And thanks to improved library monitoring and evaluation and community engagement strategies, we expect this trend to go even further so that millions more children may have access to a quality library and a quality education for years to come.

Apr 04, 2014

Erin Ganju co-founded Room to Read in 2000 with John Wood and Dinesh Shrestha and is CEO of Room to Read. This post, authored by Erin Ganju, originally appeared in theWorld Economic Forum Blog on March 27, 2014 and in the Huffington Post on March 28, 2014.

***

According to the 2014 Edelman Trust Barometer, the communications agency’s 14th annual exploration of trust in institutions and perceptions impacting trust in business and government, NGOs are the most trusted institution in 20 of the 27 countries surveyed. While in many countries, respondents’ trust in NGOs to do the right thing has risen from 2013 levels, in nine countries, trust levels stayed the same or declined, including a decrease from 70% to 61% in the US.

Building and maintaining trust is a key component in the arsenal of any business; however, social impact enterprises are uniquely reliant on trust to build sustainable partnerships with investors, governments, partners and beneficiaries. Without trust and transparency, social enterprises would not be able to operate successfully and leverage their collaboration to ensure they are truly changing the world.

Our instinct is to tout the most impressive results and significant stories that show how we are improving communities around the world and build a brand founded in success. We craft polished documents and organize curated site visits to show donors the unequivocal outcomes of their investments, because that is what we think they want.

However, could a more strategic approach to gaining trust among constituents be to “lift the veil” and spotlight the flaws in our work that are opportunities to improve our processes? Might this approach more actively engage them in helping create and be even more committed to our solutions? As the non-profit sector seeks to attract investors and partners and develop lasting relationships built on trust, organizations must communicate frequently and honestly about the state of their business – sharing both the positive and the negative openly and without refrain.

Social entrepreneurs understand the value of investing in research, monitoring and evaluation of their work to improve programme design and implementation, and efficiently target resources. We are well versed in getting a lot done with limited resources. We should ensure that we are also embracing objective evaluations of the impact we are having and conduct rigorous, unbiased research studies to provide honest assessments. Sharing this information with other practitioners in our field means that, together, we can figure out what really works and focus on systemic, lasting change.

Furthermore, we should embrace open dialogue with investors about how we are learning from these findings – particularly those that uncover flaws – and invite them backstage to witness how an entrepreneurial mindset is applied to solve the most pressing challenges facing the world today.

As CEO of Room to Read, I have embraced candid dialogue with investors and partners, inviting them on our journey, identifying challenges and failures along the way that provide opportunities to recalibrate our approach and focus time and attention on necessary improvements.

In our most recent Global Monitoring Report, which is published publicly and shared with donors, we reported evaluations of our literacy programme on children’s reading skills in four countries where we operate. Findings in Bangladesh showed grade one students in our programme were reading, on average, three times the amount of words per minute of students in schools without our programme. The results continued to improve as these students progressed with our programme in grade two, reading nearly 56 words per minute, a benchmark on track with high-income countries.

However, findings from this same study in India showed minimal progress among grade one students, indicating that the programme impact did not emerge until grade two. When presenting these findings, we explain to investors that the literacy programme in India will use this data to improve programme design and delivery at scale, increasing the pace of instruction, emphasizing components of effective reading instruction and conduct continual reading assessments to provide teachers with feedback on the effectiveness of their instruction. We are met with understanding and newfound engagement as we collaborate with stakeholders to make these improvements.

As social entrepreneurs, we strive for success but must remember that we should deliver the same accountability to our stakeholders that we expect from the businesses and government that shape our world. Failure can be a wonderful, inspiring asset and, as a community of forward thinkers, I invite you all to flaunt your flaws and welcome the vested interest you will garner from like-minded stakeholders.

Oct 22, 2013

Now more than 15,000 strong, our network of school libraries has provided millions of children with a safe place to develop and nurture a love of reading. Key to the success of each library is a standard 3-year period of monitoring and support, in which the school receives regular library training from our staff.

In order to make the three years of support more effective, our team undertook a project over the past two years to develop a rating system for libraries—a series of sequenced indicators that provide insight into how well a particular school library is functioning. In addition to providing increased visibility into how our libraries are performing over time, the use of the rating system allows our teams in each country to assess schools’ overall training needs and prioritize support.

The system’s 19 indicators are divided up into four content areas— Literate Environment, Time to Read, Professional Development and Family & Community Engagement. As data comes in about individual library performance in each of these areas, Room to Read staff use the ratings to determine the frequency and type of support that each project site needs. The indicators are also sequenced and assigned to a certain phase of library development—from the moment an initial Memorandum of Understanding is signed by Room to Read, a school and the local community (indicator #1) to a school’s creation of a long-term sustainability plan (indicator #19).

“The best part about this new system is that it provides our teams in the field with measurable, real-time information about how libraries are performing,” says Peter Cooper, senior officer for Research, Monitoring & Evaluation. “At a global level, we can use this data to identify potential improvements to our program design.”

Nov 05, 2012

The
library at Lakhonepheng Complete Primary School in Salavan, Laos is always neat
and well-organized, thanks to the school’s dedicated staff. “The selection of
our school for a library is so meaningful for our community,” says principal Bouaphan
Dokdaoheuang, who notes that the village is more than two hours’ drive form the
provincial center.

The
school has been maintaining the library entirely on its own since Room to Read’s
three-year support period ended in 2011, so when our team returned for an
annual monitoring visit to find it running as smoothly as ever, we were extremely
pleased.

The
school’s teacher-librarian is an extremely energetic 23-year-old named
Daosavanh, who says her favorite part of her job is creating interactive
activities and games related to the storybooks on the shelves. “The library isn’t
just about reading good books,” she says, “but it is also like a student center—a
comfortable place where they can explore their infinite imaginations.”

“I
never tell the children to be quiet when coming into library,” shares Daosavanh.
“Instead, I read for them and encourage them to do many activities.” Her
technique is working, she says, adding that, “their excitement tells me that
they will be back another day.”

It would seem Daosavanh is on to something with her library management style, as several
of the 4th and 5th graders have started volunteering as
book classification assistants.

When
visiting Lakhonepheng Complete Primary School recently, our team met one eager
young reader named Kingsada, who insisted upon sharing his story. Now in 3rd
grade, Kingsada comes into the library each day to check out a new book. Clutching
his latest storybook selection to his chest, he recalled his first experience
in the library. “My teacher told a story from a book with beautiful pictures
called A Bunny and a Carrot,” he shared, beaming. “I liked the story so
much and looked through it many times on my own—page by page—to see the nice
drawings, [and] it became one of my favorite books!”

Kingsada’s
story is one of many, says Daosavanh, that shows how much reading habits at at Lakhonepheng
Complete Primary School have changed over the past few years. “Many teachers say
to me that it must be hard to wear two hats at the same time—to teach as well
as to take responsibility on the library operation,” she says. “To tell the
truth, I don’t feel that at all, because I can see the dramatic improvement of
my students reading skills since we have the library.”

Oct 25, 2012

The newly released Global Gender Gap Reportby the World Economic Forum signifies progress, but reveals that there remains much to be done to ensure equal access to education, particularly in less developed countries. A trained and empowered workforce leads to economic growth, but without investing in half of that workforce – girls and women – we undermine the global economy by ceding their collective talents and leadership.

USAID estimates that when 10 percent more girls attend school, a country’s GDP increases an average of 3 percent. Yet, when the majority of the world’s labor force is female, uneducated and underpaid, it is clear that something must change if women are to lift themselves, and their countries, out of poverty.

We know that for every year a girl is educated beyond the average, her wages increase by 15 percent, which in turn benefits her family, community and society. Yet, two-thirds of the world’s 775 million illiterate are women, 98 percent living in the developing world.

There is hope. The universal approach proven to increase the status of women is education. As the leader of Room to Read, a global non-profit organization focused on literacy and gender disparity in education across Asia and Africa, I have witnessed first-hand the transformative effect of empowering girls with a high-quality education.

One example is Prem, once a scholar in Room to Read’s Girls’ Education program, and now a 21-year-old nursing student. Prem is from Nepal, which ranks 123rd in the Global Gender Gap Indexoverall and 125th for literacy, placing Nepal in the bottom 15. Prem’s mother and two elder sisters are uneducated and work daily in a riverbed to earn a meager living. Through the power of education, Prem is now creating a brighter future for herself and her family and is on her way to breaking the cycle of intergenerational poverty.

It is great to see the progress in the Global Gender Gap Report, butwe cannot afford to forget those among the bottom ranks like Prem. Nepal, as well as other countries where Room to Read works such as Cambodia and Zambia, are places where we need to strategically focus and devote our resources. I look forward to a day when stories like Prem’s are not unique, and girls are enabled to step away from a life of poverty into a world of possibility.

Apr 19, 2012

This is a guest post from Peter Cooper, Room to Read's Senior Officer for Research, Monitoring and Evaluation.

*****

As an organization, Room to Read has always been committed to the sustainability of our programs. We know that a library is only as good as the number of lives it enriches, so we design each and every one of our programs to ensure the greatest possible sustainability and sense of community ownership.

In an effort to learn more about the sustainability of our school libraries and to put our own theories to the test, we launched a multi-country study in 2011 that examined a statistically representative sample of our post-completion libraries (i.e., projects that have “graduated” from our program and are now run without support from Room to Read). Over a period of months, independent evaluators funded by the Bill and Melinda Gates Foundation visited 517 school libraries in three countries—Cambodia, Nepal and South Africa—to see how effectively each project was functioning on its own and where Room to Read might enhance our program to improve the odds of success for all school libraries.

Our team put together the infographic below, featuring highlights from the study, so that our supporters around the globe can catch a glimpse of what we found. Take a look (to view the image full-size, use this link):

Sep 20, 2011

The excitement is palpable as the world descends upon New York for today’s opening of the 2011 Clinton Global Initiative (CGI). I’ve been lucky enough to attend CGI on behalf of Room to Read for the past five years, and now sit on CGI’s Advisory Board. Every year I’m in awe of how the forum has turned into a catalyst for game-changing partnership and collaboration in the field of international development. Each year, I leave with new connections and new ideas that we have been able to implement to make a difference for millions of children across the developing world.

This year’s CGI will be no different. Room to Read will unveil an innovative report, the inspiration for which was born at a meeting convened by CGI last spring. At the meeting, Cory Heyman, our chief program officer, met with representatives from FHI 360 and Amenons Nos Filles á l’Ecole (ANFE), two other nonprofits working in the field of girls’ education. Despite the multitude of organizations around the world dedicated to girls’ education, they noted, there has been very little research dedicated to identifying the factors that contribute to girls’ success in reaching higher education.

Together they questioned: If we know that gender bias, safety concerns, early marriage, pregnancy and financial disadvantage are major roadblocks to tertiary education, then what are the success factors that can help girls in the developing world overcome them? In other words—let’s not focus on the barriers, let’s learn about the real-world solutions that actually work.

As the idea for a study identifying such factors began to take shape, The MasterCard Foundation, with whom we first connected at CGI 2010, stepped in to provide guidance and financial support—kicking off six months of rigorous research.

The fact is, even under the most challenging conditions, there are young women across the developing world charting new territory and taking the bold step of seeking higher education. We wanted to hear from them—in their own words—about the factors that were positively contributing to their success.

Our team conducted in-person interviews with 160 of these trailblazing young women in Cambodia, Democratic Republic of Congo, Ethiopia, India and Mauritania, and found that despite political, cultural, and economic differences between their communities, the girls identified very similar success factors as being instrumental in their ability to reach university.

The young women we met shared personal stories that illuminated their common attributes of determination and courage. Many cited the encouragement of their families, teachers and communities as being paramount to their success, coupled with the material and emotional support they'd received from nonprofit organizations.

For Room to Read, these findings will help shape our Girls’ Education program curriculum, which we’ve built out over the years to go beyond financial support and to include extensive life skills training and mentoring as well as family and community engagement.

By releasing the final report at this year’s CGI Annual Meeting, we hope that these girls’ voices can reach across borders and help shape best practices in the field of girls’ education for years to come.

Read the complete report, Through Their Eyes, In Their Voices: Young women in five countries share their experiences navigating tertiary education, on our website.