Could Do Better! Some Thoughts on a ‘Report Card’ for Global Learning

I recently came across a report card from my secondary school days in Ireland. It was an interesting read. My progress in areas as diverse as mathematics, singing, Irish language, and physical education was reported on in the form of marks, grades, and narrative feedback. Some teachers provided little information on my learning. Others went into detail. I was impressed by the number of areas in which my progress had been assessed (less so by my lack of singing ability, which, evidently, had been spotted early on!).

Flash forward to 2013, and there is a conversation raging in the development community about how to measure and report on learning globally. A huge concern is the fact that too often children leave school without acquiring the basic knowledge and skills they need to lead productive lives. To make matters worse, there is a global data gap on learning that is impeding efforts to better understand this crisis and how to achieve learning for all.

If we were to create a global ‘report card’ on learning, what would it look like? The multi-stakeholder Learning Metrics Task Force has been investigating this issue over the past 18 months and recently published a report that offers answers to three key questions:

1. What learning is important for all children and youth?
Based on research and consultations with stakeholders in countries around the world, the task force identified the following seven competencies as important for all children and youth: Science and Technology, Numeracy and Mathematics, Physical Wellbeing, Social and Emotional, Culture and the Arts, Learning Approaches and Cognition, and Literacy and Communication. The breadth of areas listed recognizes the need to educate the whole child –regardless of whether she lives in Dakar, Delhi, or Dublin – and that so-called non-cognitive skills are as important to individual and societal development as mathematics and science. Nonetheless, the list has been criticized by some as being blind to the gritty realities in many developing countries as well as in need of further prioritization in terms of essential versus desirable skills in each area and overall.

2. How should learning outcomes be measured?
The task force found that while global learning at the individual level is desirable, it is difficult to measure and report on in ways that have meaning across education systems and countries. Most cross-national assessment exercises focus on generating indicators in the ‘easier-to-measure’ areas of mathematics, reading, and science – and avoid the ‘harder’ areas of art, physical education (unless you count the Olympics), social and emotional skills, and so on. This does not mean that the latter areas are unimportant; rather, that our ability to measure and report on them in globally meaningful ways is lagging. In some instances, such as endangered national languages (Irish, in my case) and religious education, global indicators may have little meaning or be contentious. This means that any global ‘report card’ on learning is unlikely to provide direct indicators of learning in anything but a small subset of areas (albeit these are likely to be the core areas of numeracy and literacy on which acquisition of so many other areas depend). For the others, the emphasis will be on building capacity to measure and report on learning at the local level.

3. How can measurement of learning be implemented to improve education quality?
Most of the evidence reviewed by the task force indicates that measurement can be a highly effective tool for improving education quality and learning (not just reporting on it) if results are leveraged by students, teachers, government officials, civil society actors, and others to improve policy and practice on the ground. Effectiveness is strongly contingent on the timely feedback of results in a usable format to key stakeholders, something that is more likely with classroom-based and national-led assessment efforts (when done well) than with larger cross-national exercises. In this regard, it is important to recognize that any plans for measuring learning globally will, by default, need to emphasize local and national capacity to collect and use learning data, with the generation of global ‘report card’ indicators as a byproduct.

As we approach a new era of development goals for education, it is increasingly evident that access must be accompanied by a focus on learning (some developing countries have already demonstrated that it is possible to balance access and better learning). Given the global data gap on learning outcomes, and the key role that such data play in improving learning, it is evident that this is an area in which we all could and must do better. This week, on November 7-8, the World Bank will host a two-day Symposium on Assessment for Global Learning that brings together key stakeholders to begin to tackle how exactly we can do this. Watch the live webcast here.

Comments

Thanks for sharing such an interesting analysis of the challenges associated with measuring learning outcomes on a global scale. The recent experience with AHELO, under the leadership of OECD (http://www.oecd.org/site/ahelo/) confirms the complexity of this issue.

Thanks for bringing up the example of AHELO. The difficulties associated with measuring learning outcomes at scale is why so many efforts to evaluate the quality of higher education institutions tend to default into rankings based on reputation, publications, prizes, and so on. It’s far easier to measure these things than it is to actually see what students have learned. I think advances in cognitive science and technology, including in the areas of computer-based and adaptive testing, will really help to break down some of the barriers that until now have made it so difficult to collect and analyze large amounts of learning data across multiple contexts in a timely, robust, and meaningful manner.

After experiencing and being involved in the policy debates on learning outcomes over a few decades in a developing country, I am convinced that one cannot compare first world contexts and the developing world. However, the first world is a useful place to extrapolate in terms of benchmarks and so on. In developing countries there is a major need to reduce poverty. The children who experience major challenges are those that lack cultural and social capital. As Bowles and Gintis argued some time ago, it is about schools in general reproducing the status quo. The World Bank and other international organisations should insist on universal education with the major thrust on Early Childhood Education. There is an abundance of research that points to the limitations of deprivation in the formative years. Once this problem is resolved in terms of an early intervention, results will undoubtedly improve. Of course there are social evils that militate against progress but the the appropriate support mechanisms, change can come about. The problem is that we all want quick results. There are no quick results in education. In the education of the poor, we are in for the long haul. Schools and funding must compensate for poverty. Its worth it!

I'm a huge advocate of education for advancing civilization generally and agree with the comment above on the importance of early childhood education. But, I sometimes worry about the impacts of over testing and the long-term effects that trying to fit round pegs into square holes has on a young person's psychology. It seems like most of the methods are forcing everyone into the same boxes when we should be unearthing and celebrating the special gifts that each young person (and each of us) brings to the world. I often recall a quote from my own religious tradition, the Baha'i Faith, as follows: "The Great Being saith: Regard man as a mine rich in gems of inestimable value. Education can, alone, cause it to reveal its treasures, and enable mankind to benefit therefrom." Are we really doing this in our educational systems in a way that releases both the spiritual potential and practical capacities of each person? Probably not.

One danger of having a global learning goal is that it might be interpreted by some as requiring a global or common curriculum across countries. This is not something that we should want to see happen, although there are already hints of this tendency in some countries’ decisions to reform their national curricula to be more in line with international assessments of student achievement, such as TIMSS and PIRLS (which are seen by some as implicitly promoting a global curriculum). We need to be careful to separate the idea of a global learning goal from any notion of a global curriculum. A global goal for learning should still allow for national identify and priorities while keeping the spirit of universal learning to the forefront.

Early childhood education is, of course, a critical component of any effort to achieve universal learning, since early childhood is a key time for cognitive and physical development. A child’s brain has huge potential for learning at this stage and while neural pathways continue to be laid down over the lifetime, the early years set the trajectory of their learning for years to come.