Measures of the heart: non-cognitive skills tests

Being smart isn’t all you need to succeed.

Non-cognitive or meta-cognitive skills such as self-awareness, self-control, empathy, communication and cooperation are as important or even more important to long-term success than cognitive factors like math and reading ability. The evidence is undeniable, but US school assessment and accountability systems are currently limited to measures of cognitive skills alone. If schools don’t measure non-cognitive skills, it is difficult to prioritize them.

There are several emerging attempts to measure non-cognitive skills, aka Mindsets & Essential Skills and Habits (MESH) in schools. Most use the simplest instrument available: self-reporting surveys.

The OECD gives the Program in International Assessment or PISA to about half a million students around the world. It’s widely publicized for its rankings on math and language achievement. But in addition to the two hours of word problems or essay questions, PISA students also complete a 30-minute questionnaire asking about things like self-efficacy, self-esteem, student-teacher relationships, and the school climate, making it one of the largest attempts to measure non-cognitive skills in the world. An infographic published on the website Buzzfeed based on PISA data sorted countries about evenly into four quadrants: happiest and least happy kids, and high and low test scores. Korea stands out for its miserable kids and high scores, Singapore has the happiest kids with the highest scores, Qatar and Argentina have sad, low-scoring kids and Peru and Indonesia are whistling away their weak scores. (The US falls into the unhappy, low-scoring bottom left quadrant.)

School leaders around the world are using PISA findings on the emotional health of their students. In Korea, in February 2013 the new president Park Geun-hye announced that the happiness of citizens would become a national priority, and the Ministry of Education has taken this up as a goal.

In 2012 almost 500,000 students across the US in grades 5-12 completed the Gallup Student Poll, which asks about students’ hope, engagement, and well-being. Over the last 10 years, scores on the hope survey did a better job of predicting college persistence and GPA than high school GPA, SAT, and ACT scores. In the most recent survey, about a third of students were “struggling” and 43% were either “not engaged” or “actively disengaged” with school.

YouthTruth is a 20-minute online survey that’s been taken by about 250,000 students in grades 3 through 12 at 300 schools, covering school climate issues like bullying, classroom engagement, as well as optional topics like student motivation and grit. Schools are using YouthTruth to gather feedback not only on the school as a whole but also on individual teachers.

The State of California’s Department of Education, starting in 1998, has administered the largest statewide survey program in the nation looking at “resiliency, protective factors, and risk behaviors”, made up of the Healthy Kids Survey, the School Climate Survey, and a parent survey. In 2013-14 they updated the survey to track learning engagement, students’ connectedness to school, and a new optional social-emotional health section covering empathy, self-efficacy, self-awareness, persistence, emotion regulation, gratitude, ‘zest,’ and optimism. The social-emotional health section for middle school students asks them to identify with statements like “each day I look forward to having a lot of fun,” and “I can do most things if I try.”

Once schools have adopted one of these surveys, the next step is to incorporate them into decisionmaking processes. The Montgomery County Public Schools in Maryland has incorporated the Gallup hope survey into its strategic planning processes and goals. A group called Transforming Education is advising the California Office to Reform Education, a group of eight big-city school districts in California that has obtained its own separate NCLB waiver. They are setting up one of the first large-scale accountability systems that places non-cognitive measures as peers to test scores. Schools within these districts will be evaluated on 60% conventional test scores, 20% school culture/climate measures such as suspensions, expulsions, and bullying, and 20% social/emotional learning scores.

The idea of attaching stakes to self-reported measures of happiness is fraught with peril. Surveys that are perfectly valid when administered as research tools may be distorted when they become the basis for school and teacher ratings. It’s far easier to game a self-reported survey than, say, a math test. To overcome these problems, researchers such as Patrick Kyllonen at ETS are trying to develop more sophisticated measures, such as asking students to provide a solution to a hypothetical interpersonal problem, demonstrating their emotional wisdom. Another approach is to combine multiple measures such as suspension rates and teacher reports with student survey data to get a more complete picture of the climate at a school. Regardless of the path chosen, we are headed towards a world where students’ emotional health and wellbeing gets full attention commensurate with its importance in determining their futures.

Thanks! This was a great article that spurred me to finally do a video post on the topic. I have been mulling it over for a while. If you go to my YouTube channel linked here, it should be posted in about 30 mins or so.

[…] South Korea has one of the most competitive education systems in the world. For students in Korea it is a cut throat world, particularly in high school, where scores and class rank is as common knowledge as your last name. This competitive environment has cultivated many bright minds and has spurred educational success in the now developed nation. However, this has come at the high cost of student happiness. While they are able to achieve some of the highest test scores in the world (not the highest, Korea currently ranks in around 5th), they also rank as the least happiest. […]