Madhabi Chatterji is an associate professor of measurement, evaluation, and education and director of the Assessment and Evaluation Research Initiative at Teachers College, Columbia University.
James Harvey is the executive director of the National Superintendents Roundtable and a doctoral student in educational leadership at Seattle University.

OECD: Poverty Explains 46% of PISA Scores

There has been much discussion lately around international exams such as the PISA, and the significance of student test results for American educational policy. This important conversation, however, is distorted when educators and politicians try to draw conclusions about the significance of test scores beyond what the test itself was actually designed to measure.

Iris Rotberg, in her adjacent article, argues that international tests of science and mathematics, such as the PISA, are not related to international economic competitiveness. She argues,

International test-score rank was only one of the 113 criteria used by the IMD to measure these variables. Performance on international test-score comparisons was not even mentioned among the 114 criteria used by the World Economic Forum.

Indeed, this should come as no surprise, because the PISA was never designed to be a predictor of economic competitiveness. Such tests may measure student knowledge and skills on an international basis, but they were not designed to measure the factors that contributed to student success within a nation. In 2013, in an international study of 1700 worldwide CEOs by IBM, the following four traits stand out as critical for an employee's future success: Collaborative, Creative, Flexible, and Communicative. These four traits are not even measured on the PISA tests.

As a school superintendent for 16 years in Massachusetts, I have seen the value of state, national or international standardized tests when they provide practitioners with actionable data they can use to assess the overall effectiveness of their school or school district's curriculum and instruction. Tests such as the PISA are useful tools if used for their intended purpose - to measure content knowledge and skills that students need to master for work and academic study after high school.

While it is true that the most recent U.S. scores were average among 62 education systems, the data do not tell us why one nation scored higher or lower than another. Unfortunately, U.S. Secretary of Education Arnie Duncan recently misused the student results and entirely blamed the public schools for the nation's average PISA results. When the PISA results were released in 2013, Duncan stated:

PISA is an important, comparative snapshot of U.S. performance because the assessment is taken by 15 year-olds in high schools around the globe. The big picture of U.S. performance on the 2012 PISA is straightforward and stark: It is a picture of educational stagnation... educational complacency and low expectations.

Even the OECD authors of the PISA test acknowledge that PISA results are due to a combination of variables, including but not limited to schooling, life experiences/home environment, poverty, access to early childhood programs, and health. In 2013, the OECD wrote in one of their reports that poverty explains up to 46% of the PISA mathematics score in OECD countries. At no time did OECD claim, as Duncan stated, schools' performance on the test can be blamed on low expectations and complacency.

PISA and other international standardized tests are useful in some contexts and not in others. These tests are not either entirely predictive of everything, nor entirely useless. Let's not overstate the usefulness of international standardized tests, but let's not condemn them completely.

If U.S. education policy is really concerned about student success after high school and college, then I suggest our nation needs to establish effective policies that will reduce childhood poverty and will ensure all students a high quality public academic instruction. I also recommend K-12 educators listen to the 1,700 worldwide CEOs who need graduates who are collaborative, creative, flexible, and communicative.

Paul B. Ash, Lexington Public Schools, Massachusetts

Categories:

Tags:

Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.

Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.