The latest on each state s international standing in math and reading

Transcription

1 Globally Challenged: Are U.S. Students Ready to Compete? The latest on each state s international standing in math and reading Paul E. Peterson Ludger Woessmann Eric A. Hanushek Carlos X. Lastra-Anadón Prepared under the auspices of: Harvard s Program on Education Policy and Governance & Education Next Taubman Center for State and Local Government Harvard Kennedy School EN

2 Acknowledgements We are grateful for the Kern Family Foundation s support for this project. We are indebted to Michael Petrilli for his helpful feedback on earlier drafts; to Kathryn Ciffolillo for her careful editorial work; to Bruce Sanders and Robin Cheung for their excellent design work; to Ron Berry for table and figure construction; and to Ashley Inman and Antonio Wendland for administrative assistance and technical support. The views expressed here are our own and should not be attributed to any other party or to the institutions with which we are affiliated. Websites: hks.harvard.edu/pepg educationnext.org

3 Globally Challenged: Are U. S. Students Ready to Compete? The latest on each state s international standing in math and reading by Paul E. Peterson Ludger Woessmann Eric A. Hanushek and Carlos X. Lastra-Anadón Prepared under the auspices of Harvard s Program on Education Policy and Governance & Education Next Taubman Center for State and Local Government Harvard Kennedy School Program on Education Policy & Governance PEPG Report No.: August 2011 Websites: hks.harvard.edu/pepg educationnext.org Harvard University EN Education Next

4 Table of Contents Table of Contents Executive Summary... v Introduction... 3 Comparing U.S. Students with Peers in Other Countries... 4 Proficiency in Math... 6 Proficiency in Reading Performance of U.S. Ethnic and Racial Groups White students Students from college-educated families Are the Proficiency Standards the Same for Math as for Reading? What Do These Findings Mean? Appendix: Differences in Math Performance of the High School Classes of 2009 and Performing the Crosswalk Identifying the Class of Critiques of PISA References Biographical Sketches Figures Figure 1 Percentage of students in the Class of 2011 at the proficient level in math in U.S. states and foreign jurisdictions participating in PISA Figure 2 Percentage of students in the Class of 2011 at the proficient level in reading in U.S. states and foreign jurisdictions participating in PISA Figure 3 Percentage of students in the Class of 2011 at the proficient level in math and reading, by race and ethnicity Photography: Cover: Getty Images, Echo P. iv: Charles Vest, AP Images, Paul Sancya P. vii: Getty Images, Jack Hollingsworth P. viii: Getty Images, Wealan Pollard P. 3: President Obama, AP Images, Pablo Martinez P. 5: Vinton Cerf, AP Images, Lai Seng Sin P. 7: Getty Images, Wealan Pollard P. 11: Getty Images, Peter Cade P. 17: Getty Images, Image Source P. 18: Michael Brown, Rich-Joseph Facun, The National newspaper Figure 4 Estimated increase in U.S. annual economic growth from improved mathematics proficiency Figure A.1 Percentage of white students in the Class of 2011 in U.S. states at the proficient level in math and percentage of all students at that level in foreign jurisdictions participating in PISA Figure A.2 Percentage of white students in the Class of 2011 in U.S. states at the proficient level in reading and percentage of all students at that level in foreign jurisdictions participating in PISA ii educationnext.org hks.harvard.edu/pepg

5 Table of Contents Figure A.3 Percentage of students in the Class of 2011 with at least one college-educated parent in U.S. states at the proficient level in math and percentage of all students at that level in foreign jurisdictions participating in PISA Figure A.4 Percentage of students in the Class of 2011 with at least one college-educated parent in U.S. states at the proficient level in reading and percentage of all students at that level in foreign jurisdictions participating in PISA Figure A.5 Percentage of students in the Class of 2011 at the advanced level in math in U.S. states and foreign jurisdictions participa ting in PISA Tables Table 1 Percentages of all students in the Class of 2011 at the proficient level in math per state. Foreign jurisdictions with similar and higher percentages at the proficient level in math in overall student population Table 2 Percentages of all students in the Class of 2011 at the proficient level in reading per state. Foreign jurisdictions with similar and higher percentages at the proficient level in reading in overall student population Table A.1 Percentages of white students in the Class of 2011 at the proficient level in math per state. Foreign jurisdictions with similar and higher percentages at the proficient level in math in overall student population Table A.2 Percentages of white students in the Class of 2011 at the proficient level in reading per state. Foreign jurisdictions with similar and higher percentages at the proficient level in reading in overall student population Table A.3 Percentages of students in the Class of 2011 with at least one college-educated parent at the proficient level in math per state. Foreign jurisdictions with similar and higher percentages at the proficient level in math in overall student population Table A.4 Percentages of students in the Class of 2011 with at least one college-educated parent at the proficient level in reading per state. Foreign jurisdictions with similar and higher percentages at the proficient level in reading in overall student population Table A.5 Percentages of all students in the class of 2011 at the advanced level in math per state. Foreign jurisdictions with similar and higher percentages at the advanced level in math in overall student population GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? iii

6 America faces many challenges but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency. Charles Vest, Former President Massachusetts Institute of Technology

7 Executive Summary Executive Summary At a time of persistent unemployment, especially among the less skilled, many wonder whether our schools are adequately preparing students for the 21st-century global economy. This is the second study of student achievement in global perspective prepared under the auspices of Harvard s Program on Education Policy and Governance (PEPG). In the 2010 PEPG report, U.S. Math Performance in Global Perspective, the focus was on the percentage of U.S. public and private school students performing at the advanced level in mathematics. 1 The current study continues this work by reporting the percentage of public and private school students identified as at or above the proficient level (a considerably lower standard of performance than the advanced level) in mathematics and reading for the most recent cohort for which data are available, the high-school graduating Class of Proficiency in Mathematics U.S. students in the Class of 2011, with a 32 percent proficiency rate in mathematics, came in 32nd among the nations that participated in PISA. Although performance levels among the countries ranked 23rd to 31st are not significantly different from that of the United States, 22 countries do significantly outperform the United States in the share of students reaching the proficient level in math. In six countries plus Shanghai and Hong Kong, a majority of students performed at the proficient level, while in the United States less than one-third did. For example, 58 percent of Korean students and 56 percent of Finnish students were proficient. Other countries in which a majority or near majority of students performed at or above the proficient level included Switzerland, Japan, Canada, and the Netherlands. Many other nations also had math proficiency rates well above that of the United States, including Germany (45 percent), Australia (44 percent), and France (39 percent). Shanghai topped the list with a 75 percent math proficiency rate, well over twice the 32 percent rate of the United States. However, Shanghai students are from a prosperous metropolitan area within China, with over three times the GDP per capita of the rest of that country, so their performance is more appropriately compared to Massachusetts and Minnesota, which are similarly favored and are the top performers among the U.S. states. When this comparison is made, Shanghai still performs at a distinctly higher level. Only a little more than half (51 percent) of Massachusetts students are proficient Globally Challenged: Are U. S. Students Ready to Compete? The latest on each state s international standing in math and reading Paul E. Peterson Ludger Woessmann Eric A. Hanushek Carlos X. Lastra-AnadÓn 1. Hanushek, Peterson, and Woessmann (2010). GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? v

8 Executive Summary in math, while Minnesota, the runner-up state, has a math proficiency rate of just 43 percent. Only four additional states Vermont, North Dakota, New Jersey, and Kansas have a math proficiency rate above 40 percent. Some of the country s largest and richest states score below the average for the United States as a whole, including New York (30 percent), Missouri (30 percent), Michigan (29 percent), Florida (27 percent), and California (24 percent). Proficiency in Reading The U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. It takes 17th place among the nations of the world, and only the top 10 countries on PISA outperform the United States by a statistically significant amount. In Korea, 47 percent of the students are proficient in reading. Other countries that outrank the United States include Finland (46 percent), Singapore and New Zealand (42 percent), Japan and Canada (41 percent), Australia (38 percent), and Belgium (37 percent). Within the United States, Massachusetts is again the leader, with 43 percent of 8th-grade students performing at the NAEP proficient level in reading. Shanghai students perform at a higher level, however, with 55 percent of young people proficient in reading. Within the United States, Vermont is a close second to its neighbor to the south, with a 42 percent proficiency rate. New Jersey and South Dakota come next, with 39 and 37 percent of the students identified as proficient in reading. Students living in California (about oneeighth of the U. S. school-age population) are statistically tied with their peers in Slovakia and Spain. Data and Approach Increasingly, states, and the federal government itself, have established performance levels that students are asked to reach. A national proficiency standard was set by the board that governs the National Assessment of Educational Progress (NAEP), which is administered by the U.S. Department of Education and generally known as the nation s report card. We provide information on student performance in both reading and mathematics, but our main concern is the relative performance of U.S. students in mathematics. That information is obtained by comparing student performance on NAEP math and reading tests with the performance of students from across the world on similar examinations. If the NAEP exams are the vi educationnext.org hks.harvard.edu/pepg

9 Executive Summary nation s report card, the world s report card is assembled by the Organization for Economic Co-operation and Development (OECD), which administers the Program for International Student Assessment (PISA) to representative samples of 15-year-old students in 65 of the world s school systems. Since the United States participates in the PISA examinations, it is possible to make direct comparisons between the average performance of U.S. students nationwide and that of their peers elsewhere. But because PISA exams do not set proficiency standards in the same way that NAEP exams do, one cannot calculate the percent proficient in the various countries of the world without performing a crosswalk between NAEP and PISA. Once that crosswalk has been performed, it is possible not only to provide estimates of the percentage of students who are proficient in various countries but also to view from an international perspective the performance of students from particular social groups as well as those living in each state. A crosswalk is made possible by the fact that representative (but separate) samples of the high-school graduating Class of 2011 took both the NAEP and PISA math and reading examinations. NAEP tests were taken in 2007 when the Class of 2011 was in 8th grade and PISA tested 15-year-olds in 2009, most of whom are members of the Class of Given that NAEP identified 32 percent of U.S. 8th-grade students as proficient in math, the PISA equivalent is estimated by calculating the minimum score reached by the top-performing 32 percent of U.S. students participating in the 2009 PISA test. U.S. students in the Class of 2011, with a 32 percent proficiency rate in mathematics, came in 32nd among the nations that participated in PISA. Performance of U.S. Ethnic and Racial Groups The percentage proficient in the United States varies considerably across students from different racial and ethnic backgrounds. While 42 percent of white students were identified as proficient in math, only 11 percent of African American students, 15 percent of Hispanic students, and 16 percent of Native Americans were so identified. Fifty percent of students with an ethnic background from Asia and the Pacific Islands, however, were proficient in math. In reading, 40 percent of white students and 41 percent of those from Asia and the Pacific Islands were identified as proficient. Only 13 percent of African American students, 5 percent of Hispanic students, and 18 percent of Native American students were so identified. Given the disparate performance among students from various cultural backgrounds, it may be worth inquiring as to whether differences between the United States and other countries are attributable to the substantial minority population within the United States. To examine that question, we compare GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? vii

10 Executive Summary While the 42 percent math proficiency rate for U.S. white students is much higher than the averages for students from African America and Hispanic backgrounds, U.S. white students are still surpassed by all students in 16 other countries. 2. Quoted in the STEM Education Coalition s website Accessed June 13, U.S. white students to all students in other countries. We do this not because we think this is the right comparison, but simply to consider the oft-expressed claim that comparisons do not take into account the fact that the United States is a much more diverse society than many of the high-performing countries. While the 42 percent math proficiency rate for U.S. white students is much higher than the averages for students from African American and Hispanic backgrounds, U.S. white students are still surpassed by all students in 16 other countries. A better than 25-percentage-point gap exists between the performance of U.S. white students and the percentage of all students deemed proficient in Korea and Finland. White students in the United States trail well behind all students in countries such as Japan, Germany, Belgium, and Canada. In reading, the picture looks better. As we mentioned above, only 40 percent of white students are proficient, but that proficiency rate would place the United States at 9th in the world. What Do These Findings Mean? The United States could enjoy a remarkable increment in its annual GDP growth per capita by enhancing the math proficiency of U.S. students. Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively. Since long-term average annual growth rates hover between 2 and 3 percentage points, that increment would lift growth rates by between 30 and 50 percent. When translated into dollar terms, these magnitudes become staggering. If one calculates these percentage increases as national income projections over an 80-year period (providing for a 20-year delay before any school reform is completed and the newly proficient students begin their working careers), a back-of-the-envelope calculation suggests gains of nothing less than $75 trillion over the period. That averages out to around a trillion dollars a year. Even if you tweak these numbers a bit in one direction or another to account for various uncertainties, you reach the same bottom line: Those who say that student math performance does not matter are clearly wrong. Charles Vest, former president of the Massachusetts Institute of Technology, has warned, America faces many challenges...but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency. 2 viii educationnext.org hks.harvard.edu/pepg

11 Program on Education Policy & Governance Harvard University EN Education Next

12 Sixty-five countries participated in the math and reading examinations administered by the Program for International Student Assessment (PISA). 2 educationnext.org hks.harvard.edu/pepg

13 Introduction Globally Challenged: Are U.S. Students Ready to Compete? The latest on each state s international standing in math and reading By Paul E. Peterson, Ludger Woessmann, Eric A. Hanushek, and Carlos X. Lastra-Anadón At a time of persistent unemployment, especially among the less skilled, many wonder whether our schools are adequately preparing students for the 21st-century global economy. Despite high unemployment rates, firms are experiencing shortages of educated workers, outsourcing professional-level work to workers abroad, and competing for the limited number of employment visas set aside for highly skilled immigrants. As President Barack Obama said in his 2011 State of the Union address, We know what it takes to compete for the jobs and industries of our time. We need to out-innovate, out-educate, and out-build the rest of the world. 1 The challenge is particularly great in math, science, and engineering. According to Internet entrepreneur Vinton Cerf, America simply is not producing enough of our own innovators, and the cause is twofold a deteriorating K 12 education system and a national culture that does not emphasize the importance of education and the value of engineering and science. 2 To address the issue, the Science, Technology, Engineering, and Math (STEM) Education Coalition was formed in 2006 to raise awareness in Congress, the Administration, and other organizations about the critical role that STEM education plays in enabling the U.S. to remain the economic and technological leader of the global marketplace. 3 Tales of shortages of educated talent appear regularly in the media. According to a CBS News report, 22 percent of American businesses say they are ready to We know what it takes to compete for the jobs and industries of our time. We need to out-innovate, out-educate, and out-build the rest of the world. President Barack Obama 1. Office of the Press Secretary, White House Office, Remarks by the President in the State of the Union Address, January 25, 2011 (http://www.whitehouse.gov/ the-press-office/2011/01/25/ remarks-president-state-union-address) 2. Vinton G. Cerf: How to fire up US Innovation, Wall Street Journal, April , (http://online.wsj.com/article/ SB html) 3. STEM Education Coalition website, STEM Ed Coalition Objectives, accessed June 13, 2011 at about-us/ GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 3

14 Comparing U.S. Students with Peers in Other Countries hire if they can find people with the right skills. As one factory owner put it, It s hard to fill these jobs because they require people who are good at math, good with their hands, and willing to work on a factory floor. 4 According to a Bureau of Labor Statistics report, of the 30 occupations projected to grow the most rapidly over the next decade, nearly half are professional jobs that require at least a college degree. 5 On the basis of these projections, McKinsey s Global Institute estimates that over the next few years there will be a gap of nearly 2 million workers with the necessary analytical and technical skills. 6 In this report, we examine the capacity of American schools to meet these needs. 4. Cynthia Bowers, CBS News Report: Skilled Labor Shortage Frustrates Employees, August 12, 2010 (http:// eveningnews/main shtml) 5. Bartsch (2009). 6. McKinsey Global Institute (2011). 7. Hanushek, Peterson, and Woessmann (2010). 8. NAEP has three levels: basic, proficient, and advanced. We report here the rates of those who are at or above the proficient level. 9. See Appendix for an international comparison of the math performance of the Class of 2011 at the advanced level. Comparing U.S. Students with Peers in Other Countries This is the second study of student achievement in global perspective prepared under the auspices of Harvard s Program on Education Policy and Governance (PEPG). In the 2010 PEPG report, U.S. Math Performance in Global Perspective, the focus was on the percentage of U.S. public and private school students performing at the advanced level in mathematics. 7 Specifically, the study compared the math performance of students in the high-school graduating Class of 2009 with that of their peers around the world. The current study continues this work by reporting the percentage of public and private school students identified as at or above the proficient level (a considerably lower standard of performance than the advanced level) in mathematics and reading for the most recent cohort for which data are available, the high-school graduating Class of Just as it is critical that the United States produce a segment of students who perform at the very highest level, so is it essential that a much larger portion of the next generation be proficient enough in math and reading to perform effectively in an economy that requires ever-increasing technical skill. 9 At one time it was left to teachers and administrators to decide exactly what level of math proficiency should be expected of students. But, increasingly, states, and the federal government itself, have established performance levels that students are asked to reach. A national proficiency standard was set by the board that governs the National Assessment of Educational Progress (NAEP), which is administered by the U.S. Department of Education and generally known as the nation s report card. In 2007, just 32 percent of 8th graders in public and private schools in the United States performed at or above the NAEP proficiency standard in mathematics, and 31 percent performed at or above that level in reading. When more than two-thirds of students fail to reach a proficiency bar, it raises serious questions: Are U.S. schools failing to teach their students adequately? 4 educationnext.org hks.harvard.edu/pepg

15 Comparing U.S. Students with Peers in Other Countries Percentages and Scores for Proficient Students In the United States, in 2007, the share of 8th-grade students identified as proficient on the NAEP math examination was percent. The minimum math score on the PISA examination obtained in 2009 by the highest-performing percent of all U.S. students was estimated to be To cover a broad content area while ensuring that testing time does not become excessive, the tests employ matrix sampling. No student takes the entire test, and scores are aggregated across students. For individual student observations, results are thus estimates of performance obtained by averaging five plausible values, as PISA and NAEP administrators recommend. Comparable numbers for the other categories are as follows: Reading proficiency: percent of U.S. students are proficient on the NAEP, which corresponds to a score of on PISA. Advanced math: percent of U.S. students scored at the advanced level on the NAEP, which corresponds to on PISA. Advanced reading: percent of U.S. students scored at the advanced level on the NAEP, which corresponds to on PISA. Or has NAEP set its proficiency bar at a level beyond the normal reach of a student in 8th grade? One way of tackling such questions is to take an international perspective. Are other countries able to lift a higher percentage or even a majority of their students to or above the NAEP proficiency bar? Another approach is to look at differences among states. What percentage of students in each state is performing at a proficient level? How does each state compare to students in other countries? Those are the questions we shall explore in this report. We provide information on student performance in both reading and mathematics, but our main concern is the relative performance of U.S. students in mathematics. This emphasis is based on prior research that has identified numeracy or math skills as primary determinants of advances in a nation s economic productivity. 10 It is also, as we shall see, the subject area in which the United States performs well below many other countries in the industrialized world. That information is obtained by comparing student performance on NAEP math and reading tests with the performance of students from across the world on similar examinations. If the NAEP exams are the nation s report card, the world s report card is assembled by the Organization for Economic Co-operation and Development (OECD), which administers the Program America simply is not producing enough of our own innovators, and the cause is twofold a deteriorating K 12 education system and a national culture that does not emphasize the importance of education and the value of engineering and science. Vinton Cerf, Internet entrepreneur 10. For a discussion of this literature, see Hanushek and Woessmann (2008). In addition, it seems probable that math performances are more precisely calibrated across different languages and cultures than reading performances are. GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 5

16 Proficiency in Math 11. In the 2007 edition of the other largescale international study that compares performance of students across countries, the Trends in Mathematics and Science Study (TIMSS 2007), the United States as well as the individual states of Massachusetts and Minnesota participated. This enables the comparison of those two particular states with other countries performance for TIMSS. See note 19, below. For a further comparison of TIMSS and PISA, see Hanushek, Peterson, and Woessmann (2010), Appendix. TIMSS 2007 surveyed fewer countries in the industrialized world than PISA does, concentrating instead on gathering similar information from among developing countries. The total number of countries surveyed by TIMSS 2007 was 48, while PISA surveyed 65 jurisdictions in TIMSS content has a less applied emphasis than PISA does. Nonetheless, at the country level, scores on PISA and TIMSS are highly correlated (Hanushek and Woessmann, 2008). 12. Development of the Assessment, NAEP: available online at nationsreportcard/mathematics/howdevelop. asp, accessed June 13, for International Student Assessment (PISA) to representative samples of 15-year-old students in 65 of the world s school systems (which, to simplify the presentation, we shall refer to as countries; Hong Kong, Macao, and Shanghai are not independent nations but are nonetheless included in PISA reports). Since its launch in 2000, the PISA test has emerged as the yardstick by which countries measure changes in their performance over time and the level of their performance relative to that of other countries. Since the United States participates in the PISA examinations, it is possible to make direct comparisons between the average performance of U.S. students nationwide and that of their peers elsewhere. But because PISA exams do not set proficiency standards in the same way that NAEP exams do, one cannot calculate the percent proficient in the various countries of the world without performing a crosswalk between NAEP and PISA. Once that crosswalk has been performed, it is possible not only to provide estimates of the percentage of students who are proficient in various countries but also to view from an international perspective the performance of students from particular social groups as well as those living in each state. 11 A crosswalk is made possible by the fact that representative (but separate) samples of the high-school graduating Class of 2011 took both the NAEP and PISA math and reading examinations. NAEP tests were taken in 2007 when the Class of 2011 was in 8th grade and PISA tested 15-year-olds in 2009, most of whom are members of the Class of Given that NAEP identified 32 percent of U.S. 8th-grade students as proficient in math, the PISA equivalent is estimated by calculating the minimum score reached by the top-performing 32 percent of U.S. students participating in the 2009 PISA test. (See methodological sidebar on page 5 for details of these scores and Appendix for further discussion of the crosswalk.) Proficiency in Math According to the National Center for Education Statistics (NCES), which administers NAEP, the determination of proficiency in any given subject at a particular grade level was the result of a comprehensive national process [which took into account] what hundreds of educators, curriculum experts, policymakers, and members of the general public thought the assessment should test. After the completion of the framework, the NAEP [subject] Committee worked with measurement specialists to create the assessment questions and scoring criteria. 12 In other words, NAEP s concept of proficiency is not based on any objective criterion, but instead reflects a consensus on what should be known by students who have reached a certain educational stage. NAEP 6 educationnext.org hks.harvard.edu/pepg

17 Proficiency in Math NAEP Definition of Math Proficiency at the 8th Grade Level and PISA s Definition of Proficiency Level Three Eighth-graders performing at the proficient level should be able to conjecture, defend their ideas, and give supporting examples. They should understand the connections between fractions, percents, decimals, and other mathematical topics such as algebra and functions. Quantity and spatial relationships in problem solving and reasoning should be familiar to them, and they should be able to convey underlying reasoning skills beyond the level of arithmetic. These students should make inferences from data and graphs, apply properties of informal geometry, and accurately use the tools of technology. Students at this level should be able to calculate, evaluate, and communicate results within the domain of statistics and probability. i Roughly comparable is PISA s Level 3 standard, defined as follows: At Level 3 students can execute clearly described procedures, including those that require sequential decisions. They can select and apply simple problem solving strategies. Students at this level can interpret and use representations based on different information sources and reason directly from them. They can develop short communications reporting their interpretations, results and reasoning. ii Sample NAEP Question at 8th Grade Proficiency Level Three tennis balls are to be stacked one on top of another in a cylindrical can. The radius of each tennis ball is 3 centimeters. To the nearest whole centimeter, what should be the minimum height of the can? Explain why you chose the height that you did. Your explanation should include a diagram. If you chose 18 cm from the list of five choices, you are in the company of the 28 percent of U.S. 8th graders from the Class of 2011 who answered correctly. iii The U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. Sample PISA Question at Proficiency Level Three Mark (from Sydney, Australia) and Hans (from Berlin, Germany) often communicate with each other using chat on the Internet. They have to log on to the Internet at the same time to be able to chat. To find a suitable time to chat, Mark looked up a chart of world times and found the following: Greenwich 12 Midnight Berlin 1:00 am Sydney 10:00 am At 7:00 pm in Sydney, what time is it in Berlin? The answer is 10 am. iv i. NAEP s definitions of the different levels of math achievement nationsreportcard/mathematics/achieveall.asp. Accessed on June 13, ii. OECD (2009a). iii. Question come from NAEP s online past questions database, nationsreportcard/itmrlsx/ search.aspx?subject=mathematics. Accessed on June 13, iv. Shiel, Perkins, Close, and Oldham (2007). GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 7

18 Proficiency in Math Percentage of students in the class of 2011 at the proficient level in math in U.S. states and foreign Percentage Proficient Shanghai Singapore Hong Kong Korea Finland Taiwan Liechtenstein Switzerland Massachusetts Japan Canada Netherlands Macao Belgium New Zealand Germany Australia Minnesota Vermont Estonia North Dakota Iceland New Jersey Kansas South Dakota France Slovenia Denmark Pennsylvania New Hampshire Austria Montana Virginia Colorado Wisconsin Maryland Slovakia Wyoming Washington Norway Sweden Ohio Luxembourg Iowa Indiana Oregon Connecticut Poland Texas Nebraska North Carolina Hungary Maine Czech Rep Idaho U.K. Portugal 13. NAEP s definitions of the different levels of achievement nationsreportcard/mathematics/achieveall. asp. Accessed on June 13, OECD (2009a). says that 8th graders, if proficient, understand the connections between fractions, percents, decimals, and other mathematical topics such as algebra and functions. 13 PISA does not set a proficiency standard. Instead, it sets different levels of performance, ranging from one (the lowest) to six (the highest). A student who is at the proficiency level in math set by NAEP performs moderately above level three on the PISA, which includes students who can execute clearly described procedures, including those that require sequential decisions. They can select and apply simple problem-solving strategies. 14 (See sidebar for a detailed statement of the 8th-grade proficiency standard and sample questions from PISA and NAEP that proficient students are expected to pass.) Given the above definition of math proficiency, U.S. students in the Class of 2011, with a 32 percent proficiency rate, came in 32nd among the nations that participated in PISA. Although performance levels among the countries ranked 23rd to 31st are not significantly different from that of the United States, 22 countries do significantly outperform the United States in the share of students reaching the proficient level in math. In six countries plus Shanghai and Hong Kong, a majority of students performed at the proficient level, while in the United States less than one-third did. For example, 58 percent of Korean students and 56 percent of Finnish students were proficient. Other countries in which a majority or near majority of students performed at or above the proficient 8 educationnext.org hks.harvard.edu/pepg

19 Proficiency in Math jurisdictions participating in PISA (Figure 1) U.S. average Utah Alaska U.S. Ireland South Carolina Delaware Italy Spain Illinois New York Missouri Michigan Rhode Island Latvia Florida Lithuania Kentucky Arizona Georgia Arkansas California Greece Tennessee Nevada Dubai Russia Israel Croatia Oklahoma Hawaii Louisiana West Virginia Turkey Alabama New Mexico Serbia Bulgaria Mississippi Uruguay Trinidad and Tobago Romania Chile Thailand District of Columbia Mexico Qatar Kazakhstan Argentina Azerbaijan Montenegro Brazil Albania Jordan Peru Colombia Panama Tunisia Indonesia Kyrgyzstan level included Switzerland, Japan, Canada, and the Netherlands. Many other nations also had math proficiency rates well above that of the United States, including Germany (45 percent), Australia (44 percent), and France (39 percent). Figure 1 presents a detailed listing of the scores of all participating countries as well as the performance of individual states within the United States. Shanghai topped the list with a 75 percent math proficiency rate, well over twice the 32 percent rate of the United States. However, Shanghai students are from a prosperous metropolitan area within China, with over three times the GDP per capita of the rest of that country, so their performance is more appropriately compared to Massachusetts and Minnesota, which are similarly favored and are the top performers among the U.S. states. When this comparison is made, Shanghai still performs at a distinctly higher level. Only a little more than half (51 percent) of Massachusetts students are proficient in math, while Minnesota, the runner-up state, has a math proficiency rate of just 43 percent. 15 Only four additional states Vermont, North Dakota, New Jersey, and Kansas have a math proficiency rate above 40 percent. Some of the country s largest and richest states score below the average for the United States as a whole, including New York (30 percent), Missouri (30 percent), Michigan (29 percent), Florida (27 percent), and California (24 percent). (See Table 1 for a comparison of each state with performances abroad.) 15. Our results are qualitatively similar to those reported by TIMSS 2007, which tested a representative sample of students in Massachusetts and Minnesota. Five countries that had higher average scores than Massachusetts on TIMSS 2007 also took the PISA test. Four of those countries Taiwan, Korea, Singapore, and Hong Kong are identified in Figure 1 of this report as outperforming these states on PISA. Japan also outperformed Massachusetts on TIMSS, but we found Japan s performance to be statistically indistinguishable from the Massachusetts one. Minnesota (whose performance was consistently lower than that of Massachusetts) trailed all five of the above-named countries on both tests, but it outperformed Australia, Sweden and Norway on TIMSS 2007, even though we identified it as not having done that well. In sum, the Massachusetts and Minnesota performances reported here resemble those reported by TIMSS, though Minnesota students seem to have done modestly better on TIMSS than reported here, while the reverse is true for Massachusetts students (Mullis, Martin and Foy, 2008, p. 38). GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 9

21 Proficiency in Reading Proficiency in Reading According to NAEP, students proficient in reading should be able to make and support inferences about a text, connect parts of a text, and analyze text features. 16 According to PISA, students at a proficiency level four, a level of performance set very close to NAEP s proficient level, should be capable of difficult reading tasks, such as locating embedded information, construing meaning from nuances of languages critically evaluating a text. 17 (See sidebar on page 7 for more specific definitions and sample questions.) As can be seen in Figure 2, the U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. It takes 17th place among the nations of the world, and only the top 10 countries on PISA outperform the United States by a statistically significant amount. In Korea, 47 percent of the students are proficient in reading. Other countries that outrank the United States include Finland (46 percent), Singapore and New Zealand (42 percent), Japan and Canada (41 percent), Australia (38 percent), and Belgium (37 percent). Within the United States, Massachusetts is again the leader, with 43 percent of 8th-grade students performing at the NAEP proficient level in reading. Shanghai students perform at a higher level, however, with 55 percent of young people proficient in reading. Within the United States, Vermont is a close second to its neighbor to the south, with 42 percent proficiency. New Jersey and South Dakota come next, with 39 and 37 percent of the students identified as proficient in reading. The District of Columbia, the nation s worst, performs at a level that cannot be distinguished statistically from that of Turkey and Bulgaria. Students living in California (about one-eighth of the U. S. school-age population) are statistically tied with their peers in Slovakia and Spain. See Table 2 for a comparison of how each state fares internationally. Performance of U.S. Ethnic and Racial Groups The percentage proficient in the United States varies considerably across students from different racial and ethnic backgrounds (see Figure 3). While 42 percent of white students were identified as proficient in math, only 11 percent of African American students, 15 percent of Hispanic students, and 16 percent of Native Americans were so identified. Fifty percent of students with an ethnic background from Asia and the Pacific Islands, however, were proficient in math, placing them at a level comparable to all students in Belgium, Canada, and Japan, if lower than that of all students in Korea and Taiwan. In reading, 40 percent of white students and 41 percent of those from Asia and the Pacific Islands were identified as proficient. Only 13 percent of African Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively 16. NAEP s definitions of the different levels of reading achievement is available at nces.ed.gov/nationsreportcard/reading/ achieveall.asp. Accessed on June 13, OECD (2000). GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 11

22 White Students Percentage of students in the class of 2011 at the proficient level in reading in U.S. states and foreign Percentage Proficient U.S. average Shanghai Korea Finland Hong Kong Massachusetts Singapore Vermont New Zealand Japan Canada New Jersey Montana Australia New Hampshire Connecticut Belgium Maine South Dakota Minnesota Pennsylvania Ohio Iowa Kansas Nebraska Colorado Netherlands Washington Oregon Virginia Wisconsin Wyoming Maryland France New York North Dakota Switzerland Idaho Germany Norway Iceland U.S. Indiana Missouri Delaware Poland Sweden Utah Liechtenstein Ireland Illinois U.K. Hungary Estonia Michigan Florida North Carolina American students, 5 percent of Hispanic students, and 18 percent of Native American students were so identified. White Students Given the disparate performance among students from various cultural backgrounds, it may be worth inquiring as to whether differences between the United States and other countries are attributable to the substantial minority population within the United States. To examine that question, we compare U.S. white students to all students in other countries. We do this not because we think this is the right comparison, but simply to consider the oft-expressed claim that education problems in the United States are confined to certain segments within the minority community. This is equivalent to the claim that the overall performance of the United States in international comparisons does not take into account the fact that the United States is a much more diverse society than many of the high-performing countries. While the 42 percent of math proficiency rate for U.S. white students is much higher than the averages for students from African American and Hispanic backgrounds, U.S. white students are still surpassed by all students in 16 other countries. A better than 25-percentage-point gap exists between the performance 12 educationnext.org hks.harvard.edu/pepg

23 White Students jurisdictions participating in PISA (Figure 2) Kentucky Texas Taiwan Rhode Island Alaska Italy Denmark Israel Oklahoma Georgia Tennessee Arkansas Portugal Slovenia Greece South Carolina Arizona Luxembourg Austria Czech Rep West Virginia Slovakia Spain Nevada California Alabama Latvia Dubai Macao Croatia Hawaii Louisiana Lithuania Mississippi New Mexico Turkey Russia Bulgaria District of Columbia Trinidad and Tobago Chile Uruguay Serbia Brazil Qatar Argentina Romania Mexico Montenegro Colombia Kazakhstan Panama Jordan Thailand Albania Tunisia Peru Kyrgyzstan Indonesia Azerbaijan of U.S. white students and the percentage of all students deemed proficient in Korea and Finland. White students in the United States trail well behind all students in countries such Japan, Germany, Belgium, and Canada (see Figure A.1). White students in Massachusetts outperform their peers in other states; 58 percent are at or above the proficient level in math. Maryland, New Jersey, and Texas are the other states in which a majority of white students is proficient in math. Given recent school-related political conflicts in Wisconsin, it is of interest that only 42 percent of that state s white students are proficient in math, a rate no better than the national average. In reading, the picture looks better. As we mentioned above, only 40 percent of white students are proficient, but that proficiency rate would place the United States at 9th in the world. This proficiency rate does not differ significantly from that for all students in Canada, Japan, and New Zealand, but white students trail in reading, by a significant margin, all students in Korea, Finland, and Singapore. In no state is a majority of white students proficient, although Massachusetts comes close with a 49 percent rate. The four states with the next highest levels of reading proficiency among white students are New Jersey, Connecticut, Maryland, and Colorado. (See Figure A.2 for the ranking of all the states.) GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 13

24 NAEP Definition of Reading Proficiency at the 8th Grade Level Eighth-grade students performing at the proficient level should be able to provide relevant information and summarize main ideas and themes. They should be able to make and support inferences about a text, connect parts of a text, and analyze text features. Students performing at this level should also be able to fully substantiate judgments about content and presentation of content. Sample NAEP Question What is an acceptable way to place a $1 Bargain Basement ad in this newspaper? 1. Phone in the ad, pay by credit card 2. Phone in the ad, pay by money order 3. Mail the ad, pay by cash 4. Mail the ad, pay by check If you chose answer four, you, along with 31 percent of 8th graders, got the question correct. Question from PISA corresponding to the NAEP proficiency level in reading: R236: New Rules BARGAIN BASEMENT Use this coupon for items over $25 but not more than $100 3DAYS FOR $1 We ll insert your classified ad for 3 consecutive days in the BARGAIN BASEMENT section. To qualify, the advertised item must be over $25 but not more than $100 and each item must be priced. Enclose check or money order with coupon. 1. Print one (1) letter in each space. 2. Allow one (1) space between words. 3. Include punctuation marks within the appropriate letter space. 4. ALL ADS MUST HAVE PRICE AND PHONE NUMBER IN THEM. 5. No businesses, individuals only qualify for this rate. 6. Maximum of 3 orders (9 insertions) per item. FIRST LINE Comparable PISA Question Question: Underline the sentence that explains what the Australians did to help decide how to deal with the frozen embryos belonging to a couple killed in the plane crash. i (Answer underlined in red in text to the right.) i. Cosgrove, Sofroniou, Kelly, and Shiel (2003). EDITORIAL Technology Creates the Need for New Rules Science has a way of getting ahead of law and ethics. That happened dramatically in 1945 on the destructive side of life with the atomic bomb, and is now happening on life s creative side with techniques to overcome human infertility. Most of us rejoiced with the Brown family in England when Louise, the first testtube baby, was born. And we have marvelled at other firsts most recently the births of healthy babies that had once been embryos frozen to await the proper moment of implantation in the mother-to-be. It is about two such frozen embryos in Australia that a storm of legal and ethical questions has arisen. The embryos were destined to be implanted in Elsa Rios, wife of Mario Rios. A previous embryo implant had been unsuccessful, and the Rioses wanted to have another chance at becoming parents. But before they had a second chance to try, the Rioses perished in an airplane crash. What was the Australian hospital to do with the frozen embryos? Could they be implanted in someone else? There were numerous volunteers. Were the embryos somehow entitled to the Rioses substantial estate? Or should the embryos be destroyed? The Rioses, understandably, had made no provision for the embryos future. The Australians set up a commission to study the matter. Last week, the commission made its report. The embryos should be thawed, the panel said, because donation of embryos to someone else would require the consent of 3DAYS SECOND LINE Name Address SPECIAL OFFER the producers, and no such consent had been given. The panel also held that the embryos in their present FOR state had FREE no life or rights Items and must thus be $25 or less could be destroyed. The item commission must be priced. members were conscious of treading on slippery legal and Times. The Times reserves the right to limit quantity of free ads in any given publication. ethical grounds. Therefore, they urged that three months be allowed for public opinion to respond to the commission recommendation. Should there be an overwhelming outcry against destroying the embryos, the commission would reconsider. Couples now enrolling in Sydney s Queen Victoria hospital for in vitro fertilization programmes must specify what should be done with the embryos if something happens to them. This assures that a situation similar to the Rioses won t recur. But what of other complex questions? In France, a woman recently had to go to court to be allowed to bear a child from her deceased husband s frozen sperm. How should such a request be handled? What should be done if a surrogate mother breaks her child-bearing contract and refuses to give up the infant she had promised to bear for someone else? Our society has failed so far to come up with enforceable rules for curbing the destructive potential of atomic power. We are reaping the nightmarish harvest for that failure. The possibilities of misuse of scientists ability to advance or retard procreation are manifold. Ethical and legal boundaries need to be set before we stray too far. Follow the above instructions & mail us this coupon to insert your free ad for 3 consecutive days in the BARGAIN BASEMENT section. The advertised item must be $25 or less and each $1 ads and free ads accepted only on this coupon. $1 ads and free ads will not be accepted by phone. No cancellations or refunds. coupons also available at the CLASSIFIED Counter of The Mail to: The Times Newspaper, BARGAIN BASEMENT, P.O. box 847, Trenton, NJ educationnext.org hks.harvard.edu/pepg

25 Students from College-educated Families Percentage of students in the class of 2011 in the U.S. at the proficient level in math and reading, by race and ethnicity. (Figure 3) 50% Math Reading Percentage Proficient White African -American Hispanic Native American Asia and the Pacific Islands White African -American Hispanic Native American Asia and the Pacific Islands Students from College-educated Families An elite segment of the U.S. population that the NAEP data allow us to isolate consists of students who have at least one parent who has attended college. Given the benefits that accrue to most of those who live in bettereducated families, that segment can be expected to outrank all students in other countries. It may be helpful to think of it as the upper bound of what the U.S. education system has delivered in terms of student performance. Significantly, not even among students from college-educated families can we find a majority of students crossing the proficiency bar in math (see Figure A.3). Only 44 percent of such students did so. In Massachusetts, 61 percent of students from college-educated families are proficient in math. Seven other states have a majority of students from college-educated families performing proficiently in math: Vermont, Minnesota, Kansas, Pennsylvania, Virginia, New Jersey, and Colorado. In reading, 42 percent of U.S. students from college-educated families in the Class of 2011 are proficient. In two states a majority of these students are proficient in reading: Massachusetts with 57 percent and Vermont with 53 percent. Other high-ranking states include New Jersey, Connecticut, Pennsylvania, Oregon, and Ohio. (See Figure A.4 for the ranking of all the states.) GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 15

27 Are the Proficiency Standards the Same for Math as for Reading? Are the Proficiency Standards the Same for Math as for Reading? Has NAEP set a lower proficiency standard in math than in reading? If so, is the math standard too low or the reading bar too high? At first glance it would seem that the standard is set at pretty much the same level. After all, 32 percent of U.S. students are deemed proficient in math and 31 percent are deemed proficient in reading. But that coincidence is quite misleading. When compared to peers abroad, the U.S. Class of 2011 performed respectably in reading, trailing only 10 other nations by a statistically significant amount. Admittedly, the United States trails Korea by 16 percentage points, but it s only 9 percentage points behind Canada. Meanwhile, U.S. performance in math is seriously disappointing. It significantly trails that of 22 countries. Korean performance is 26 percentage points higher than that of the United States, while Canadian performance is 17 percentage points higher. Judged by international standards, the U.S. Class of 2011 was clearly doing worse in math than in reading, despite the fact that NAEP reports similar percentages proficient in the two subjects. A direct comparison of NAEP s proficiency standard with PISA s levels three and four (out of a total of six proficiency levels) also indicates that a lower NAEP bar has been set in math than in reading. To be proficient, one needs to perform at or near the fourth level on PISA s reading exam, but only modestly above the third level on its math exam. From these findings, we infer that the NAEP experts set an 8th-grade math proficiency standard at a level lower than the one set in reading. Perhaps this is an indication that American society as a whole, including the experts who design NAEP standards, set lower expectations for students in math than in reading. If so, it is a sign that low performance in mathematics within the United States may be deeply rooted in the nation s culture. Those who are setting the common core standards under discussion might well take note of this. Of course, it could be argued that the math proficiency standard is correct but the reading standard has been set too high. In no country in the world does a majority of the students reach the NAEP proficiency bar set in 8th-grade reading. NAEP experts set an 8th-grade math proficiency standard at a level lower than the one set in reading. What Do These Findings Mean? Many have concluded that the productivity of the U.S. economy could be greatly enhanced if a higher percentage of U.S. students were proficient in mathematics. As Michael Brown, Nobel Prize winner in medicine, has declared, If America is to maintain our high standard of living, we must continue to innovate. GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 17

28 What Do These Findings Mean? If America is to maintain our high standard of living, we must continue to innovate. Math and science are the engines of innovation. With these engines we can lead the world. Michael Brown, Nobel Prize winner in medicine 18. Quoted at the STEM Education Coalition s website stemedcoalition.org/, Accessed June 13, Hanushek. and Woessmann (2008). Math and science are the engines of innovation. With these engines we can lead the world. 18 But others have argued that the overall past success of the U.S. economy suggests that high-school math performance is not that critical for sustained growth in economic productivity. After all, U.S. students trailed their peers in the very first international survey undertaken nearly 50 years ago. But that is the wrong message to take away. Other factors contributed to the relatively high rate of growth in economic productivity during the last half of the 20th century, including the openness of the country s markets, respect for property rights, low levels of political corruption, and limited intrusion of government into the operations of the marketplace. The United States, moreover, has always benefited from the in-migration of talent from abroad. Furthermore, the United States has historically had far higher levels of educational attainment than other countries, with many more students graduating from high school, continuing on to college, and earning an advanced degree. It appears that in the past the country made up for low quality in elementary and high school by educating students for longer periods of time. As we proceed into the 21st century, none of these factors remain as favorable to the United States. While other countries are lifting restrictions on market operations, the opposite has been occurring within the United States. The U.S. has also placed sharp limits on the numbers of talented workers that can be legally admitted into the country. Our higher education system, though still perceived to be the best in the world, is recruiting an ever-increasing proportion of its faculty and students from outside the country. Meanwhile, educational attainment rates among U.S. citizens now trail the industrial-world average. Even if some of these trends can be reversed, that hardly gainsays the desirability of enhancing the mathematical skills of the U.S. student population, especially at a time when the nation s growth in productivity is badly trailing growth rates in China, India, Brazil, and many smaller Asian countries. Eric Hanushek and Ludger Woessmann have shown elsewhere that student performance on international tests such as those we consider here is closely related to long-term economic growth. 19 Assuming that past trends continue, the country could enjoy a remarkable increment in its annual GDP growth per capita by enhancing the math proficiency of U.S. students. Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively (see Figure 4). Since longterm average annual growth rates hover between 2 and 3 percentage points, that increment would lift growth rates by between 30 and 50 percent. 18 educationnext.org hks.harvard.edu/pepg

29 What Do These Findings Mean? Estimated increase in U.S. annual economic growth from improved mathematics proficiency. (Figure 4) 2.5 Shanghai Added annual GDP growth Current U.S. 0 0 U.K. France Germany Canada Korea Finland Increased % of students who are proficient in math Note: Country points indicate the additional % of students proficient in the United States in order to reach the country s level (X-axis) and corresponding additional growth to be expected on reaching that level (Y-axis). When translated into dollar terms, these magnitudes become staggering. If one calculates these percentage increases as national income projections over an 80-year period (providing for a 20-year delay before any school reform is completed and the newly proficient students begin their working careers), a back-of-the-envelope calculation suggests gains of nothing less than $75 trillion over the period. 20 That averages out to around a trillion dollars a year. Even if you tweak these numbers a bit in one direction or another to account for various uncertainties, you reach the same bottom line: Those who say that student math performance does not matter are clearly wrong. Given the integration of the world economy, a global perspective is needed for assessing the performance of U.S. schools, districts, and states. High-school graduates in each and every state compete for jobs with graduates from all over the world. Charles Vest, former president of the at Massachusetts Institute of Technology, has warned, America faces many challenges...but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency For a thorough explanation of this calculation, see Hanushek and Woessmann (2011). 21. Quoted in the STEM Education Coalition s website stemedcoalition.org/, Accessed June 13, GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 19

30 Differences in the Math Performance of the High School Classes of 2009 and 2011 Appendix Differences in the Math Performance of the High School Classes of 2009 and 2011 Students are identified as advanced by NAEP only if they score well above the proficient level. Seven percent of U.S. students in the Class of 2011 performed at the advanced level in math (See Figure A.5, Table A.5), a gain of 1 percentage point over the 6 percent identified as advanced in the Class of That slight gain is a modest accomplishment, especially given the decline in performance in many other countries. While most changes were small, the percentage of advanced students declined by 2 or more points in the Czech Republic, Austria, Korea, Finland, United Kingdom, Ireland, and Lithuania. Only two of the higher-performing countries, Denmark and Portugal, showed improvement of 2 percentage points or more. Given all these changes, the relative position of the U.S. Class of 2011 improved from the 31st place held by the U. S. Class of 2009 to a tie for 26th place (with Poland, Hungary, Norway, the United Kingdom, Portugal, Italy, and Sweden), despite the inclusion of two new high-scoring PISA participants, Shanghai and Singapore. Within the United States, little change could be observed between the Class of 2009 and the Class of 2011, apart from the astonishing shift upward in the already high-performing state of Massachusetts, where the percentage advanced rose from 11 percent to 15 percent, a gain unequalled by any other state. Minnesota s performance came in second place in both years, but its students performance budged northward by only 0.7 percentage points to 11.5 percent for the Class of In four other states, scores improved by 2 percentage points or more: Vermont, Maine, North Dakota, and Wyoming (which made a 3.0 gain, the largest gain outside of Massachusetts). It is remarkable how concentrated in certain parts of the country these gains are to be found. If teaching to the talented is a skill, the teachers getting better at the task seem to be concentrated in a few states in New England and the northern plains. Indeed, the picture that we see of little change in the relative performance of the United States is one that is consistent with the broader trajectory of the United States in international comparisons, which is at best flat and at worst in slight decline over time. Regardless of whether the United States is actually improving in its performance, it is clear that its relative standing with respect to other developed countries is in the bottom half of the OECD countries. 20 educationnext.org hks.harvard.edu/pepg

31 Performing the Crosswalk At the same time, we have noted above how a number of other countries, most notably Asian countries such as Taiwan, Singapore, and Japan but also others such Finland and Switzerland or Canada, are significantly ahead of the United States. Performing the Crosswalk Our aim is to compare how students in the different states in the United States are doing with respect to their peers internationally. To obtain this information, we perform a crosswalk between NAEP and the Program for International Student Assessment (PISA), which was administered by the Organization for Economic Co-operation and Development OECD, to representative samples of 15-year-old students in 65 of the world s school systems, which, to simplify the presentation, we shall refer to as countries. (Hong Kong, Macao, and Shanghai are not independent nations but nonetheless are included in PISA reports.) The crosswalk is performed by looking at the percentage of U.S. students who reach the proficient level on the NAEP assessment and at the equivalent cutoff score in PISA for that percentage of U.S. students. This gives us the equivalent of the PISA proficiency threshold, allowing us to estimate comparable proficiency rates for all countries and to compare student performance in each of the states within the United States with that of their international peers. Our analysis relies on test-score information from young adults collected by NAEP and PISA. 1 NAEP is a large, nationally representative assessment of student performance that has been administered periodically since the early 1970s to U.S. students in 4th grade and 8th grade, and at the age of seventeen. Since 2001, it has provided achievement data for students in each of the 50 states and a select number of urban school districts. PISA is an internationally standardized assessment of student performance in mathematics, science, and reading established by OECD. It was administered in 2000, 2003, 2006 and 2009 to representative samples of 15-year-olds in all OECD countries as well as in many others. 2 NAEP is governed by the National Assessment Governing Board (NAGB), which consists of 26 educators and other public figures appointed by the U.S. Secretary of Education. In 2007, NAEP tested representative samples of 8th-grade public and private school students in each of the Data for NAEP come from the official website [accessed May 15, 2011], ed.gov/nationsreportcard/. NAEP has also tested periodically a representative sample of students in several other subjects. 2. The OECD, which administers PISA, is an international economic organization encompassing most of the high-income, developed countries of the world. In 2009, it had 30 members; three new members (Chile, Israel, and Slovenia) were added in Sixty-five countries/economies participated in PISA in 2009 (up from 57 in 2006). Data for PISA 2009 come from the PISA microdata (http://www.pisa.oecd.org/). The PISA assessments build upon earlier international testing, most importantly those of the International Association for the Evaluation of Educational Achievement (IEA) now known as Trends in Mathematics and Science Survey (TIMSS). IEA has conducted assessments since the mid-1960s and is responsible for the TIMSS testing that is discussed below. See Historical PISA scores and those of TIMSS are summarized in Provasnik, Gonzales, and Miller (2009), which also contains references to the original publications for TIMSS. GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 21

32 Identifying the Class of 2011 states, in 10 large public school districts, and in the United States as a whole in math, science, and reading. For each of these jurisdictions, NAEP 2007 calculates the percentage of students who perform at three levels: basic, proficient, and advanced. Using the NAEP and PISA data for the United States as a whole, the crosswalk exercise identifies an estimated PISA score of for math proficiency, as defined by NAEP, and a score of for reading proficiency, as defined by NAEP. With the PISA data, we can obtain an estimate of the percentage of students in those countries above the cutoff, i.e., those who reach the level equivalent to the proficient level in 8th-grade math on NAEP The shares of students who reach the proficient level in 8th-grade math in each U.S. state are taken directly from NAEP It is assumed that both NAEP and PISA tests randomly select questions from a common universe of mathematics knowledge. Given that assumption, it may be further assumed that students who scored similarly on the two exams will have similar math knowledge, i.e., students who scored points or better on the PISA test would have been identified as proficient had they taken the NAEP math test. The scaling of PISA straightforwardly reveals that a score of points is 31 percent of one standard deviation above the average OECD student score on the PISA, indicating that a somehow accomplished group has been found. Some of the calculated differences in performance across countries may simply reflect sampling uncertainty or measurement error. We therefore calculate whether the observed differences among states and countries are statistically significant (at the 5 percent level). The requisite standard errors are computed using the methodology described by the OECD. 3 These standard errors account for both sampling uncertainty (including the two-stage sampling design employed by PISA) and test unreliability (as captured by the five plausible values that represent the underlying probability distribution). NAEP 2007 standard errors are obtained from the NAEP website OECD (2009b). See Chapters accessed May 18, National Center for Education Statistics, Report , Table C-1. Identifying the Class of 2011 This crosswalk identifies the relative performance of the Class of NAEP examinations are given to 8th graders, January through March, when most students are 13 years of age. PISA examinations are given to a random sample of students at the age of 15, the age at which approximately 70 percent of U. S. students are in 10th grade. 5 To track the Class of 2011 we rely upon the 2007 NAEP test and the 2009 PISA test. In comparing the performance of the Class 22 educationnext.org hks.harvard.edu/pepg

33 Critiques of PISA of 2011 on the NAEP and PISA tests at these two different points in time, we assume that no event happened between 8th and 10th grade that significantly altered the performance of American students relative to that of students in other countries. Our previous report relied on a similar crosswalk between the PISA 2006 results and the NAEP 2005, a less preferable cohort match but the best possible given the information available in November With the appearance of the latest PISA wave in December 2010, we were able to match cohorts more closely, as NAEP data was available for 8th graders in 2007 while PISA data was available in 2009 for 15-year-olds who are typically in 10th grade. While we are pleased that the cohort match reported in this report is more precise, the findings from this report are consistent with those presented in the previous report. Inasmuch as there are only small differences from one year to the next in the performance of students in particular countries, small variations in the years used to execute a particular cross-walk do not alter findings materially. Critiques of PISA Questions have been raised as to whether any cross-country comparison using PISA data can be meaningful. Prais and others point out that PISA s testing focuses on real-life circumstances and on students capacity to enter the labor market with core skills, such as literacy and numeracy, more than the testing of any specific curriculum. 7 This inherently favors some countries specific sequencing of items in the curriculum. For example, Germany s introduction of supplementary questions in the PISA math test (that focused more on arithmetic skills) presents a case of a country that thought that the basic PISA math test was not providing an accurate evaluation of its students math skills. But too much emphasis on specific questions ignores the commonality of welldesigned tests of student achievement. The TIMSS and PISA tests have quite different designs, but the performance of countries on TIMSS and PISA are highly correlated and both are strongly correlated with a country s economic performance. (See our previous report for further discussion.) 8 Further, the U.S. National Center for Education Statistics (NCES) is deeply involved in the development of PISA, as it is an active participant in OECD discussions of PISA design; presumably, that office is satisfied that the test is not unfair to U.S. students. Beyond the appropriateness of the specific PISA questions, worries have been voiced about whether excessive focus on such exams can take away from other educational values that arguably cannot be so readily measured 6. Hanushek, Peterson, and Woessmann (2010). 7. Prais (2003). 8. Hanushek, Peterson, and Woessmann (2010), Appendix. GLOBALLY Challenged: ARE U.S. STUDENTS READY TO COMPETE? 23

44 Biographical Sketches Biographical Sketches Paul E. Peterson is the director of the Program on Education Policy and Governance in the Taubman Center for State and Local Government at Harvard s Kennedy School of Government. He is the Henry Lee Shattuck Professor of Government in the Faculty of Arts and Sciences at Harvard University and a senior fellow at the Hoover Institution at Stanford University. He is the editor-in-chief of Education Next: A Journal of Opinion and Research. His most recent publication is Saving Schools: From Horace Mann to Virtual Learning (Harvard University Press, 2010). Websites: hks.harvard.edu/pepg educationnext.org savingschools.com Ludger Woessmann is professor of economics at the University of Munich, head of the Department of Human Capital and Innovation at Ifo Institute for Economic Research, and coordinator of the European Expert Network on the Economics of Education (EENEE). His most recent book, School Accountability, Autonomy and Choice around the World, considers sources of international differences in student achievement. Website: cesifo.de/woessmann Eric A. Hanushek is the Paul and Jean Hanna Senior Fellow at the Hoover Institution of Stanford University. He is chairman of the Executive Committee for the Texas Schools Project at the University of Texas at Dallas, a research associate of the National Bureau of Economic Research, a member of the Koret Task Force on K-12 Education, and the area coordinator for Economics of Education of the CESifo Research Network. He currently serves as a commissioner on the Equity and Excellence Commission of the U.S. Department of Education and was the past chair of the Board of Directors of the National Board for Education Sciences. His most recent book, Schoolhouses, Courthouses, and Statehouses, describes how improved school finance policies can be used to meet our achievement goals. Website: hanushek.net Carlos X. Lastra-Anadón is a research fellow with the Program on Education Policy and Governance in the Taubman Center for State and Local Government at Harvard s Kennedy School of Government. He recently earned his Masters in Public Policy from Harvard s Kennedy School. In addition to contributing to PEPG s U.S. Math Performance in Global Perspective report, he is co-organizing a conference at PEPG on that subject in August of His most recent paper, State Standards Rise in Reading, Fall in Math, co-authored with Paul E. Peterson, was published in the Fall 2010 issue of Education Next. Before that he worked in international education reform with McKinsey and Company in the Middle East and Europe and worked on the launch of Empieza por Educar, the Spanish version of Teach for America and a member of the Teach for All network. 34 educationnext.org hks.harvard.edu/pepg

Constructing a High-Performance Tertiary Education System in Finland Jamil Salmi, 10 December 2014 The future of tertiary education? A world of science fiction? social and economic progress is achieved

BUSINESS DEVELOPMENT OUTCOMES Small Business Ownership Description Total number of employer firms and self-employment in the state per 100 people in the labor force, 2003. Explanation Business ownership

Impacts of Sequestration on the States Alabama Alabama will lose about $230,000 in Justice Assistance Grants that support law STOP Violence Against Women Program: Alabama could lose up to $102,000 in funds

HIGHER EDUCATION FINANCE: TRENDS AND ISSUES STATE COUNCIL OF HIGHER EDUCATION FOR VIRGINIA SEPTEMBER 14, 2009 Paul E. Lingenfelter President, SHEEO Forces flattening the global playing field Fall of Berlin

are Centere on Questions of o Deserves at? an at Val ues Soul e Honor? e ave to figt for ile we on t promie ucceoen t epen on outcome of our life.. tuent life outie of cool. One of New Mexico trengtiitabunance

NON-RESIDENT INDEPENDENT, PUBLIC, AND COMPANY ADJUSTER LICENSING CHECKLIST ** Utilize this list to determine whether or not a non-resident applicant may waive the Oklahoma examination or become licensed

New York State Trends in Student Financial Aid and Cost of Attendance Presented to the Higher Education Committee of the New York State Board of Regents January 10, 2011 Joseph Frey Glenwood Rowse Office

Alabama Alaska Arizona Arkansas California Colorado Connecticut Delaware District of Columbia Effective 10/16/11: Producers holding a life line of authority on or before 10/16/11 who sell or wish to sell

Englishinusa.com Positions in MSN under different search terms. Search Term Position 1 Accent Reduction Programs in USA 1 2 American English for Business Students 1 3 American English for Graduate Students

Licensure Resources by State Alabama Alabama State Board of Social Work Examiners http://socialwork.alabama.gov/ Alaska Alaska Board of Social Work Examiners http://commerce.state.ak.us/dnn/cbpl/professionallicensing/socialworkexaminers.as

Taking stock of the digital divide Dr. Sharon Strover University of Texas at Austin Prepared for the Lisbon Research and Policy Workshop: Technologies, contents and services for social inclusion: Facing

PISA FOR SCHOOLS How is my school comparing internationally? Andreas Schleicher Director for Education and Skills OECD Madrid, September 22 nd PISA in brief Over half a million students representing 28

State Tax Information The information contained in this document is not intended or written as specific legal or tax advice and may not be relied on for purposes of avoiding any state tax penalties. Neither

Brought to you by Alamo Insurance Group Supreme Court Strikes Down DOMA, Clears Way for Same-Sex On June 26, 2013, the U.S. Supreme Court announced decisions in two significant cases regarding laws affecting

State Pest Control/Pesticide Application Laws & As Compiled by NPMA, as of December 2011 Alabama http://alabamaadministrativecode.state.al.us/docs/agr/mcword10agr9.pdf Alabama Pest Control Alaska http://dec.alaska.gov/commish/regulations/pdfs/18%20aac%2090.pdf

Real Progress in Food Code Adoption The Association of Food and Drug Officials (AFDO), under contract to the Food and Drug Administration, is gathering data on the progress of FDA Food Code adoptions by

American C.E. Requirements Alaska Board of Nursing Two of the following: 30 contact hours 30 hours of professional nursing activities 320 hours of nursing employment Arizona State Board of Nursing Arkansas

Methodology for 2013 Application Trends Survey F rom early June to mid-july 2013, the Graduate Management Admission Council (GMAC ) conducted the Application Trends Survey, its annual survey of business

Senate Select Committee on College and University Admissions and Outreach November 14, 2007 Sacramento, CA David Longanecker President Western Interstate Commission for Higher Education (WICHE) Community

Select a state below to display the current regulation and requirements, or continue to scroll down. Light grey text signifies states that have not adopted an annuity training program. Alabama Illinois

Page 1 of 7 (https://www.insidehighered.com) Data show key role for community colleges in 4-year degree production Submitted by Doug Lederman on September 10, 2012-3:00am The notion that community colleges

World Leader Great schools, great teaching and how to get them Professor Chris Husbands, Director, Institute of Education Professor Chris Husbands, Professor Director, Chris Husbands Institute Director,

Pursuant to Section 104 of the Intercountry Adoption Act of 2000 (IAA) (Public Law 106-279), the U.S. Department of State submits the FY 2014 Annual Report on Intercountry Adoption. IAA 104(b) Report Elements:

State Tax Information The information contained in this document is not intended or written as specific legal or tax advice and may not be relied on for purposes of avoiding any state tax penalties. Neither

The Obama Administration and Community Health Centers Community health centers are a critical source of health care for millions of Americans particularly those in underserved communities. Thanks primarily

Issue Brief November 2007 Changes in the Cost of Medicare Prescription Drug Plans, 2007-2008 BY JOSHUA LANIER AND DEAN BAKER* The average premium for Medicare Part D prescription drug plans rose by 24.5

MEASURING UP 2008 THE STATE REPORT CARD ON HIGHER EDUCATION Pennsylvania What Is Measuring Up? Measuring Up is a series of biennial report cards that provide the general public and policymakers with information

University System of Georgia Enrollment Trends and Projections to 2018 Introduction: Projections of USG Headcount Enrollment Enrollment projections use past trends and information on other variables to

2009-10 STATE AND LOCAL GOVERNMENT TAX AND REVENUE RANKINGS By Jacek Cianciara Wisconsin Department of Revenue Division of Research and Policy December 12, 2012 TABLE OF CONTENTS Key Findings 3 Introduction

Attachment A The following table provides information on student teaching requirements across several states. There are several models for these requirements; minimum number of weeks, number of required

State Insurance Department Websites: A Consumer Assessment By J. Robert Hunter Director of Insurance November 2008 1 of 10 EXECUTIVE SUMMARY This report analyzes Internet websites for the nation s 51 major

Detail on mathematics graduation requirements from public high schools, by state as of June 5, 2013 State Year in Effect Algebra II required Years of Math Alignment Comments/Explanations Alabama 2011-12

MEASURING UP 2008 THE STATE REPORT CARD ON HIGHER EDUCATION Maryland What Is Measuring Up? Measuring Up is a series of biennial report cards that provide the general public and policymakers with information

School Readiness: High-Quality Early Learning Head Start $10.5 $9.5 $10.1 +$1.5 +17.7% $8.5 $7.5 +$2.1 +27.0% $6.5 for fiscal year 2010 Included in the budget is $1.078 billion to ensure that every Head

FELONY DUI SYNOPSIS The information in the following charts was compiled by examining the felony DUI laws in all 50 sates and the District of Columbia. The analysis focuses on the felony DUI threshold,

Fast Facts on Education in America updated december 2010 The climate of postsecondary education in America has changed dramatically since Scholarship America s inception 50 years ago. Access to college

2014 IMD World Talent Report By the IMD World Competitiveness Center November 2014 IMD World Talent Report 2014 Copyright 2014 by IMD: Institute for Management Development, Lausanne, Switzerland For further

International Call Services Affordable rates for business calls. Wherever you are in the world. We ve got plenty to say when it comes to staying in touch when you re overseas. We have agreements with 443

IMMIGRANTS IN WASHINGTON STATE MARK ELLIS PROFESSOR OF GEOGRAPHY AND CENTER FOR STUDIES IN DEMOGRAPHY AND ECOLOGY UNIVERSITY OF WASHINGTON QUESTIONS: ELLISM@U.WASHINGTON.EDU TABLES AND CHARTS AVAILABLE

NAIC Annuity Suitability Requirements by Specific Alabama Alaska 10/16/2011 TBD Arizona Arkansas If you obtained a life insurance license prior to 10/16/11, you must complete the NAIC course by 4/16/12.

BY STATE INFORMATION This information is being provided to assist in your 2014 tax preparations. The information is also mailed to applicable Columbia fund non-corporate shareholders with their year-end

GOVERNMENT-FINANCED EMPLOYMENT AND THE REAL PRIVATE SECTOR IN THE 50 STATES BY KEITH HALL AND ROBERT GREENE November 25, 2013 www.mercatus.org 0.7 2.4 4.2 FEDERAL CONTRACT FUNDED PRIVATE-SECTOR JOBS AS

Consent to Appointment as Registered Agent This form is used by the person or business entity that agrees to act as the registered agent for a limited liability company. Every person or business entity

NAIC Annuity Suitability Requirements by Specific Alabama Alaska 10/16/2011 TBD Arizona Arkansas If you obtained a life insurance license prior to 10/16/11, you must complete the NAIC course by 4/16/12.

UNDERSTANDING THE CONTEXT The graduation rate in NM is below the U.S. average and it has been flat for two years Gaps persist in graduation rates between different ethnic/racial groups in NM Increasing

COMPARE NEBRASKA S BUSINESS CLIMATE TO OTHER STATES Selected Business Costs for Each State Workers Compensation Rates Unemployment Insurance Costs Electricity Rates Gas Rates Average Hourly Earnings in

College Completion in : The Impact on the Workforce and the Economy February 2012 Investing in Individuals and Society through College Completion 30 25 20 15 10 5 0 Higher Levels of Education and Skills

NORWAY Key findings Norway performs around average in mathematics, above average in reading, but below average in science. Norway s mean performance in mathematics declined since the previous PISA assessment