Welcome to the new Becker-Posner Blog, maintained by the University of Chicago Law School.

09/10/2006

Our Bad Math Scores--Posner's Comment

The Program for Individual Student Achievement (PISA), the source of Becker's statistics, does triennial international comparative studies of 15-year-olds' educational achievements. The latest results to be reported are those of the 2003 survey. The United States came out in the middle of the European pack in reading literacy, but in math proficiency we were below the European average, as well as below several Asian countries in the sample.
There are three questions to ask about these results: (1) Are they meaningful, in the sense of providing an accurate picture of relative math proficiency? (2) If so, are there economic or other real-world consequences? (3) If there are, what if anything should be done to increase our rank score?
(1) The answer to the first question appears to be "yes," partly because of consistency with other studies. There is a good article on this issue by Paul E. Peterson, available on the Web, called "Ticket to Nowhere." See also an article by Mariann Lemke and colleagues in a recent issue of the Educational Statistics Quarterly, also available on the Web. Math skills appear to be deteriorating in the United States as well asto be inferior to the average of the 40 countries in the PISA sample; and this is true even if blacks and Hispanics, who on average do poorly (especially blacks) on these tests, are excluded from this comparison, although such exclusion does increase our international rank somewhat (but a proper comparison would require similar exclusions from some other countries in the sample). I agree with Becker that it would be interesting to see what difference it would make if college rather than high school students were tested. But I would not expect much difference, because little emphasis is placed nowadays on math in American colleges.
(2) But so what? Here I agree with Becker; perhaps I am even more emphatic than he, that better education in mathematics would not have substantial effects on social and economic welfare. Very few jobs nowadays require even simple math skills ("deskilling" is the story of modernity); almost all computation is automated. Even fewer jobs require advanced math skills--and kids cannot acquire those skills by education; they are innate.
What would be socially and even economically useful would be to instruct high school students in the rudiments of statistical theory. That would help them learn to think straight about a range of public policy issues, as well as to avoid certain recurrent mistakes in everyday life. People are terrible at handling probabilities. For example, most people, including otherwise quite intelligent and well educated people, don't understand that randomness is not regular alternation--that a typical random pattern is 1000110110001, not 101010101010. And this mistake leads them, for example, to give undue weight to the recent performance of a mutual fund (e.g., 1101). But whether to teach statistical theory in high school is an issue of educational policy rather than a matter of raising the scores on math tests.
It would also be helpful to the United States, mainly from a public policy standpoint, if more of our people were scientifically literate; and it would help them to be so if they knew some math, because modern science is heavily mathematical. In my book Catastrophe: Risk and Return (2004), I examined the issue of scientific literacy briefly, pointing out that only a third of American adults (adults, not 15-year-olds) know what a molecule is, that 39 percent believe that astrology is scientific, that 46 percent deny that human beings evolved from earlier animal species, and that almost 50 percent do not know that it takes a year for the earth to revolve around the sun (many do not know that the earth revolves around the sun). These are amazing statistics, and yet, according to the materials I consulted, the scientific literacy of the U.S. population actually exceeds that of the European Union, Japan, and Canada.
In the age of the computer and the Internet, school education probably has rather limited effect on job performance, marital stability, happiness, or other measures of welfare, except perhaps for elite people--but they attend elite educational institutions and probably get a better education than is available in any other country; our best colleges and universities are the envy of the world. And because of compulsory schooling, a very bright child almost always will be spotted even if he or she comes from a poor or educationally deprived home, and will be shunted onto the elite track.
So I do not think that the low quality of public education matters a great deal from an overall social standpoint, except that our public schools seem needlessly costly, and also unresponsive to the special needs of very poor students. These are reasons why I strongly support school voucher programs.
(3) If this is wrong and our poor international standing in math proficiency is hurting the United States, the solution is to teach more math and less of something else. It should not be to drill the kids in the 2003 PISA math test so that they can do better on the next one. It is always possible to improve scores on standardized tests by orienting instruction to the tests, by tutoring, or, if worse comes to worst, by withholding the test from the weakest students!

Comments

You can follow this conversation by subscribing to the comment feed for this post.

"and kids cannot acquire those skills by education; they are innate."

No, the capacity to *aquire* those skills is inate. The skills themselves are most assuredly aquired by education, even if a lot of it will be self-directed for the reqlly tallented.

At the very least, if we're going to do some kind of tracking for math education, (And I agree that the average person really doesn't need much algebra, let alone caculus.) we need a good system for identifying *early* the children who do posess those inate capacities. Which suggests that what little math everyone is taught, they should be taught as soon as practical.

I'd be interested to see whether the international scores correlate at all with the IQ studies, which show a strong association between IQ and the level of economic development.

East Asians do better than Caucasians, who do better than American Blacks who do better than Africans.

Some of this is due to things like nutritional problems in early childhood in Africa and other environmental factors. I'll get flamed for this, but some is probably due to heredity.

And does the math differential have anything to do with the fact that we are importing engineers and scientists to the point that native English speakers complain that they can't understand the teaching assistants who in fact do the teaching.

I also can't agree with your seemingly unalloyed praise of our universities. With some exceptions, they are a sordid oligopoly run by incompetents and staffed by addled malcontents. Just ask Lawrence Summers.

Gary, missed your conclusion: To be sure, a significant number of prominent "American" scientists and some business leaders were born abroad and immigrated to this country. This attraction to skilled immigrants must be taken into account in assessing the quality of the human capital that enters the American economy, although it may reflect unfavorably on the quality of education provided to American students in math and the sciences. Still, if America allows India and other countries to pay the cost of training many of the engineers and other skilled workers who end up in the American economy, that is a pretty effective human capital production "system" when considered in its totality.

Wondering what you meant by it - should the US educational system be measured on the final numbers - that may include the foreign educated? and secondly - where would their costs be traced - in the native lands or in the US? I thought there was a need to accrue a notional level of costs - benefits US derives from these intakes. Love to know your mind;

I agree that average math skills, or average reading skills, or average abilities at any particular subject area are not a good measure of anything but the average of skills in that area. A far more useful statistic would be the scores of the top performers from each country. After all, since math skills are not required for most jobs in the economy, those who aren't good at math simply end up in other jobs. Those who are good at math become superstars in the field. Thus, the poor average performance on a test doesn't affect the economy proportionately, since the bottom performers don't have to perform those calculations on the job. The reason I'm curious about the scores of top performers is because they are the ones whose skills in these areas are actually reflected in economic output.

"except that our public schools seem needlessly costly, and also unresponsive to the special needs of very poor students."

This statement is not in line with the experiences of my family or with what I have seen from my girlfriend's teaching career in some of the poorest rural schools around.

Public schools, in southern Indiana for example, absorb social welfare programs that the tax base otherwise would not support. (counseling, juvenile delinquency remediation, health care, free lunch and breakfast, afterschool day care, special needs care). These schools appear more costly than private or parochial schools because those schools do NOT take on this much community service, nor do they take many of the students that public schools must contend with.

A single blind, autistic, or severely abused child can absorb ten times the teacher and school resources as a "normal" child. But we have covered this ground in previous discussions about school vouchers.

"It is always possible to improve scores on standardized tests by orienting instruction to the tests, by tutoring, or, if worse comes to worst, by withholding the test from the weakest students!"

All of these things are happening in epidemic proportions in our schools because of the heightened focus on testing since "No Child Left Behind." Fill a teacher's class with special needs students who can't pass the test, tell the teacher they are fired unless the students pass the test, watch what happens. Tutoring is not bad, unless the added expense provokes criticism of the school's fiscal policy. Teaching to the test and withholding or cheating on behalf of bad students are very destructive of the educational environment.

Art, music, gym, and other electives are already being taught less. In many cases, they are gone from the public schools entirely. Students might end up knowing more math and sentence structure, but they are going to be more boring people. Tests standardize around FORM, because substance is often subjective (and thus difficult to test).

Or to put it another way, in the world of standardized testing, no one understands the literary and political value in the statement "two and two make five."

"I'd be interested to see whether the international scores correlate at all with the IQ studies, which show a strong association between IQ and the level of economic development."

If that is true, then the IQ test is suspect. Unless you can tell us why "intelligence" would be correlated to "how much money I have." Oh, but you said...

"I'll get flamed for this, but some is probably due to heredity."

No, no flames, just quiet sighs of complete disapproval and sadness, and resolve not to let the next generation be so enamored of tests that they are willing to believe in hereditary racial inferiority rather than question the possible cultural or socio-economic bias in the measurement.

Should we talk, scientifically, about just how one normalizes a test that is being given across international linguistic, cultural, economic, and political barriers. Perhaps the process of translating the test introduced a systematic error eh?

I would be curious to see how the people on this blog evaluate their own education experience prior to age 15 (so, for most Americans, grades 7-9).

For me, I remember the tremendous incentives for short-term learning and forgetting - the cram and dump syndrome. Why spend 3000 hours covering information that no one thinks is worthy of long-term retention? Surely all that instruction was not just expendable fuel for the encouragement of critical thinking, discussion, and creativity skills? Suffice is to say that I think those years could have been much better spent than they were.

Perhaps we ought to ask a sampling of the American 15 year olds who took the test, "What did you think of the test and did you take it seriously?" We might find that there was nothing to be personally gained from taking it, that they just blew it off as being unimportant and of no consequence so why bother. That's a teenager for you, especially an American one.

If they weren't taking it seriously, that's probably an even better way of assessing ability - the areas of weakness won't have been crammed for and the areas of strength neglected, so it will reflect actual ability a lot better. I know that in the exams I did when I was 15 my marks were more or less the same in all subjects even though I was much better at some than others, just because I focussed so hard on what I was bad at.

"Corey, tests can be--and have been--adjusted for all the things you mention.

And we take your word for that because? There was a fun study at Berkeley on the LSAT that corrected for education, socio-economic background, number and status of parents, etc... still found a ten point racial score gap. How do you explain that? Undertheorized test bias is the likeliest answer, especially given the frequency with which we hear:

"Unfortunately, when results don't fit with what people wish to hear, they make excuses and demand the test be changed."

And you don't object because the results reinforce your own feeling of superiority. Everyone below you on the standardized test hierarchy is an incentive not to question the utility of the test.

This is why the caste system persisted in India in some form for thousands of years. Create enough subcastes and everyone but the bottom caste is slightly better off than they would be if they joined with the untouchables in revolt. All you need is a religious belief in objective karma (or objective merit here in the US) and the recipe for entrenched discrimination is complete.

That is, the more you believe that objective merit exists, the less incentive you have to question any purportedly merit based sorting.

I would be able to make a whole different group of people "smart" just by changing the questions on the test.

So then you need a theory for why the way the test is now is better, one that explains demonstrable racial and economic score gaps, and accounts for the obvious fact that it is possible to improve performance by teaching to the test.

If Americans teach to the test better than say, the French, and we beat them at math scores, what have we proven? Greater ability to sacrifice content to the gods of objective criteria? A less than proud accomplishment.

Slate covered this, and I think they are on to something. There is no incentive for kids to do well on this test, and if there is one thing we believe in America, it is that people only respond to incentives.

http://www.slate.com/id/2124163/

===
You could conclude from these exams that American high-schoolers are ill-taught and ill-prepared for the competitive global economy. But what if you look at these tests like a capitalist rather than an educator? Nothing is at stake for kids when they take the international exams and the NAEP. Students don't even learn how they scored. And that probably affects their performance. American teenagers, in other words, may not be stupid. It could be that when they have nothing to gain (or lose), they're lazy.

The fact that 8-year-olds and 17-year-olds have different attitudes toward low-pressure exams isn't going to come as a surprise to anyone who has raised a teenager‚Äîor has been one. The NAEP is used to judge school systems and overall student performance, but the test doesn't matter at all to individual kids. In 2002 nearly half of the 17-year-olds tapped to take the national NAEP exam didn't bother to show up. Students who did show up left more essay questions than multiple-choice questions blank, an indication that they weren't going to be bothered to venture an answer if it required effort.
===

Posner contends that the typical American does not understand that "randomness is not regular alternation--that a typical random pattern is 1000110110001, not 101010101010."

I agree that randomness is not the same as regular alternation, but the idea of a "typical random pattern" escapes me. Each of those two sequences would occur with equal probability in any truly random number such as a the representation of a series of coin tosses or in the binary representation of pi, for example. I do know, however, that if I bet on a "0" to show up next in the second sequence, I would be the winner at the table, which indicates that the typical American considers the second sequence to lack "randomness."

I think we should require remedial math for all members of Congress. Then, perhaps, we would have a balanced budget.

Seriously though. In my job, as in most, there is rarely a need to compute the area under a curve. Still, math education is useful. First, it encourages logical thinking and problem solving, which are important in any field. Second, some people will become our future math teachers, scientists, engineers, etc. In grade school, we are not yet sure who those will be. If everyone has a strong math education, then more people will be able to choose technical careers down the road. High school students should have a strong background in all disciplines, so they can choose, in college and beyond, what occupations to pursue. Also, it gives them a broader understanding of the world.

Finally, I strongly take issue with this comment by Judge Posner:

"So I do not think that the low quality of public education matters a great deal from an overall social standpoint, except that our public schools seem needlessly costly, and also unresponsive to the special needs of very poor students. These are reasons why I strongly support school voucher programs."

In contrast to Posner's thesis, a well educated public holds far more potential than a poorly educated one. And, hopefully, an educated public will do a better job of selecting leaders and will be less fooled by demagoguery. Or by so-called "statistics." As the saying goes, there are lies, [expletive deleted] lies, and statistics. Beyond that lie watered down high school level statistics. :)

As to Posner's voucher point, I once again chalk that up to a misplaced ideological plug. The public vs. private school question is an entirely separate debate. As someone who attended public schools and also an elite private university, I see merits (and demerits) to both approaches.

The assertion that the American high school education is part of a "build-up" design is not convincing. This build-up, build-down theory is somewhat dubious, in fact. It sounds to me more like we are trying to make a bad situation look not as bad. There is nothing that says that in a system where students are more rigorously prepared in their high school will necessarily slow down in their college years. My money is on that they will, in fact, be more prepared and more motivated to acquire more knowledge and skills.

A significant portion of high school "graduates" do not continue their formal education, most of them therefore have little incentive to select the more demanding subjects and to work hard. This mal-educated lot are the ones who meander among low-pay jobs for years, have little analytical abilities and, as such, are most gullible to the dishonest merchants and pandering politicians alike, and finding many problems in life beyond their ability to solve.

Even many of those who plan to go to higher education are often mal-educated under the easy system of high schooling. As a result, they sit through four years of college, not learning and thinking (the theorized "building-up" is just too hard for them) but partying and emoting. So along with the engineers, scientists, economists, lawyers, etc., the universities produce a huge crop of graduates who have little or no marketable skills.

Surely the great judge knows that mathematics is not just about computation. "Deskilling" may be the story of modernity, but "de-skill" should not be equated to "de-knowledge." More accurately, modernizing causes people (at least the productive ones) to specialize rather than to dumb down.

Oh, I was also intrigued by the great judge's comment that 46 percent of the people deny that human beings evolved from earlier animal species. I am not a "creationist" myself. But is it not true that while evolution is amply demonstrated in the adaptation of the species to the environment, we still have neither proof nor verifiable procedure to ascertain that evolution has produced any new specie? Not that I don't think that evolution is our "best" theory so far.

it is all about digital vs analog:
(1) when a five year old asking for a digital clock, because she can not figure it out the analog type, you got problem. (most likely not happen in other place)
(2) when grade 11 asking about sin and cos and graphic calculator, without fully understand what is real meaning of sin. cos. tan... you got bigger problem (logic thinking process problem too)..(definitely not happen in other place)
(3) when the MBA or Marketing got fat paycheck compare to the innovator and engineering in 30 to 40x... you got motivation problem... the abstract out weight the reality..
(4) when you have a MBA program called "Science and Engineering MBA", but it is the program try to avoid the challenge of math teaching to the other candidate (non-science and engineering), you got a problem presented on a grand scale. Just think the next gen. of leaders with lack of basic concept of math and science is like a downhill skii slop... (send the same test to MBA school final year class will prove my point...even all of them pass the GMAT with additional Math during the MBA program... no teamwork, just nomal 2hr standard tests after graduation)..

Taking education as something absolutely utilitary is wrong. Yes, purely specialized education gives higher returns instantly, but in the long run this will lead to catastrophe. One bad teacher can make tens of talented students hate the subject or even science in whole.

I studied in cheap elite high-school in Israel and I think, that mentality and "clustering" of good students are much more important, than funding. Even with equal funding received by all schools there would be self-segregation as some children are naturally stupid. They are not "little slow", nor "minimally exceptional", just stupid.