Don’t use the SAT or ACT to undermine education accountability

May 26, 2019

Pamela Burdman and Christopher Edley, Jr.

Pamela Burdman and Christopher Edley, Jr.

May 26, 2019

Providing all students with the chance to take college admissions tests sounds like a logical way to pull disadvantaged students into the college pipeline.

Pamela Burdman

Unfortunately, the way state lawmakers are moving to do this is a huge mistake that will undermine accountability for excellence and equity. For several years, a number of school districts and others have been pressuring the state to administer the SAT or ACT in lieu of the Smarter Balanced tests in English Language Arts and math that all 11th graders are expected to take each spring.

Last year, the Legislature actually passed a bill to do just that, but it was vetoed by Gov. Brown. A new bill (AB 751) now moving quickly through the Assembly has brought the idea back into play.

There are so many reasons why this is wrong.

First, rather than measuring students’ learning or achievement, the admissions tests contribute to preserving an inequitable status quo. Their baked-in assumption is that student aptitude is distributed according to a bell curve. Test makers write and score questions to create that curve and rank-order students.

Christopher Edley, Jr.

State standards tests (like California’s Smarter Balanced test) have a wholly different purpose. They’re designed to evaluate schools by measuring whether students are meeting educational benchmarks established by state education policymakers. There is no artificially imposed curve, so it is theoretically possible for most or all students to earn high scores, signaling that schools are doing a good job of educating their students.

Second, doubling down on college admissions tests like the SAT flies in the face of consistent research showing that scores on these tests are weaker predictors of college performance than high-school grades. Moreover, demographic factors — including family income, parental education and race/ethnicity — explain a large proportion (39 percent) of the variance in SAT scores, according to a recent study of University of California applicants. Such factors don’t cause disparities in test scores, the study said, but are tied to other more proximate factors — like quality of school attended, availability of test prep services or stereotype threat — that do.

Likewise, SAT-type tests don’t cause inequality. They perpetuate it with a technical design that ensures, not intentionally, that the demographic distribution of scores is virtually identical from year to year. This hinders efforts by colleges to recruit a class that is racially and socioeconomically diverse.

Last week the College Board, owner of the SAT, went farther than ever to acknowledge publicly the uncomfortable influence of students’ background characteristics on test performance. It released a so-called “adversity index” intended to put SAT scores into context by characterizing the applicant’s school and neighborhood environment. The implication is that universities should weigh contextual factors in evaluating SAT scores — something they should already be doing because a person is not a test score. The new adversity index confirms the limitations of college admissions tests, undermining the idea that they could replace accountability tests.

A third problem: research makes clear that the SAT and ACT are more stratifying in terms of race and income than the Smarter Balanced test. Colleges should consider adopting more equitable tests. This is especially important in California, where affirmative action cannot be used to compensate for the test’s inherent inequities, and a UC task force is currently studying whether to move away from using the SAT in admissions.

Fourth: SAT-type tests won’t work as the measure we need for accountability. The state’s Smarter Balanced test, which the legislation would make optional in 11th grade, unlike the SAT, is based on the curriculum standards California and many other states now use.

It hard to imagine how policymakers or parents would compare high schools’ performance if some use Smarter Balanced and others use the SAT. And if students’ scores are to be taken in the context of their “adversity index,” what about schools? How would state policymakers weigh the scores of high schools with high adversity levels?

The bottom line is that no matter how well teachers teach and how much students learn, college admissions tests are calibrated to rank order students from the 0th percentile to the 99th. Admissions tests are designed to ration college opportunity, whereas Smarter Balanced is designed to ensure K-12 opportunity.

Helping more students take the SAT and ACT has merit. But that is something school districts can already choose to do — and several have — without undermining California’s school accountability system in the process.

•••

Pamela Burdman is the founder is the founder of Just Equations, a project of the Opportunity Institute to reconceptualize the role of mathematics in ensuring educational opportunity. Christopher Edley, Jr. is a professor and former dean of the UC Berkeley School of Law, a faculty director of Policy Analysis for California Education, and president emeritus of the Opportunity Institute.

The opinions expressed in this commentary represent those of the authors. EdSource welcomes commentaries representing diverse points of view. If you would like to submit a commentary, please review our guidelines and contact us.

Comments Policy

Cynthia Schmeiser3 months ago3 months ago

The May 26, 2019, commentary in EdSource has some fundamental inaccuracies that negate the authors’ argument that the SAT shouldn’t be used for accountability. The biggest problem is that the authors discuss the old SAT and ignore the new SAT, which was first introduced to students in March 2016.
First, the new SAT was redesigned to be an achievement test. It measures what students are learning in school that is essential for college and career readiness. … Read More

The May 26, 2019, commentary in EdSource has some fundamental inaccuracies that negate the authors’ argument that the SAT shouldn’t be used for accountability. The biggest problem is that the authors discuss the old SAT and ignore the new SAT, which was first introduced to students in March 2016.

First, the new SAT was redesigned to be an achievement test. It measures what students are learning in school that is essential for college and career readiness. The thousands of teachers and curriculum specialists across the nation who participated in the redesign process or who are involved in the test development process have examined the test specifications for the new SAT (go here) and concluded that it measures skills and concepts essential for college and career readiness. The skills and concepts measured by the new SAT were identified by empirical research. (This evidence is posted at the above website as well.) That’s why there is a very close alignment of the SAT to the California state standards and standards of those states committed to college/career readiness.

Simply put, all vestiges of the old aptitude approach used to develop the early SAT 100 years ago have been eliminated. Any claims that the redesigned SAT is not an achievement test are false.

The article claims that the SAT’s technical design ensures that the distribution of student test scores is virtually identical from year to year. That is not true. The redesigned SAT is developed and assembled to measure the content and skills in the test specifications. That is the highest priority. Contrary to the authors’ claim, it is not assembled primarily on the basis of statistics to either maximize differentiation among students or fit them on a bell-shaped curve. Because there is no artificially imposed curve used to score the test, it would be possible, theoretically, for most or all students to earn high scores and to meet and exceed the College Board’s College Readiness Benchmarks (go here).

Second, the only national validity study using national data on the new SAT has just been released by College Board (go here). Studies of new SAT scores in California show that the SAT and high school GPA are independent and equally strong predictors of first-year performance at the University of California (UC), and when these measures are combined, there is a 16% boost in predictive power over high school GPA alone. Moreover, the UC study shows that the SAT adds additional predictive value over high school GPA that, if ignored, would mask students who would likely be successful at UC. That’s only one of the several reasons the claims made by the authors about the SAT being a weaker predictor of college performance than high school GPA are inaccurate.

Third, another troubling error made by the authors involves the College Board’s Environmental Context Dashboard (ECD), which they claim confirms a limitation of college admissions tests. This claim completely misrepresents what the ECD tool offers and why it’s being piloted with colleges. The ECD presents environmental factors that contextualize the high school and home neighborhood of students. These factors affect their scores on any achievement test, including the Smarter Balanced Test. Environmental and school factors clearly impact student achievement and success and add important insights about the context of students’ academic achievements that colleges and universities may want to consider when reviewing applicants.

Finally, the authors claim that the SAT is “calibrated to rank order students from the 0th percentile to the 99th.” Again, this is false. This is not the way the new SAT is developed, designed, or assembled as an achievement test. The SAT measures student achievement just as a thermometer measures temperature. When we see differences in test scores, those differences are not caused by the test: what we are seeing is how well students were instructed and how well they learned essential content and skills in high school. While College Board has many strong disagreements with the authors, the authors would likely agree with us that achievement tests like the SAT have exposed inequities in the quality of instruction in our schools for decades.

Addressing that injustice is one of the greatest challenges we face as a nation, and one of the most important.

el4 months ago4 months ago

Check out this study that wonders if gender performance differences on standardized tests may also be functions of temperature:
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0216362
"We study performance in math, verbal and cognitive reflection tasks and find that the effects of temperature vary significantly across men and women."
Maybe we need to include the quality of the HVAC system on school dashboards.
I agree that the SAT is not aligned to the curriculum, and I tend to think that the SBAC tests are both … Read More

Check out this study that wonders if gender performance differences on standardized tests may also be functions of temperature:

“We study performance in math, verbal and cognitive reflection tasks and find that the effects of temperature vary significantly across men and women.”

Maybe we need to include the quality of the HVAC system on school dashboards.

I agree that the SAT is not aligned to the curriculum, and I tend to think that the SBAC tests are both better indicators of what we ask the kids to learn, and more fairly administered (like that they are not time-limited in the same way).

If you want 11th graders to care about the test, it seems straightforward to allow it to give positive benefits when they do well – for example allowing it to be used in place of a math or English placement test for college, or allowing a good score to bump the student’s grade positively in the relevant English or math class. Or maybe UC/CSU should allow SBAC scores to be submitted in lieu of the SAT to competitive programs, to the extent they want test scores.

Dennis Higgins4 months ago4 months ago

Thanks for pointing out the fundamentally different purposes (aptitude vs mastery) of the SAT/ACT and the Smarter Balanced Tests. Teachers and schools can't do anything about aptitude. Mastery is what they work on, and so that should be what they measure. But I am uncomfortable when well-meaning educators try to avoid talking about bell curves (which are a fact of life) because they reflect negatively on the measured aptitude of Black and … Read More

Thanks for pointing out the fundamentally different purposes (aptitude vs mastery) of the SAT/ACT and the Smarter Balanced Tests. Teachers and schools can’t do anything about aptitude. Mastery is what they work on, and so that should be what they measure. But I am uncomfortable when well-meaning educators try to avoid talking about bell curves (which are a fact of life) because they reflect negatively on the measured aptitude of Black and (to a lesser extent) Hispanic students. That’s not a subject to avoid but to embrace.

I have tutored more than a hundred Black high school students over 20 years, and there is no difference that I can tell between the distribution of their aptitude as compared to that of other racial groups. Black and Hispanic students do perform worse on aptitude tests like the SAT and ACT, but it’s because, as a group, they lack self-confidence. And they lack self-confidence because, as a society, we have yet to come to terms with the widespread marginalization they endure in a still largely racist society.

Self-confidence is extremely important when you tackle a math or language problem, and nothing undermines a person’s self-confidence more than being repeatedly told (in subtle and not-so-subtle ways) that, “unfortunately and through no fault of your own,” you are deficient.

Karl Pister4 months ago4 months ago

My plaudits to the authors for their excellent explanation of what the use of the SAT should be limited to, if used at all. A UC study found that high school gpa was a better predictor of academic success than SAT scores as noted by the authors. An increasing number of institutions have dropped the SAT from the admission protocol.

Dr. Bill Conrad4 months ago4 months ago

SAT and ACT scores should definitely not be used as an accountability measure as they are not closely aligned with the Math and ELA standards that schools are expected to ensure that their students achieve. The SBAC is a better more and standards-based measure.
11th grade scores on the SBAC are embarrassingly low compared with Grades 3-8 performance. Check out results for Silicon Valley School Districts at http://sipbigpicture.com District Administrartors blame the students … Read More

SAT and ACT scores should definitely not be used as an accountability measure as they are not closely aligned with the Math and ELA standards that schools are expected to ensure that their students achieve. The SBAC is a better more and standards-based measure.

11th grade scores on the SBAC are embarrassingly low compared with Grades 3-8 performance. Check out results for Silicon Valley School Districts at http://sipbigpicture.com District Administrartors blame the students for not taking an interest in these assessments rather than looking in the mirror and understanding their role as the grownups to make sure that students are prepared to take the test and do their best work.

We should not be redirecting our accountability to the SAT and ACT where many affluent students take prep courses and are thus their results are not a true reflection of how well the system is preparing students for success in college and career readiness. The state tried to provide cover for the school districts by initially keeping the 11th grade SBAC scores off of the Academic Dashboard and then using a convoluted scale score averaging system that redistributes excess scale score points from high performing to low performing students creating a biased view of actual performance.

It is time for school districts to get their acts together and begin taking the SBAC seriously as a measure of student achievement of ELA and Math Standards. Their attitude should be bring it on rather than trying to weasel out of their responsibility to successfully prepare their students for success on the SBAC.

Bo Loney4 months ago4 months ago

To add to my comment, the academically gifted are not being given the same opportunities to be challenged. If they were they wouldn't be rolling on your floors in boredom and being used as teacher assistants to keep them busy. But we don't want to go there right? It's ok to recognize and foster athletic ability but not intellectually gifted of all races.
You know there are some kids that were … Read More

To add to my comment, the academically gifted are not being given the same opportunities to be challenged. If they were they wouldn’t be rolling on your floors in boredom and being used as teacher assistants to keep them busy. But we don’t want to go there right? It’s ok to recognize and foster athletic ability but not intellectually gifted of all races.

You know there are some kids that were totally failed in the current state of education from the gates of kindergarten, not hot housed and had no test prep that still come out with scores in the 1% on the SAT? Some before they even hit 7th grade and haven’t received any of the curriculum of high school. Keep ignoring this minority whose support has been decimated after NCLB though.

Bo Loney4 months ago4 months ago

Hedge Fund companies use SAT scores in their hiring process. Because raw horse power, when given the same opportunities at public school, matters more than inflated or deflated grades. Keep spinning though. Thanks for putting out an article supporting that it’s all about race and opportunities than acknowledging that there are academically gifted students.