Researchers Call SAT Alternative Better Predictor of College Success

Everybody knows that a little creativity, wedded with a good amount
of common sense, can go a long way in life, whether it's in a pickup
basketball game or the thorniest corporate board meeting.

Yet those attributes aren't always easy to quantify—and they
aren't measured with much success in today's most widely used college
admissions test, according to a prominent scholar and team of
researchers working out of Yale University.

Robert J. Sternberg, a professor of psychology at Yale who has
probed the norms and nuances of intelligence and wisdom for years, has
completed the first phase of a project he believes could produce an
admissions test that is a stronger predictor of success in college, and
fairer to underrepresented minorities.

His premise is the belief that the SAT, taken about 2.6 million
times by students each year, measures primarily memory and analytical
ability, rather than creative or practical skills—the latter of
which is usually associated with common sense.

Creative and practical talents are important for determining an
individual's success in college, and in life, Mr. Sternberg contends.
He directs the study, called the Rainbow Project, which is
financed by the College Board, the New York City-based nonprofit group
that sponsors the SAT.

"The system fails to identify kids who might succeed in ways the
[current SAT] doesn't suggest they will," said Mr. Sternberg, the
director of the PACE Center at Yale in New Haven, Conn. He is also the
president of the American Psychological Association.

For the project's first phase, researchers designed an alternative
exam, one that Mr. Sternberg believes eventually could augment, rather
than replace, the SAT. The new test was given to 1,007 student
volunteers at two high schools, five community colleges, and eight
four-year colleges across the country, mostly from April to June of
2001.

Because researchers sought to gauge the predictability of success in
college, not high school, the first phase of the study primarily used
the data collected from the 793 first-year college students among the
group of volunteers. Elena Grigorenko, the deputy director of the PACE
center, took the lead in coordinating much of the testing
process.

New Sections

The exam evaluated students in three overall areas: analytical,
practical, and creative skills. It used a combination of
multiple-choice questions and performance-based sections, in which the
participants worked out their own answers.

Some of the sections, such as one asking students to figure out word
meanings in the context of written passages, bear a resemblance to the
current version of the SAT. Many other sections, though, share little
with today's model.

In one section, students were given cartoons, purchased from the
archives of The New Yorker, and were asked to write captions for
them. Test-takers were also asked to write two stories, taking 15
minutes each, based on a list of fictitious titles. A team of judges
rated those stories on originality, evocativeness, complexity, and
descriptiveness.

Other sections presented students with various troublesome
situations they might encounter in everyday life and asked them to rate
numerically the best solutions.

A supervisor has assigned you to work with an employee who is "rude,
lazy, and rarely does a proper job," according to one sample on the
"common sense" questionnaire. Students were then asked to rate the
quality of different responses, such as telling the colleague off, or
working with him in a businesslike manner.

The current SAT is a good predictor of college success, Mr.
Sternberg asserts, but he believes the Rainbow Project's test does even
better. He also says the new test showed smaller gaps between the
scores of students from different racial and ethnic groups. In theory,
the new exam could offer a college-admissions test that more accurately
predicts postsecondary achievement, with fewer worries that the exam
was biased in favor of one racial or ethnic group or another.

Diane F. Halpern, a professor of psychology at Claremont McKenna
College, in Claremont, Calif., who assisted with the project, arranged
to give the exams by computer to about 150 students at the campus where
she used to work, California State University-San Bernardino. She saw
value in the study.

With today's technology, students have the ability to access more
information more quickly than ever, Ms. Halpern noted. But the
challenge in college, and in life, she said, is approaching data
critically and gauging what is most important.

College-admissions tests, she argues, need to reflect the demand for
those skills.

Creativity in a Can

One researcher familiar with Mr. Sternberg's study, Paul R. Sackett
of the University of Minnesota, disputes some of the project's
conclusions, though he says he admires its broader goal.

By giving the test to first-year college students, the Rainbow
Project was evaluating an overly restricted population of test-takers,
said Mr. Sackett, a professor of psychology at the university's Twin
Cities campus.

And in trying to gauge the success of the participating students in
college, the Yale project did not go far enough in accounting for the
differences in grading at the particular schools, Mr. Sackett
contends.

Overall, while he sees merit in the study, Mr. Sackett believes that
its claims about predicting college success, and reducing disparities
between racial and ethnic groups, are overstated.

But Mr. Sternberg counters that his test-taking population was quite
diverse, since it included high school and community college students,
who might not get accepted or even choose to go to a four-year college
at all. And he said adjustments had been made to account for college
grading.

A central question, Mr. Sackett noted, is whether the project has
any value for a broad swath of the college-going population. He had
doubts: The ability to assess students' creative and practical
abilities could suffer from college applicants' determination to
manipulate and beat the system.

"Will it still measure creativity when people can go to two-day
workshops aimed at writing cartoon captions?" Mr. Sackett said.

But Mr. Sternberg asserts that because sections of the test grade
students on innovation and the novelty of their ideas, their scores
would suffer if they were simply to repeat stale answers suggested by
test-prep courses.

Rather than supplanting the SAT, Mr. Sternberg envisions that a
portion of college-bound students could take the new test, either
because certain colleges asked them to or because the students
voluntarily signed up for it. The second phase of the project will
focus on giving the test to a larger sample of students—anywhere
between 5,000 and 10,000—and shortening its time from about three
hours, among other improvements.

Mr. Sternberg is optimistic about where the project is headed.

"I'm very pleased," he said. "We see this as not just a project, but
a step to transform education ... that will change the way we teach in
the classroom."

The American Council on
Education offers a brief
summary of a National Urban League survey report finding that
standardized admissions tests "fail to measure the qualities that are
most needed to succeed in the corporate world."

Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.

Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.