Blog

Listen Closely: The Avant Annual Proficiency Report

Thursday, December 07, 2017 by Carl Falsgraf

Data whispers.

We need to get close and sit quietly to discover its secrets. And we must be good listeners. If we show up with a headful of opinions and assumptions, we might miss subtle truths and glorious surprises.

As a service to the field, Avant has released its annual report on STAMP test results from 2016-17. The proficiency ratings in this report represent the best national snapshot we have of student performance. Some of these stories are familiar, but others have surprise endings. Let’s start with two common beliefs about language learning and see what the STAMP data can tell us.

1. Two years of high school language study equals one year of college

Many colleges use this rule of thumb to place incoming students, but is it accurate? Let’s look at the case of Spanish, since we have a large number of test scores to inform this question. STAMP reports proficiency scores on a numeric scale that relates to ACTFL levels as follows:

1 = novice-low 4 = intermediate-low

2 = novice-mid 5 = intermediate-mid

3 = novice-high 6 = intermediate-high

Here is a comparison of average proficiency scores for second year high school students and first year college students who took STAMP tests:

Reading

Writing

Listening

Speaking

High school: second year

3.5

3.2

2.7

3.2

College: first year

4.2

3.6

2.9

3.4

This data indicates the “two years high school = one year college” is not a bad estimate. The difference in reading scores also explains why some college teachers complain that students are not well prepared for college. But there is more to the story.

If two years of high school study equals one year of college, then fourth-year high school students should perform at the same level as second-year college students. Here is a comparison of their STAMP scores:

Reading

Writing

Listening

Speaking

High school: fourth year

5.5

4.6

4.4

4.3

College: second

5.9

4.4

4.2

4.1

Here we see that fourth-year high school students actually outperform second-year college students in every area except reading. Why would this be? The best explanation is that many students who take two years of high school Spanish, especially those who do poorly, do not go to college. Those who stick with language study for four years, however, are more likely to attend college. So among high school graduates who attend college, the “two years high school = one year college” rule seems to slightly underestimate students’ abilities.

The real answer to this problem is to set aside seat-time estimates and administer a proficiency-based placement test, such as Avant PLACE. This will not only measure each individual on a proficiency basis, but help the college program orient towards proficiency outcomes from placement through graduation.

2. For English speakers, Asian languages are harder than European languages.

This seems like a no-brainer. All those crazy characters, lack of cognates, and grammar that differs so drastically from English must make it harder to learn Asian languages.

How do students of Chinese and Japanese actually stack up against students of Spanish? First, let’s look at elementary-level immersion students. To make a fair comparison, we will just look at writing and speaking. STAMP presents learners of all languages with the same prompts and grades them according to the same criteria, so this should give us a pretty good cross-linguistic comparison of third, fourth, and fifth grade immersion students.

While Spanish learners score slightly better in writing, Chinese and Japanese learners actually score a bit higher in speaking at grade 4, and are very close in the other grades. At least in elementary immersion programs, it doesn’t seem that Asian languages are more difficult to learn than Spanish. But what about older students?

Here is the same comparison for high school learners:

For high school learners as well, writing and speaking scores for Chinese and Japanese are about equal to those for Spanish. How could it be that students learn “hard” languages like Chinese and Japanese at the same rate as an “easy” language like Spanish? The data doesn’t even whisper an answer, but here are a few hypotheses.

1. Perhaps Japanese and Chinese students are higher achievers. This may sound attractive at first because it fits the stereotypes we have about students of these languages. But the immersion data also shows no evidence of a difference in difficulty and kindergartners don’t choose which language to learn. This alone does not account for this curious finding.

2. Perhaps Chinese heritage speakers skew the data. There are also lots of heritage speakers in Spanish immersion programs. And there are relatively few in Japanese programs. This is not likely to be a significant factor.

3. Maybe it really is not that much harder to learn an Asian language. For English speakers, Japanese is fairly simple to pronounce and Chinese lacks those complex verb conjugations and tricky pronouns of French and Spanish. Using a keyboard to write Japanese and Chinese means students don’t have to handwrite every stroke. It may be that we have over-estimated the relative difficulty of Asian and European languages, at least at the novice and intermediate levels.

Data whispers, but it also mumbles. These averages obscure huge differences in programs and individuals. Schools that use STAMP may not be a fair sample of all schools. Different programs administer the test at different times of the year. We rely on schools to accurately report grade levels and program model. We have to throw out some data that is just too messy. All of these factors, and more, mean that we have to be cautious about making bold statements based on this one report. Nevertheless, listening carefully to this national proficiency data can raise questions and force us to rethink assumptions.

STAMP data has more stories to tell. In our next blog, we will look at whether the proficiency pyramid is really upside down and how much of an advantage heritage speakers have in immersion and traditional high school programs. Keep your ears open for the next whisper!