Schools with many AP tests but lousy scores

We education watchers are gradually waking up to the fact that a very small but growing number of educators are using Advanced Placement, originally designed for only the best high schools, as a shock treatment to improve instruction at some of our worst high schools.

This is not, to say the least, a well-understood trend. Some of the smartest AP people in the country do not like it. Others do. I think it has great potential benefits, but it is too soon to draw solid conclusions. So I have appointed myself the unofficial scorekeeper for such schools, and have created a special category for them -- what I call the Catching Up schools -- in my annual Challenge Index ratings. This includes my ranked list of all public high schools in the Washington area, published in The Washington Post, and a separate list of schools nationally that have the highest AP test participation rates, best known as America's Best High Schools in Newsweek.com.

I am giving this such attention because when I have looked at schools using this wild approach, it seems to be working for them. Students and parents like the challenge and don't care if they are unlikely to pass many of the tests. The teachers are energized. The fears of critics that using AP with low-performing students will create false expectations and low self esteem seem unfounded.

The Catching Up lists are separate from the main Challenge Index lists. (I am about to update the Newsweek list, by the way, so any school that is not on the list now but thinks it qualifies because it gave at least as many AP, International Baccalaureate or Cambridge tests as it had graduating seniors in 2009 should e-mail me at mathewsj@washpost.com.)

We were slow this year in getting the national Catching Up list on Newsweek.com, but it will soon be there, so I wanted to explain what it shows about this odd group of schools that think they can help students who are below grade level with a program designed to mimic courses given in the first year of college. Here is the national Catching Up list of 33 ranked schools, with a short introduction:

AP programs have evolved in an unusual way in some high schools with large numbers of impoverished students. Educators at these schools have concluded that although few of their students are likely to achieve passing scores on the three-hour college-level AP exams, many would benefit from taking AP courses and tests that acquaint them with college standards and help them build academic muscle for college. Once students are involved in AP, they say, teachers can help them catch up to the AP standard through improved instruction and more challenging programs in lower grades.

This approach to AP is so different from most high schools that Newsweek has decided to provide a separate list for those schools where less than 10 percent of AP tests earn passing scores. Each school is ranked as usual by its Challenge Index rating, the ratio of AP tests to graduating seniors. The list gives the name of the school, followed in parentheses by the passing rate on all AP exams taken at the school, followed by the school's location, its percentage of students who qualify for federal lunch subsidies and its Challenge Index ratio.

There were 31 schools on last year's list. Only 18 of them remain on this year's. Some of those that didn't appear this time failed to qualify because their index ratio fell below 1.000. Some failed to respond to my requests for their 2009 data.

But six of last year's Catching Up schools improved their passing rate on the AP exams enough to move from this list to the main list. Here they are, with their change in passing percentage:

Englewood, San Bernardino and McKinley Tech had somewhat lower index ratios in 2009 than they had in 2008. The other three, Columbia Heights, R.L. Osborne and Thurgood Marshall, had higher ratios. That means they not only gave more AP tests but had more tests with passing scores.

As you can see, this experiment affects just a few schools in a small number of districts -- Jacksonville, Orlando, Tampa, Miami, the District, Kansas City, Houston, Fort Worth, Prince George's County, Charlotte, Avondale Estates and Donaldson, 12 in all. Sixty-four percent of the schools have more than half of their students coming from low-income families.

I set the target for moving off the Catching Up list at 10 percent of all AP tests with passing scores, because at that point these schools produce as many passing scores as schools with average passing rates and average AP test participation rates. Enough of them are making progress to lead me to think that their principals, no matter how wrong-headed they might look to some critics, might be doing something right.

So I will continue to keep score and see where this goes. I welcome messages to mathewsj@washpost.com from any educators or students at any of these schools. Tell me how this unusual approach looks to you from the inside.

I don't have a strong opinion one way or another about schools pushing lots and lots of unqualified kids into AP courses. In an ideal world we would obviously invest more resources into pre-AP preparation to insure that larger numbers of kids taking the AP courses are qualified. But in schools where you get 1 out of 20 or so exams with passing AP scores, it's tough to fault the administration for at least trying something fairly radical.
I would like to see some data though (not anecdotes) about effects of having high numbers of unqualified kids in AP classes upon kids who are marginally qualified.

It would also be helpful for you Jay to publish passing percentages (based upon test takers) for all the high schools on your list. I know that you believe that pushing the kids to take college-level courses and exams should be the goal, but lots of people would, I'm sure, like to see just how well the high participation schools are doing in terms of passing percentages.

Thanks for creating this list Jay; I do think it is helpful. You mention the percentage of students in some of these schools with higher than 50% free and reduced lunch, but on your "regular" challenge index a number of schools (like Stuart in FCPS) have a very high free and reduced number and consistently appear fairly high up on your challenge index. So, I assume you aren't using that (free and reduced) as a reason for some of these schools on the new list to have such low passing rates.

I do wonder why you can't put the passing rates on your challenge index since you seemingly have that data? Now that it has been, what a decade, since you created the list couldn't you add that figure?

Your point, that challenging the students (hence the name) was the reason for your index has well, kind of been made. People opposed to your reasoning are still opposed a decade later, so change it up and create one list, with passing rates included. You added the equity number a couple of years ago, right? And that only added value to the challenge index. (i.e those opposed appreciated it).

So, why can't you add the passing rate to the original challenge index?

For patrick and researcher2, who always have useful thoughts on this. --
I have the passing rates on AP and IB for every school, and could make those available in some way, but the Web site people have argued that I already have too much data on the list for most people to absorb, and that adding another line would also make it difficult to keep the list up and healthy, or something like that.
I will discuss this point with them when we do the list next year. Thanks very much for raising it. I think keeping the list fresh with new dimensions is a very good idea. My thought for next year was to experiment with adding dual enrollment course test participation to the formula, as i do on the local WP Challenge Index.
Tell me your reaction to this---I have long thought that the equity and excellence percentage is a better measure of mastery than the percentage of tests passing because it did include an element of participation, but did reflect strongly how many kids were passing the tests. You can look at the list now and find schools with weak e and e percentages, and thus know that they are getting by with low scores. Would adding the passing rates for AP really add anything that the e and e does not already tell us?

If you take a class where no-one or almost no-one passes the AP exam, have you really taken an AP class, or have you taken some other class, better than the "honors" class offered in your high school, but not really an AP class? If so, I have no problem with that for the students who took the class even though they weren't ready for it; there is a downside, however, for the students who were ready but didn't actually get an AP class, therefore didn't get AP credit (because they failed the AP test or got a 1 or a 2), and might have passed the AP test if they had been given a more traditional AP class.

for jane100000--- good question. if you are a student at McLean High and one of the 5 percent who get only a 2 on the AP US history test, have you taken an AP course? I would say you have. Particularly since we have data showing that kids who get 2s do better in college than similar kids who dont take AP.
In the schools on this list it would be the VERY rare student who was all set to get a 5, but didnt get pushed enough because the course was full of slow kids . In nearly every case an AP course and test are going to be, as you say, much more challenging that whatever else any student there would have had at that school, so it is unlikely that anyone was all primed for a 5 because their previous courses would have been pretty low level.

Second, you need to track whether the bulk of the failures were 2s or 1s. In most cases, the answer is most of the students are failing with the lowest grade possible.

What is the problem, Jay asks blithely, with letting students take tests they aren't prepared for?

It is a COLOSSAL waste of money. In most cases, given that these are low income students, the state is paying for unqualified students to take a test they haven't a chance of passing.

The externalities are creating profound inequities. Most colleges give an extra GPA point for taking AP courses on the grounds that the student is taking harder courses *than the average high school student*. But these students are taking much easier courses that are simply *called* AP. So they are getting an unfair GPA advantage, over far more qualified students of *all races and income levels*, mind you, because their schools and teachers are fundamentally committing fraud all to get Jay's approval.

Jay is only looking at one tree: unqualified students are spurred to work harder, he says. He could care less about the forest. Never mind that they are using a system that was not intended for unqualified students. Never mind that he's participating in a process that involves fraud. Never mind that in a time when schools and states are bankrupt, we are wasting millions, if not billions, of dollars on a fraud--all because Jay wants to piggy back unqualified, unready students on a system that never thought of setting up rules to keep them out (and now it's politically impossible). None of that matters because hey, kids are working harder!

Tell you what, Jay. Let's take all the millions of dollars wasted on unqualified kids taking AP, and ask society--not parents, not kids, not the schools committing fraud--but the taxpayers who fund this farce. Let's ask society if we could be doing something better with the money than committing fraud and promoting unfairness. Let's see if they can come up with some other way of "challenging" kids who are wholly unprepared for the academics they're being shoved into.

Would adding the passing rates for AP really add anything that the e and e does not already tell us?

Posted by: Jay Mathews

Yes, it would clearly state the percentage since the e and e is a formula, and not really the percentage of whom passed (I remember you made it a point that it was confusing when you first reported the e and e because people thought 45..or whatever, meant a 45% passing rate).

I am amazed actually that you state schools moved to the challenge index when they achieved 14% passing rates...is that all it takes to be on the "regular" challenge index???

Your editors don't need to worry about an additional column being too much data; for those opposed that column would be key to feeling the list meant something.

I do think Cal makes some valid points about paying for the tests. Though I do believe eligible students shouldn't be excluded from taking the tests due to inability to pay, I don't think we should be pushing unprepared students into classes and paying for their tests.

If students' reading ability is not at the high school level, and/or they have yet to pass Algebra 1, they shouldn't be pushed into AP courses, and taxpayers shouldn't pay for their tests.

I am clearly not saying they should be "college ready" when stating they only need to read at a basic high school level and have basic Algebra skills. Students meeting these requirements would still be Challenged, and would be the students AP normally doesn't "target" but they would have a chance of passing the AP exam (if in a true AP course).

The number of different AP courses that a school offers should be a function of the overall passing rate - say 50% to put a mark on the wall. Proliferating the number of AP courses available while letting the passing rate plunge is a broad indictment of a school.

If I was in the admissions department of a selective college, I would be highly suspect of any high school that had a high Challenge index score coupled with a low passing score rate. I would much rather see a high AP passing score rate coupled to a Challenge Index score greater than about 3.0.

As a rule of thumb, I might multiple the passing score rate by the Challenge Index score. If the product was greater than 1.5, I would conclude that the subject high school probably has a high quality AP program.

I am a little torn here. I find the assumption that a class has to be AP to be challenging to the students odd. As a teacher, I can set the bar high and challenge my students regardless of the label on my class. I teach accelerated middle schoolers, the majority of whom go into AP, IB, or Cambridge programs. Our local AP school does not allow students into the pre-AP classes unless they get As and Bs in their high-level middle school classes. Even underachieving gifted kids are kept out without a history of working hard. This makes sense to me, but at the same time I don't like the idea of a gate, telling non-typical AP students that they can't even try. Maybe the answer is that we need to intervene earlier. If students in elementary and middle school realize they have the opportunity for the college track and are challenged, they will be (more) ready for pre-AP and AP (or similar programs) in high school. Our goal should be that ALL students have the opportunity for the best education possible, that they learn to push and stretch themselves.

Jay, "doing better in college if you've gotten a 2" does not equate to "AP." Let me piggyback on what jennypalmer1 has said: you can make a course more challenging without calling it AP. The label should not matter. Why, I ask, are we so dependent on the label AP? Why aren't teachers teaching hard, and students studying hard, in classes labeled "honors?"

for jane100000--because it is very hard in the social pressures of an average high school to keep standards high without incorruptible external assessments like AP or IB.

for jennypalmer1---i would be very grateful for an email to mathewsj@washpost.com telling me which AP school that is. I promise not to reveal my source.

for Cal---I really think you and others with a visceral reaction to yr image of what is going on inside these schools need to actually visit them before you draw conclusions. You are a great teacher and I think would see the value here once you saw what is happening at Columbia Heights in DC, for instance. I realize that is a stupid argument since it is hard for most people to get to these schools, but seeing them makes a difference.

for fairfaxguy and Cal---It seems a little premature to worry about kids at these schools getting a GPA bump and getting an advantage in the college race. Take another look at the schools on the list. These are not kids competing for the Ivies. They go to colleges that take almost everybody.

My reaction isn't "visceral". I simply object to your piggybacking a cause onto an existing system and actively participating in its misuse.

"--because it is very hard in the social pressures of an average high school to keep standards high without incorruptible external assessments like AP or IB. "

And yet, incorruptible external assessments is the one thing your Challenge Index completely ignores. You give schools with dismal results the highest ratings, provided they shove students into testing seats.

AP is supposed to be a challenge, not a privilege for the few. Critics like Lanier would never have let Jaime Escalante teach his course because those classes initially would have a low pass rate. Although AP was started as a way to exclude and classify, smart teachers and students have morphed it into a way up.

Those ELD kids or those kids whose parents did not go to college should be encouraged to take AP classes by their districts. This is one of the jobs of education.

Good AP teaching and tough "regular" classes open the eyes of students to see what is possible. In creating high standards it raises the level of all classes in the school as AP teachers take those techniques and apply them in all their classes, in their departments and filtering down to the middle schools. Yes it takes time - but give these schools a chance.

I would be interested in a new list too. That list would have all the scores of schools with 50% or more free and reduced lunch who made your Challenge Index. That is where the real movement is going on. Keep it up!

AP isn't the only objective measure. The SAT II or the CLEP tests could be near the end of the appropriate class AND the scores are returned much faster, so grades could be adjusted appropriately to reflect test scores; no more As for kids scoring 400.

momof4: You forget that there are many students out there that are poor test takers in high stress situations. They might be amazing workers in the classroom, who ask questions and refuse to quit until they understand the material, but who freeze up in the artificial test-taking environment. As a teacher, I am not going to change my classroom grade, based on their work in my room, because of a score on a standardized test. So yes, you can pass the class, but not reach a high enough score on the AP to receive college credit. Remember that the point of AP college credit is to place a student out of the lower level college classes. Perhaps these students that receive a 2 on the test will now have to take that lower level college class, but will be more prepared for it than if they had not been in AP.

"You forget that there are many students out there that are poor test takers in high stress situations. "

Cite. I'm a test prep instructor with 7 years experience, teaching students at every point of the income spectrum. I would say maybe 1-5% of the population is stressed out by tests to the point that they score 10% lower than their actual knowledge.

PC psychobabble aside, "test stress" is extremely rare. The reason that many students do well in school but poorly on tests has more to do with their poor cognitive abilities (or teacher's lies) than test stress.

"I would say you have. Particularly since we have data showing that kids who get 2s do better in college than similar kids who dont take AP. "

Those two statements don't follow.

The AP test is not supposed to be an individualized achievement test, but an absolute standard.

The teacher might have spent the entire time focusing on reading and writing in an AP US History class, for example, which improved the student's skill enough to get a 2. But was the class college level? No. And if the class wasn't college leve, the student shouldn't get credit for taking a college level class.

I agree with Cal: In general, absent specific learning disabilities such as dyslexia, most people who "don't test well" are people who don't know the material well. Classroom practices such as regular, quizzes (maybe only 10") and tests and increased weighting of same (vs. homework) may help students; classroom practices like rare quizzes and tests and inflated homework weighting, projects and groupwork may make standardized testing more stressful.

My ten years in the classroom - private and public school, 3rd grade through high school - has consistently shown me that some students are better test takers than others. Those that are weak can be taught skills that will serve them well in the future, but it is not natural or easy for them.
As well, study after study has shown that the one link to success on standardized tests is parental income. So do we just write off those students of lower socio-economic status, who are of many different cultures? These students may not have the support at home for their education. They don't have the opportunity to go to a camp for the gifted at UVA like my older son, or space camp like my younger child. Does this mean they shouldn't be permitted in AP or IB, because they don't have the background that supports success? If these kids are willing to try, are willing to put forth the effort to succeed that these classes require, why do you want to keep them out?

"My ten years in the classroom - private and public school, 3rd grade through high school - has consistently shown me that some students are better test takers than others. "

You don't need more than one or two years IN grade school--as a student, not a teacher--to know that much. But your original claim was:

"You forget that there are many students out there that are poor test takers in high stress situations. "

And you went on to assert that they would be *so* bad at testing that it would misrepresent their actual knowledge. That's quite a different assertion.

Would a student who has a 5 knowledge get a 1? No. Would a student who has a 4 knowledge get a 2? No. Would a student who had a 3 knowledge get a 2? Only if they were in the small percentage I mentioned, and even then, that's unlikely.

"So do we just write off those students of lower socio-economic status, who are of many different cultures? "

No. Ideally, without Jay and Newsweek creating bogus incentives, we put them in classes that will meet their academic abilities. In some cases, that will be AP classes. In others, the classes will be less advanced. A student who reads at the 8th grade level shouldn't be in AP. I don't care if they improve to reading at the 9th grade level. Their presence in the class is a lie, it's distorting a good system, and it's giving them advantages over kids with equal or better skills whose teachers weren't willing to commit fraud.

1. The current popular trend in education is the idea that high expectations create high achievement. This data seems to contradict that idea.

2. AP tests seem to have become a Frankenstein's monster of sorts. Conceived of and created at Kenyon College, that college, along with many others, no longer accepts AP tests for college credit. Do the inventors know something about the flaws in the system?

Stop fidgeting with numbers and percentages; humanize the issue. This boils down to a matter of equal access to educational resources. AP should be open enrollment. In my experience I have seen open enrollment cause AP teachers more work and added frustrations. Unfortunately some under prepared students simply don't do the expected work. On the flip side, some marginal students step up their work ethic. That said, everyone should get a chance to take an AP class if they desire. How are Americans going to become more educated and more socially mobile if they are not given the opportunity to break the cycles of ignorance and poverty?

It says something significant about our public education system that apparently the only way students can be challenged is in AP classes. Why should we not have a system where every kid is challenged (but not overwhelmed) every day, starting in kindergarten? If we did that, far more kids would be prepared to do AP work in high school, after taking appropriate prerequisites.

The United States is the only country that tries to educate all of its citizens. Mainstream classes are composed of a range of students with skill levels ranging from far below basic to advanced. Some mainstream students have learning disabilities, many are English learners, and some just have very low skill sets. Mainstream classes have to strike a balance between the needs of all of these subgroups. There are many students who become overwhelmed by the existent level of rigor in mainstream classes.