Finding the better high school

On the second page of the Post's Metro section, and on this Web site, you see the results of the 12th annual Washington Post survey of high school student participation in college-level tests, what I call the Challenge Index.

The ranked list of public schools -- both the Washington area version in the Post and the national version in Newsweek each June -- gets lots of attention, but the outrage and acclaim usually swirls around the issue of whether ranking schools is good for you. With much support from Advanced Placement and International Baccalaureate teachers around the country, I think it is. But how can you use it?

I invented the list to show that some schools in good neighborhoods don’t deserve their great reputations, and some schools in poor neighborhoods don’t deserve their terrible ones. Opening up AP and IB courses to everyone who wants to work hard -- the philosophy of the teachers who inspired me to do this -- is a relatively new idea. Ten years ago, most schools in the United States did not let students take these courses unless they had strong grade point averages or teachers’ recommendations.

Many still have those rules, or at least don’t encourage students to challenge themselves as much as they could. The list helps you find the ones that have shed those old, bad habits.

Let’s say you live in the District, and have heard that the public high schools are bad. Some are. But on the list you will find one school, Columbia Heights Educational Center, with a typical D.C. student body that is 84 percent low-income students, but a very high Challenge Index rating -- third in the region.

If you are a D.C. parent of modest means, if you don’t speak English well but have great ambitions for your child, Columbia Heights (previously known as Bell Multicultural) is a place to check out. It is the only public school in the region that requires all students to take AP English. Its students struggled with the difficult exams at first, but the quality of the teaching is increasing and the school now has more students not only taking the AP exams but passing them.

Last year, for instance, 24.6 percent of Columbia Heights seniors had passing scores on AP tests, higher than suburban high schools such as Long Reach (11.1 percent of seniors passing AP) in Howard County or Key in Carroll County (13.3 per cent of seniors), both of which have much lower percentages of low income students. The average U.S. senior class has about 15 percent of students with passing AP scores.

The standard way to rate schools is by test scores. That turns out to be mostly a measure of average family income. On the index, you can find which schools of similar family income level try harder to prepare students for college. Stone High School in Charles County, where 22 percent of students are low income, has a rating of 0.970 on the index. But Fairfax High in Fairfax County, with 21 percent low income, and Dominion High in Loudoun County, with 18 percent low income, have Challenge Index ratings three times as high.

There is much more to it. To see how schools have done in previous years, and examine other facets of rating the energy of school staffs rather than the size of parental incomes, find the new lists here and my blog at washingtonpost.com/class-struggle.

So here's what I know about Columbia Heights from your stats. The average kid who enrolls there will take between 5-6 AP tests by the time she graduates. About 1 in 4 of the students will pass an AP exam.

Wouldn't it make more sense to offer a lot less exams and work on making sure the students who do take them are better prepared?

Also, it would really be helpful if you would publish not only the percentages of seniors passing an AP exam but the percentage of tests that received passing scores.

I graduated from Richard Montgomery High School, ranked #2 in this list, in 2004. I took 10 AP tests there, which got me a total of 39 college credits. And when I think back to my high school days, one thing stands out both when I remember IB and non-IB students: Their rampant cheating.

The International Baccalaureate program, which got RM its high score, was composed almost entirely of students who were forced into the program by their parents and who did as little work as possible. They would often set up cheating rings and e-mail each other assignments.

The faculty of RM was well aware of all this cheating -- teachers would walk by before class and see students copying homework -- but they did absolutely nothing about it. In the 2002-2003 school year the History department, where cheating was perhaps most common, encouraged sophomores to take AP test for the first time without taking any steps to address academic dishonesty.

It seems entirely possible to me that, before this ranking system got so much publicity, it was a way to find the best high schools. Maybe the schools that scored the best at the beginning really were great at giving all of their students the opportunity to challenge themselves. But this list has a very different effect now. Students who have neither the desire nor the ability to complete college-level coursework as high school juniors, and who are known by faculty as cheaters, are encouraged to sign up for AP English so that the school principal can brag about Challenge Index rankings at graduation ceremonies. And the few students who are actually qualified to take advanced courses are forgotten as instructors dumb down discussions and teach to the tests.

It is no surprise to me that Churchill HS, in the top 10, has a rampant cheating scandal on its hands and a student body that supports the cheaters! The same students who are telling reporters that the cheaters should not be punished because an unfair amount of pressure is placed on students to succeed academically, are the students who are making faculty members happy by getting their school a #8 ranking in the Challenge Index.

In today's academic climate, the Challenge Index should not be published alone. Either the schools should be ranked by their average AP scores as well, or some sort of Cheating Index should be devised to measure academic integrity, and should be published alongside it. Schools should not just be rewarded for "giving students the opportunity" to take AP tests (i.e., pressuring them to), but also for cutting down on the academic dishonesty that they see every single day.

Your attempt to take socioeconomic status into account through objective measures is an admirable goal. As I looked at these statistics, however, I already suspect that the % Lunch Subsidy does not fully take this into account.

The wonderful results of Columbia Heights are astounding when you look at other high schools with a similar % Lunch Subsidy. A high school on the list of catching up schools with the same % Lunch Subsidy is Ballou high school. Anyone who knows the two neighborhoods, however, knows that the Columbia Heights neighborhood is undergoing gentrification, with lots of money and attention being pumped into the neighborhood. This is not happening around Ballou High School.

The fact that Ballou has a higher % Lunch Subsidy than the other schools ranking below it on the list shows the advances that are occurring at Ballou. Their next challenge is to get someone to pass the AP exam. The staff at Ballou is working on this challenge and hopes to achieve it soon, but the challenges due to socioeconomic factors are significant. The temptation to punish the staff for these scores needs to be avoided, but it seems likely that there might be room for improvement in some way at Ballou. Forward progress is being made, however.

Underestimating the challenges in making forward movement in areas with socioeconomic disadvantages can easily destroy advances that are being made. It is easy to see the socioeconomic factors creating chicken and egg problems that are especially difficult to deal with, and make simplistic analyses very dangerous.

Great to get some fresh ideas on this. For ranasandra, cheating on classwork is sadly rife in US high schools, but you can't really cheat without getting caught on an AP or IB exam. They are proctored, the scoring is done by outside experts who have seen every trick in the book, and since so much of the exams are essay questions, it is very hard to game them. Your view that we should restrict AP or IB to carefully screened students is widely held, but all the research--and the experience of just about every successful AP or IB teacher I have ever interviewed---says you are wrong. Read this column and let me know what you think:
http://www.washingtonpost.com/wp-dyn/content/article/2009/01/09/AR2009010901085.html

For peterbb, thanks for the kind words. Indeed the neighborhood around Columbia Heights Educational Center is being gentrified, and the neighborhoods near many schools full of impoverished students have been, but the research shows that that generally has no effect on the school if the school is still full of poor kids. 84 percent low income is a huge drag and no fancy row houses across the street are going to change that. What works instead is great leadership, like Columbia Heights long term principal Maria Tukeva, and great teachers. Ballou on the other hand has had a succession of struggling principals, and never any clear vision of what its teachers should do.

for tslats and someguy100, your analysis works in theory, but I have been hearing those arguments for 12 years and have yet to find a school where either of your scenarios could come close to occuring. Someguy100s percentages of kids taking 6 to 8 exams is WAY too high. Cut them at least in have, and in the real world once you open the door to AP participation spreads to much more than 30 percent of the students. Remember, in Moco at least twice that percentage of kids go to college, and that makes AP very attractive to them. For tslats, NO high school would tolerate the kind of stunt you suggest. The teachers and the parents would be up in arms, and the principal would be quickly fired. There are plenty of low income schools where lots of kids get 1s and 2s, but they get them after a full year of study and teaching, and that has proven to be better than the normal, or subnormal, courses they would have gotten otherwise.

for lxlopez27--no such private school list exists because the best known ones refuse to release AP data, but I am thinking trying to compile such a list anywhere. There are some privates here who have given me their very impressive numbers in the past, like the Washington International School that has had a rating of 6.000 and St. Anselm's Abbey, about the same.

for Patrickmattimore: we do give the passing rates on the catching up list, because that is the reason for that list, to show schools that have very low passing rates. The passing rate for regular school can be very deceptive--yr passing rate looks better, as you say, if you restrict access to AP. But you are ignoring the fact that a 2, a non-passing score, leads to better performance in college. And many AP teachers will say some of the kids who struggled and got 1s also got something out of it better than an unchallenging regular course would have done. And I think one more number added to the three we already provide would make the list too confusing.

Jay, when will you stop with the charade of Maria Tukeva as a great leader. No school can claim to have great leadership when it recruits some of the best teachers in the city and has over half of them leave every year. This year will be even more than that. That school is lacking in leadership. It is a testament to the teaching and the students at CHEC that progress in AP and DC-CAS scores continues despite the chaos, dysfunction, fear and intimidation that exists in that building. Imagine how much greater they could be if teachers and students had longer to work with each other. But most teachers have never worked in a more unprofessional environment than that toxic place.

Having taught in an excellent school I find it difficult to read about the flaws of the system,however,I do think there is a measurement which should be done by each and every educational institution.
They should evaluate the students from their schools at the higher level.
Do elementary students do well in junior high school?Do the junior high schools produce good high school students? Do the high schools produce successful undergraduates? Do the colleges produce successful graduate or professional students? Finally, all the schools should get a report card regarding the employment of their students,their marital status,and their income levels after 10 years.
In short,more follow up needs to be done.
This may sound difficult,but since each child is now given a social security number,most of this can be handled on a computer.

Why are so many assuming that the only choices should be an AP class or a low-level, worthless class? Why are we not talking about honors-level as a worthwhile challenge, and regular-level as a solid class with an amount of homework keyed to the average student? Solving the deficiencies of honors-level and regular-level classes by shoving all students ito AP makes no sense.

Jay - could you provide specific AP information on Columbia Heights? I have heard in years past that its high number of AP tests and passes is mainly a result of making all the kids take AP Spanish (just as you say they all must take AP English). Of course most of them are native Spanish speakers and the test is meant for native speakers of English, so it wouldn’t be surprising if most of them passed – in fact it would be disturbing if they didn’t. While I’m glad they are getting good academic instruction in their native language, it’s somewhat disingenuous to count it as a great achievement, when their skills are already much higher than what the test is geared to.

The DC Public schools AP site provides aggregate data for Hispanics. I suppose most of them are from CHEC, but don’t really know.
http://www.collegeboard.com/prod_downloads/student/testing/ap/sumrpts/2009/xls/DC_Summary.xls

Among Hispanic public school kids taking AP Spanish, the site lists 12 fives, 21 fours and 17 threes, and 13 each of twos and ones. However, for Spanish Lit, which they are less likely to be learning at home, all 12 public schools Hispanics who took it got ones.

As for the wonderful administration, again, I’ve heard a very different story. For more information, please check this blog http://filthyteaching.blogspot.com/
It’s written by a young teacher who just quit at the end of the first semester.

For specific info on AP classes, please see this page on the blog http://filthyteaching.blogspot.com/search?updated-max=2009-11-05T20%3A55%3A00-05%3A00&max-results=7

I appreciate your response, but I think that both points you made characterized my comments incorrectly. Nowhere did I say that students cheated on the AP or IB exams themselves. (I am well aware of the fact that these tests are proctored!) Students cheat in their classes, and then get a 1 or a 2 on the tests. They don't cheat to raise their AP grades, they cheat to raise the grades on their high school transcripts. The high pressure students are under to take AP classes only increases cheating among the unprepared and the lazy.

The study you cited in the linked article is not just unconvincing because it was funded by the College Board, but also because they apparently couldn't find any reputable journal to publish it in. (As I said, I took plenty of AP tests myself. So I am not saying this as a "critic of AP" but rather as someone who understands the value of the peer review process. If the same study had been published in Current Issues in Education my opinion would be very different, with or without College Board funding.)

Also, I don't recall saying "that we should restrict AP or IB to carefully screened students." What I said is that it is irresponsible for high schools to actively encourage students who they know are cheaters to take tougher course loads. The fact is, I don't think that we should carefully screen students, or screen them at all, before allowing them to sign up for AP classes. But I do think that schools should take disciplinary action against students who are caught cheating, which they currently do not do. The proportion of AP students who cheat is so incredibly high that this matter really should be taken into account before praising schools for high AP test-taking rates. The two issues are certainly not disconnected. Schools worry so much about getting students into AP classes in the name of high school rankings that they have turned a blind eye to academic dishonesty.

The downside not discussed is that from 1989 to 2009, the percentage of AP exams earning a 1 (lowest score) has doubled, from 10% to 20%. Over that same period, the mean score declined from just over 3 to 2.89. This tells me a lot of kids are being pushed into a AP classes for which they aren't ready or motivated.

Euphemistically, I suppose one could say they have been "over-challenged" but are nonetheless wiser for the experience. Right. They'd be better off in a bona fide high school class that provides rigor they can handle AND that prepares them for college. Schools that push the unprepared into college-level material do them no favors.

How many schools give students an opportunity beforehand to see an AP course syllabus and hear a discussion of the pace and the demands of the course? That would be eye-opening for many. They are young adults and their choices/apprehensions should be respected.

But even the score of 3 is overrated as a benchmark of success. Many colleges give only advanced standing rather than actual credit for an AP course with a middling score.

When HS stops being HS and becomes the college next door, there's a population not being served. They're the unfortunate students who want to attend college but aren't ready yet because they're still in high school. They score 1's and wonder why they were mis-advised.

semitone and jane100000 are right to wish for honors or regular courses that set a good standard that moves students to the next level. But they are going to be disappointed. Without the external check of something like an AP or IB exam, they are prone to the usually dumbing down tendencies of regular high schools---kids complaining about low grades and too much work, parents complaining to the principal, the principal asking teachers who are trying to set a high standard to please dial it back a bit. Spend some time talking to AP and IB teachers who have succeeded in raising average kids to a new level. They will tell you how much that external standard protects them from the natural forces of course dilution, and how much more even that kid at the grade 5 level can learn in a well taught AP or IB course. You have to get into the classes and talk to teachers who are making them work before you can reach valid conclusions on these issues.

JM -- my point is valid because the data are. A lot more people than ever before are flunking AP tests pretty badly. You must believe that only by virtue of being an AP course that a solid F (a 1 score) is a good thing, even better, say, than a C or a B in an honors or regular course. I don't buy it, feel good discussions with teachers notwithstanding.

But there's another lesson being taught here. Many teachers will award at least a C despite a poor AP test result. Are you prepared to say that grade inflation of this magnitude is good for the student too? Because they'll be in for a shock in college, when an F on a major exam earns an F or a D for the course.

I see a downrange effect to pushing increasing numbers of students to take AP. High schools eventually will have two tracks: the college bound track, with standards set by a private organization; and a college/other track, with especially dumbed-down curricula since everyone with even the slightest ability will have been steered to AP. This is a poor model because it will fail to serve those with greatest need (LD, Asperger's, and students who belatedly started getting serious about school). Teachers are fooling themselves if they think that teaching over the heads of the un-ready/special needs population is good practice.

Hello Jay. I went to Eleanor Roosevelt High School, and can attest to the fact that it is the science magnet school for the northern part of PG County. Just like TJ, there are admissions tests and GPA requirements for admission. It has students that aren't part of the magnet program, but the 250 or so seniors there for the magnet program definitely skew the numbers. It's no accident that they rate way above any other PG county schools in your index. Why do you exclude TJ but include Eleanor Roosevelt and its southern PG sibling, Oxon Hill?

"They will tell you how much that external standard protects them from the natural forces of course dilution, and how much more even that kid at the grade 5 level can learn in a well taught AP or IB course. You have to get into the classes and talk to teachers who are making them work before you can reach valid conclusions on these issues." JM

Here we go with the Jaime Escalante/Magic Bullet school of proselytizing with a new wrinkle. You can only know that it can be done (that is, essentially raise the reading levels of kids 8 grade levels in 1 year- from 5th grade to college-)if you speak with the "legion" of superstars who are making this happen on a daily basis as Jay has. And if you happen to speak with the majority of AP teachers who disagree and think AP access has gone too far (Fordham Survey 2009) http://www.nytimes.com/2009/04/29/education/29class.html
well then we all know the problem: "bad teaching. Bad teaching is nearly always the problem." JM
Jay's response to me 2 years ago when we debated some of these very issues.

There are a couple of things to consider here. First, the challenge index is not a measure of AP achievement. It is a measure of a school or system's commitment to opportunity. That opportunity exists in the exposure and immersion into advanced placement thinking which puts a premium on evaluating ideas, systemically analyzing root causes and application of knowledge to solve problems. Unfortunately, advanced placement and the associated test has been converted into a competitive exercise to continue to sort, select and rank students. Under such an atmosphere, it is extremely difficult to measure student achievement through attainment of knowledge. In reality, you measure how much prior knowledge is present. Using a more constructivist environment that does not require one student to compete against another for knowledge and achievement fosters a classroom where cheating is not necessary. Cheating is the natural outgrowth of competition. It is also not valid to suggest that the competitive environment prepares students for college or the real world. Cooperation is necessary in the real world.

Look at the challenge index as a commitment to providing an opportunity for the majority of students. That is also what drives systems to pay for the exams for students. If we erect the artificial barrier of who can and cannot afford the test cost, we limit the opportunities to be exposed to the highest level of rigor. Students may not be successful in their first AP test but they can learn from the experience, regardless of the score.

Scores are only meaningful in competition, better left to the playing fields.

So, Jay, you're saying that only with external standards and evaluation can schools and teachers (and parents and students) be held to reasonable performance levels. Then the answer is not to shoehorn all (or 50%) of students into AP classes; it's to add those features to non-AP classes.

Ranking schools is an improper practice because of the poor nature of the measures and the use of ordinal scales.

That said, I'm pleased to see you pay attention to the segregative effects of tracking. This is probably the most hidden yet pervasive forms of segregation. Solid research on this issue can be found at

patrickmattimore is right, although bad teaching can become good with the same teachers if they get the right training and support, just like bad students. Jane100000 is also right. But I fear it will be very difficult to create the kind of system she suggests. Worth trying though. Maybe we can make the existing state tests better.
Great comment by justlistening. I wonder if the new AP courses and tests will take us in the direction you suggest.

I love the challenge index despite all it's warts. Any test has many of the same warts which is AYP is going to go by the wayside ust like the functional exams did in Maryland. It is merely one indicator to consider when one looks at a school and one indicator of how many upper level classes are available. Before I buy a car, I'm gonna read more than one review.
But, on the other hand, I would like to see some sort of measure of how well a high school is doing with it's less challenable students--its average and below average students. Are they passing the HSA test, are they grduating, are they college bound? I think that tells much about a school, too.

Wrong again, It is obvious that you don't know what you're talking about when it comes to Rhee utilizing the 3 rules process to making better schools. The first rule: hired strong leadership (principals). The strongest one we had, she got rid of her. The men are weak in their disciplinaruy techniques, and had no experience in dealing with inner city black youths. I don't even need to go on to step 2 and 3.

I have had this argument with people so many times over the years that I tire of the exposition so here are my points:

1) While you can do the arithmetic needed to calculate this index it is no more a valid indicator of student achievement than is the average SAT score. Interestingly enough the College Board seems to be making more money than ever since these two numbers started to be reported in the press.

2) Your index is flawed at a fundamental level. Your list only has merit if it includes tests that are passed (3 or higher) BUT

3)Many colleges no longer take a 3, and if the AP test is in your major field of study it is often meaningless as the school wants you to take their introductory course with their professors.

4) Your index (and its publication) have watered down the AP program laughably since the emphasis has shifted to participation rather than success

5) To get their number up school systems are spending millions of dollars on these tests which could clearly be put to better use in this economy

6) I have personally seen documentation of a high school principal who reverse engineered your formula to figure out the number of AP tests that needed to be taken to obtain the Challenge Index number that he wanted. There was then a school wide initiative to recruit new kids into AP classes regardless of their prior knowledge, desire, or abilities.

In short, in my opinion, the so-called Challenge Index has done far more harm to education than good.

You continue to make the mistake of vastly overrating the value of offering many AP courses at the expense of many other equally valuable indicators of how well a school serves the specific needs and abilities of its students. AP does not automatically indicate the best possible placement, even for the most motivated student. Similarly, that motivated student may learn more in a non-AP class appropriately geared to his level of preparation and skill.

If you can't evaluate how well the students do, then your challenge index is worthless. Offering AP classes to students who then fail the test will still earn a high rank on your Challenge Index, but that school may have failed those kids.

A parent who takes your advice and chooses a school based heavily on your Challenge Index may well be sorely disappointed when her child can't keep up.

Look at ALL the factors rather than an artificial index that overemphasizes something that is often not the best for everyone.

Bowen, Chingus, and McPherson, in the largest study to date on successful college completion "Crossing the Finish Line", link student background, achievement and HS factors to completion. HS grades are found to carry the overwhelming part of the explanation, across many sub-populations; and the authors can offer no better explanations for this than what every teacher and thoughtful lay person would say. Demonstrated persistence and adoption of the norms and behaviors of school work, including study habits and work completion will create a template for and result in best college outcomes: timely graduation with above average grades in college.

For grades to offer so much explanation and SAT-ACT scores so little implies that teacher ARE already doing a good job of assessing student HS work and providing encouragement,and of preparing students for further schooling.
Nothing Jay has learned and written for several years on his Challenge (and now Challenged) Indexes and no research literature persuades me that AP courses have substantially improved HS education more than they have enriched the College Board (and supported the career of Jay Mathews). Columbia Heights HS presents among the stronger cases for pernicious effects of promotion of Jay's Index as a valid and reliable indicator of school quality. Why? Because it explains too much, leaving Jay to ignore the rest of the program at that school; and in this series of posts to denigrate schools about which he, the education reporter, knows NOTHING except for the small fractions of students taking AP tests.

Bowen, Chingus, and McPherson, in the largest study to date on successful college completion, "Crossing the Finish Line", link it to student
background, prior achievement and HS factors. HS grades are found to carry the overwhelming part of the explanation, across many sub-populations; and the authors can offer no better explanations for this than what every teacher and thoughtful lay person would say. Demonstrated persistence and adoption of the norms and behaviors of school work, including study habits and work completion will create a template for and result in best college outcomes: timely graduation with above average grades in college.

For grades to offer so much explanation and SAT-ACT scores so little implies that teacher ARE already doing a good job of assessing student HS work, of providing appropriate encouragement,and of preparing students for further schooling. Contrary to Jay's belief, AP courses don't provide much by way of differentiating the successful from the unsuccessful students. My own speculation, not addressed by Bowen et al is that failure in AP courses and tests by under-prepared students may cause an increasing fraction of the "undermatching" (students going to a less selective college than they might) in the study. That's a serious matter, because undermatching is associated with lower successful completion.

Little Jay has learned and written for several years on his Challenge (and now Challenged) Indexes and no research literature persuades me that AP courses have substantially improved HS education more than they have enriched the College Board (and supported the career of Jay Mathews). Columbia Heights HS presents among the stronger cases for pernicious effects of promotion of Jay's Index as a valid and reliable indicator of school quality. Why? Because it explains too much, leaving Jay to ignore the rest of the program at that school; and in this series of posts to denigrate schools about which he, the education reporter, knows NOTHING except for the small fractions of students taking AP tests.

Incredulous,
In your otherwise insightful comment, I would take issue with the following:
"For grades to offer so much explanation and SAT-ACT scores so little implies that teacher ARE already doing a good job of assessing student HS work, of providing appropriate encouragement,and of preparing students for further schooling."
Bowen et al may indeed have found HSGPA to be a much better predictor of college grad rates than SATs and ACTs but that in no way leads to the second part of your sentence.
In fact the abysmal college grad rates suggest just the opposite.
In addition, Bowen et al do not pooh-pooh all standardized testing. They in fact praise AP tests and SAT Subject tests which have nothing to do with HSGPA.