Challenging the Challenge Index

Jay Mathews is the dean of American education journalists, a great reporter and a committed advocate for making schools better. He's also a great guy. And you know already that I'm gearing up to rip him.

But really, it's not Jay I propose to rip, but his annual Challenge Index, which the Post plastered all over Page One yesterday (am I just jealous?) To rank area high schools, Jay takes the number of Advanced Placement and International Baccalaureate examinations taken by students at each school and draws conclusions from those numbers about how serious that school is about adhering to rigorous standards and giving all students a chance to shine.

This year, Jay's story focused on how schools in the Washington have leaped well past any other metro area in the nation in the number of AP and IB exams that kids are taking. That number has jumped by a staggering 69 percent since 2002. An executive at the College Board, which administers the AP exam, told Jay: "I don't know of any area of the country in which the concentration of AP exam-taking is so high, or where the concentration of districts paying for the AP on behalf of their students is so great."

The difference between the D.C. area and the rest of the country could barely be overstated: Only 5 percent of schools nationwide manage to hit Jay's benchmark of giving as many college-level tests as they have graduating seniors. But in our region, a whopping 70 percent of schools hit that mark.

Jay theorizes that this is a result of the decision by many Washington area schools to pay the testing fees involved in the APs and IBs.

I beg to differ: I think the reason for the discrepancy between the D.C. area and the rest of the country is Jay Mathews' Challenge Index. And while I think those college-level courses can be a very good thing to add to the curriculum of the weakest and least demanding schools, I think it's a big mistake to use those tests as a measure of good or excellent high schools. In fact, as many school principals and teachers argue, using APs as a ranking tool puts enormous pressure on schools to add more and more AP courses, squeezing out creative classes, limiting what the best teachers can do in their classes, and turning high school into a dull grind for too many of the better students.

I am far from alone in this interpretation of the Index and its impact on local schools. I've heard one principal after another bemoan the effect the Index has on their own curriculum and the atmosphere of their schools. So why don't those administrators just go their own way? Why do they cave to pressure from a newspaper story? Good question, but given the enormous and corrosive impact that the US News college rankings have long had on higher education, why would we think that status-conscious high schools, always eager to please parents and politicians, would be immune to such pressure?

Principals tell of parents who see a middling ranking in the Challenge Index and insist on more AP courses, even when that means eliminating the really cool Shakespeare elective or the uniquely rigorous European History seminar that made that particular school someplace special. The Index pushes a one-size-fits-all mentality on schools that dare to go their own way--and in doing so, win the hearts and spirits of students. And isn't that what schools should be most concerned about, not producing more numbers on more standardized tests?

Needless to say, Jay does a splendid job of defending his work against critiques such as this. And I've invited him to respond here on the big blog later today; he has graciously accepted.

But until I see more nuance in the Challenge Index--more of a recognition that what's good for low-performing schools is not necessarily good at all for schools that were already rigorous--I can't help but think that what appears to be a sign of healthy and improving schools is actually an indication of the good old American propensity to keep up with the Joneses, even when it means diminishing the quality of your own product.

Comments

We need an index that can incorporate the creative as well as the AP-type stuff. Unfortunately corporate education fights anytime anyone mentions trying to determine their actual results. And who says that AP-type courses can't be "creative", that they don't "win the hearts and spirits of students." Maybe they just don't win the hearts and spirits of the dullard and lazy?

Posted by: Stick | December 8, 2006 8:47 AM

As a graduate of a local high school who took plenty of AP classes, I agree that they should not be used so heavily to determine a school's standing. They didn't prepare me for college any better than the other classes I took - my experience was that my college classes were VERY different from any of those I took in high school. Nor did I receive any college credits because my school would not accept them. AP is just one more example of how students are taught to take tests these days and not necessarily how to analyze and think critically.

Posted by: rmd | December 8, 2006 9:42 AM

Thank you! It has become so frustrating to see every aspect of education homogenized. Do parents really want every child to learn the exact same thing, so they can take the exact same test, so they can ultimately march robot-like into college? Is there no place for those students that learn at a different pace, or have different interests and strengths? Give me a break...

Posted by: Jan | December 8, 2006 9:45 AM

Rmd's got it right. I remember my AP classes being neither more difficult than regular honors classes nor particularly good at preparing me for college. What I do remember is "stressed out" grind students who were obsessed with grades and ranking among their peers. The funny part is that now, 15 years out from HS, it is completely clear that those that obsessed over grades and ranking are no better off than those who took it easy. Ha! Revenge of the slackers...

Posted by: Double 0 Zero | December 8, 2006 9:56 AM

There is concern among college faculty that the AP courses and exams aren't really as challenging as college courses--as rmd points out based on personal experience. Nevertheless, Jay Mathews has repeatedly produced statistical evidence that strongly suggests that students benefit from the more rigorous AP and IB curricula, even if they don't take or don't pass the exams. This, to me, raises a question: If these curricula are so good--and I'm in no position to say that they aren't--why aren't states and school districts making their curricula more like the AP and IB courses? Shouldn't all students benefit from a challenging program in high school?

The answer may be that the very titles--ADVANCED Placement and INTERNATIONAL Baccalaureate--have a cachet about them that status-conscious Washington area parents and the students they raise simply cannot resist. People really do want all of their children, like those of Lake Woebegon--to be above average.

Posted by: prof | December 8, 2006 10:11 AM

My DOD contracting firm at one time just made recruiting trips to the top 50 colleges and universities based on US News and World Report rankings. I left those details up to my HR manager. As a result we ended up with a bunch of morons. Top students do not equate to top employees. HR manager gone and we recruit recent college grads locally and a few other select schools to add diversity. this has resulted in a better quality employee and one can contribute instead of a management headache. The only exception is we no lober recruit at the U of MD because of the poor quality of their graduates. U of MD just doesnt prepare their students in a variety of majors for the real world. GMU, JHU, Howard Coppin State, Towson, W&M, JMU, Radford, VA Tech, VCU, Norfolk State, VA State, GU, GWU have all provided us with quality recuits who are well prpepared for the real world.

I took AP courses because I had to to get into college. Did I learn anything worthwhile in these courses no except that I had to certain things to get somewhere. I would have better off taking English 101 my senior year at GMU.

Also some kids just arent college material. Nothing wrong with being a auto tech, carpenter or plumber. A aut tech after two years in school makes almost $50k at Lexus or BMW dealership with no loans since their education is covered.

Posted by: vaherder | December 8, 2006 10:36 AM

Oh the wisdom of rmd. My daughter who is a top student does not seem very interested in AP either. She just sees it as more work, but no additional learning.
Frankly, when I went to college almost 40 year ago, I was skeptical of AP also. The Calculus exam did not impress me then. The college I went to did not take the AP calculus courses either.
Though I encourage taking Calculus in H.S. from a well qualified effective teacher by those who may be ready, I don't think, on the whole, that such courses are comparable to college. Frankly, taking such a course without the pressure of AP may produce a more beneficial learning experience.

Posted by: Peter Roach | December 8, 2006 10:43 AM

"Jay theorizes that this is a result of the decision by many Washington area schools to pay the testing fees involved in the APs and IBs."

Do most places still pay? As early as 1993, Anne Arundel had quit paying for AP tests. I remember mainly because I had to pay for all of mine!

As for whether they truly mean anything...was I prepared for college work? Yes, but mainly because of how my teachers taught, not because of the AP tests themselves. My teachers demanded a lot of us and taught us how to take notes from a class or textbook, run a lab, write up a lab, and how to prepare for class discussions. The AP material provided the topics to learn about, but ultimately it was my teachers who helped me be ready for college.

The greatest value to me of the AP classes was getting out of a lot of my core requirements in college and entered as a sophmore, so I was freer to take more electives and get a double major. With college getting more expensive, I would imagine some students take AP so they can graduate sooner.

Posted by: AG | December 8, 2006 10:48 AM

vaherder writes "Top students do not equate to top employees." In some cases, top high school students don't make top college students either. A few years ago, during registration at the college where I teach, I was complaining to one of our counselors about advising students who had good test scores but couldn't seem to follow the steps necessary to register. "You're talking about the GT, AP, and IB students," she said. "They already know everything and they're pretty much ineducable until after they flunk a couple of courses."
The experiences my kids had in AP courses were much like those that others have described here: more work instead of more challenge.

Posted by: prof | December 8, 2006 11:35 AM

I think the reason so many schools in this area haved upped their programs is that, for all intents and purposes, this is a one newspaper town & what the Post prints has an enormous effect. My niece (a smart girl and a good student)took an AP US History course at HB Woodlawn in Arlington last year. Yet, when I asked her a few simple questions - How many Representatives in Congress? How many Supreme Court Justices? How many Senators? - she had no idea. It seems to me that just having the courses available doesn't mean that the kids are actually learning, or more importantly, retaining what they've been taught.

Posted by: lucy | December 8, 2006 11:42 AM

Oh, I just loathe the "Outstanding Scholar" hiring practices that the federal government uses. They work for a few months in our office (administrative), then quit because "the work is too hard". The program might work in the scientific discipline, but not in administrative.

Posted by: MoCo | December 8, 2006 11:58 AM

I graduated from a local, college prep, private high school in 1998.

APs are a joke. As a sophomore, I got a 4 on the AP European History exam, even though I didn't take the AP class, and as a senior, I got a 3 on the AP Latin exam, even though I made stuff up in the translation sections and used tricks unique to multiple choice questions to answer that section. At the end of that exam, I thought I'd be lucky if I earned a 1, and somehow that sloppy work garnered a 3, a passing grade to some colleges.

One of the best offered classes was honors molecular biology taught by a part-timer who also worked at NIH. She taught by using case studies and current medical journal articles, much like my college biology class. That honors class was not AP, and I learned more in it than in my World Lit. and American History AP classes, though I got 5s on those exams.

I do have to say that the AP Art History class was an ambitious comprehensive introductory survey, and I felt so lucky for the opportunity to take it. However, I would have appreciated taking it in tandem with an art history class that focused more in depth on one movement.

I think the biggest problem isn't AP vs. no AP, it's discussion vs. lecture. None of my high school classes were discussion based, they were all lecture. Every class I took in college was a discussion.

And knowing what I know now, I would have killed for a Shakespeare or philosophy class in high school. I also would have appreciated a New Historical approach, studying art, literature, politics, philosophy all in historical context. That's why my high school freshman sister is going to a public school instead of a college-prep private school; she has so many more options.

Posted by: Anonymous | December 8, 2006 11:58 AM

I'm always really surprised in this country that debates over the merits of AP courses never look at the class divide it creates at the college level. I grew up in, and went to school in Canada in the mid-90s. The elite private schools in Toronto were at that time just starting to use AP courses (I had never heard of them at the time, despite going to a very good public HS). When I got to University, I was surprised and appalled that all the rich Toronto private school kids had garnered enough credits in advance of their Freshman year that they were able to pretty much skip most of the dumb, 300 person 101 courses that many of us loathed in that first year. I remember distinctly that one of my best friends had almost a full year of AP credits when she arrived. It meant she got to take more interesting courses at University, and freed up a lot of her time each semester to do non-class activities. She ended up in a tremendous internship, where she was able to make excellent contacts, learned a huge amount, and in the process figured out exactly what she wanted to do with her life (now doing a niche phD at Harvard in a field that she loves). Despite being in the same school as she, I didn't feel like I had the same opportunities to explore my interests at such an advance level.

In this country, AP courses are readily offered at the best high schools, while many poor areas don't offer them at all. There is already a terrible gap between these socio-economic groups of quality of education and readiness for college, so why exasperate the gap by letting the rich kids skip so many of their courses?

Posted by: Washington, DC | December 8, 2006 12:26 PM

If you are really concerned about high school education I wish you would set aside this contrived argument with Jay Matthews and beat a drum in support of Shawn Hearn. Does it really matter how many AP/IB courses a school offers when a thug can attack the principal and the principal gets fired?

Posted by: Paul | December 8, 2006 3:26 PM

Some days, Marc Fisher makes sense. Other days (like yesterday), he doesn't.

Today, he makes sense.

Posted by: countyofmo | December 8, 2006 4:17 PM

I totally agree with what Marc Fisher says. The issue isn't how many people take advanced placement courses but how all school courses challenge children. What Jay's index leaves out is how well students do in these courses and how many of the courses have been dumbed down to accomodate all the children taking them who are really not prepared to take an advanced placement course and really benefit from it.

Many schools like the Fieldston School in New York have dropped advanced placement in favor of their own rigorous courses. Jay has really bought into the whole PR of the College Board which has promoted Advanced Placement for all students as they make alot of money on each course. It would be interesting to find out whether some of that money could be spent on developing more appropriate and challenging curriculum for some of the students who are now in advanced placement coursed but not ready or don't take full advantage of them.

Posted by: peter dc | December 8, 2006 4:26 PM

As a high school student (too many years ago), I took every AP and honors class offered for one primary reason: It got me out of the classes that were filled with students who didn't want to be there and didn't want to learn. Some of the AP classes were better than others. The skills and passion of the teacher were a more important yardstick than the AP tag. However, the worst, least inspiring teachers that I had were in the middle of the road sort of classes that were filled with nice, but underachieving sorts of students.

My children are too young to be in AP classes. However, some of my friends have children who are entering middle and high school. They find that the AP/IB classes and the prepatory classes for students on that track are moderately challenging. The regular classes are not, as they mostly focus on getting the most students to pass the state standardized tests. It's the cult of mediocrity at its finest.