Through the lens of social science, eduwonkette takes a serious, if sometimes irreverent, look at some of the most contentious education policy debates. (Find eduwonkette's complete archives prior to Jan. 6, 2008 here.)

The NYC High School Progress Reports Meet Credit Recovery

Yesterday, the NYC Department of Education released its high school progress reports - 83% of high schools received A or B grades. Like the K-8 reports, 60% of the grade is based on "student progress," which in the case of high school includes credit accumulation.

We know that many students fall of the wagon, so to speak, early in high school and fail enough courses that it makes it hard for them to catch up. So tracking students' credit accumulation closely - and intervening when students fall behind - makes a lot of sense.

But holding schools accountable for credit accumulation creates a number of perverse incentives, and readers have provided a number of examples of how this is unfolding in their schools. The central issue is how "credit recovery" is being used - and in some cases, abused. For the uninitiated, credit recovery involves "letting those who lack credits make them up by means other than retaking a class or attending traditional summer school." (See this NYT article on credit recovery.)

Teachers have complained that they've been pressured to change grades (more than they have in the past) because of these credit accumulation measures. Other teachers have reported that students who fail a course in the first term are allowed to sign "contracts" that promise that their grade will be changed to a passing grade if they attend tutoring two hours a week, but tutoring attendance is never monitored. Still other teachers have reported that students who've failed their courses are given simple tasks - i.e. a packet of math problems - that students can complete to get credit.

Readers know well that I generally come down on the side of keeping kids in school (see this exchange about the dropout age, for example). But some of these wild year-to-year jumps in the fraction of students earning 10+ credits do make me wonder what's happening with credit recovery on the ground. For example:

* At the Secondary School for Journalism, 6% of first year students earned 10+ credits last year; this year, 60% did.

* At the Rachel Carson High School for Coastal Studies, 11% of first year students earned 10+ credits last year; this year, 68% of students did.

* At Canarsie High School, 10% of first year students earned 10+ credits last year; this year, 41% did.

* At the Law, Government and Community Service High School, 17% of first year students earned 10+ credits last year; this year, 44% did.

* At the Cobble Hill School of American Studies, 29% of first year students earned 10+ credits last year, this year, 57% of students did.

It is, of course, possible that changes in the student population from year-to-year may explain some of these jumps, or that schools made substantial changes that led to real increases in course passing rates. It could also be the case that students are better off in a world where educators are paying close attention to credit accumulation, even if it does lead to some practices that many educators would frown upon. Alternatively, we could end up making schools look better than they really are, and students find themselves in a lurch when they come face-to-face with kids who actually did master these courses.

At the very least, let's hope that reporters make use of the new data available and try to find out how these increases are being produced. Readers, where do you draw the line on credit recovery? Got insights on how credit recovery is being used in NYC schools or elsewhere? Leave a comment below.

Categories:

10 Comments

Sometimes I wonder how much is perverse incentives and how much is just perverse schools. It seems as if no system is too tight to be manipulated and gamed, although I am not certain what incentives are operating in either direction.

Not certain that it's happening, but in my experience, many course failures could have been prevented by earlier interventions. When my daughter was in high school, the school that failed a student was under no obligation to allow the student to re-take the class. The options included summer school or adult evening classes. They have now moved to a "credit recovery" option--which is a computer based "mastery" program--offered after-school or summer. All of these represent, to my thinking, an out of sight, out of mind approach to getting kids through. Let's just suppose that half of the kids failing 10th grade English are failing because they get overwhelmed by the term paper, which is introduced in the second semester. Most of these kids get weeks behind during the outline and research portions, and by the end of the year, when it is due, and parents are finding out for the first time what their student was supposed to be working on, there are lots of Fs, and no one can do anything about it--too late. This could go on forever--with kids just making up the credit over the summer.

Or, if someone is suddenly paying attention, maybe some changes could be made in the system. Maybe the assignment could be moved to the fall--so that there would be time for make-up if kids fall behind. Maybe the home-school communication could be vastly improved.

So--a school that suddenly admits that it has a problem might be able to show heroic improvement. Or a school that continues to believe that all solutions to problems lie somewhere outside their sphere of influence might come up with some crafty ways to show "improvement" without really doing anything different.

When a kid fails, so does their teacher and their school. WE have a problem. We have got to get past this feeling that teachers have some inalienable right to fail kids. I don't think changing grades only to show improvement is any kind of solution--but neither do I think that is the only possibility available.

I worked at a small high school in NYC for a year. During that time our administration began brainstorming credit recovery options and eventually initiated one. Essentially, this program allowed credit-lapsed students to come to the school for a few hours on Saturday for several weeks (unsure of exact number - 6-8?), during which time they would do research towards completion of a paper and presentation, due at the final meeting. From what I gather, this approach was marginally successful at attracting students who had - presumably after a personal epiphany or a nudge from a teacher or counselor - realized that they wanted to graduate and became motivated to do what was prescribed in order to reach that goal.

Determining its success is a matter of perspective, depending on what you view the objectives of the credit program to be.

Is the objective to provide the equivalent amount of work and opportunities for learning as the student would have received if he’d successfully completed the course during the regular school year?
If so, then – again – it depends on your perspective and on the school – sometimes a student isn’t successful in “regular” school because of disruptive behavior of peers that the teacher may have no means of disciplining; a more intimate setting like Saturday would be better for the student and achieve the desired result.
Yet, is attendance and completion of tasks enough? Sometimes the amount of work required in the regular classroom was so minimal (at the school where I taught) that while the students may, in fact, have been producing the same amount of academic work in an abbreviated format, in both cases it was an amount and quality – when compared to prestigious private schools to competitive public schools – that is embarrassing.

Is the objective to get as many students to graduate as possible?
If so, then just set the bar low enough and it can be met. The city puts pressure on principals to graduate students, who in turn put pressure on teachers. If the teacher “fails” the principal by “failing” the student (among teachers, it is common to hear, “teachers don’t fail students; students fail students”), then a credit recovery option – as ambiguously designed as it is – can achieve the desired result, which explains the spike at some of the schools mentioned in your blog.

These programs can be looked at in a positive way - if you are driven by numbers; however, if you scratch the surface, you realize they are a band-aid approach to one of the many problems of our under-funded, under-esteemed public schools, which ultimately cheat the students out of quality education.

Schooling is very important to us. The educational authorities in New Hampshire are pushing for some high school students to graduate by their 10th grade year. This means that these students will be out into the workforce sooner, and into the real world and paying bills, like rent, utilities, cable – internet, and possibly into the world of payday loans, and also into the world of responsibility. The idea is to administer state board exams to sophomores, and those that pass can move onto community or technical colleges, and foregoing the last two years of high school. Many people are doubtful that it is a good idea. Do you really think that a 16 year old can handle the pressures that many of the rest of us have to cope with? I'm not necessarily talking about the responsibility that comes with the reading and writing required, rather the pressure and responsibility that come with being financially independent and therefore responsible for you. Most people are barely ready at 18. The youth of America don’t need to be pushed out of the nest any earlier, and if they were, they couldn’t even get payday loans to help cover books, tuition, or rent if they fell behind.

Absenteeism is not as sexy a topic as a possible credit recovery scandal, but it is clearly a driving factor in school performance. Recently, The New York Times reported on a study of chronic absenteeism in New York City schools. The Progress Report data released by the DOE show just how significant an effect absenteeism has on student progress. Using the high school data, we can correlate the attendance rates with the measures used in the Progress Reports.

Let's start at the top. The correlation between Attendance Rate and Overall Progress Report Score is .63527 (n=284 schools, p

.56214 with Environment
.54298 with Performance
.60334 with Progress

If there was any doubt, we now know: Attendance drives the Progress Report scores.

Look at all of the great ideas being promoted by the Gates Foundation: national curriculum standards, more effective ways of attracting and evaluating teachers, and other generally wonderful ideas. But none of that will have any effect unless students are in school. A boring fact, obvious perhaps, but true.

Until we get a handle on the effects of attendance, we need to ask anyone who presents data: What were the effects of attendance on this measure? How did absenteeism affect our AYP this year? How did the students often absent in my classroom affect my value-added score or the scores of my fellow teachers? How did the fact that Student X had 12 unexcused absences affect his Regents Math score? You might want to know the answer before you buy special tutoring software, fire a teacher (or reward the others), or demand a school restructure plan.

When you ask about the effects of absenteeism and the data gurus shrug off your question or throw out some mumbo-jumbo about how they have sophisticated statistics that take care of that, you can assume that they don't know...or are afraid to tell.

I can see how "credit recovery" can be beneficial, and I do not question that it has worked well in some (many) cases. In my experience, it has just been a "fig leaf" for fabricating data, and CYA. In my school, students refer to credit recovery as "excercizing their right click finger." They know its mostly a ruse to deliver kids to "out of sight, out of mind." Maybe, I'd phrase it a little differently. I think adults also embrace credit recovery so they won't feel so guilty when they fabricate data.

I would think it would be easy to identify when a credit recovery program - whether it was successful or not - was created sincerely or when it was bogus from day#1. If almost no money is devoted to the program, when no serious planning is attempted, when kids are sleeping or picking up trash and they still graduate, then ...

Margo/mom, combining your comment here and the other recent one about parents, shows where we are similar but different. I don't really blame the administrators who order those programs. They have to make the numbers come out right or the district (not to mention their careers) will be in jeopardy. The enemy is that old "culture of compliance." To improve, we need to assist parents. I would think parents activists would want accurate data, and the truth about programs so they can work for their kids. Remove sanctions, and data will become more reliable. Lower the incentives for bogus credit recovery, and perhaps we can devise solutions, including well-rounded credit recovery efforts.

By the way, in our divided world of NCLB, theorists, administrators, and consultants, tend to blame teachers while pushing for their solutions. But teachers share plenty of the blame. All to often we use credit recovery and similar programs as an excuse. Too many teachers ask, "why should I keep knocking myself out trying to raise standards when 'they' will just find a way to pass kids on?" Its true that we will always have destructive institutional pressures, but that is no excuse for giving up. If nothing else, the students still need to see us fighting to raise standards.

Mr. not-so-sexy, I agree with you like I agree with the 26 year old retiree.

In my experience, the two are linked. In addition to burnouts, older teachers have more health problems and inner city schools can't keep their best young talent. Its not just the stress of dealing with dysfunctional policies, its the dishonesty that poisons the well for young teachers (and students).

An accountability system should measure the success of schools. That measure should be based on factors schools can control. It would be unfair to hold schools accountable for what they cannot control. You cannot manage what you cannot manage.

Two factors alone can account for almost 50% of the differences in NY's Progress Report scores for high schools. A simple regression model using Enrollment and Attendance to predict the Overall Score results in an r-square of .4528 (n=284). The same model results in an r-square of .4984 for the unadjusted Overall Score, so any adjustment for those factors did not make much of a difference.

How important are these factors? The high school recently mentioned by the New York Times as a successful school certainly does have challenges. The High School for World Cultures is 30.4% overenrolled. It is very overcrowded. But it also has only 299 students and has an 88.0% attendance rate. Their attendance rate is almost a half a standard deviation better than the city's 85.6% average. The high school is a relatively small school with relatively strong attendance, statistically very likely to do well in the ratings. In fact, there were only 9 high schools that had 300 or fewer students and an 88% or better attendance rate. Of those, 7 earned Overall A's and 2 earned B's.

What does this mean to a non-statistician? Very large schools with poor attendance aren't good schools. About half of the complicated Progress Report score is just attendance and how big the school is. And the unfortunate truth is that these are factors that teachers, even principals, can do very little to control.

If I were among those teachers and principals who did not get a cash bonus this year, I might want to know more about how those two factors affected my score. If I were the President of the United States of America, I might want ask a few questions about what NCLB and all of these expensive tests and statistical models are telling us that we didn’t already know.

You know more about statistics than I,so if I'm in the large schools my two questions are going to be:

1. did the small schools cream off the best and most motivated? and
2. When they leave a greater critical mass of challenging students, are the administrators going to tie our hands further in dealing with students with disciplinary and attendance problems?

I find it to be particularly frustrating when teachers in particular are held accountable for a student’s lack of motivation. Having teachers be pressured into changing grades so that students can move onward only creates an army of lazy students who come to learn how to exploit the system instead of actually learning enough to move on to the next grade.

I recall in high school various students who fell behind due to sheer laziness, not people with legitimate reasons, being given opportunities to earn their credits through make up work that was not nearly as challenging or useful as the original work was. It also cheapens the experience had by the students who put forth their best effort and forms a certain level of resentment amongst the student body. After all, how could you possibly keep students motivated to do the original work when they see that to get an easier chance at the same credits, all they have to do is slack off for an entire year?

I find it especially appalling that even instead of the basic at best summer school option, students are given just a packet of problems to make up for an entire class that they failed. Summer school in it of itself is generally a joke to students and teachers alike. Attendance is generally enough to get pushed through to the next grade and arrive ill prepared for what will be required setting in motion an endless cycle of being pushed through grade by grade with no student accountability in the process. In recent years terminology has changed and no longer is a student lazy or less intelligent than any other. There is a veritable cornucopia of psychological conditions that prevent a child from participating even on a level close to their peers. Granted there are plenty legitimate cases but often times you see a student diagnosed with something due to only their laziness, and they ride out the extra test time given straight through graduation day.

I find that holding the school itself accountable for credit accumulation is counterproductive for basically everyone involved in the process. It puts unnecessary pressure on the teachers to keep students motivated when in reality

I feel as though that should fall on the parents. I realize that a large percentage of parents are unable to thoroughly help their children with schoolwork because of various constraints but that doesn’t place the sole responsibility on the individual teachers. There should be programs in place to help with things like this.

Overall I feel that accountability needs to fall more on the student and less on the school. Withholding money or other resources doesn’t solve any problem as these threats generally only lead to more failing students and less money. Threatening teachers is only going to cause them to lower their own standards in an attempt to get more students to pass not solving the problem at all. In fact in reality all that threats accomplish is a lowering of standards. Credit recovery should be up to the student to fully make up.