Kindly Log In

Los Angeles Unified, union cut deal on test scores and evaluations

November 30, 2012

November 30, 2012

Facing a court-ordered deadline, Los Angeles Unified and its teachers union have agreed on a framework for evaluating teachers that will include using student scores on local and state standardized tests ­– but only to a limited, as yet undetermined extent.

The tentative agreement announced Friday, responds to a Superior Court ruling in June that found the district had failed to comply with a state law requiring that measures of student academic progress be factored into a teacher’s performance review. Although Los Angeles County Superior Court Judge James Chalfant’s ruling applied only to Los Angeles Unified, the state’s largest district, many, if not most, districts in California also ignore that provision of the Stull Act. These districts should “take notice,” said Bill Lucia, president and CEO of the Sacramento advocacy organization EdVoice, which sued the district and United Teachers Los Angeles a year ago on behalf of a half-dozen unnamed parents.

LAUSD Superintendent John Deasy

Superintendent John Deasy called the tentative agreement “historic” and said in a statement that it “stands as testament that working together, LAUSD and UTLA can resolve difficult professional issues, while providing models for the state and the nation on any number of necessary transformative practices.”

The technique, called Academic Growth Over Time (AGT), had become a flashpoint in UTLA relations with Deasy and a source of controversy nationwide. Deasy had proposed using it as a significant factor in evaluations; UTLA opposes it as unsound. In the agreement, AGT scores of individual teachers cannot be part of a teacher’s evaluation, although school-level AGT data, thought to be more reliable, can be included among many factors.

Two years ago, the Los Angeles Times published its own version of the district’s AGT data for individual teachers based on California Standards Test results provided by the district. Under the terms of the agreement, that data, linked to individual teachers, would become part of personnel files, no longer available to the public and the news media, and the district would defend that position in court.

Observations of teachers’ classroom practices will continue to count the most in an evaluation. The district has been developing a comprehensive and uniform observation process for two years and training principals in using it, although it has not been negotiated with UTLA for districtwide use.

Student progress will be another component. The California Standards Tests, along with “future criterion-referenced state-mandated replacements” (by implication, Common Core math and English language arts tests starting in 2014-15) will be among the multiple measures. The agreement mentions others, from which teachers and their principals can choose: district benchmark tests, various reading tests and curriculum-based exams, and school-level measures such as attendance and suspension rates, Advanced Placement passage rates, English language reclassification rates and class grades. Locally developed assessments will determine students’ progress for teachers in middle and high schools whose students don’t take state standardized tests. A six-person committee, with the district and UTLA choosing three members, will recommend which measures are appropriate in each grade and help resolve evaluation disputes.

The agreement, an addendum to the teachers contract, must still be approved by the school board and UTLA members, who will vote in January. But the union and administration were facing a Dec. 4 deadline set by Judge Chalfant to reach a deal on the use of standardized test scores and other measures of academic progress. Both sides understood that “if we did not respond, we risked having something imposed on us,” said Fletcher.

It will be a “logistical challenge” to include performance measures in this year’s evaluations since teachers’ goals and objectives have already been set, Fletcher said. Under the agreement, a “significant number” of teachers will be able to push their reviews back to another year. Because the new system will require extensive planning and training, teachers with 10 or more years experience will have the authority to push their reviews back an additional one to three years.

Lucia said he’d be watching to see how the agreement is implemented, but at least on paper he is satisfied with what he sees. “Finally, there will be recognition that there needs to be multiple measures of student progress and that it is OK to put a pupil learning component into the evaluation of an adult.”

In a statement, Deasy said, “This agreement strikes a balance that is much needed in the country right now in terms of using student measures of academic progress as both a vehicle to improve instruction, and to hold us accountable for the achievement of students in our schools.”

But the agreement will not end the debate over the use of student test scores, whether raw CST scores or complex valued-added algorithms like AGT. An effort to rewrite the Stull Act earlier this year ended bitterly, with hard feelings between the California Teachers Association and groups such as EdVoice. Whether to require state test scores as a component in evaluations was one point of division.

Comments Policy

Manuel5 years ago5 years ago

I did not opt my kids out of the CSTs because they do fine taking these silly tests. Plus I knew that if I did I would run the risk of getting the same type of scolding that Bea got. I just told my kids not to worry about their performance because it did not matter to their grades. (BTW, Bea, your kid is just one of those who will never do well in the … Read More

I did not opt my kids out of the CSTs because they do fine taking these silly tests. Plus I knew that if I did I would run the risk of getting the same type of scolding that Bea got. I just told my kids not to worry about their performance because it did not matter to their grades. (BTW, Bea, your kid is just one of those who will never do well in the CSTs. This says nothing bad about your kid, but says much about how reliable the CST is in terms of truly measuring academic achievement.)

As far as the “agreement,” well, I don’t know what happens behind closed doors so I cannot tell you if my kids’ teachers and/or administrators discuss the CSTs in as much depth as Navigio has outlined and is now called for in the agreement. What I do know is that I have informed both UTLA and AALA officials that the score distributions show that they can do nothing about them: they will roughly get 50% of the students to be proficient and it won’t matter what Deasy demands and what they do (other than cheating). I have also discussed this with three board members and they all agree that this is very serious. Unfortunately, all of them also agree that this is an issue that must be taken care of at the State Board of Ed/State Superintendent level and seem to not be able to publicly state that “the emperor has no clothes.”

Meanwhile, LAUSD, UTLA, and AALA were required to come up with an agreement to meet the December 4th deadline imposed by Judge Chalfant and came up with their respective MOUs which, when read closely, do not say what exactly will be done about the CSTs and the derived AGT. I suspect, cynically, that this just kicks the can down the road. And, what do you know, the CSTs sunset in less than two years. By the time the dust settles, LAUSD will be trying to figure out what it will do about Common Core assessments and how to fend off the next lawsuit.

We will live in interesting times!

(For the record, I no longer have kids in the k-12 system. But I fear for the future of public education and the economic well-being of the state.)

Bea5 years ago5 years ago

@David, I too kept my broader concerns in check and have not opted my children out*. I'm glad because it gave us a real world look at the results. My youngest, currently a high school senior with a 4.5 GPA, who scored a '5" on the AP Calc test scored barely proficient in math last year and the year before. She didn't intentionally blow it off or make patterns on the answer sheets. But those … Read More

@David, I too kept my broader concerns in check and have not opted my children out*. I’m glad because it gave us a real world look at the results. My youngest, currently a high school senior with a 4.5 GPA, who scored a ‘5″ on the AP Calc test scored barely proficient in math last year and the year before. She didn’t intentionally blow it off or make patterns on the answer sheets. But those results gave her pause enough to wonder if there are problems in the scoring. She’s fairly accurate in her self-assessment as borne out in her SAT and ACT results. In other words, had she felt she had done poorly, she would have expected those scores. It leads us to wonder how her excellent teachers would be graded on these scores given that she mastered the content, aced the AP test and is bound for a four year college to study pre-med.

*I did opt my oldest out her junior year when she had 5 AP tests, the SAT, the ACT and 2 research papers in one 2-week window in May. For that, I was publicly scolded by the principal that my Nat’l Merit Scholarship kid had not “helped her school” by taking the CSTs.

We may need the makers of Race to Nowhere to make a version for teachers subject to these stresses. Let’s hope the “other measures” used in evaluations are selected to accurately reflect actual learning.

David B. Cohen5 years ago5 years ago

Navigio,
I think I understood that you weren't promoting CSTs but were talking about the gaps in rhetoric, policy on paper, policy in practice, appearances vs. realities, etc. Regarding your concern about going through the motions, I think you're right, and in a way, I endorse it. It's a vicious cycle. If states/districts impose more more cumbersome or misguided policies on schools, and try harder to enforce the wrong approaches to the wrong … Read More

Navigio,

I think I understood that you weren’t promoting CSTs but were talking about the gaps in rhetoric, policy on paper, policy in practice, appearances vs. realities, etc. Regarding your concern about going through the motions, I think you’re right, and in a way, I endorse it. It’s a vicious cycle. If states/districts impose more more cumbersome or misguided policies on schools, and try harder to enforce the wrong approaches to the wrong goals, they get a very predictable, superficial result. People working at the school level don’t feel there’s enough reciprocal accountability: teachers and principals are expected to produce, but their input and professional judgment are not respected and their work is inadequately supported. The system is not accountable to the people working in it, and as a result, they feel less accountable to the system.

The short answer about not opting my kids out of testing is that, whatever my reservations about the big picture of testing, I feel that my kids’ schools and teachers have not, to my knowledge, done anything I would consider inappropriate around testing. Very little test prep, no pressure, no pep rallies, no rewards/punishments, etc. To the extent that opting out might have any impact on their schools, I’ve decided not to bring my broad-level concerns into play at this most local and personal level. For now.

navigio5 years ago5 years ago

Hi David, I held off on responding because I wanted to think a bit about your post.
First off, thanks for your response about the opting out. I didnt mean to put you on the spot with a personal question. We have many parents who feel as you do in our school. Interestingly, some who would have opted out were told by their kids that they didnt want to opt out.
I agree with you on the … Read More

Hi David, I held off on responding because I wanted to think a bit about your post.

First off, thanks for your response about the opting out. I didnt mean to put you on the spot with a personal question. We have many parents who feel as you do in our school. Interestingly, some who would have opted out were told by their kids that they didnt want to opt out.

I agree with you on the lack of reciprocal accountability. Ironically, imho realization of that fact often tends to push people toward more local (and by definition, less societal) solutions. I think prop 38 was a direct reflection of such an attitude (though a broad implementation of it). I think charter school policy is to some extent as well. Parcel tax loopholes, etc.

But admittedly your comment about endorsing going through the motions is somewhat concerning to me. IMHO, when we do this, we reinforced a lack of accountability at the district administration level (which I think is probably the place we need it most right now). Although it sometimes sounds like I am arguing two different positions, my concern is largely consistency. I would not mind knowing that we are going through the motions if it were admitted and the community were then allowed to decide whether we should continue to do that. Instead, we are told the process is important, we spend time and money (sometimes lots of both) to try to implement it, but then its either poorly implemented or the results ignored altogether. Same goes for CST results or API scores. Why dont we just admit what they really mean instead of pretending they mean something else and continuing along the path of middle ground vacillation.

The whole notion of accountability is meaningless if the input to the process lacks integrity.

Manuel5 years ago5 years ago

Navigio has put it in very stark terms: "The whole notion of accountability is meaningless if the input to the process lacks integrity."
That's what I have been arguing in a roundabout way when I state that the CSTs will never allow for "growth" because the Bell Curve is immutable.
How did we get to this? IMHO, it is because "the system" allowed the PSAA to be passed and then "implemented" by an appointed State Board of … Read More

Navigio has put it in very stark terms: “The whole notion of accountability is meaningless if the input to the process lacks integrity.”

That’s what I have been arguing in a roundabout way when I state that the CSTs will never allow for “growth” because the Bell Curve is immutable.

How did we get to this? IMHO, it is because “the system” allowed the PSAA to be passed and then “implemented” by an appointed State Board of Education which never questioned the validity of the CSTs as instruments that measure true educational achievement. The SBofE then doubled down by decreeing that all schools must attain a metric that puts them somewhere above 60% of all other schools. How could 100% of schools be above the 60% level? This demand certainly lacks integrity.

Now, because of the push to essentially privatize public education by certain sectors, “the public” is demanding that schools be held “accountable” to a metric that can’t possibly be met given the rules of the game. Never mind what this says about education, it is all about the politics and the potential profits to be made.

Is it a surprise that the biggest school district and its teacher and administrator unions are dancing around the issue under a legal threat even though the testing program is about to sunset? Is it a surprise that many “Rheeformers” have set up camp in Los Angeles and are trying their best to get their candidates to win seats in the next Board election? They are not content in having siphoned off 10% of LAUSD students, they want the whole enchilada.

And so it goes…

David B. Cohen5 years ago5 years ago

Just out of curiosity, Navigio, do you have experience with the tests, or with your own children getting CST results back? I don't mean that in a snarky way, but sincerely would like to know more about where you're coming from on this. In my experience teaching high school English, the CSTs offer nothing useful, and nothing I couldn't figure out myself in terms of a student's general reading skills. There are … Read More

Just out of curiosity, Navigio, do you have experience with the tests, or with your own children getting CST results back? I don’t mean that in a snarky way, but sincerely would like to know more about where you’re coming from on this. In my experience teaching high school English, the CSTs offer nothing useful, and nothing I couldn’t figure out myself in terms of a student’s general reading skills. There are many misleading scores for individual students, and the subtests are useless (not just my opinion, but also a key finding cited by Robert Marzano looking at the work of Jay Cizik, who reviewed many state tests and found subtests worthless as a tool for guiding instruction). When I see my sons’ scores, I laugh at the idea that the tests would be useful. So few questions per standard, and such lousy methods of measurement. If I found out that my sons’ teachers were actually planning individualized/differentiated instruction based on CSTs, I’d opt them out in a heartbeat. (Why I haven’t done so yet is a topic for another night).

navigio5 years ago5 years ago

Hi David, I think people are misunderstanding my comments. My intention was not to promote CSTs rather to point out the inconsistency of the ostensibly 'new' process with that of existing ones.
Although the CSTs obviously do correlate pretty strongly with something, what that is and whether its meaningful to a teacher or policy maker is a valid discussion. I do think that discussion is particularly important for deciding whether the LAUSD/UTLA agreement makes any sense, … Read More

Hi David, I think people are misunderstanding my comments. My intention was not to promote CSTs rather to point out the inconsistency of the ostensibly ‘new’ process with that of existing ones.

Although the CSTs obviously do correlate pretty strongly with something, what that is and whether its meaningful to a teacher or policy maker is a valid discussion. I do think that discussion is particularly important for deciding whether the LAUSD/UTLA agreement makes any sense, so its relevant to this thread. But I wanted to focus on clarifying my comments here instead.

You did say something interesting in that you would decide to opt out (I assume you meant of testing, but perhaps you also meant out of the school, since even using that process for other kids in the same classroom would likely impact the instruction of someone who opted out–actually, I would be interested why you didn’t opt out anyway given your opinions?). So I am interested in whether you would make the same suggestion to all LAUSD parents given that this new agreement requires exactly that.

From the agreement itself:

“a. The Teacher’s CST Results: The results of the teacher’s previously-assigned students on CST (California Standards Tests) from recent years as available, and especially as indicated by those students’ year-to-year CST results and the content strand data from the previous year, are to be reviewed and considered in the formulation of objectives and related strategies to be reflected in the initial planning sheets.
b. Group CST Results: In order to include the context in which individual teacher CST results may occur in any individual situation, CST results are also to be reviewed and considered at the school for the applicable subject matter/grade levels/ departments/school-level as part of the initial planning process.
c. Currently-Assigned Students’ Previous CST Results: The CST results of the teacher’s currently-assigned students in the classrooms of their previous teachers, especially as indicated by those students’ year-to-year CST results and content strand data from the preceding year, shall also be reviewed and considered in the formulation of the teacher’s performance objectives and related strategies to be reflected in the initial planning sheets. Applicable reports indicating such results include the “Elementary/Secondary Roster: My Students, Prior Year Data.”
d. School-Level Results: School-level CST (see paragraph b above) and school-level AGT data/reports (and other school-level data such as API) relating to the performance of all teachers at the school whose assigned students participate in the same CST subject matter tests as the individual teacher, shall also be reviewed and considered in the establishment of the individual teacher’s performance objectives and related strategies, as reflected in the initial planning sheets. School-level AGT data and reports may also influence, or be adapted into the objectives of employees for whom there are no CST results but whose services still contribute to improved progress of students as measured by school-level reports such as those mentioned above.
e. Individual AGT Results: Individual AGT scores (as distinguished from the school- level AGT results) are to be used solely to give perspective and to assist in reviewing the past CST results of the teacher, and shall neither form the basis for any performance objectives/strategies nor be used in the final evaluation.”

So clearly, this is not dancing around the issue by any means. There is a whole other section on non-CST related assessment data as well, which may or may not be as troubling to people. Note that this includes individual student results and all of these are used to set objectives, strategies, and, ostensibly, eventually performance.

For the record, the biggest problem I have with the standardized testing and accountability system we have is the fact that we define policy based on the assumption that it matters but then the actual implementation is more like ‘Whisper, whisper, we’re just going through the motions. Everyone knows we just ignore this stuff. Oh, and dont tell the parents.’

In the end, Fletcher and Deasy make how much per year? Now we are talking about them as if they were actually doing something (productive or not I guess depends on one’s perspective). IMHO, that was the goal. Who exactly is this helping? UTLA (Teachers) say its the kids. Hopefully Deasy thinks (or says) the same. Most other people, and the current actions of the educational system say ‘nope, its actually hurting the kids, so we’re just going to (continue to) ignore it.’

Bleh..

Gary Ravani5 years ago5 years ago

Nav:
There is an assumption here that all/many teachers have great confidence in the CSTs and that they provide some guidance, minimally for instruction, or (the "meta question") for student learning. They don't and they don't.
That's why the multi-billion (that the system doesn't have by the way) investment in new assessments
. Read More

Nav:

There is an assumption here that all/many teachers have great confidence in the CSTs and that they provide some guidance, minimally for instruction, or (the “meta question”) for student learning. They don’t and they don’t.

That’s why the multi-billion (that the system doesn’t have by the way) investment in new assessments
.

navigio5 years ago5 years ago

Hi Gary. I agree that teachers do not have great confidence in the CST, though many do seem to use it as a way to corroborate other measures, such as quarterlies or similar. I dont know any teachers who rely on it as a sole piece of meaningful information. However, my comments were more intended to highlight the fact that we sometimes seem to want to make it look like something is happening, so we … Read More

Hi Gary. I agree that teachers do not have great confidence in the CST, though many do seem to use it as a way to corroborate other measures, such as quarterlies or similar. I dont know any teachers who rely on it as a sole piece of meaningful information. However, my comments were more intended to highlight the fact that we sometimes seem to want to make it look like something is happening, so we redefine processes that are already defined. Whether using CSTs is right or not, the fact that the process that was agreed upon by these two parties already should largely be happening, including through principals and certificated admins doing their current jobs as well as through bodies that are mandated by state law and take hundreds of hours of staff and parent time, seems like a cruel joke. Either that means this agreement is a fake, or our accountability system is a sham. Or both. Either way seems bad for our kids.

The question of CST validity was something I intentionally avoided in my comment, though it obviously brings a whole new realm to the discussion, including what UTLA is thinking. Deasy, I know, but Fletcher?

navigio5 years ago5 years ago

I sometimes get the sinking suspicion that districts implement 'reforms' for no other reasons than to make it look like they are doing something. I remember once hearing a superintendent say something along the lines of, 'we dont need to do anything new, we just need to do the things we are already supposed to be doing, and properly.'
Once again, I am dismayed to read over this 'agreement' to find a whole list of things … Read More

I sometimes get the sinking suspicion that districts implement ‘reforms’ for no other reasons than to make it look like they are doing something. I remember once hearing a superintendent say something along the lines of, ‘we dont need to do anything new, we just need to do the things we are already supposed to be doing, and properly.’

Once again, I am dismayed to read over this ‘agreement’ to find a whole list of things I thought were already happening everywhere. Let’s see…

– The teacher’s previous years’ CST scores: dollars to donuts, all teachers and the principal are aware of every teacher’s previous years’ CST scores, and likely even design class makeup somewhat on that information. In some schools, I expect even the School Site Council is aware of this data and discusses with staff and principal on how best to address any failings in that regard.

– Group CST results: This is, again, essentially looking at the individual teacher results as compared to the group with similar instructional duties. Both staff and SSC, likely already do (or should be doing) this.

– Currently assigned students’ previous CST results: So this is a really interesting one. This is actually looking at the cohort data on a student by student basis. If our schools are not already doing this in order to decide how to implement instruction and intervention, then we are in trouble. Whether this shows up on an evaluation sheet is probably beside the point if its not already happening. There is one additional point to be made here that a parent once asked me: when a school fails to achieve a goal set out in its plan, what actually happens? Of course the answer is nothing. Well, nothing other than you simply try harder next time. But many of the goals set in the plan (including the unreasonable AYP) are not taken seriously in the first place, let alone assessed in hindsight.

– School level CST and AGT results: CSTs are perhaps the most likely single thing that all SSCs currently look at, next to API (though API is obviously a direct reflection of CST performance). Although AGT per se is probably not used anywhere currently other than in a few areas in LAUSD, the concept of value add is absolutely used with SSC discussions. Looking at grade level CST results, it is clearly possible to see whether a certain group/grade of students has a lower ‘baseline’ for performance, and to take that into account when thinking about setting targets or assessing results.

– Individual teacher AGT results: while not used in the evaluation process according to this agreement, these are still to be used to provide some context. Again, I think this would be happening at the teacher/principal level already (I hope so), and in some extreme cases, in the SSC context.

– And maybe most importantly, Non CST/AGT results: Every year, each student is given from 3 to 5 ongoing assessments (maybe called ‘quarterlies’?). They are also evaluated for many other things such as fluency, literacy, etc. These are almost constantly available to both teachers/principals and School Site Councils. One of the reasons in the first place that the SSC’s site plan is a ‘living document’ is so that it can be adjusted to address changes as the year goes on, specifically things that can be seen using these kinds of assessments. If this is not happening already, we are lost.

I will just also add that there is one significant difference to this context than anything we have been doing in the past, and that is that teachers evaluation results are, in theory, based on whether they now meet these metrics. BUT IF THAT IS ONLY THE CASE NOW, WHAT HAVE PRINCIPALS BEEN DOING THIS WHOLE TIME? This is exactly the kind of data principals already have access to and use to not only evaluate the effectiveness of teachers, but design their curriculum. (admittedly, the use for evaluation may be have been legally tricky in the past, but principals havent turned blind eyes to this stuff, even up to now.