Louisiana leads U.S. in grading teacher training, but report cards are inconclusive

Gerald Carlson's heart sank when he received word several years ago
that a controversial statistical analysis had declared his program one
of Louisiana's weakest in preparing educators to teach English language
arts.

"We thought we had a good program," said Carlson, dean of the
education school at the University of Louisiana's Lafayette campus. "We
were shocked when we saw the results." Carlson has since worked with his
staff to revamp the school's curriculum: adding a new class in reading
and English language arts, requiring more writing across the board, and
beefing up professional development for the university's professors.

He is confident that when the latest results come out this spring --
the first to be released in over a year and a half -- the university will
fare well.

Story by

Sarah Carr

The Hechinger Report

Scores of teacher-training programs across the country could face
similar scrutiny in coming years. Following Louisiana's lead, policy
makers in a growing number of states are evaluating programs based on
the test scores of the students their graduates teach. So far, eight
states have policies requiring them to do a similar analysis, most of
them adopted in the last few years, according to the National Council on Teacher Quality.

"This is a policy movement that's sweeping the country," said Charles
Peck, a professor of special education and teacher education at the
University of Washington's College of Education.

Related efforts to evaluate individual teachers based on student test
scores have sparked a flurry of publicity -- and led to a major lawsuit.
But efforts to evaluate teacher-preparation programs, including
well-established university-based education schools and less traditional
programs like Teach For America, have gone comparatively unnoticed and un-scrutinized.

As other states climb aboard the teacher-accountability bandwagon,
Louisiana's experience speaks to the promise and peril of the new
approach. Some programs, like Carlson's, have used the data to make
significant changes. "We should be doing this analysis anyway, whether
we're forced to or not," Carlson said.

But the data only capture institutions with a large enough cohort of
graduates (25 in a given subject area) for the results to be
statistically meaningful. As a result, not a single New Orleans-based
teacher-training program -- including those at the University of New
Orleans, Xavier University, Southern University at New Orleans, and Our
Lady of Holy Cross -- is consistently included in the study. In recent
years, only UNO has had a large enough cohort, but just in its
undergraduate social studies and math-certification programs.

The New Orleans institutions still go through traditional
accreditation processes, in which a team of experts visits the campus
and gauges program quality by interviewing students, staff and course
content. But notwithstanding Louisiana's growing reputation as a state
that holds its education schools accountable, the quality of many local
institutions remains comparatively unknown.

Others worry that even when the cohort size is large enough the data
can be overly simplistic and at times misleading. For instance, do low
reading scores recorded years after a group of teachers enters the
classroom mean their training program had a bad curriculum or weak
instructors? Or did the program admit weaker candidates from the start
or perhaps send them off to schools with less supportive principals?

"It's kind of like having a fire alarm go off in your house, but not knowing where the fire is," Peck said.

Digging deeper into the data

The concern voiced by Peck and others prompted state officials to
break down the data for each training program in a more detailed way,
said Jeanne Burns, the Louisiana Board of Regents' associate
commissioner for teacher and leadership initiatives. Over time, training
programs could see how they fared with low-income students, for
instance, or on which specific skills their graduates fell short.

"Our goal has always been to use the data to help our campuses improve," Burns said.

Starting in 2003, the Louisiana studies
tracked student performance in grades four through nine, tying it back
to the teacher-preparation programs. By using what's known as a
"value-added" analysis, researchers homed in on the amount of growth
seen in individual students, no matter their starting point. They then
compared the overall student growth in the classrooms of recent
graduates of different training programs to the growth achieved by
veteran educators. No new results have been issued in over a year
because state officials have been working to change the system to align
it more closely to the state's comparatively new evaluation process for individual teachers.

Louisiana officials requested the analysis more than a decade ago
partly in the hope that it would weed out the state's weakest
teacher-training programs. If programs consistently rate low, the state
can shut them down. But so far, no program has been forced to close as a
direct result of the new system.

"I don't view it so much as an accountability mechanism because none
have been shut down," said Tim Daly, president of the national
organization TNTP (formerly The New Teacher Project), which trains many of New Orleans' Teach For America instructors, and whose organization has fared fairly
well in the studies. But he says the analysis has improved
transparency, including for prospective teachers shopping for a training
program.

It has also prompted self-examination and improvements at a few
programs, including not just the University of Louisiana at Lafayette
but also the Louisiana Resource Center for Educators, a private teacher-certification program based in Baton Rouge.

The initial round of results showed that the Resource Center was
"doing a lousy job of teaching reading," said Nancy Roberts, the
executive director. But the initial data did not provide any information
about which of the center's graduates posted the weakest results, which
aspects of reading their students had failed to master (phonemic
awareness? overall comprehension? something else?), or which grade
levels struggled most.

"It was like shooting darts in the dark," Roberts said. "But we decided to roll up our sleeves."

After looking at the more detailed data, the Resource Center began to
tailor its literacy instruction more precisely, based on the age group
the graduate would be teaching. It also increased the amount of overall
class time spent on reading and literacy, from less than 20 percent to
about a third.

George Noell, the Louisiana State University researcher who designed
the evaluation system, said he is pleased that some of the
lowest-performing programs are making improvements, although "across the
whole spectrum, it's not as clear the data have helped people as much
as I hoped."

"I assumed it would not be as long a journey from seeing results to
figuring out what to do with them," he said. "But I consider the fact
that we are even talking about it huge progress."

The selectivity factor

Some experts say that determining how much of a young teacher's
success or failure can be tied to his or her training program is like
asking which came first, the chicken or the egg.

If a program's graduates post weak results, it could be that the
program is "doing a fabulous job but they are not selective enough in
terms of who is admitted," said Kate Walsh, president of the National
Council on Teacher Quality. She pointed out that many of the programs
that have scored best in Tennessee and Louisiana evaluations also have
highly competitive admissions.

Daly disagreed that the selectivity of a program significantly
affects its graduates' performance in the classroom. "While selectivity
may play a small role, what we do to train teachers plays a big role,"
he said.

Katrina Miller, the director of Tennessee's federal First to the Top
grant, said she considers selectivity something training programs can
control. "You are choosing who to admit," she said. "Selectivity, since
it's chosen by the program, is part of the training."

Some programs are far choosier than others, however. And it's easier for Teach For America -- to which more than 10 percent of all Ivy League seniors apply -- to be more selective than your average state school.

That said, in addition to revamping part of its curriculum after that
initial evaluation, the teacher program at the University of Louisiana
at Lafayette has also managed to raise its admission standards modestly,
Carlson said. The university now requires a 2.5 cumulative GPA (which
includes all of a candidate's past grades) instead of a 2.5 adjusted GPA
(which only counts the higher of two grades if a student had to repeat a
course). "That's eliminated a lot of people," Carlson said.

Brian Beabout, a former New Orleans public school teacher who now
works as an assistant professor at the University of New Orleans, said
he worries that a teacher's success in his first two years on the job
says more about the quality of his K-12 education, which spans 13 years,
than the quality of his preparation program, which can be as short as
six weeks. And as time passes, teachers sink or swim largely based on
how much support they get on the job, he said.

"One of the dangers of single rankings is that we let school
districts off the hook for providing career-long support and
development," he said.

Missing the small programs

Perhaps the greatest limitation of Louisiana's data-centered analysis
is that it fails to encompass most of the state's small
teacher-training schools, a mixture of private and public university
programs. All told, at least seven of the state's programs did not have
large enough enrollments for any results to be reported in 2010-2011.
Fourteen programs had at least some results released.

In New Orleans, many of the programs were already small before the
dual blows of Katrina and state budget cuts further reduced their size.
In 2000, 172 students completed the University of New Orleans' teacher
certification programs, compared to 104 in 2009. The drop-off was even
more dramatic at Southern University at New Orleans: 87 graduates in
2000 compared to 12 in 2009. UNO's young graduates have performed fairly
well in social studies and even better in math, but those were the only
two areas where the cohort size was large enough to be captured in the
study.

Burns said even New Orleans' smaller programs, like SUNO, eventually
will be captured in the value-added analysis. That's because the state
plans to wrap together multiple years of results from different
institutions to achieve meaningful sample sizes for statistical
evaluation. Meanwhile, the smaller institutions have been held
accountable through national accreditation processes and a requirement
that obliged all teacher-training programs to undergo re-approval over
the past 10 years.

Few advocate including programs so small that their value-added
results aren't statistically meaningful. But some experts say Louisiana
should develop a more qualitative component of its evaluation that can
capture small programs -- and provide a more multi-dimensional look at
larger ones.

Larger institutions can be left with a false sense of superiority if
they post high value-added scores in states where the overall quality of
teacher-preparation programs is weak, Walsh said.

"We need multiple measures to see if a program is effective," she
said. "Our view is that performance by these institutions is not good
across the board and even the highest performing institution is not
something you want to hold up as a model."

This story was reported by The Hechinger Report, a nonprofit, nonpartisan education-news outlet based at Teachers College, Columbia University, and it was originally published by The Lens, an independent, nonprofit newsroom serving New Orleans.