Schools crunch data to drive improvement

Russ Stolberg, eighth-grade science teacher at Olivet Middle School, leads students through an exercise on calculating atomic mass. Stolberg can find out quickly if students have grasped a concept by posing a question, offering several answers and asking students to “vote” for the correct answer using handheld clickers.

How many 10,000s are there in 200,000?

When math teacher
Jennifer Ball included that question on a test earlier this year, she knew not
every fourth-grader would choose the correct answer. (It’s 20). But she also
knew that wrong answers had the potential to tell her as much as right ones.

So, as conventional
and charter public school teachers across Michigan are increasingly doing, Ball
loaded the results into a database and used a computer program to find out which
wrong answers were selected most often and by whom. Using the same program, Ball
also could compare her students’ answers with similar questions on past Michigan
Educational Assessment Program tests, determine if her test reflects what the
state says should be taught in fourth-grade math and, in some cases, compare her
results with those of other math teachers.

Doing each of those
analyses individually for dozens of students would overwhelm a teacher, but
having access to a warehouse of data and the software to manage it makes the
difference, according to Ball, a teacher at Olivet Middle School.

"It’s amazing to me
to think I have that system available," she said. "It really brings forth
patterns and trends."

When teachers get
together to analyze those patterns, it can lead to insight and improvement at
the student, classroom and building level, teachers and administrators told
Michigan Education Report.

Like Olivet, other
schools also are adopting "data-driven decision making," a process of digging
into assessment results to find out exactly what students know and when they
know it, and hopefully to shed light on which students haven’t grasped certain
concepts and why.

"The deeper we dig
into things, I think the more we are learning," teacher Kristina Priesman said
of the math team at Olivet Middle School, which currently is focusing on
improving instruction in fractions. "We can actually focus on the specific areas
of fractions that students are struggling with and not just spend more time on
fractions in general."

Ball and Priesman
both participated in a pilot project to foster data-driven decision making
throughout their intermediate school district and, eventually, statewide. Funded
by a $1.5 million federal grant, the project so far has resulted in creation of
a Web site that helps educators analyze Michigan Educational Assessment Program
scores, followed up by intense professional development on how to translate
their findings into improvement plans.

Educators who
participated in the project have established professional learning communities
in their own schools — teams of teachers who meet regularly to review data, set
goals and even develop lesson plans together, trying out each other’s best
practices, according to Rebecca Rocho, assistant superintendent of general
services and legislation at the Calhoun Intermediate School District. The
Calhoun ISD is leading the grant project in consortium with the Macomb and
Shiawassee intermediate districts.

"It’s those
conversations that this project is really about," Rocho said. "We’re trying to
break down those isolated classrooms and get teachers talking."

Schools have always
collected volumes of academic and demographic data about students, Rocho said,
but haven’t necessarily used it to drive school improvement. Some information
was stored in separate databases that didn’t "talk" to each other. Other data
was accessible, but mainly to tech-savvy teachers who understood how to query
it. And given the volume of data, it could be hard to tell which facts and
figures were important.

"Data alone is often
not meaningful," Rocho said. "As you look at data, you need to know what the
questions you should be answering are."

Collaborating with
the Center for Educational Performance and Information in the Office of the
State Budget, and with local educators, the Calhoun ISD identified six key
questions to put to MEAP data. Those questions are now the basis of Data for
Student Success, a Web-based program that educators can use as a tool to analyze
their MEAP scores in more detail than many have done in the past.

The program goes
beyond the typical breakdown of students into proficiency levels, and adds
reports on how many students are "near proficient" as well as proficient on
specific concepts within each subject. For example, educators can quickly learn
how well their students read narrative text or calculate length and area. The
system can report on individual students or cohorts of students over time, and
academic records can be compared to demographic information like the student’s
age, ethnicity and attendance record.

Now in the second
year of a three-year development and rollout plan, a final version of Data for
Student Success is expected to be available to all teachers and administrators
in 2008-2009.

While the Data for
Student Success model focuses on MEAP scores, some schools also are building
local data warehouses to track each teacher’s classroom assessments throughout
the year. A number of districts now ask all educators who teach the same subject
in the same grade to give the same quarterly or year-end assessments so the
results can be compared. They also keep records on what’s been tried with
struggling students in the past — tutoring, after-school study groups — and
which approaches resulted in academic improvement.

Being able to analyze
results as assessments are given, rather than once-a-year MEAP scores, is a big
advantage, Rocho said.

"The really important
data is what is going on with a student right now," she said.

Some schools also are
turning to more frequent tests. Olivet third- through 10th-graders take the
Northwest Evaluation Association’s Measure of Academic Progress test at least
once a year. Students at National Heritage Academies, a network of 55 public
charter schools in six states, including 35 in Michigan, take it three times a
year. NWEA is a nonprofit association based in Oregon that offers educational
testing services, data analysis and other academic resources.

At South Arbor
Charter Academy, an NHA school in Ypsilanti, students take the computerized NWEA
exam in language arts, reading and math. The results show not only where South
Arbor students currently stand compared to their peers nationwide, but also
predict how much academic growth that student should be able to achieve in a
given year. The results are broken down into subcategories which relate to
specific academic skills.

"We’re able to use
that to drive instruction," Principal Tim DiLaura said. "Teachers can track data
and watch students progress from test to test. We’re able to calculate the
percent of growth a student has achieved."

Knowing the students’
strengths and weaknesses early in the school year — teachers receive a test
report within 24 hours — helps them target instruction, he said.

"More and more
schools are realizing that we need to manage achievement," DiLaura said. "We
need to know where we’re starting at."

At Walters Elementary
School in Marshall Public Schools, Principal Susan Townsend schedules regular
"data checkpoint" meetings with each teacher to talk about how students did on
recent assessments and to discuss improvement plans. She makes time for teachers
to prepare data reports by taking over each classroom for several hours a month.

Teacher collaboration
is the key to using data well, said Principal Mark Bensinger at Olivet Middle
School. Every teacher at his school is now a member of a Collaborative Action
Team that meets monthly to review data and set specific goals in a given
subject.

The team process has brought teachers together, he and others said.

"The culture and
climate of our school is not just a little better. The change is incredible,"
Bensinger said. "Implementation of professional learning communities has allowed us to develop a culture of mutual accountability and ownership for decisions."

"Nothing we have done
is incredibly technically difficult," he added. "Once we overcame some of our
traditional views of staff meetings … things started clicking."

Similarly, South
Arbor Principal DiLaura said that once the data points out strengths and
weaknesses, "That’s where we want teachers to bring their craft to the
challenge."

But the ability to
analyze data apparently comes with its own achievement gaps among Michigan
schools.

In a presentation to
the state board of education in November, Linda Hecker of the Michigan
Department of Education said that administrators in some underperforming schools
don’t understand achievement data.

"Auditors found that
many schools don’t have any idea why they didn’t make AYP (adequate yearly
progress), so they can’t focus as well on improving themselves," said Hecker.
She added later, "They got their (MEAP) data, but sometimes they weren’t skilled
in reading the data. Sometimes there were three or four subgroups in different
areas and they didn’t understand how to put those pieces together."

The state hopes that
intermediate districts will lead the way in serving as data warehouses and
offering professional development on how to use data, as several already do,
according to Bruce Umpstead, director of the Office of Educational Technology
and Data Coordination at the Michigan Department of Education.

"We are primarily
looking to our ISD partners to support local districts," Umpstead said,
responding by e-mail to questions from Michigan Education Report. School
districts aren’t required to use data-based decision making in setting school
improvement goals, but "state and federal performance requirements are making
the effective use of data imperative," he wrote.

No Child Left Behind
legislation has led school districts nationwide not only to test students every
year, but to develop data systems to track individual student performance over
time, noted Margaret Ropp, director of CEPI and Michigan’s representative to the
National Center for Education Statistics.

"We’ve always been
getting data in," Ropp said, but now the emphasis is on "getting data back out
to local districts securely and longitudinally."

Though Data for
Student Success focuses on K-12 education, Ropp said supporters of data-driven
decision making nationally see a need to link educational data from preschool
through college. A "P-20" system, as it’s called, would point out such things as
how many students from a given high school needed remedial coursework in
college, or whether preschool attendance is related to reading scores.

Struggling students
are not the only ones to benefit from such programs, noted Townsend, the
principal at Walters Elementary. Data analysis also points out students who need
more challenge.

"So often when we
look at improvement, it’s for the lower end," she said. "These warehouses are
wonderful. You have more time to get to the nitty-gritty."

That’s the challenge
facing Russ Stolberg and other science teachers at Olivet Middle School, where
92 percent of eighth-grade students tested proficient in science in the fall of
2007.

Stolberg analyzes his
students’ performance across the MEAP, looking for clues to strengths and
weaknesses that might affect science scores. Learning that some students didn’t
do well at reading informative text, he and his colleagues decided to add a
timed reading — reading a scientific article and answering questions about it —
to one of their own classroom assessments.

"We really have to fine tune our approaches," Stolberg said.

#####

Lorie Shane is the managing editor of the Michigan Education Report, the Mackinac Center’s quarterly education policy journal. Permission to reprint in whole or in part is hereby granted, provided that
Michigan Education Report is properly cited.