Test monitoring offers look into city's efforts to preempt cheating

A test security practice that city officials devised to deter cheating before it happens is also being used to preempt schools already suspected of misconduct.

Each spring, as part of its test monitoring program, the Department of Education disperses a small team to schools on testing days to scrutinize and enforce security guidelines. Some schools are picked randomly, but others were flagged by the department because allegations were lodged by school staff and test score data showed “anomalous” results in recent years, officials said.

During this year’s six-day elementary and middle school testing period in April, education department employees paid 41 visits to 37 schools, according to records obtained by GothamSchools in a Freedom of Information Law request.

The city would not specify which schools were the subject of a targeted monitoring visit, as opposed to a random one. But an analysis of test score data for the schools that had monitors visit showed that many had large increases in 2011, a year when the citywide pass rate barely budged. When monitors visited the schools for the 2012 tests, some of them saw sharp drops on its scores — even while the citywide average increased.

Not all monitored schools saw declines this year and, in fact, some saw large gains. But of the schools that made significant gains on either English or math in 2011, more than half regressed to some degree in 2012. One school’s math proficiency rate dropped by more than 40 percentage points.

The previously undisclosed details about the monitoring program comes at a time when state and federal education officials are increasingly focused on devising policies to improve the integrity of tests in the wake of cheating scandals that have erupted in other cities. The number of schools listed in the monitoring program also provides a limited glimpse into the scope of cheating allegations that the city education department receives and is able to deal with.

(View which schools the city monitored during 2012 tests: http://www.scribd.com/doc/109627524/F8639-testingviolations-updated092812)

Despite calls for more comprehensive test security measures, the test monitoring program in New York City is shrinking. The number of schools canvassed in 2012 represents about 3 percent of schools that tested students this year, down from about 9 percent last year.

In almost all of the schools, monitors found no evidence of wrongdoing on the days of their visits. They follow a checklist that lists 20 state guidelines to make sure that schools in compliance. Was the test packet’s shrink wrap unmolested and kept in a locked room? Check. Are teachers making sure kids don’t receive inappropriate help? Check. Is the door cracked open and the windows left unobstructed?

But city officials say the program isn’t meant to catch principals or teachers in the act of misconduct. Instead, they say, the program is a successful strategy to stop it from happening in the first place.

“These unannounced visits are effective largely because of their deterrent effect – even schools that don’t actually get a visit they know that they could get a visit at any moment and must demonstrate that their test administration is consistent with regulations,” said department spokeswoman Connie Pankratz.

Officials wouldn’t say if they thought the monitoring presence had a direct effect on how schools performed, but many of the schools did see sharp drops.

At P.S. 270 Johann Dekalb, in Clinton Hill, the percentage of students who passed the English exam dropped 20 points this year, from 63 percent to 43 percent. The school’s math scores fared even worse. The number of students who passed the math dropped from 83 to 38, which yielded a proficiency rate that was 40 percentage points lower that the year before.

P.S. 270 was one of four schools that monitors visited twice in April — one day for a portion of the English exam and one day for a portion of the math exam. The three others were Esperanza Preparatory Academy, in Harlem, P.S. 091 The Albany Avenue School, in Crown Heights and Satellite West Middle School, in Fort Greene. Monitors did not visit charter schools in 2012.

Another monitored school, Choir Academy of Harlem, dropped 23 percentage points on math proficiency in 2012, just one year after it increased 16 percentage points in 2011. And at P.S. 375 Jackie Robinson School in Brooklyn, the passing rate in math increased 25 points in 2011, then dropped 11 points this year.

Often, the abnormal scoring peaks and valleys didn’t show up when looking at school wide proficiency rates, but did when comparing a grade level from one year to the next:

At the Cornerstone Academy for Social Action in the Bronx, the percentage of fifth graders rated proficient in math spiked from 31 percent in 2010 to 72 percent in 2011. When monitors visited in April, proficiency dropped back down to 26.9 percent.

Cornerstone’s fifth graders improved significantly on their English exams in 2011 as well; their proficiency increased 22 percentage points from 17 percent to 39 percent, then dropped back down to 25 in 2012.

The number of sixth graders at Esperanza Preparatory Academy who were proficient on their English tests more than doubled in 2011, from 13 to 27. The number then dropped in 2012 back down to 14 students.

The number of fourth grade students who passed their math test nearly tripled in 2011 at P.S. 044 David C. Farragut. This year, the number dropped by more than 40 percent.

The number of eight graders who passed math at P.S. 096 Joseph Lanzetta more than doubled in 2011, then dipped by 60 perent in 2012.

Calls and emails to the schools and principals were not returned this week. The week is a vacation for most school-based department of education employees.

Officials said they were first inspired to look at the schools by allegations of cheating or other test security violations. They did not respond to questions about whether the schools were under investigation, saying more time was needed to provide an answer. Records of the monitoring visits show that no violations were observed that required a referral for an investigation.

Monitors once had a larger presence in schools, but it began dwindling years ago when the Bloomberg administration restructured around networks. Before that, district offices had the responsibility of monitoring and deploying staff members to schools on test days.

Former Regional Superintendent Kathy Cashin said she blanketed her district of more than 100 schools with monitors on exam day. She scoffed at the size of the city’s systemwide monitoring program this year.

“It’s ridiculous,” Cashin said. “I mean, we had monitors in every single one of our schools.”

Test security makes for complicated politics at a time when districts increasingly lean on gains to touttheir policies. But in the wake of cheating scandals that erupted in large cities around the country, Education Secretary Arne Duncan believed that test integrity was important enough of a reform issue to send a guidance memo to state superintendents last summer, urging them to adopt stricter measures.

“As I’m sure you know, even the hint of testing irregularities and misconduct in the test administration process could call into question school reform efforts and undermine the State accountability systems that you have painstakingly built over the past decade,” Duncan wrote to top state education chiefs.

But one reform that has stayed on the table is test monitoring. The decrease in New York City’s program comes at a time when the state is hoping that districts will start ramping up their own programs.

“We’re looking to beef up the monitoring program,” said Tina Sciocchetti, who oversees test security at a statewide level.

Labor Day used to signal the end of summer break and the return to school. That’s no longer the case in Tennessee, but the long holiday is a good time to catch up on all that happened over the summer. Here are 10 stories to get you up to speed on K-12 education in Tennessee and its largest school district.

TNReady is back — with a new test maker.

Last school year ended on a cliffhanger, with the State Department of Education canceling its end-of-year tests for grades 3-8 in the spring and firing testmaker Measurement Inc. after a series of missteps. In July, Commissioner Candice McQueen announced that Minnesota-based Questar will pick up where Measurement Inc. left off. She also outlined the state’s game plan for standardized tests in the coming year.

But fallout over the state’s failed TNReady test in 2015-16 will be felt for years.

The one-year void in standardized test scores has hit Tennessee at the heart of its accountability system, leaving the state digging for other ways to assess whether all of its students are improving.

Speaking of accountability, Tennessee also is updating that plan under a new federal education law.

The state Department of Education has been working with educators, policymakers and community members on new ways to evaluate schools in answer to the federal Every Student Succeeds Act, or ESSA, which requires states to judge schools by non-academic measures as well as test scores.

Meanwhile, issues of race and policing have educators talking about how to foster conversations about social justice in school.

In the wake of police-related killings that rocked the nation, five Memphis teachers talked about how they tackle difficult conversations about race all year long.

School closures made headlines again in Memphis — with more closings likely.

Closing schools has become an annual event as Tennessee’s largest district loses students and funding, and this year was no exception. The shuttering of Carver and Northside high schools brought the total number of district-run school closures to at least 21 since 2012. And more are likely. This month, Shelby County Schools is scheduled to release a facilities analysis that should set the stage for future closures. Superintendent Dorsey Hopson has said the district needs to shed as many as two dozen schools — and 27,000 seats — over the next four years. A Chalkbeat analysis identifies 25 schools at risk.

Exacerbating the challenges of shifting enrollment, families in Foote Homes scrambled to register their children for school as Memphis’ last public housing project prepared to close this month amid a delay in delivering housing vouchers to move elsewhere.

The new school year has officially begun, with the budget approved not a moment too soon for Shelby County Schools.

District leaders that began the budget season facing an $86 million shortfall eventually convinced county commissioners to significantly increase local funding, while also pulling some money from the school system’s reserve funds. The result is a $959 million budget that gives most of the district’s teachers a 3 percent raise and restores funding for positions deemed critical for continued academic progress.

The district also unveiled its first annual report on its growing sector of charter schools.

With charter schools now firmly entrenched in Memphis’ educational landscape, a Shelby County Schools analysis shows a mixed bag of performance, while calling on traditional and charter schools to learn from each other and promising better ways to track quality.

Ten stories you might have missed over the summer (and should read now as a new school year begins)

There is no such thing as time off from covering education. While school doors were shuttered, plenty happened this summer on the Colorado education beat. Here, we’ve compiled stories that we hope prove useful as you ease back into your fall routines.

We’ve got your immunization data right here …

For the second year, Chalkbeat tracked down immunization data for more than 1,200 schools in Colorado’s largest school districts. Our database revealed that Boulder remains a hotspot for the anti-vaccination movement, students in districts with racial and income diversity are more likely to get their shots and nearly half of schools in the database did a better job this year tracking students’ immunization records. Read our news story about the findings, check out these six charts that dig into the numbers and search for school-level data here.

In June, Denver Public Schools’ longtime schools chief returned from a six-month unpaid sabbatical in South America with his family. “It made us appreciate the extraordinary resources we have here,” he said in an interview about his experience.

A milestone for Colorado charter schools on diversity, but not so much on integration

But the University Club has a lovely lunch menu (and squash courts, too)…

What if the State Board of Education held a not-so-public meeting with the education commissioner at a private club downtown to prioritize goals, but didn’t get much of anything accomplished? That happened.

Despite opposition from advocacy groups, Colorado appears headed toward lifting a seven-year ban on diet soda in high schools. The rule change would clear the way for diet soda to be sold in high school vending machines and school stores, though districts could decide not to stock the drinks. We covered the issue before and after the State Board of Education’s initial vote.