Deborah Meier is a visionary teacher, author, and founder of successful small schools in New York City and Boston. Harry Boyte, senior scholar at Augsburg College, is founder of the youth civic empowerment initiative Public Achievement and a leader in the movement to democratize higher education.

Lies, Damn Lies, and Statistics

I enjoyed seeing you honored as a hero of education by FairTest last week, which established an annual award named for you. If anyone had told me five years ago that I would be at that event, I would have thought them mad. This is what “Bridging Differences” has done for me, I suppose.

At the event, I was surrounded, not surprisingly, by educators who have long believed that standardized tests are more wrong than right, or that they are a crime against children’s nature, or worse. I continue to believe that we can get valuable information from standardized tests and that they can help with diagnosing problems and needs. The information derived from testing can be useful, but lately I have begun to see how often test scores are being misused to punish kids, teachers, and schools and to mislead the public.

As it happened, New York state just released the results of its annual tests of English language arts and mathematics, and the scores soared across the state to an extent that was literally unbelievable.

The state Education Department released the math scores last week. From 2006, when the current testing regime’s trend line begins, to today, the percentage of kids meeting state standards (that is, scoring a 3 or 4 on a four-level proficiency scale) has gone from 65.8 percent to 86.4 percent. In 8th grade, the proportion meeting or exceeding standards leapt from 54 percent to 80 percent. The gains for black and Hispanic students across the state were huge—for black students, from 45 percent to 75 percent, and for Hispanic students, from 51 percent to 79 percent. White and Asian students are already close to the ceiling, at 92 percent and 95 percent respectively.

Some districts saw increases that defy anyone’s wildest dreams: In Buffalo, the proportion passing flew up from 28.6 percent to 63.3 percent; in Rochester, it went from 33.1 percent to 63.4 percent; in New York City, from 57 percent to 82 percent; and Syracuse nearly doubled from 30 percent to 58 percent. All in four short years! At this rate, everyone will be proficient well before NCLB's deadline of 2014.

The news media reported the dramatic gains with a straight face. The superintendents of Rochester and Buffalo basked in the limelight. Mayor Bloomberg said the scores proved the value of his one-man control of New York City's schools, although surely his reign had nothing to do with the even larger gains in other cities in the state. Only the Rochester newspaper asked in an editorial whether these gains made any sense.

Now the New York Daily News has done an analysis of the math tests and concluded that the state tests got progressively easier from 2006 to 2009. Kudos to reporters Meredith Kolodner and Rachel Monahan, who beat The New York Times to this statistical scandal. Kolodner and Monahan had the smarts to turn to Jennifer Jennings of Columbia University, who was formerly the blogger for Education Week known as eduwonkette; Jennings analyzed the tests and discovered that the state has been testing only a fraction of its math standards, and teachers are able to predict which standards will appear on the tests.

Jennings also found that nearly identical questions have appeared every year. “In 2009, at least 14 of the 30 multiple-choice questions on the seventh-grade exam, for example, had appeared in similar form in previous years,” said Jennings. Teachers and principals chimed in and agreed that the questions were predictable and students are taking frequent practice tests that teach them the format.

A teacher explained to me recently that “we drill down into the state test to predict what will be tested," and then students practice those questions, again and again.

My guess is that if the students in New York state were given a math test from another state—one that they had not been primed for—their scores would be much lower.

U.S. Secretary of Education Arne Duncan keeps telling states and districts that they are “lying” to kids when they tell them that they are doing just fine, but they really are not. Just last week, Duncan said that many states are "lying to children and their parents, because states have dumbed down their standards." New York is a perfect example of what Duncan means. The proportion of students who pass the tests keeps going higher and higher every year, but when the 2007 NAEP scores were released, the state had flat scores in everything but 4th grade math.

What we see in New York state is institutionalized lying, according to Secretary Duncan’s definition. The state is well on its way to becoming a national laughingstock if it keeps up this Ponzi scheme whose victims are its students.

Mark Twain wrote, “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: 'There are three kinds of lies: lies, damned lies, and statistics.'"

The New York State Education Department is showing how easy it is to lie with numbers.

Categories:

82 Comments

If anyone had told me five years ago that I would be at that event, I would have thought them mad. This is what “Bridging Differences” has done for me, I suppose. ( Diane )

It has been uplifting to see what a true dialog between people who respect each other and listen to each other can lead to. Not either/or thinking but a combination of the best from both as the dialog progresses.

The information derived from testing can be useful, but lately I have begun to see how often test scores are being misused to punish kids, teachers, and schools and to mislead the public.
( Diane Ravich )

Totally agree with this except in my experience it has been going on for years now.

The problem that you describe with tests and scores varies state to state yet the issue is deeper. The current situation has states paying lots of money for data banks that house fairly meaningless test score data that can be manipulated in many different ways.
( love your twain quote! )

The quick answer to this problem is to standardize the curriculum and testing state to state. This would provide us with information that we already have and this would be a move in the wrong direction. This is however, the direction we seem to be heading.

As Deb mentioned in her last post:

And the latest headlines about 46 states joining together to decide year by year school curriculum (and tests) is not the way to decide this. ( Deb )

In effect….

This direction would mirror what we do with AP classes right now. The question for me is….

Do we think that the AP model of schooling ( tight curriculum and testing) is something we want to try and replicate as the exemplar to “schooling” in America?

Will this type of “schooling” lead us toward where it is we want to go?

Will this set up the meritocracy that our current leadership seems to be trying to create?

Can't figure out why the Times, Post, or Daily News doesn't demand NAEP results in an attempt to corroborate NY state tests. This would prove whether Bloomberg/Kline are really getting the job done or merely pulling the wool over the eyes of the taxpayers, parents, and students.

Paul--nobody needs to "demand" NAEP tests for corroboration. Just a few mouse clicks and you are there. As it happens, NYC is among a short list of urban districts for whom there is data--most over 3 administrations, some only 2. You cannot go school by school--and the most recent scores are 2007, so they won't help much in looking at any big upticks. But, generally, there has been significant (statistically) improvement on NAEP from 2003-2007--looking at average scale scores--a bit more specific indicator than the number who fall on on the upper side of the proficiency mark (and I'm sure that scale scores are or soon will be available for the NY tests). Of interest also is that not all of the urbans sampled have gone up, some are flat and others erratic--which leads me to rather suspect that those who have gone up have actually performed better. There is lots of ways to look at the data. I found it interesting to look at comparisons of the 25th, 50th and 75th percentile scale scores in the various districts. Almost without exception the lines of change were parallel--whether upward or downward. Reminder that our best efforts have not done much to narrow the distance from top to bottom--which should concern us, because it appears that the leading countries not only have higher average achievement, but greater equity.

But I would urge us, as we get very specific about what the data tells us, and doesn't tell us, that we remain vigilent as well about the specificity of our words and descriptors in general with regard to the effects and outcomes of testing. Despite a lot of rhetoric to the contrary, I know of very little that I would consider to be "punishment" of students, schools or teachers through the misuse of test scores. That statement reminds me of folks who opposed desegregation on the grounds that the kids in the all white school shouldn't have to be punished to make up for slavery and Jim Crow. Every consequence is not punishment. Every change is not punishment. A change in diet when a blood sugar test is too high is not a punishment--even though it might mean foregoing well-loved treats.

I have no idea whether the decision-makers in NYC are snakes in the grass or the reincarnation. But I observe that there are lots of opinions on that--and we ought not let them enter into our evaluation of data.

Thanks for the ACHIEVE report and the data on the American Diploma Project (ADP). Interesting to note it's been up and running since 2005 yet the state of New York has not yet chosen to jump on board. Wonder why?

Got an email from Diane a few minutes ago saying the owners of the New York Post and the Daily News are both close personal friends (and billionaires) of Mayor Bloomberg. Isn't that an interesting tidbit.

I'm not sure a common test for all states is a concern. I think it's a blessing...for the kids.

New York won't have to join Achieve Inc. because they are on board with this already:
The Common Core State Standards Initiative, a state-led process to develop common English-language arts and mathematics standards. The Common Core State Standards Initiative will be jointly led by the National Governors Assoc. Center for Best Practices (NGA Center) and the Council of Chief State School Officers (CCSSO).

Be careful what you wish for... we are producing systems with pretty predictable winners and losers.

Margo/Mom: Despite a lot of rhetoric to the contrary, I know of very little that I would consider to be "punishment" of students, schools or teachers through the misuse of test scores.

Margo there are more than a few states that require kids to pass end of year tests to move to the next grade...regardless of performance during the year. Here is an example of large scale mis-use of testing that hurts kids:

Last year, 132,744 Florida public school students were classified as "non-promotions" -- that is they didn't move up to the next grade in August. That is down from a high of more than 208,000 retained kids at the end of the 2002-03 school year.

"That was the year the third-grade retention law kicked in, requiring schools to hold back many kids who scored a level 1 on the five-level FCAT reading test. The law was part of a series of education reform measures pushed by former Gov. Jeb Bush's administration.

Mike,
Here are some of the demographics you about which you were wondering....

The total number of non-promotions in Florida equates to 5% of the state's entire public school population. The two grades most affected are the 3rd to 4th grade (6.6%) students and the 9th to 10th grade (10%) students who were not promoted from the lower to the higher grade level.

Along other demographic lines... as would be expected....
3.8% of all white students were retained
7.6% of all black students were retained
5.2% of all Hispanics
2.4% of all Asian students
4.5% of all Native American students.. and...
4.4% of those students who identified themselves as multi-racial.

Retentions at all grade levels were mostly for reading problems, with the exception of 6th grade, which had a larger percentage of students with math problems.

However, this does not take into account that the state has created AT LEAST a dozen loopholes in the non-social promotion agenda. This allows for parents with ANY wits whatsoever, to force the school to use alternative methods for assessing their child's progress and thereby, avoiding retention.

The issue, for me is.... of the 132,744 Florida public school students who were retained in 07-08....
did it do anything to remediate the problem?

My experience and the research indicate that retention does not work.

Here is a sample of some of the research:

Systematic reviews and meta-analyses examining research over the past century (studies between 1911–1999) conclude that the cumulative evidence does not support the use of grade retention as an intervention for academic achievement or socio-emotional adjustment problems (Holmes, 1989; Jimerson, 2001).

Research also fails to find significant differences between groups of students retained early (kindergarten through 3rd grade) or later (4th through 8th grades).

What is most important is that, across studies, retention at any grade level is associated with later high school dropout, as well as other deleterious long-term effects.

When comparing retained students with similarly under-achieving but promoted peers, research indicates that retained students have lower levels of academic adjustment in 11th grade and are more likely to drop out of high school by age 19 (Jimerson, 1999).

In fact, retention was found to be one of the most powerful predictors of high school dropout, with retained students 2 to 11 times more likely to
drop out of high school than promoted students (Jimerson, Anderson, & Whipple, 2002)

Furthermore,the retained students are less likely to receive a high school diploma by age 20, receive poorer educational competence ratings, and are also less likely to be enrolled in post-secondary education of any kind. These youth also receive lower educational and employment status ratingsand are paid less per hour at age 20 (Jimerson, 1999).

It is another example of the irony we seem to be living with. NCLB talks about the use of "scientifically based stategies" yet we seem intent on using interventions that are anything but research based.

What concerns me is....what is the long term effect of this policy on the hundreds of thousands of kids in our country currently being held back for not passing tests.

If the above research and my experiences are accurate...seems like it does have a negative impact on kids.

Amazingly, the best we can do on the data is 2005-2006. Imagine that. In June of 2009, we can figure out who graduated in 2006.

Imagine if Fedex knew only where its packages were three years ago. Imagine if Walmart could tell you how many of its stores made their sales targets 12 full quarters in the past. Imagine if the US Army couldn't yet tell us how many units reported to the Surge because it occurred six months after mid-2006.

42% of those going to college enrolled in remedial courses. That's more of the bad news.

Better news is that 20 states are now defining what it means to be "college ready". Better, but we're still way, way behind the curve.

Also, I always find the national map of state grad rates interesting.

"progress has been more rapid in areas where graduation rates have historically languished. High school completion rates have risen much faster in high-poverty school systems as opposed to more-affluent ones, and in urban vs. suburban communities. Stronger improvements have also been found in larger districts and, to a lesser extent, in those serving majority-minority student populations"

No word on how many of the additional graduates come from places like WhiteHat's Life Skills Centers, and the 4,600 charter schools serving over 1.4 million children who have saved so many kids.

Mike--I absolutely agree that grade retention is not an acceptable intervention. I would also point out that grade retention is often the result of other indicators (not to mention wrongheaded thinking) besides standardized tests. My own state has a "ninth grade bubble" despite the fact that this is a non-tested grade. This has spurred its own silliness in the form of "credit recovery" programs--or online "mastery-based" courses. This allows students to put in their "seat time" in a class, but get the "learning" the following year from a computer. And we do this with a straight face. My son was retained in first grade--something I probably shouldn't have allowed, but, when I felt helpless to bring about any of the things that might meet his needs (or even locate an appropriate forum where they could be identified and discussed rationally), it seemed to be the better or poor options.

I think that you included in your post an aside that bears looking at--before we decide to ditch the testing programs. That is the NCLB requirement for scientifically-researched methods of intervention. As you point out--there is no research to support retention--but we do it anyway (my state had the kind of loophole-ridden requirement that Rory describes at one point before giving it up, Chicago is a famous example, as well as, I seem to recall NYC at one point, pre-NCLB). However, near as I can tell, the only real "intervention" that is available--at least in my district--is a Saturday morning test prep for six weeks prior to testing. It is very generic--not targetted to any particular student needs, and as far as I can tell, the district's goal is to accomplish a huge turn-out, rather than to identify kids with specific needs. Frankly, I am of the opinion that if the need is that broad, then we are not talking about intervention at all--we are talking about a need for school reform. But that is perceived by teachers to be a punishment levelled at them, or their school.

Further--Diane's accusation of test data being misused in this way (to punish)--follows from a description of something totally opposite. Manipulating the data (if in fact that is what has occurred) to make things look better than they are, is not an example of test data being used for punishment, but the opposite. Again, if the charges are true (and I have all kinds of respect for Jennifer Jennings, but to take her analysis as substantiating that the tests have gotten easier--well, not so sure that's there), this would be an example of the data being used to reward, not punish.

Agree, data is being used in all sorts of ways.... by some to avoid punishment...usually at the district level, by some to punish...usually staff and kids in very poor ways. Principals are being fired or removed, teachers are being forced to move toward "drill and kill" practices.

Many schools seem to be providing those "shot gun" boot camp like pre-test strategies that you describe.

Over all our use of data inside and outside the school systems is at a very low level. Data used wisely can be a major help to schools and kids... but in its current form it is doing more harm then good. We all have much to learn in this regard.

I believe it is time to step back and re-think our current attempts at reform.
The system needs to be transformed not reformed. I continue to also believe that public schools have an important function in our society.

To that end, i find myself drawn more and more toward a system that would not allow some districts to function at poverty rates close to 90% while others that are very near by have very little poverty at all.

Kahlenberg said that research starting with the 1966 Coleman Report, a seminal study led by sociologist James Coleman, consistently shows that the biggest influence on student achievement is the socioeconomic makeup of his or her school—followed by the socioeconomic status of the student's family.

A recent national study, Kahlenberg said, using data from the No Child Left Behind program, found that low-income kids in high-poverty schools—defined as having more than 50 percent of students eligible for the free or reduced lunch program—were performing on average two years behind low-income kids in low-poverty schools.

And integration based on income can yield racial integration. African-American and other minority students are almost three times as likely to be low-income as white students.

For example, among fourth-grade students nationally in 2005, 24 percent of whites were eligible for federally subsidized lunch, compared to 70 percent of African-Americans and 73 percent of Latinos.

Furthermore, poor blacks in particular are more likely to attend high-poverty schools than poor whites.

The Civil Rights Project at Harvard University found that in the 2003-04 school year, 76 percent of schools with minority rates at or above 90 percent were high poverty, compared with only 15 percent of schools with minority populations at 10 percent or lower.

Wondering.... why this is rarely mentioned in the dialog concerning school reform?

Mike: "i find myself drawn more and more toward a system that would not allow some districts to function at poverty rates close to 90% while others that are very near by have very little poverty at all."

Perhaps you would lay out for the kind folks here what you have in mind.

mike--I think that it is rarely mentioned for the same reasons that we have a really hard time discussing the fact that schools are segregated--primarily these days by means of income, with correlations to ethnicity and parent education. Not only to those that have carry a load of guilt at having more that they rationalize--but they are loathe to give it up. Listen for awhile to the gifted student lobby carry on about why their kids deserve additional funding and separate classes. There are shrill voices depicting the children of the poor as "disrupters" who don't really want to learn, or who lack proper home trainiing in socialization--or are just poor victims of poverty who can't be helped very much by schools.

We do not have a culture that accepts responsibility in social ways. We are very individualistic, believing that those who succeed do so because of some factor of being deserving. There has been a flap on the news about a School Board that cancelled graduation ceremonies over a cheating scandal in the senior class. Apparently the cheating was widespread, although not universal. No diplomas were withheld--so no lasting harm was accorded to any who might be termed innocent. But the uproar has been widespread. Many cry loudly for the "right" of those who did not cheat to be celebrated in the traditional fashion. Even those who might have known but did nothing are regarded as "innocent."

I have spent a good bit of time in group work situations--where we put a high level of emphasis on the unity of the group. In those settings, not only would it make no sense to single out only those who received benefit under false pretenses, but it would be inappropriate to celebrate an unresolved state--regardless of the date on the calendar. But, in our American-ness, we have a hard time looking at things this way. We are an every man/woman for him/herself kind of place--and have a hard time accepting the idea that this hurts us all in the long run.

I think that it is this limitation to our thinking--rather than the measures provided by the tests--that leads us to casting improvement attempts as "punishment," as well as to doing some of the silly things that we do in the name of improvement. What is the solution to truancy? Why, punish the parents, of course. How do we make students "behave" in class? Punish them with suspension or expulsion (if we cannot paddle them), or punish their parents--send them to court, send them to Saturday school, send them to jail.

In my experience, as well as according to research, these things don't have the intended effect--but we are deeply wedded to them, because they put the focus on fixing the individual, making them want what we want them to want--which is to do what we want them to do--with as little impact on those around them as possible. This is not terribly different from all of the blind alleys gone down, in the hope that scores will go up or that administrations will change and they will go away. Eliminate recess, only teach reading and math, start earlier. These things don't work--and we know that they don't work. But doing them allows us to point back to the futility of doing anything that might rock the boat of our beliefs.

I guess you've caught me on a cynical day. Unless we get serious about wanting to change--to build more equity into education--it doesn't much matter what measures we use.

Diane, thanks for your excellent article. Fortunately there is good news on the horizon. The media are finally catching on to the widespread cheating that is prevalent with high-stakes testing. Lately there have been several articles questioning local scores (See "Leader or Cheater?" Colorado Springs Independent, June 4).

I've had several experiences that have helped me understand the testing situation as you have described it:

Several years ago I walked up to a state senator and complained, "Everyone is just cheating on these tests!" Without missing a beat, she responded, "Oh, we know that."

At about the same time a very honest and intelligent sixth grade teacher shared her testing concerns with me, saying, "Even though I'm drilling the kids on the word 'hyperbole' every day, I know some of them will get it wrong." When I told her she was invalidating that item, she looked perplexed and answered, "But if I don't teach it, how will the children know it?"

My third experience occurred just the other day. While riding in a car with my 31-year-old son and his professor wife, I started complaining about testing. He and his wife both hold doctorates from prestigious universities. Both of them disagreed with me. "I never knew what was on those tests," said my son (who always scored at the 99th percentile). "James, don't you remember how you'd bring copies of the test home so you could practice?" No, he did not. (To be fair, no one would be that blatant nowadays.) His wife said, "I know they don't do that with the SAT." No, indeed they don't.

And so I believe that politicians know what is going on and they know that their constituents don't know. Ambitious people (Bloomberg, Rhee, Klein) are probably using the information about "miraculous" improvements to advance their careers. When Michelle Rhee bragged that her students went from the 13th percentile to the 90th, she knew that most people would believe this - and she was right. Did anyone even ask for documentation?

Many teachers do not understand the concept of sampling, especially if they never took a course on testing and evaluation. Also, many of them, like my teacher friend, are told by their principals to drill the children on the test items. Many citizens, even highly educated ones, just do not know what is going on with these tests. My guess is that most journalists just accepted at face value the pronouncements of superintendents and mayors that test scores had risen dramatically.

Of course, the victims have been children. The public has been fooled to believe that simplistic and inexpensive measures (i.e. fire the teachers, give frequent tests, demand uniforms)will take the place of expensive solutions such as health care, infant monitoring, preschool, highly qualified teachers, interdistrict permits and so on. Hopefully with a new administration that has acknowledged the importance of non-school factors, we'll begin to see some authentic reforms.

Thanks for the responce... very well said and you describe it very well.

I think that it is rarely mentioned for the same reasons that we have a really hard time discussing the fact that schools are segregated--primarily these days by means of income, with correlations to ethnicity and parent education.

Not only to those that have carry a load of guilt at having more that they rationalize--but they are loathe to give it up. ( Margo/Mom )

It is my belief that it is time to put it back into the mix...

Ed... here is a sample but what i believe to be important is to take this issue and put it under the light of day.
Not opposed to charters and magnets being part of this broader and deeper attempt to transform our system.

Here is a sample:
from Richard D. Kahlenberg

Numerous studies subsequently confirmed Coleman’s findings that con-centrations of poverty tend to defeat good education programs. And yet, until recently, most school districts consciously ignored the research, worried that “busing” to mix rich and poor kids would be politically toxic.

Spurred in part by increased state and federal pressure to raise overall student achievement and to reduce the achievement gap between groups, a growing number of districts are pursuing policies of socioeconomic school integration.

The list includes Wake County, N.C.; San Francisco; La Crosse, Wis.; Cambridge, Mass.; St. Lucie County, Fla.; Rochester, N.Y.; San Jose, Calif.; and several others.

Most of these districts rely primarily on a system of magnet schools and public school choice, rather than compulsory busing, to achieve their goal of socioeconomic integration.

In Wake County, for example, an extensive system of magnet schools in the city of Raleigh helps the district reach toward its goal that no school have more than 40 percent of students eligible for free and reduced-price lunch or more than 25 percent reading below grade level.

While most of these programs are fairly new, the early signs are promising. In Wake, for example, low-income and minority students perform better than low-income and minority students in other North Carolina districts that fail to break up concentrations of poverty.

In 2005, on the state’s High School End-of-Course exams, 63.7 percent of low-income students in Wake passed, compared with 48.7 percent in Durham County, 47.8 percent in Guilford County, and 47.8 percent in Mecklenburg County.

The research suggests that while it is possible to make high-poverty schools work—we all know of such schools—it is extremely uncommon. An Economic Policy Institute study, for example, found that middle-class schools (those with fewer than 50 percent of students eligible for free and reduced-price lunch) are 24 times as likely to be consistently high performing as low-income schools (those with 50 percent or more of students eligible for subsidized lunch).

Not to burst your bubble, but I just looked up my own district's level of reported poverty (free and reduced price lunch). We are hitting 98%. A quick run through report card info isn't coming up with any buildings lower than 97%. There isn't any way to dilute this without some merging with suburban districts. And, yes, it probably is too toxic to deal with. We have an incredibly complex set of school district boundaries based on legislation that at one time prohibited cities in the state from annexing any newly developed areas into their school districts. As a result, many newer areas of the city were annexed into suburban districts. This legislation had a sunset clause back a couple of decades ago and the suburbs and near suburbs had a panic attack for fear that their kids would be moved into the urban district. A facilitator was hired to bring the districts into some kind of agreement. I think that the district is still receiving some pay-off money that allows kids on the fringes to continue to live (and pay taxes to and receive services from) in the city but be a part of the suburban school system.

More recently, there was a move on the part of a county housing agency to build low-income housing outside the city--in one of the other districts. Again--public outcry, with regard to keeping those folks where they "belong" (as if this small town cum bedroom community has no low income folks of its own)--and particularly the potential of upsetting the apple cart for the school system--public housing not yielding the same property tax base as residential owner-occupied housing.

We have a whole lot of notions about where certain folks "belong." And when it's kids that we feel threatened by in some way, where they belong always seems to be somewhere else.

"We are hitting 98%. A quick run through report card info isn't coming up with any buildings lower than 97%. There isn't any way to dilute this without some merging with suburban districts." ( Margo )

In my opinion.... this issue is the elephant in the room that is rarely even mentioned. Yet again, the research is clear and goes back to the Coleman Report. By the way, you describe perfectly what just about all our urban area's went through and is talked about at length in: Gerald Grant's book...Hope and despair in the American city ( Why there are no bad schools in Raleigh ) Well worth reading.

2 other key points by you:
1.back a couple of decades ago and the suburbs and near suburbs had a panic attack for fear that their kids would be moved into the urban district.

2. More recently, there was a move on the part of a county housing agency to build low-income housing outside the city--in one of the other districts. Again--public outcry, with regard to keeping those folks where they "belong" (as if this small town cum bedroom community has no low income folks of its own)--

What i am suggesting is that these conversations and issues need to become a major part of the discussion concerning school reform in America.

"We have a whole lot of notions about where certain folks "belong." ( Margo )

I believe we need to put these issues and others front and center... let people speak to them so we can see what the real issues are...instead of just focusing on "band-aide" solutions for a minority of children. Let our govenerns, amyors, education secretary, and many others ask...

Are public schools that serve student populations with extreme levels of poverty in the best interest of our country?

Will some richer communities continue to create ways to get around fair housing laws?

Not naive... but is it not time to begin to have this conversation and many others as we so intently want to prepare our kids for the 21st century?

Is it OK for public schools in America to have poverty rates as high as you describe Margo?

Traditionally affluent school districts have fought hard to prevent impoverished children from transferring to their schools. However, in many states, this is changing. The economic situation is actually helping because many of these high-achieving districts need the money that the out-of-district children bring to the receiving school.

In my opinion, getting low-achieving children out of failing schools and into districts with high-achieving children, is the best way to accelerate the learning of the low-achievers. Yes, there is a lot of resistance on the part of the affluent parents, but if decisions are made with them, and not in secret, hopefully they'll want to help children who are less advantaged than their own. The American people are a generous people, if given the opportunity.

The worst thing for poor children is to be trapped in high-poverty, low-performing schools.

In this day and age, by far, the poor and those with special educational needs receive the lion's share of federal resources. That is not equality.... that is fairness.

The leadership within many of the various minorities so frequently mentioned on this blog are calling for a boycott of the 2010 census. The ONE Constitutionally mandated procedure for directing national revenue to where its needed most. Why do minorities keep biting the hand that feeds them? Because they are experts at making the fortunate feel guilty. That is the new industrial revolution, the new cottage industry.

All of the data clearly points out that minorities and the poor (who most often are the same demographic) do not achieve academically as well as the white, middle class. The possiblity is that all this data may very well be proving that minorities (as a whole) are less capable .... or better yet.... less willing.

The same types of issues are creating similar problems in the emerging economies of the World. Brazil, India, China, and many Southeast Asian countries are reaping the benefits of globalization without having to go through the serious societal pains of the three major revolutions which occurred in Western Europe and North America - Industrial, Agricultural, and Technological (Medical). Now, they have a rapidly developing society WITHOUT changing the cultural norms for producing offspring. Result: Overpopulation without the infrastructure to deal with all the newly created problems.

Judge Sotomayor's decision in the New Haven, CT. firemen's case is proof that the data is being manipulated to somehow prove the system works against minorities and the poor. It is likely her decision will be overturned... WHY? Because the test was not a problem until the results showed the white applicants performed better. Talk about the end justifying the means.

Could it possibly be that the whites studied/prepared/worked harder?????

And.... what would be wrong if the whites achieved higher scores and DIDN'T have to work harder?????

My brother designed the computer programs for the Hubble Space Telescope... when that bored him at age 40, he went back to college and earned straight A's in medical school and became a neurologist. I will not ever achieve that. It is folly to think that just because we're both human and that we both live in America, that somehow we have the same potential, deserve the same recognition, or deserve the same attention, or deserve the same compensation.... that would make us equal..... that would be Communism.

The data does not indicate that being poor or being a minority means one's rights are being violated... just that the minorities and the poor do not make as many achievements. So... the question becomes:

SO WHAT???

The only people who can change that are the people who find themselves on the wrong side of the gap. Schools CANNOT EVER change the culture. Particularly if members of the culture DO NOT want it changed.

Think about this one. President Reagan offered amnesty to the illegal aliens in this country. The result: 3 times as many people entered the country illegally in less than 40% of the time that it took to create a the world's most generous naturalization process in 1865 until the day President Reagan worked his "magic." Why do so many claim the "counter-culture" is the fault of the culture?

Free and reduced lunch???? Some of the fattest kids in the schools qualify for free and reduced lunch... that makes NO SENSE.

Here are the biggest lies of all....

1. All Americans belong in academic settings
2. All Americans can achieve acceptance into college
3. All Americans deserve to go to college.
4. The government's job is to create the PERFECT living circumstances and/or opportunities for every individual.

Do suburban children count at all in these considerations? Is there any data out there describing what happens when district boundaries are changed, kids are bused back and forth between "poor" schools and "rich" schools? Are the non-poverty-stricken children just anonymous pawns to shuffle around in order to achieve someone's idea of equality?

I support the ideas behind public schools, but the lack of realistic considerations why suburban parents might panic at the thought of their children being shipped to the inner-city makes me wonder if I should plan to homeschool my three kids.

What you failed to mention is WHY the affluent parents do not want the poor kids in the school. Their poverty is not the reason. Their lack of manners, proper conduct, proper repsect, and tendency toward unprovoked, violent behavior against those who are more fortunate than they are the reasons. And... having grown up affluent to have my family lose it all and become poor..... being poor is no excuse for angry, anti-social, threatening, illegal behavior.
How much more generous are the American people expected to be? The middle class already pays for the schools in the high-poverty areas. The middle class already provides many of them with a roof over their heads and two meals a day. Unless we are willing to remove the poor kids from their parents..... the justified fear will and should remain.

I suggested that impoverished children be allowed to request transfers to affluent school districts, but no one would be forced to do anything. Many low-income parents prefer to keep their children near home. I'm sure affluent parents would not request a transfer to the inner-city. I am not talking about busing, but rather a voluntary transfer situation, where a child could apply to attend an affluent school and that school could accept or not accept the child. It definitely could not be forced because we tried that and we know how it turned out.

Another idea would be for the federal government to sponsor magnet schools on the outskirts of big cities. These schools, (science, music, etc.) would be designed to attract children from all socio-economic backgrounds.

I taught poor children for almost forty years. The vast majority are sweet children who are well-behaved and want to learn. It is in everyone's best interest (economically as well as otherwise) to give these children a fair shot at a good education.

Here's a test (borrowed from "LitStart"): Read the Sentences and answer the questions--

"Jan bought a new stecker at the hardware store. She needed a stecker for her minkle. Everybody knows that a minkle won't dreep if it doesn't have a good stecker."

What did Jan buy?

What did she need it for?

What would have happened if Jan hadn't bought it?

Testing only measures the number of questions a student can answer correctly (or hand-wave about sufficiently to get partial credit) on some random Tuesday in March. Just because a student answers questions correctly, it doesn’t necessarily mean s/he understands the material, particularly in math. (The opposite is also true--sometimes a student knows the material but doesn’t test well for some reason.) Certainly there is a correlation between test scores and (real) proficiency, but it is NOT the absolute litmus test we take it for. One of my daughters faked her way all through 5th-grade math but still scored at grade level on the state test. What she actually learned in math over the course of the year was appallingly little, and what she retained was even less.

As a math major at a large university (where the foreign grad students outnumber the Americans in math), future teacher, the parent of two middle-schoolers--one of whom I homeschooled for a year and both of whom I tutor in math--and researcher on educational issues, I have seen the subject of math education ‘up close and personal’ from all sides.

Under pressure to produce good test scores, the teachers increasingly teach 'recipe math' ("Just learn these steps to get the right answer...") and the students memorize the recipe for producing the right answers to different kinds of problems without necessarily understanding the material. Even if they understand it more or less, they cannot apply the concepts to a problem unless they have seen one exactly like it.

By the time many of the students get to high school and they try to do more complex math that builds on the basic concepts, their weak foundation crumbles because they never really knew the material to begin with (regardless of having 'passed' the state tests in math at the lower levels.) This is born out in comparisons of high school students in the US versus their international counterparts.

One day in one of my upper-division math classes this past semester when we had a particularly challenging homework assignment due, we were waiting for the professor to arrive, and one of the (American) students bemoaned that she was ok with problems when she had seen something like them before, but if a problem was unlike anything she had ever seen, then she had no idea where to start. (And this was a college math major...)

This inability to problem-solve on a higher level is common, and it is becoming more pervasive as students are taught how to produce the right answers to specific types of problems (a skill they will not retain) rather than to think mathematically (an ability that would translate into critical thinking in all subjects.)

Teaching to the test that is invariably happening to the extreme in the districts described in the article is actually harming the kids in these schools (as opposed to just wasting their time) as it short-circuits their ability to learn how to think critically. Even my average middle-class suburban school district, which does not spend all year doing blatent test prep, has now eliminated the most rigorous AP math class because they no longer have enough students qualified to take it. (There were only a couple of kids even interested in signing up.)

The high-stakes testing movement has been a shot in the foot for math education rather than a shot in the arm.

I live in the inner-city. This is where I have chosen to raise my family. The really eye-opening experiences have not been the kids that my kids have experienced, but rather the degree of hostility towards kids (and parents)that I have at times (far too many times, I am afraid) experienced within the schools. Playing on people's fears for fun and profit is certainly nothing new. I recall block busting and how it operated during the 1960s. Someone, by chance or design, brought about the sale of a house to the first black family experienced in a neighborhood. Shortly thereafter, other families would receive inquiries from realtors about whether they were interested in selling their house, before the values dropped. Of course, this led to panic selling and whole communities changed demographics overnight. I also recall that there were some principled groups (my contact was through church--but I am certain that there were other organizations). There were discussion groups and films to discourage such panics, with honest presentations of the facts. My own community acted legislatively to ban the posting of for sale signs in front yards (and later moved to see that any school redistricting efforts avoided segregated buildings).

So, in that vein, Jennifer, here are the facts. Any research that I have ever read on the topic (and there has been research) has determined that attempts at mixing students across SES, or across ability levels (or by ethnicity) tends to benefit those at lower levels without any harm to those at the top. You will hear much to the contrary--as Rory demonstrates. But, be careful what you listen to--evaluate the source. Temper your fears with facts.

I appreciate your reply. I'm in my 30s but my parents grew up on the southside of Chicago and experienced some of the racial changes in the 60s you describe. I grew up in the suburbs and experienced dramatic negative changes in high school when they redrew district lines and bused urban students in. That is one reason I am interested in any data out there. Personal and anecdotal experiences from people who were in different regions are more often negative. I've also volunteered in very disadvantaged schools (East Palo Alto in the Bay Area) and agree that the young elementary school children were wonderful to work with.

But I worry when I read comments criticizing suburban segregation or the parents who are not eager to have 'high-achieving' children mixed with 'low-achieving' children. These parents are easily dismissed as racist or selfish, but are usually just reluctant to have their kids be guinea pigs in a sociological experiment. Often they were products of low-achieving school districts and the stereotypical negative urban community, and worked hard to place their children in a different environment.

I worry when I see some groups of children treated as a means to some end. There is a lot of attention given to the disadvantaged - and there should be. But the better-off children matter too and too often there is an attitude that "oh, they can fend for themselves" or worse that they don't deserve what they have/should be less selfish and accept a worse situation for the sake of equality.

Jennifer,
I have taught for 17 years in districts where, as a white male, I have been the miniority. I have been all over the research on such issues. I have, until the last few years, taught Seriously Emotionally Disturbed students who were being "mainstreamed" thanks to the Feds.... Talk about social "experiements."

Margo/Mom is correct AND incorrect.

1. Anecdotal information has been proven to be just as valid as any statistic. Why? All of it, regardless of type, can be manipulated to prove just about any point. The difference between anecdotal data and "statistical" data is that anecdotal is assumed to have more bias because it cannot be categorized as easily. However, researchers at Johns Hopkins Univ., and others, have proven methods to diminish this presumption.

2. Margo/Mom should certainly know that there is significant STATISTICAL data about violent/often criminal behavior in urban schools... and its effects on student achievement. Nationally, minority schools experience far more of such activity. The STATISTICS also prove that whites, when they are the minority in a school, tend to be the victims of a high percentage of this negative conduct.

3. Studies have shown white students, who were part of the majority in elementary school, but who are minorities in their middle and high schools, DO SHOW declines in their achievement. The scary part... even though there is decline, they STILL fare better than their cultural minority peers.

When the data is not shown on a trend graph, it can be "interpreted" (actually misinterpreted) that "low achieving" moniority students fare better, while "higher achieving" whites are "done no harm."

Linda/Retired Teacher,
I teach in an urban school district. A significant majority of our students "qualify" for free and reduced lunch. We have magnet schools within our LARGE COUNTY which is a single district (30 public high schools... three are in the nation's 10 largest with student populations of over 6,000 each).

We catch THOUSANDS of parents trying to fake addresses within our district boundaries, because the neighboring county is even MORE "urban" and even MORE of a disaster.

The magnet schools were all placed on the Eastern end of our district (the city). The Federal Govt pays millions of dollars for an extra set of buses to drive all over our county to pick up and drop off kids at our magnet schools. And better yet.... the afternoon buses run again at 5:30pm for kids who want to stay after school for tutoring, sports, clubs, or other after school programs. We have students who must catch the bus at 5:30AM.... yes.... AM .... and, if they stay after school, often do not get home until 7:30pm.... that's right... PM

The result: All the black kids from the Western suburbs and all the black kids in the poorer, Eastern, city neighborhoods come to our one school.... most parents admit, not because the kids are interested in Pre-Law, or possibly going to Cambridge Univ..... no.... its because we have the fewest number of gun incidents each year.

Our International Baccalaureate High School magnet program is about to be dissolved. Why? A school of 4,500 students with (get this) 5 principals and even more AP's, along with 3 full time police officers, is so violent.... parents with kids who qualify for the program refuse to send their kids to that school.

Rory's image of a school with 4500 students, 3 FT police officers and so much violence the intended students won't come. Why?

And Margo. Whatever happened in the 60's, that was 40 years ago. Why do northern cities remain so segregated today? If the jobs and money all moved to the suburbs, as is so often cited, why haven't urban Blacks followed them over the past 40 years?

The African American population here is 0.25%. Broaden the circle a bit and it grows to a whopping 4%. Yet move the circle to Cleveland and it's 51%.

Is this the government's fault? Is it something government can fix?

Housing units sit empty here. The law says if you want to move in, you can't be denied.

The schools here are decent if a bit impecunious. They're certainly safe. The neighborhoods are quiet. (Where there are neighborhoods).

When test scores go up dramatically, the leaders should investigate the gains carefully before proclaiming the success of their reforms. They should do what Jennifer Jennings did: compare tests from one year to the next. They should interview those who scored tests. They should ask for insights from teachers, students, parents, and principals. They should look at a sampling of actual students' tests to see what sort of work gets a passing score.

Instead, they set a tone of pomp, and schools follow suit. Schools, too, want to look good; if their scores have gone up, they, too, want some of the acclaim. For all the "inquiry teams" across the school system, schools are naturally averse to the sort of inquiry that would call their test score gains into question. No one wants to burst the bubble. No one wants a bad grade on the report card. And so everyone is drawn into the success pageant. When all the confetti has fallen, we see that not too much has changed; there's just more paper to pick up.

We need leaders who set a different example: leaders who are more concerned with the substance of education than with PR, leaders who strive to come closer to truth, even if it means less glory for them. We need leaders who can listen to the concerns of others, who can express their own doubts honestly.

This is not to say that drastic improvements can't happen and that they are necessarily false. But when there are clear contradictions--when students who pass the tests can't form clear sentences,spell simple words, identify key historical events, or solve basic problems--we know something is amiss.

One of the problems i ponder is an over focus on test scores and looking for better and better ways to measure learning by using tests.

Found this over-view- from...

TEST SCORES, POVERTY AND ETHNICITY: THE NEW AMERICAN DILEMMA
Donald C. Orlich and Glenn Gifford
Washington State University

A Short Review of Published Literature
Ronald C. Nyhan and Mohamad G. Alkadry (1999) statistically analyzed the relationship of class size, expenditure per student and socioeconomic status on student achievement test scores in three south Florida counties. Poverty was the primary determinant of student achievement.

A parallel finding was also reported when English, Scottish, Welsh and Irish student test scores were analyzed (McCallum and Demie, 2001).

Mark Hornbeck (2001) wrote that one in five or about 600 Michigan schools would fail to meet that state’s standards. Again, there are correlations with family income.

Similarly, Alan
H. Schoenfeld (2002) provided data showing that economic status has a negative learning impact
on poor children and children of color.

Alan Gottlieb (2002) discussed research commissioned by the
Piton Foundation. Results showed high concentrations of low-income children in Denver’s
neighborhood schools kept their achievement levels below what they should be. Data illustrated
that low-income elementary school children in Denver performed “. . . significantly better on
standardized tests when they attend schools where fewer than 50 percent of the students are
poor” (p. 1).

It goes all the way up the testing ladder:

The ACT and SAT

Similar findings tend to be found when examining SAT scores (Fleming and Garcia,1998; Adelman, 1999-2000; Nairn, 1980). Data published in 2004 regarding the SAT scores of college bound high school seniors form a linear function in which there is a very strongly positive correlation between parental income and students’ SAT scores (Fair Test Examiner, 2004).http://www.fairtest.org/nattest/ACT%20Scores%202004%20Chart.pdf.

Poverty is a powerful force in educational deficits, and you will not find advocates of high-stakes testing discussing that social issue, including Achieve, Inc. or The Business Roundtables.

One simply has to ask,
“Why the silence?”

One could speculate that Americans have
been subtly conditioned and misled into believing that childhood poverty is really not our social problem.

Yes, there are high-poverty schools that seem to beat these odds...yet to not look at the data and recognize that children living in area's with high levels of poverty also are at-risk of living in area's with limited “Social
Capital.”

Some people think that these NCLB tests are so poorly constructed that the scores are virtually meaningless. I guess New York proves the point.

Tests measure the content of the curriculum by measuring a random, representative sample of the content taught. The "random" requirement means that any part of the curriculum could be on the test and no student or teacher knows before the test which items will be selected this year. The "random" feature allows people to say that if a student earned 70% correct on that set of items, the student probably learned about 70% of the entire curriculum. However, if the test doesn't measure a random, representative sample, the test cannot be valid measure of the curriculum. Scores can be calculated correctly, but the interpretation is meaningless.

Jennings' analysis proves that, for the past few years, nothing that has been said about education in New York based on these tests means what data-driven educators and administrators have imagined. This includes everything that has been said about NCLB, AYP, charter schools, teachers' unions, paying teachers, paying students, reforms, experiments, and every student's every score. Every argument based on the idea that these tests were measuring what teachers were teaching and students were learning must be reconsidered -- just as if you had found out that half of the state had been cheating for two or three years. The effect is the same and the problem has affected everyone.

The test results are not even valid for the teachers who did not try to predict the content or focus on the small slice of content tested. No single form of the test appears to be a reasonably random, representative sample of the curriculum. Given that many students had a significant advantage and many did not and we don't know which are which, comparisons between two individual students are meaningless. No statistical procedure can erase the effect of one group of students having had such a significant advantage over another. Even saying that any individual student is "proficient" is suspect because the score for proficient is dependent on analyses of the test items taken by all students and assumptions underlying those analyses have been shown now to be completely false. You can calculate a cut score using the correct equation and procedures, but that number is completely meaningless. Lost in translation.

Jennings has revealed a serious defect in these tests. This kind of problem is worse than an incorrectly scored item or misprinted score table or score report. Those kinds of technical errors can be corrected. States can exclude items or recalculate scores. The problem that Jennings has detected is more difficult to detect and, once found, impossible to correct.

Jennings has shown without a doubt that the content of the New York state tests is very limited and the difficulty of the items has been compromised. There is only one reason why no NY state legislator has called for an investigation and demanded the tests be nullified: Almost everyone's scores are up everywhere. Let the good times roll. Jennings must understand now how the guy who told the SEC about Madoff felt when his warnings were ignored.

But some good may come of this. New York has inadvertently provided Arne Duncan with the first real test of his willingness to hold states accountable. There are requirements in the NCLB law requiring that NCLB assessments measure state standards. These don't. Other regulations address the idea that test forms must be secure and equivalent. If the second and third and fourth form look just like the first and everyone knows that, they are not secure. Even if two forms had the exact same items, knowing that means the forms cannot be statistically equivalent. Any test you have already seen must be much easier than a test you have not.

These tests clearly violate the letter and spirit of NCLB. What will Duncan do?

Since he took office, Duncan has been pushing New York to connect student test scores to teachers. Maybe the state legislators who outlawed use of the data understood the problems with the tests. Perhaps they realized that a merit pay program would trigger a legal challenge drawing attention to the fact that the accountability system had become a sort of fraud.

Maybe some other state should file a federal lawsuit charging that it has been unfairly denied its share of Title I funds because other states like New York have knowingly allowed a kind of institutionalized cheating. Why should Arizona have to use secure tests that attempt to measure state math standards if New York can just use the same 14 math problems over and over and over again?

Or, maybe all of the states are doing it. About a year ago on his Dayton Daily News blog, Scott Elliott reported on an Ohio teacher who had been accused of cheating. Her students had said that, just before the NCLB tests, she had prepared them by reviewing test items. Those items looked just like the ones on the test. The students thought they were cheating. After a review, the teacher was found to be innocent of cheating. The items were not exactly the same as the items on the test. Not much more was said about it.

anon has a lengthy post, and it makes me wish again that NY Daily News had published a link to some actual work by Jennifer Jennings. As I have said previously, I have a lot of respect for her. But, I have many more questions, in general, about journalists reporting on statistical matters. Included in the news report is the information that only 54% of curriculum had been tested over a three year span. This brings about OMG's from an audience already prepped to discount state tests as "meaningless." But, one really has to ask, in statistical terms, what percentage of the curriculum must be tested annually in order to achieve validity (that is a sound relationship between the test and the curriculum) and reliability (that is, a reasonable ability for the same instrument to produce similar results, given similar conditions, repeatedly). These are things that Jennings is up on. I would suspect that the average reporter for the NY Daily News is not.

anon suggests that in order to be statistically meaningful (skirting the troublesome words reliable and valid), a test must offer a RANDOM sampling of the curriculum. This is not a concept that I have encountered in anything that I have read previously--but, it is an interesting idea--and seems to be the one that Jennings is hanging her hat on (as reported by the NY Daily News) when she reports (or is quoted as reporting) that the tests have gotten easier due to a predictability factor. I rather suspect that the predictability factor (or getting used to the test--as an MSU prof termed it in the article) is likely to bring about a slow increase in scores, rather than the rapid upticks reported. It is certainly true that a test may be considered valid without randomization. Consider, for instance, the driving tests administered by states, or the GED tests used to determine equivalency to a high school diploma. These are both widely accepted as valid measures of the things that they purport to be measuring--yet the content is assuredly predictable.

But--even the idea of rapid upticks is something to be expected in a press release from an administration trying to gain public trust. What has been dropped in this consideration (although Jennings, and Aaron Pallas, have dealt with it in the past) is whether this enormous uptick is in fact a dramatic improvement in scores--or the result of a large number of "bubble kids" (those just below proficiency), making it across the proficiency line. The two have very different implications. The second may indicate a somewhat higher average across the board or focused instruction for those with the greatest chance of passing (perhaps not the most desireable, but certainly not cheating, and nothing new in the field of education). The first MIGHT indicate "cheating" or focus on some assumed pre-knowledge of what is to be on the tests, or might reveal some actual dramatic shift in the quality or focus of instruction.

Jennings' assertion that there are "more" questions that belong in an earlier grade is interesting, and again and OMG for test skeptics, but I am afraid that this is perhaps where she is out of her depth. Is there any confirming evidence from any actual content experts regarding this mis-alignment? What I would expect Jennings to be looking at here, would be the reports on the state's equating process. These are publicly available documents, and I have found them before on NYC's website. This is where a statistician can truly be helpful--and most of the rest of us just go bleary eyed. But, in brief, this is the technical process by which one year's scale scores are adjusted up or down to provide statistically equivalent numbers to previous years. My understanding is that this process is based on field testing of items--and the comparison of each to the previous scores who got other items correct. Gets pretty technical and I don't pretend to understand it--but I know that it exists--and that without some consideration of the soundness of that process, charges of a test being "easier" or "harder," don't have much meaning.

But, as regards actual cheating, or violations of secure testing procedures, there are in fact teachers and principals who have lost both jobs and licenses. As with any kind of fraud, not everyone gets caught. Not everyone suspected is found guilty. I believe that Indiana is currently looking into a case of someone (not students, perhaps not teachers) changing student answers. As many as 40 erasures on a single test. Profound impact on outcomes at that school. So--apparently there are some measures and at least sometimes they work. Can't speak to what happened in Dayton--but I also recall a charter school in the Dayton area that hired someone to coach kids for the test--using materials too similar. I thought that the contractor in that case at least lost the job, perhaps more.

Margo,
When a state test includes 14 out of 30 multiple-choice questions that were used on a previous state test, that suggests that students were practicing by answering the exact same questions that were on a previous test. That comes close to institutionalized cheating. Doesn't that trouble you?
When a district jumps from a pass rate of 28% to a pass rate of 63% in four years, it suggests that many more than the "bubble kids" are involved.
When 92-95% of the Asian and white kids reach proficiency, it suggests that the definition of proficiency is very much below the NAEP definition of proficiency and probably well below the NAEP definition of basic.
In my experience of large-scale assessments, gains of the magnitude recorded in NY in the past four years don't happen.
Diane

Diane--the actual information is: “In 2009, at least 14 of the 30 multiple-choice questions on the seventh-grade exam, for example, had appeared in similar form in previous years,” This is a far cry from identical questions. Does this include just the previous three years, or all years? A question appearing in similar format might indicated that every year there were questions that tested the ability to add three digit numbers vertically and select the appropriate answer from four possibilities (or provide the correct answer in a fill in the blank). This may lead to concerns about the rigor of the test (not using the ability to add three digit numbers in an application, for instance)--but does not in any way constitute cheating. Comparisons to NAEP are another issue--and one of cut scores, or performance standards. But again, not evidence of cheating. As Jennings herself has pointed out, reporting in relationship to the fixed point determined to indicate "proficiency" has its own limitations--both for NAEP and for state tests.

But the question that no one has yet answered is the one with regard to the exact magnitude of gains. What is the magnitude of the gains--in scale score terms? And what would be a reality-based expectation for gains? Is there really no improvement possible? Do none of the measureable input differences add up to any possible outcome differences? Is schooling irrelevant? Is quality in schooling a myth?

Margo
Of course we should hope and expect to see improvement based on instruction. Solid gains tend to be incremental. We should be thrilled with steady gains. However when phenomenal gains occur across an entire state, it raises reasonable doubt about the test and/or the scoring.

One hears of miraculous teachers, even once in a while a miraculous school. But a miraculous state? That defies reason and experience.

Even if the experts could come up with a distribution equation that would equalize per-capita spending how would it be possible to implement it without creating a more tyrannical force than what already exists? Who would get to define 'equal'? "Quis custodiet ipsos custodes?"

At what border would you delineate this program- a town, city, county, region, state, section, country, time zone, hemisphere, techtonic plate or solar system (and that was only spatial...)?
Equality applies to all humanity and most humans live below the government's poverty line, right? Time to divvy it up?

Diane--how "steady" is "steady" and how "phenomenal" is "phenomenal?" If I suddenly (stranger things have happened) found myself to be the parent of a kindergartner, ought I to sell my house and move to the suburbs (knowing what I know about the experiences of my actual children)--or might one hope that things could in fact improve?

I still do not know what the actual improvement has been in NY. Is the increase in scale scores as "phenomenal" as the increase in percentage proficient? Not to mention that not all "phenomenal" increases are the result of cheating, as one recalls from Jaime Escalante's experience.

Measures to try and fix inequality presuppose political control of the economic means which comes with consequences that almost certainly always lead to yet more inequality.

First of all, political distribution is always suboptimal. The political apparatus does not create wealth, it only takes it from one, pockets a chunk, and passes it on to an accomplice. Therefore, the non-politically connected will surely be abused. The irony is that kids in impoverished districts are the ones most hurt by political seizure of wealth. On balance, much of the government allocations end up in middle class and rich hands- the teachers, bureaucrats, politicians, connected corporations and students of politically connected families. Don't the stats reflect this? It is certainly true of much welfare.

The easy part of equalization is writing the plan. Getting public opinion behind it a little more tricky.

But the important questions are: where would it be implemented? By who? And how would such authority be circumscribed when total authority is necessary as part of any plan of equalization?

Here's a NYTimes article about a "detracking" experiment in a Connecticut Middle School http://www.nytimes.com/2009/06/15/education/15stamford.html?pagewanted=1&_r=1 Not surprisingly the tracking--based on test scores--served to effectively separate the better to do and whiter kids from the lesser to do and darker kids. Apparently the experiment paid off in improved engagement, behavior and grades for the lowest performers. There is now a petition circulating to end the mixed classes. Seems the parents of kids at the top kinda liked it that way.

Margo/Mom.... yes the trend toward wanting to continue to "track" is continuing to move down the grade levels. Thanks for the link.

Daniel:
The easy part of equalization is writing the plan. Getting public opinion behind it a little more tricky.

But the important questions are: where would it be implemented? By who? And how would such authority be circumscribed when total authority is necessary as part of any plan of equalization?

My reading of the distrcits that have explored this time of integration by SES seemed to be done in a county wide manner.

For example...in Hope and despair in the American City, Gerald Grant puts it this way:

Raleigh uses busing to achieve socioeconomic integration in its schools, mixing high- and low-achieving students. They seem to want to keep all the schools below 40% poverty rate.

Students from middle-class families bring with them a set of expectations, support networks and attitudes - what Mr. Grant calls "social capital" - that is conducive to learning.

"Social capital is the yeast that makes a good school rise."

Wake County is using busing policy to redistribute students by socioeconomic class.

As a result, it is achieving what Mr. Grant variously terms "the right balance," a "healthy balance" and a "workable balance" of race and class in each school.

Some highlights from Barack Obama's Speech on Race.....

Segregated schools were and are inferior schools; we still haven't fixed them, 50 years after Brown v. Board of Education. And the inferior education they provided, then and now, helps explain the pervasive achievement gap between today's black and white students.

Legalized discrimination — where blacks were prevented, often through violence, from owning property, or loans were not granted to African-American business owners, or black homeowners could not access FHA mortgages, or blacks were excluded from unions or the police force or the fire department — meant that black families could not amass any meaningful wealth to bequeath to future generations. That history helps explain the wealth and income gap between blacks and whites, and the concentrated pockets of poverty that persist in so many of today's urban and rural communities.

In fact, a similar anger exists within segments of the white community. Most working- and middle-class white Americans don't feel that they have been particularly privileged by their race. Their experience is the immigrant experience — as far as they're concerned, no one handed them anything. They built it from scratch. They've worked hard all their lives, many times only to see their jobs shipped overseas or their pensions dumped after a lifetime of labor. They are anxious about their futures, and they feel their dreams slipping away. And in an era of stagnant wages and global competition, opportunity comes to be seen as a zero sum game, in which your dreams come at my expense. So when they are told to bus their children to a school across town; when they hear an African-American is getting an advantage in landing a good job or a spot in a good college because of an injustice that they themselves never committed; when they're told that their fears about crime in urban neighborhoods are somehow prejudiced, resentment builds over time.

But I have asserted a firm conviction — a conviction rooted in my faith in God and my faith in the American people — that, working together, we can move beyond some of our old racial wounds, and that in fact we have no choice if we are to continue on the path of a more perfect union.

The reason that statistics are worse than damned lies is because statistics can seem so authoritative. You have looked at statistical reports and admit that you don't understand everything about them, but you are a smart, well-educated professional. You don't need an EDD or PHD to understand that, when a large number of test takers can predict exactly what will be on a test, those test results can't tell you much more than how well students can learn to answer those kinds of questions asked that way.

By the same account, Jennifer Jennings doesn't need to be a curriculum expert to see that many of the items on the New York tests look exactly like other items. If the state has data showing that obviously similar or even simplified versions of previous items were equally difficult, the state has bigger problems than even Jennings has suggested.

The fact is that statistics rarely provide definitive proof of anything. Statisticians have fancy ways of saying "stuff happened," but they rarely know why. Equating constants, item parameter drift, model fit, errors associated with likelihood estimates. Stuff happened. But all of the procedures were followed according to plan, so the end result must be okay. Almost everything problematic about test statistics can be explained away by a skilled statistician. A lot of data are not reported or reported with no explanation, leaving much to be overlooked.

It doesn't matter if you are using proficiency rates or scale scores; the numbers from the New York tests can't possibly tell you much about what teachers are teaching and students are learning. You could calculate the scale scores to the fourth decimal, but the data still wouldn't be useful for any sincere accountability program. As for measuring change, using multiple meaningless measures won't add up to anything of value.

Considering Ohio, do you know how the Dayton charter school consultant seemed to know what would be on the next form? Does the state release previous versions of the tests? If you studied a couple of old versions of the test you were going to take, would you have a significant advantage over a student who had not? How many points do you think you could pick up just by studying the old tests?

If you had all of the previous tests, like Jennings did, you could see if the tests had become predictable -- narrower content, simplified items, item clones, more answer cuing. Now that there is proof that such problems exist in New York, invalidating the meaning of those scores, the question remains as to how widespread this problem is.

If someone has a better suggestion for a state, speak up. Otherwise, Ohio should be next. If Jennifer Jennings won't do the job, maybe Mike Petrilli will.

Mike, in one sense it is good after 18 months to begin to hear some sense of what you think could be a solution.

However, I have to point out that Raleigh may not be the most useful of models re the solution vectors you have in mind.

Raleigh is 67% white and Asian, with a median household income of $46,600. Cleveland, by comparison, has 43% white or asian averaging $25,900. Eleven percent of Raleigh's citizens are below poverty; 26% of Cleveland's. So its gonna be a bit harder to spread that "socioeconomic integration" model to other places.

Moreover, 44% of Raleigh's citizens have a BA or higher. Its 11% in Cleveland!!

Good points Ed.... sorry it took so long for you to see a bit of my thinking... may be part of the issue with threaded discussions.

I have attempted over time to move toward solutions.... feel free to put yours out as clearly as possible too Ed... for i am not real clear on your possible solutions after 18 months either :)

The statistics you reference are good. What they have done in and around Raleigh is move from city schools to county wide schools... i believe this was done back in the 1970's.

If you looked outside of the man made boundaries around Cleveland... into the suburbs that surround our major cities it is there you will find populations of higher SES.

As we try and look at solutions, for instance, your recent post on national graduation rates.... what you will find is that our graduation "crisis" is not everywhere in America.

The highest concentration of dropout factories is in large cities or high-poverty rural areas in the South and Southwest. Most have high proportions of minority students. These schools are tougher to turn around because their students face challenges well beyond the academic ones.

Utah, which has low poverty rates and fewer minorities than most states, is the only state without a "dropout factory."

Is it time to re-think the boundaries we have created in some of Americas school districts?

As many states move toward "consolidation" of school districts, usually for ecconomic reasons, is it time to also discuss issues concerning levels of poverty with-in school districts?

anon--I don't know much about how a television works (does the color still come from lots of little groups of three color dots?), but I can still change the channel and watch. I don't understand how to carry out an equating process, any more than I can remove an appendix. That doesn't mean that the folks that I rely on to do that must be charlatans. And it doesn't mean that I cannot rely on their ability to ensure that this years version of scores are equivalent to last years verion of scores.

Back in the day, when I was a college applicant, we could be absolutely certain that word analogies were included on college aptitude test. You know: hammer is to nail as screwdriver is to ? That type of question, measuring whatever it is supposed to measure (vocabulary, I suppose, with a dash of reasoning thrown in) was on the test year after year. Did this skew the results? Were certain students unfairly advantaged if they studied the materials that the test company put out with samples of this kind of thing? Nobody seemed to think so. Answering the question correctly required a knowledge of vocabulary--definitions of words, as well as some kind of abstract thinking process to apply those definitions.

There is a huge difference between looking at scale scores and looking at the percentage proficient. The percentage proficient only looks at the student score in terms of a single point. In essence, it boils the entire test down to one True/False answer. Is the student above the point of proficiency--yes or no. This has great utility in an accountability system that seeks to hold schools and districts accountable for ensuring that a basic level of education is attained by all students. It is nearly useless in determining if the level of advancement is "phenomenal" or merely "steady."

In considering Ohio--the state does release a portion of items annually. Originally all items were released--this is a very expensive proposition as released items cannot be re-used. There was a great outcry from the teaching profession when the legislature deemed it acceptable to only release a portion of items every year. New York has been in the testing biz far longer than Ohio (or any other state, I believe). The Regents is an institution that goes back generations. My understanding is that they both release and re-use all items. I suppose that this is possible if you have an immense test item bank--as New York must have at this point. As I pointed out earlier, one could pull together all released items and memorize the answers--and bet on seeing more used than new answers on the test--and probably come up with a decent score. This would, however, be WAY more difficult than teaching or learning the right material to begin with.

The Dayton consultant who was accused, was suspected of (possibly in collusion with the school principal) breaking the seal on secure test items to gain access to the current year's test--a very different feat than what is being suggested in NY. Also very different from the principal (I had referred to Indiana--but it was Georgia) who is suspected of erasing and fixing answers after the fact.

My biggest concern is that any gain, particularly if it involves those kids we don't believe that we can teach much to, sparks charges of fraud, cooking the scores, making the test easier. What does this say about what we believe about our children?

Mike,
Thanks for the recommendation for Gerald Grant's new book. I ordered it immediately. His earlier book "The World We Created at Hamilton High" is one of my favorite books. If you haven't read it, you should.
Diane

Anon,
I have really enjoyed your contributions, and you remind me about why I hate anonymity on the internet. I would love to respond to you directly but don't know who you are!
In case you want to remove the mask, my NYU email is [email protected]
I am reminded of a New Yorker cartoon from many years back. It shows a dog in front of a computer screen, and he says to another dog, "On the Internet, no one knows you are a dog."
Diane

Thanks for the background on the cheating scandal at the Ohio charter school. I missed the details of that story. Do you recall where it was reported?

As for Ohio, the release policy presents an ideal opportunity for a naturalistic study. We should let a thousand research questions bloom.

What were the scores or gains before and after the release of forms became more limited? Was statewide progress slower after the reduction in released items? Within a particular grade-subject test, were there bigger gains the following year for items that were more like the released items (format, content) than the non-released items? Does the state monitor that and report it in statistical reports?

If one grade-subject test had more items like those released than another grade-subject, were there greater gains for the grade-subject with the clones? Was there ever a situation in which the released items looked nothing like the next test? Was there a sudden decline in performance or slowdown in gains? Or do the tests measure such a limited slice of the curriculum that some of the released item content will always be on the next test?

The potential effects are enormous. What if you had set AYP progress targets or growth goals based on the released period and then had to live with those targets when helpful hints were less available? Would progress just stall out or flatline?

Does Ohio analyze teachers using student test scores like New York city? If so, how do they account for use of the tests by some, but not all teachers? It may not be cheating, but wouldn't that throw off the curve, just like if some students in a class had cheated on a test and others had not?

I might be overestimating the effects, but you said that the teachers seemed to regard the released tests as very important. The open teacher protest over limiting the release is fascinating. We can be certain, just like in New York, that a very large number of teachers were using the released items. We just don't know which ones and what effect the use had on student performance. Wouldn't you expect that teachers who had used the forms heavily had students who performed better on subsequent tests than teachers who did not? How big an effect that is would be dependent on the degree to which you could predict one form from another. To know that, you would have to do the laborious analysis Jennings did.

Of course, you couldn't rule out that many teachers had their own personal test bank with all of the previous items saved, stockpiling items before the rationing took effect. No statistic could tell you that and I doubt anyone is offering confessions.

I don't expect you will set out to answer all or any of these questions, but I hope someone does. You have been kind enough to provide as much information as you already have. But if you know where to find more on the cheating scandal story, I would be interested in a newspaper link, if you recall where it was reported.

Thank you for the kind words and e-mail invitation, but I prefer to remain anon. The way the web works, you have probably already received 1,936,142 e-mails from people claiming to be "anon." I fear your inbox has gone full tilt. I can only imagine the e-mails written defaming the good name of "anon." Not a one is or will be from me.

Whatever I have to say, I can say here. I don't have any answers, just questions.

You raised an important issue on this post concerning validity. Reliability is easy to understand because it usually comes with a number and people put a lot of faith in a number. People who calculate those numbers don't put as much faith in them as the people who just read them, but a number has a certain power to it. Validity is hard because it is the meaning of the number. When you have a very strong case of a clearly invalid test, a test not measuring what it claims, that is worth discussing, even if it draws away attention from other important issues like poverty and social justice. I appreciate your drawing this issue to the attention of your readers, myself included.

I am flabbergasted that no one in charge in New York is treating this issue like a four-alarm fire. Shouldn't they say something more than, well, the statistics fixed it. Because statistics cannot fix this. Someone should ask New York's statistician under oath whether statistics can fix this.

And then we find out that it seems to be the same case in Ohio.

Does everyone everywhere have a copy of the test and the only schools where students are not proficient are the ones where the students don't show up for class most days or started the year so far behind that they could never catch up?

I am beginning to believe that our entire educational system rests atop a massive fraud. It is exactly like the banking fiasco and will end the same, but the damage will be worse and longer lasting.

In the midst of this, you get someone like Jennifer Jennings who is yelling fire in a crowded theater that is really on fire and the only thing that happens is the manager comes out to tell her to sit down and shut up because everyone else is trying to enjoy the movie.

I sincerely hope others like Jennings will pursue this topic. New York, Ohio, who's next? Someone on this blog must have a suggestion.

Thanks again for the kind words, but I will politely decline the e-mail. Be grateful: I may come on like a golden lab, but I'm really just a toy poodle/hound dog mix. And that isn't pretty. Woof.

You are making some pretty strong accusations here, and I am not at all certain that they hold up. For info regarding the Dayton Charter, I would check into the Dayton Daily News--they live on scandal and since it was in their area, I certain that they led the coverage. However--important to point out: the alleged test violations, were in fact violations. Codified. The perps broke the law. And they were caught. They weren't going through old tests to look for likely to appear items. They were previewing secured test items that only a few school employees were only allowed access to--and those few violated code to actually open and view the tests.

This is very different from what Jennings has suggested. And again--I would prefer if Jennings actual findings were published--rather than a newspaper interpretation--to determine the actual applicability of her research. It is a great leap from pointing out similarities in test items to the assumption that these similarities 1) can be used to advance scores without advancing learning AND 2) that this is the actual practice--particularly in those schools/districts named as having made advances too profound to be the result of increased learning.

You are right about the potential for naturalistic study. There is material there for many a dissertation. Might I point out, however, that such study has not yet (to my knowledge) been undertaken. And yet, much of this discussion assumes that there are gospel answers to some of the questions that you pose. It is entirely possible, to my mind, that what Jennings has discovered is not a fire in the theater but a red herring.

The practice of test preparation using "practice tests" of varying kinds is nothing new. Many companies hang their had on this kind of preparation for "high stakes" tests, the like of which we never blink an eye at: college aptitude, GRE, LSAT, bar and state medical licensing exams. I don't know of any test that matters that does not have a cottage industry of some kind purporting to boost scores--and I would suggest that most bank on some level of predictability in the format and content of the test. Are these folks "cheating?" And whom do they cheat? Are they in fact artificially inflating scores? Do they have an impact on scores at all--and if so, is it because their students have advanced knowledge, or merely that they are able to take advantage of the predictability of the tests? I don't expect we will have an answer based on research, simply because these companies don't release even a portion of their tests.

Popham--one of the recognized testing gurus--actually advocates for full disclosure of all test items following a state administered tests. He also seems to advocate for a higher level of predictability--to foster a better match between specific testing objectives and the taught curriculum. Again--I would point to state drivers license exams--highly predictable, and also widely regarded as valid (that is, measuring what they are intended to measure).

Determining validity, as I understand it, is largely a matter of ensuring that the test development process is a sound one, that questions receive review from appropriate experts not only in content, but also from psychometricians who are able to screen for such things as a "wrong" answer predictably chosen by others who get questions of similar rigor correct, and bias experts--who can screen out possible diversions--cultural unfamiliarity, things that muddy the waters (insulting the test taker), etc. I would guess that NY (and Ohio) has a multitude of documentation on their test development process--and further that the generally accepted processes for arriving at valid and reliable tests are being adhered to. If not, they should be rectified.

But, I don't see anyone making these claims examining this documentation. I do see a lot of amatuer sleuthing by folks who are enjoying the gotcha game. It ain't that simple. And--it ain't that hard to understand, if one has the time to go through all of the stuff and compare it to textbook descriptions of recommended practices. But then, where's the fun in that?

Yes, our entire educational system does rest atop a massive fraud, and has done so for years now, but what shocks me is that the media have not caught on. Or perhaps they simply don't care. To me, this is the big news in education; everyone is just drilling the children on the test (yes, the exact items).

During my final years of teaching, I did a lot of complaining about the testing situation and discovered that almost all educators know this (cheating)is going on and so do the politicians and the "department of education;" but strangly, the public does not know. The only thing I can figure out is that the ordinary citizens (including journalists) remember when their teacher gave them a spelling list on Monday and tested them on that same list on Friday. So perhaps they think it is permissible to do the same with standardized tests. I know for a fact that some teachers think they're supposed to drill the kids on the exact items because "the principal said it was OK" or "If I don't teach it, how will the students know it?"

And yes, I do believe that most stakeholders (teachers, administrators, politicians) want this unconscionable fraud to continue undetected.

I apologize if I have made any unfounded accusations. I am merely responding to the information you are providing, making inferences and asking questions.

For example, you say that a study of Ohio would make a good dissertation. I say that dissertations take to long. I say it would make a nice new series in the Dayton Daily news or Ed Week starting next week. You provided the background and I supplied the follow up questions. We did it together. I could not have done it without you.

But I do wonder about your last post. I have always thought of you as the mother of a special needs student who had been neglected by an urban school -- a child advocate, a "mom." I cannot fathom why a child advocate would attempt to make such a strong defense of the test prep industry. This is the same situation where you have written something and I am making inferences, but I can't make any sense of that.

Are you really such a strong supporter of the test prep industry? Do you really want to defend teaching to the test? How is all of this consistent with your concerns about poor and minority students in urban school districts being given a sound education? Will the real Margo/Mom please stand up?

Actually the driving test is a good example of the testing situation. In my state there are several different forms of the test and the person is not permitted to take his form home. Most people have not seen the test before they take it, but they do receive the booklet so they can study for the exam. For these people who study (or don't study) the booklet, the test results would be valid. But it is possible to obtain all forms of this test ahead of time if you know someone who works at the DMV. My uncle knew such a person, got all the copies, and scored 100% on the test. And that pretty much sums up what is happening at many schools. The more important the test (law, medicine) the more secure tests are. The tests that children take have almost no security.

Sorry I don't fit neatly into a box for you. Let me explain to you why I am an advocate for the tests. Prior to the standardized tests--and the requirement that scores be reported and that students with disabilities be tested and their scores reported--what I had to rely on, as a parent, was a mishmosh of teacher reported progress (often heavily imbued with behavioral "stuff" folded into the same approximation of what my kid had learned), incomprehensible and ever-changing diagnostic tests--and no indication at all of whether other kids were also being dragged along at the same low levels as those reported for my kid.

In order to make sense of the numbers, I had to bone up on things like the difference between criterion-referenced and norm-referenced tests, how to compare across years (especially when the test changes every year), what is a raw score, a scale score and a whole lot of other interesting stuff.

It was very important when I began to get some school-level data on the schools that my kid was being sent to. Turns out that kids with disabilities were doing very poorly across the board--but in the "special schools," the ones that were supposed to help my kid catch up by meeting his special needs, there was next to no measureable learning going on. This despite the fact that every kid was required to have a written IEP with specified goals and a requirement to report to parents on the progress towards those goals on a regular basis. The first year that all this went public was a big wake-up call for the district. The last time I saw such scrambling to pay attention to special ed was the year that the district was sued for physically abusing the kids in one of the "special" schools and they fired the alcoholic principal and moved in a bunch of actual supportive resources.

Now--all this is happening as teachers are loudly complaining that the tests are not "valid," that they shouldn't be trusted, let's go back to how it was before. "Teaching to the test" by incorporating drill and kill, or cancelling recess, art and music--these things do not raise scores appreciably. Why do them then? Well, it helps to garner troops for the anti-testing movement. Do a lot of stupid things and blame it on the test.

Over and over again it has been demonstrated that the things that really raise scores are more comprehensive, more child-centered, more inclusive of higher-order thinking skills. Why not do them then? I suspect that we have a lot of folks who don't know how. I didn't see a lot of it going on in the pre-testing days for my kid--I saw lots of low-level worksheets, very little cohesion, no articulation or communication from one grade to the next--in fact the "resource rooms" were "broadly graded" catchalls of kids who were making little to know progress of any kind--taught by teachers who could not possibly approach grade-level content across the number of grades and levels of need that they were expected to be "teaching."

Back in the dark ages I aspired to be a teacher. Instead, I became a back-door social worker in an agency where I worked with children in other settings. I know that had I entered a school as a teacher at that time, I would have been a walking disaster. No worse than any other beginning teacher, for sure, but just not knowing enough. What became clear to me, looking back on the road not taken, is that no school could possibly have offered me the growth and support that I garnered outside of schools. Teachers are not supported well in learning and growing. Some do. Many just learn to cope. Some leave. This is the state of education. The kids who suffer most are those at the lower end of the scale--the poor, the disabled, minorities.

So--just as I have learned to check and re-check anything that schools tell me about IDEA, I have gotten very careful about listening to complaints about testing.

I have my "mom" hat on right now so I am going to agree with much of what you've written. As a mother I also wanted accountability for my sons and appreciated the standardized tests because they gave me a good idea of how my children were doing compared to other children of the same age. In the lower grades, where there was a lot of "test prep," both my sons got very inflated scores, but as they grew older the scores became more legitimate because (in my opinion) older children become witnesses.

So many of us are upset with these tests because they are being used to mislead the public, especially about "miracle" schools. However, you are very correct in pointing out that many parents were misled BEFORE this testing craze.

I too have a question for you. I've had it for a long time but didn't have the nerve to ask. Now "Anon" has given me the courage:

We know that "suburban" (i.e. schools with a high percentage of college educated parents) and private schools are generally better than high-poverty urban schools. One of the reasons is that the suburban schools have many parents like you who keep everyone on their toes. They (parents) represent the best "accountability" for the students. Knowledgeable parents, even poor ones, generally know how to get their children into "better" schools. If such a school does not exist for their special needs students, these parents are the ones who get the district to pay for a deluxe private school. You seem to be one of these super-intelligent and assertive parents, so why have you not found a school for your son that is acceptable to you?

There are probably many answers. One is, don't believe everything that you hear about getting the district to pay for a super-expensive school for your kid. It ain't that easy. There might have been one option, geographically speaking, that might have been an improvement--and I know some similarly situated parents who were able to access it, but not through the district. They had access to another funding source that can no longer be used for this purpose. It also would have meant accepting a non-inclusive environment, something I have always hoped to be able to get away from.

The parents who ultimately succeed in getting district payment for private schools--based on decisions that I have read--are the ones who have the bucks upfront to move their kid and then fight things out in court. I don't have that kind of money. I am mostly a social worker who knows how to access things through the system.

But, many years ago--before I knew who my children would be--and not expecting to be the parent of a child with disabilities (who is?)--I chose to live in the city, rather than the suburbs, because of the things that the city offered. At that time, it offered racially integrated schools, and a pretty good system of alternative (magnet) schools. I really had some moral misgivings as well about being a part of the "white flight" to the suburbs that had followed a court order to desegregate the schools.

My daughter did fairly well through the alternative system. By the time my son started school, the court was out of the picture and begun to dismantle the desegregated attendance areas. As he began to have difficulties (in one of the alternative schools) I also discovered that students who don't do well had never really been desegregated. Resource rooms were heavily used, they only existed in certain schools (generally the neighborhood rather than the alternatives), they were racially isolating, not performing well and on no one's radar screen.

I suppose at that point I might have sold my house and moved. I knew a lawyer, however, who was a strong advocate, had children with disabilities, lived in the suburbs and had represented families across the state. Her advice was that the suburbs were not doing much better for kids with disabilities. Now--it may be that her view was somewhat jaded, as she was really dealing with all of the worst cases.

But, I am really much more of a stand and fight kind of person. My experience in advocating for my son has taught me that there is very little value to the improvements that can be brought to a single child, if the overall system is unsupportive. I have "won" many battles in the timeframe of one year, one classroom--only to have to start all over again the following year with a new teacher or principal. And we're not talking biggies. We are talking about things like following through on an IEP by reporting on the goals at report card time--as required by federal law.

The things that I have learned about Positive Behavior Support have convinced me that it is futile to try to "fix" the behavior of a single child, with a personalized plan, when they are in the middle of unrecognized chaos all day, every day.

But, I'm not that big a hero. At the end of the day, there has been many a time I would have leaped at the chance to go to that "better" school, had it been available.

For a long time now, when people said that NCLB was problematic because teachers were "teaching to the test," I thought that meant that teachers were teaching to the general content of the test. I could not have imagined that teachers were really teaching to THE test, a real copy of the test. I am still having trouble believing that, or believing that anyone would defend that practice. The adults and children may not be cheating, but the adults are cheating the children.

When people said that the NAEP results could not be used to judge states because the NAEP tests didn't measure state standards, I wondered what could be so different about reading from one state to the next. That did not seem like a very good explanation for why state test scores kept rising so quickly when NAEP results stayed flat, but I believed in the idea of criterion-referenced standards. I thought differences in curriculum and teachers and so on could actually produce these staggering gains on state tests. I feel very foolish about that. I was wrong.

I did not know that many states were apparently releasing actual test forms and teachers were using them extensively. That is an enormous difference between the two tests. In fact, you could really say that the state test shows what Johnny can read when he already knows exactly what kinds of questions will be asked on the test and the NAEP shows what he can do when he doesn't.

All of this time I thought that the state test meant more when all it meant was that Johnny had been prepped so that when he saw a non-fiction text and someone asked about the author's intention, he would choose "to inform," whatever that means. Johnny hadn't learned any standard and might not even understand the word "intention," but he had learned to find "to inform" and bubble that.

And now I feel like everyone, even some of the accountability hawks like Margo, already knew that and I am just the last to know.

And once you know you have been cheated, you want to know the whole story. You don't want to know that someone you love has betrayed you, you want to know with whom, when, how many times, and why. You don't actually want to know, but you ask anyway, because you can't help but ask. I suppose it is a side effect of the shock. So, you ask.

So, here is the last thing I would like to know. List all 50 states in the order of the biggest difference between state gains and NAEP gains since 2001. Now, identify which states release all, none, or some of every state test form. If a state like Ohio has changed its policy, note that. I don't know how you can find this out, but I would guess you could call each state's assessment office and get a quick answer. Publish that list.

I know New York and Ohio, but I want to know who else, when, and how many times. I am very sorry to say that I already understand why.

Anon,
Like you, I feel a growing sense of outrage and frustration as I see how fraudulent the state gains of the past several years have been. Even more outrageous is the complacency that greets those gains as real, and even celebrates them. Witness the report this past week by the Center for Education Policy that applauds the gains on state tests as if they represented real improvement, rather than a sick and lying system by which we fool ourselves.
Diane

Don't put words in my mouth, please. Getting to where you seem to be (and believe that I am there with you) requires several very nimble (and unacknowledged here) leaps. Make no mistake--many states, as well as NAEP (not to mention PISA and TIMSS) release either individual items or entire tests once they have been used. SAT and ACT release "sample items." Once, as a teachaer in an adult ed program, I was one of four teachers who put together a program to prepare factory workers for an aptitude test to determine eligibility for promotion. All that we had to work from were sample tests (in other words--no learning objectives, no statement of what was being assessed). From the sample tests we had to back out some assumptions of what was being assessed and try to teach this assumed material to our students. Believe me, it was not an easy task. It would have been far easier to work from knowledge of what the test was assessing than what the test had looked like in the past.

What Jennings has uncovered that my have some significance is not the shocking news that previously used forms are available--but that 51% of the items are similar in form. This is not the same as having the same answer. It is not the same as being the same question. In fact, it could be as broadly construed as the tests having been developed from the same test blueprint--which, as I understand it, is not only standard practice, but not a bad practice. Such blueprints are available on state websites if you poke around long enough. Try looking one up and see if it would help you in either teaching students, or preparing students to cheat.

What Jennings IS saying, if the press is to be believed, is that in the three years that she examined, not all possible content was tested, and that there was considerable overlap. She suggests that this has led to predictability as to the content that will be covered from year to year. This in turn (assuming that teachers also did the same comparative work, or had access to someone who did) has led to a certain "predictability." So--based on the material on the previous two tests, she asserts, teachers have predicted the material to be tested on the third, hit these topics heavily and voila, improved scores.

Now, this may or may not be happening in actual fact. Jennings' analysis shows that it is possible. Is it probable? Likely? Did it happen? We do not, from what has been presented, have any idea. Further research needed. And if true, what are the implications? The implications are that rather than covering the full breadth of material required by standards, students have been exposed more solidly to certain material presumed to be on the test, possibly to the excusion of other material (about which we know nothing at this point). The further implication is that the State of New York has willfully set up this situation--that is, have requested test preparers to limit the content and stick to predictable forms of questions.

What Linda is suggesting (and Diane seems to be willing to conflate) is a whole other can of worms--not at all related to Jennings' research. What Linda claims is that cheating by way of opening the boxes of tests when they arrive in order to set up marathon rounds of memorization exercises with students in order to boost scores is more the rule than the exception. And she further claims that everyone, including teachers, principals and even legislators--in short everyone but us poor slobs in the general public--is in on it. Now, I will admit that I see one or two news stories every year about charges of this sort. A couple of years back, I believe that there was even a newspaper in Texas using some kind of computerized score analysis to assess the possibility of cheating. Some of these apparently find nothing. Some result in firings and loss of licenses. Seems to be adequately disincentivized to me. But, if everyone "on the inside" knows, how come no one is talking? (including students--who really love to squeal on others from time to time) Do we need anonymous tip lines? Do we need better ethics screening for educators? Or is this maybe just a wishful illusion perpetrated by folks who were happier before anyone could take a peep into the results of the education system?

Please do not be so naive as to think that those who are in a position of power are not EVER going to use it to their advantage.

Please do not feign ignorance to the notion that as long as education is politicized, testing results will be manipulated by the incumbent AND the challenger.

Please be realistic about how successful the education system will be. Politicians who actually WANT the education system to funcion satisfactorily in the eyes of the citizens are likely lame ducks. Any challenger WILL FIND FLAWS and publicize them for political gain.

Lose a category in the collection of campaign battle cries???

What person has NOT done at least ONE of the following?
1. Embellished career history on a resume, or during an interview, for a job opportunity
2. Creatively presented results of a professional task.
3. Screened a potential reference to gauge that person's opinion and/or to gain an "idea" of what that person might say if a potential employer were to call.
4. Decided that getting involved was too much of a hassle, and so therefore, you didn't.

There can be no "inside" if the members of the community meet their responsibilities. There HAS NEVER BEEN a time when the public could not "take a peep" into the education systems. It is not public otherwise.

The public CREATED, and are simutaneously the OWNERS AND CLIENTS of their education systems. Citizens elect boards made up of their fellow community members who are charged with hiring a CEO to run the company. Almost every member of Congress FAILED at their constitutional job of oversight and investigation into the banking and insurance industries.... and yet.... most were, and will be, re-elected.

1. Attend the school board meetings, workshops, PTA meetings, etc. regularly and consistently.... and pay attention. Remember, as a citizen, NO ONE is the ADVOCATE for ONLY his/her own child. Every citizen has an ETHICAL responsibility for the education advocacy of every child.

2. Be versed NOT JUST on what you believe are your child' rights, but be equally as versed on EXACTLY what procedures the school/personnel are expected to carry-out (principals, guidance counselors, teachers, secretaries). BE VERSED on how often the rules/policies for some students come into conflict with other policies designed for other students and understand that the school must BALANCE the needs of the individual against the needs of the whole. This societal FACT creates, by its very nature, winners and losers unless the citizens are willing to provide a BOTTOMLESS well of funds and resources.

3. This nation was founded on the concept of REPRESENTATIVE GOVERNMENT. The citizens, through their civic participation (or apathy), IMPLICITY give THEIR CONSENT TO BE GOVERNED. Citzens, through their vote, EXPLICITY give those who are elected the RIGHT TO GOVERN. A Government is THE RULING AUTHORITY of a community. Citizens agree (even if they dissent about certain govenment decisions) to remain LOYAL to their government.

So......vote for the proper policy-makers, policy enforcers, and policy interpreters (governors, state legislatures, local school boards)... and if there are no proper candidates... become the candidate

4. NO ONE is constitutionally guaranteed a "right" to a high school diploma, to be prepared for the "workforce," or to be fully prepared for higher education.... only the OPPORTUNITY to attempt the achievement of one of these three goals.... with equal opportunity to fail at that attempt.

5. Our nation was FOUNDED on the concept of LIMITED GOVERNMENT. Realize that as long as the teachers are trying and succeeding 60% of the time, they are doing their contracted jobs. If you want teachers who do their jobs BETTER, be ready to PAY BETTER.

This nation elected a CEO who chose to hire people with PROVEN questionable ETHICAL shortcomings into his Cabinet. These "experts" receive big, 6-figure salaries and EVEN BIGGER expense accounts. Yet.... we want ethical teachers at $33,000

6. Understand that prior to this economic recession, MOST DISTRICTS in the nation had TEACHER SHORTAGES. Hmmmmm.... with unemployment at a low of approx. 4% a little more than a year ago.... with colleges reporting record-breaking enrollments....AND all the while experiencing a nation-wide teacher shortage.... SOMETHING MUST BE VERY UNAPPEALING about the Teaching "Profession"

BIG REMINDER - if you have ever so much as, made a mistake, miscalculation, or even committed a little "fudging of the math" on your income tax return.... you ARE NO MORE ETHICAL than a teacher who opened the standardized testing materials early. Why? Simply... you have prevented the government from being able to PROPERLY provide the service of education.

If you KNOW someone who has "cheated" on his/her taxes and you did not report that person to the proper government authorities... you HAVE PREVENTED the government from being able to PROPERLY provide the service of education... and... you have aided and abetted criminal behavior.

With all due respect, please be aware that you are engaging in mythological invention and logical fallacy concerning consent to be governed and the duty of taxation. As I just posted on another BD entry, here is the great Lysander Spooner on consent:

"Our constitutions purport to be established by 'the people,' and, in theory, 'all the people' consent to such government as the constitutions authorize. But this consent of 'the people' exists only in theory. It has no existence in fact. Government is in reality established by the few; and these few assume the consent of all the rest, without any such consent being actually given."

You might be thinking, “But you use the roads and benefit from society.” This logic on your part would be akin to blaming the victim. Besides, as long as government is running the show then society is an oppressive and suboptimal place, relatively speaking.

Taxation is also in the same boat. It makes no sense to exempt any person or institution, government or otherwise, from equal justice. There is no a priori right to another person’s labor or property when it comes to individuals qua individuals. Just because a group of people with guns calling themselves “government” demand your property does not change the golden rule. Taxation is theft and no one has a duty to submit to them.

Of course there are many other things to comment on in your post, like the fallacy of ownership concerning public schools or anything public, but two issues are enough for the moment.

Many contributors here point to flaws in the system of government and how those flaws have created the "achievement gaps." Some frequently cite the government, and the people within it (including teachers), as the culprits. They frequently offer solutions that include the "advantaged" taking on even greater amounts of responsibility and care of the "disadvantaged."

I often try to offer up the ideas that:

1. a lack of personal responsibility is more often the culprit DUE to the amount of "responsibility weight" already being carrried by others..... and.....

2. it is not always reasonable to expect 300+ million people to collectively be responsiblile for themselves AND every other individual who often "makes the bed in which he lies."

If we eliminated those things which you write are illegal, or better yet, are fictional, then there would be NOTHING left but the laws of nature. Everyone WOULD HAVE TO fend for themselves.

While I understand what you have posted, the information in my previous post is far more realistic than your crticism of it. According to your own theories, as presented, victims do not exist.... so how can I blame one?

The creation of government is instinctive. It is inherent in all living things. Humans have used reasonable thought, sometimes, to elevate it . Granted, more often to distort it.

I do not know your personal history. I will assume you were educated in America. So, if that is the case, the only reason you are capable of offering the criticism and alternative thoughts is due to the fact that some "government," somewhere, organized the means necessary for you to obtain the knowledge.

Your parents/guardians were your government during your childhood. You depended upon them for your survival; they expected your loyalty. There was likely very little democracy in that relationship. It was mostly dictatorial until you were capable of taking full care of yourself.

Which, brings us to what I see is the purpose of this blog.... to engage in the discussion (albeit written) of what we all believe are real problems and realistic solutions for what CAN be accomplished within the perimeters of our existence.

Thank you for supplying the "food for thought" about government. How does it relate to improving this country's system of education?

Have had a bit of an epiphany over the past several months regarding schools and segregation. Don’t know why it hit me so late in life or under these circumstances. Why do so many poor/minority families wind up in urban areas and so many middle class families wind up in the suburbs? Margo has talked about this somewhat.

I'm sure there are numerous variables that contribute to this phenomena but one has really come to the fore recently in my experience(s). Golf course memberships. I am avid golfer in my retirement. Why, when I play at the Irish country club are there 85%-90% Irish members? Why, when I play at the Jewish country club are the numbers also similar? The same holds for the Protestant country club, the Catholic country club, etc, etc. For some reason over these past few months this message has really jumped out at me. Why do people of similar backgrounds tend to congregate towards each other, whether it's at a golf course, a community, a neighborhood, wherever? Is it comfort? Compatibility? Homogeneity?

Yes, I know. Many poor people don't have the choice(s) afforded to others and that can be one of the variables I eluded to previously. But I see no hue and cry emanating from urban families wanting so desperately to get out into the suburbs.

So what exactly has contributed to the divisions in this land of e pluribus unum - out of many, one? Is Jim Crow still alive and well in most or all of America more than a half century after Brown? Or are people simply more comfortable living with others with similar likes and dislikes, similar interests, similar patterns? I don't have an answer. But if the latter is the case, how is this country ever to survive with these polarized cultures? What do these differences, polarizations, and preferences have to do with the achievement gap and how is that ever to be addressed?

If I have offended anyone with these questions/observations I apologize up front. It was not my intention. I am simply trying to expand on many of Mike’s queries on this blog.

Some of the country clubs that you suggest have arisen from outright discrimination (still permissable in such private clubs). There was a time that people who were Jewish would never be presented for membership (and if pressented would never be accepted) at such existing bastions of society. As various groups have become more economically successful, they have frequently opened their own clubs to rival those social venues from which they were excluded. The social clubs which organize floats for the Mardi Gras have a segregationist history which has developed long-lived clubs based on race--even though the lines may be less well defined than in the past. Churches remain largely segregated--having developed denominations along racial lines that have developed varying cultures and ideologies.

This is why, to my way of thinking, the districts that committed only to "following the law," of desegregation, but did not embrace any of the lessons of diversity, did a great disservice. In some cases kids developed their own new and shared culture--in others they merely co-existed.

As usual, you are quite insightful. Many of these country clubs were exclusionary in the past. Fortunately today, that trend seems to be disappearing. But it still amazes me the way many of these clubs seem to attract such a homogeneous group of people even today.

Which gets me back to my original post - why is this such a pattern and what can we do in our schools to minimize/eliminate it, if that's ever possible?

Hi Rory,
Thanks for the sharp reply. Sorry for the delay- it’s work week for me. My responses are after your quotations.
-Dan

“I often try to offer up the ideas that:
1. a lack of personal responsibility is more often the culprit DUE to the amount of "responsibility weight" already being carrried by others..... and.....”

Subsidizing irresponsibility, or anything, only brings more of it. The welfare mentality is a plague on society. Since the government has become the ‘end all be all’ it is now possible for neighbors to rob neighbors without punishment while control accrues more and more to government.

“2. it is not always reasonable to expect 300+ million people to collectively be responsiblile for themselves AND every other individual who often "makes the bed in which he lies."

Communism, the idea that each individual owns every individual, is unreasonable indeed. The consequences of such collectivist thinking are always tragic on a mass scale.

“If we eliminated those things which you write are illegal, or better yet, are fictional, then there would be NOTHING left but the laws of nature. Everyone WOULD HAVE TO fend for themselves.”

Let’s address the State of Nature argument. True, life is ‘brutish and short’ before man decides to give up the law of the jungle for civilization, as Thomas Hobbes, the great social contract theorist, wrote. Hobbes, like you, believed that society needed an absolutely privileged power that could stop any actor from threatening to put society back into a state of nature. Where Hobbes, who was big into the mechanistic theories of his day (17th century), went wrong was that he did not satisfactorily answer the question, “Quis custodiet ipsos custodes?” (Who will guard the guards themselves?).

I am sure you are familiar with Lord Acton’s Dictum “Power tends to corrupt…”. Here is the full quotation so you may have context:

"I cannot accept your canon that we are to judge Pope and King unlike other men with a favourable presumption that they did no wrong. If there is any presumption, it is the other way, against the holders of power, increasing as the power increases. Historic responsibility has to make up for the want of legal responsibility. Power tends to corrupt, and absolute power corrupts absolutely. Great men are almost always bad men, even when they exercise influence and not authority: still more when you superadd the tendency or certainty of corruption by full authority. There is no worse heresy than the fact that the office sanctifies the holder of it.” (Acton)

Are there alternatives to official authority?

People come together to form society because they discover that voluntary cooperation, sharing in the division of labor, behooves them. Assuming that the need for justice, security and a means for caring for the weak are inherent, the question is whether cooperation is better aided via the slave master’s whip or liberty.

In the market place, where property right abrogates the need for government, there naturally arises off-setting powers that mitigates the possibility of tyranny. Competition in human endeavor is unlike the Darwinist struggle in nature. Rather, it is the way of cooperation and serving needs sans coercion. If you don't serve others you go out of business. This is obviously untrue concerning government (although majority opinion tends to reign in the long run even for government).

It is unfortunate that Hobbesian thinking still reigns. The belief that government has to exist is akin to the slave mentality. Why do people accept the state? It is an institution that claims, by the force of the gun, to be the final arbiter in even its own disputes!

This fault has to do with the fact that the Founding Fathers, who recognized the brilliance in Hobbes but also noted the flaws in giving special power, did not go far enough. The FF’s, as great as they were, did not overcome the Hobbesian contradiction. The Federalist Papers are loaded with Hobbesian thought tempered by somewhat libertarian modifications. Madison’s Fed #10 is a great example. By making sure interests compete for control or favor of the government then tyranny would be impossible, says Madison. Madison thought it wise to advocate only a partial competition and, unfortunately, left the 600lb gorilla in the middle of the room unattended. Government is inherently non-competitive by its own claims.

But I hear you saying, 'But didn't the FF's use a mixture of democracy and republic to limit the government?' Yes, the FF's instituted democratic mechanisms but that has only proven to be fuel on the fire. The expansion of democracy has led to the worst wars, including Lincoln's, Wilson's and you name the 100's of others. Now groups democratically 'compete' and conspire for control of the tyrannical mechanism.

Madison, like most of the FF's, could not make that one last logical leap toward cooperative society.

“While I understand what you have posted, the information in my previous post is far more realistic than your crticism of it. According to your own theories, as presented, victims do not exist.... so how can I blame one?"

What do you mean? Can you clarify?

"The creation of government is instinctive. It is inherent in all living things. Humans have used reasonable thought, sometimes, to elevate it . Granted, more often to distort it.”

Regardless of the origin of government (I admit that war and conquest do not make up all historical data)- you would do well to remember that the mafia, sports teams and slave plantations are also natural organizations. Government’s privileged nature is its defining characteristic.

Privilege, like crime, is unnecessary for human thriving.

As already said, humans cooperate because they discover that it is in their own self interest. The market is the highest form of cooperative order yet discovered. It presupposes property and the freedom that comes with it. Government, or any other institution that makes coercive territorial claims of monopoly on the use of violence, the right to tax, and final arbitration in all disputes- even the ones involving itself, is plainly immoral.

All the basic functions that people ascribe to government-courts, police, schools, security, and you name it- can be done cooperatively.

You can just call the market order "society" because government at its core is anti-social, anti-society.

I could get into a deeper economic argument concerning market derived prices, their super-importance for the expansion of cooperation, and how the government destroys their existence- but maybe that is for another time.

"I do not know your personal history. I will assume you were educated in America. So, if that is the case, the only reason you are capable of offering the criticism and alternative thoughts is due to the fact that some "government," somewhere, organized the means necessary for you to obtain the knowledge."

I am becoming educated in spite of the attempts by the government to thwart such activity. Government has no interest in helping people think outside its perpetuation.

“Your parents/guardians were your government during your childhood. You depended upon them for your survival; they expected your loyalty. There was likely very little democracy in that relationship. It was mostly dictatorial until you were capable of taking full care of yourself.”

It does not follow that as an adult I would then need a political/economic parent like the state. Based on your thinking, how would you explain away the following contradiction in the democratic/state belief:

A) Humans are too incompetent to create society without the parental force of government, but B) humans are competent enough to vote for these political guardians, and C) these overseers are themselves human.

“Which, brings us to what I see is the purpose of this blog.... to engage in the discussion (albeit written) of what we all believe are real problems and realistic solutions for what CAN be accomplished within the perimeters of our existence.
Thank you for supplying the "food for thought" about government. How does it relate to improving this country's system of education?”

The fact that you engage the phrase “this country’s system of education” is very telling. A Soviet planner would applaud your aggregations (dehumanization) of children, assumption of unitary “system”, and uncritical acceptance of state propaganda creating the illusion that education is the primary goal of public schools.

Almost 90% of American children attend government school. Millions of kids are more or less compelled. Billions of dollars are forcibly extracted from citizens to feed these self-serving monoliths that retard children’s and, therefore, society’s development.

I am traveling now so I've just had the chance to see what Margo wrote about my comments on cheating. Before I say anything, I'd like to reiterate that I am not alluding to tests such as the PSAT, SAT or ACT. These tests are very secure, at least as far as I know.

The type of cheating that is going on is very sophisticated, and that is why students don't tell. Teachers know that erasures can be detected by programs designed for that purpose so they don't do anything blatant. They certainly no longer xerox old tests and send them home, as they did for my sons. What they do is drill the students on old tests, knowing that many items will be the same or almost the same. The other common practice is to look at a section of the test and drill the children on that section a day before or even an hour before the test is given. Because teachers might have the test in their rooms for a week, this is easy to do. If there are only 20 math problems, it's fairly easy to drill students on those problems in the days or hours preceding that section of the test. Teachers also warn one another of an unexpected test item. A second grade teacher might say something like this: "I didn't prepare my kids for telling time to the minute, but there is a question that asks them to identify 12:25." Of course, each second grade teacher would go back to the classroom and quickly teach a lesson on telling time to the minute, with special emphasis on 12:25. As a first grade teacher, I didn't administer the test, but I was a witness to the actions that I am describing. Also, the other teachers admitted their stategies to me. Some did not realize they were wrong, especially since more than one principal (and even an assistant superintendent) told the faculty that it was OK to teach specific items. (Yes!)

As children became older, they did sometimes pick up on "techniques." My good friend, a sixth-grade teacher, pointed to a boy's response and said, "You know the answer to that one." He responded with "You're not supposed to be telling me that."

Here's a good specific example of what is going on across the USA:

In my last year of teaching, our low-income school was placed on "school improvement" because our English-language learners had not made enough progress on the state test. The principal took some of us to another school to observe. We couldn't figure out what the other school was doing that was superior to our school until the other principal asked, "Who is administering your test?" Our principal responded, "Our instructional aides have been trained to do it."

"Oh, no," was the response. "You have to have a TEACHER do it." Everyone exchanged a knowing look because we all knew why she emphasized the word "teacher."

Back at our school the principal asked a grandmotherly teacher to administer the test. This woman was known for "helping" the children on tests because, as she put it, "I just know they know the answer." She meant well and probably did not see her methods as cheating. Subsequently she administered all the tests (a substitute was hired to take her class) and our English-language learners showed "significant" improvement at the end of the year. We were taken out of "school improvement" under NCLB.

I don't doubt that some of what you are alluding to occurs, because I have read about it. In the newspapers. In connection with teachers and principals being investigated and losing jobs and licenses. It's not terribly sophisticated. It's blatant, out and out cheating. Any teacher who engages in it and thinks otherwise should think very carefully about whether they would allow a student in their classroom to engage in such "strategies." They should also recall their mother's admonition that they shouldn't do every stupid thing that their friends do.

And just to be abundantly clear about what I think--NCLB does not make anyone dishonest. People make that choice themselves. I also believe that it is possible to better educate the kids that we actually have, and to demonstrate that education through improved test scores. Every high, or improved, test score is not the result of dishonesty.

I am willing to bet that in your state there is a place to report any of the nudge, nudge, wink, wink behavior that you are alluding to. I would suggest that you have an ethical obligation to do so.

It is human nature to identify with a group and then desire to "stick close to one's own kind."
EVERYONE practices discrimination. Otherwise, you would not thump the mellons at the grocery store.

Proof? Although the USA is one of the youngest countries of the World, it is also THE MOST culturally, racially, ethnically, and socio-economically diverse country in the World. Countries that are centuries older, are FAR LESS diverse. According to our own statistics, NO immigrant comes FROM a more diverse country TO the U.S.

The phrase "E pluribus unum" was NOT originally intended to be symbolic of the diversity of the people. The people of the USA were not, at the time, culturally diverse. It was "coined" (pardon the pun) by Ben Franklin to signify the 13 various "independent states" which joined together in a confederation for the common purpose of defeating Great Britain.

Leaders of each state government felt that, upon winning independence, they would continue the already established colonial practice of self-government. The loose confederation of independent states would cooperate with each other for two exceptional purposes: conducting inernational affairs and for the raising of a military in times of war.

Each independent state had its own system of currency (imagine having to exchange currencies every time you crossed state lines), its own constitution, its own complete system of government, and its own Bill of Rights.

Each independent state (with the exception of Rhode Island) had an established Christian religion to which one must vow allegiance in order to hold government position or employment. Frequently, communities would ex-communicate anyone who did not practice the established religion.

Hence the establishment of the 1st Amendment and the right to assemble peacefully and associate with whom we choose. It was included to protect Loyalists (those who were against the Revolution) from being discriminated against, from having their property seized, or from being jailed by state governments. It is, of course, also why the BOY SCOUTS, the NAACP, the NRA, Mothers Against Drunk Driving, the Society for the Veritcally Challenged, are allowed to exist.

The Founding Fathers likely had a limited sense of what this could actually mean... but we applaud them for the idea that government should be LIMITED in its scope and should not intrude upon the choices made by citizens anymore than is necessary. Otherwise, you could not create a book club, a neighborhood watch, a bowling league, or community sports teams. Do we REALLY want our sons and daughters sharing the same locker rooms at the community pool?

The immigrants who arrived in America during the 19th and 20th centuries CHOSE to move into ethnic neighborhoods for the comfort that comes with FAMILIARITY - people who spoke the language, held the same traditions/customs, could help new arrivals with the transition, and where all were trying to achieve the goals for which they made the journey.

However, by the third generation.... no one can claim that their cultural differences are being used to prevent their progress. In fact, someone who can claim minority status receives MORE opportunity and resources from the government than anyone who is a white male of European descent.

This practice still exists. Here in South Florida, more people are moving here from the islands, Central and South America than are moving here from other parts of the U.S.

Why cities? Cities hold more opportunity than rural areas. It has only been during the last half of the 20th century that a majority of America's population has transitioned from rural to city/suburban life. According to most Human Geographers, "White flight" occured more because of increased economic ability of the "baby boomers" than due to racial disharmony.

The Constitution CLEARLY puts education in the hands of the state governments (part of the compromise between Federalists and Anti-Federalists necessary to gain ratification). It has only been in OUR lifetimes that Congress has used its IMPLIED powers to start getting involved in education. It has only been during OUR lifetimes that individual citizens have used the courts to redefine the words FAIR and EQUAL. Thus.... giving politicians (such as Presidents) yet another campaign battling cry... "Elect me and I will fix education"

The truth is that the most effective form of government for achieving the goals of the community is the LOCAL government. Hurricane Wilma was our smaller version of "Katrina." (Although going without electricity in 90+ degrees and 75+% humidity for 4 weeks was harsh). The City of Ft. Lauderdale had ice, water, and food rations up and ready at numerous checkpoints 3 days before FEMA could get organized.

The national government has ALWAYS had the least effect on the daily lives of citizens. It is folly to think that will change. And as long as there are limited resources, there will be limits to what the national government can accomplish. The Supreme Court, Congress, and the President can make law, enforce law, and interpret law all day long WITHOUT funding any of it. Why educators/education advocates keep looking to the national government for proper action is beyone me.

Case in point, it would appear that President Obama has already spent his political capital and what resources the taxpayers are willing to part with until they see some actual results. His healthcare reform agenda is hitting some pretty big obstacles... number one... WHO will pay for it? I, for one, am in no mood to part with more of my hard-earned money (possible tax on benefits) just so my employer can DROP my existing benefits as a business-cost-saving measure and say, "Oh... well... you can go on the fed. government health plan."

Unless we want to do away with capitalism, there will ALWAYS be an underclass in this society. As long as the system keeps getting the blame, the less privileged will continue to have a finger of blame to point.

We are still LIVING the "Great Experiment."

Daniel,
I think we are, in a sense, on the same side... but for different reasons. Concurring opinions, if you will.

Margot: The sixth grade teacher I mentioned in my last post is traveling with me. She just read your post and reminded me that I have often talked about an idea for making these tests more secure: Each teacher would test a class at another grade level, and, more ideally, at another school. We both doubt that teachers would go for that but it would certainly improve the situation. After all, as things are now, the tests are a huge waste of money because they give us false information. I DO believe in standardized testing so I would not advocate doing away with them.

My friend also reminded me that I did quite a lot of complaining. I wrote a letter to each member of the board of education, describing the "testing strategies" in general terms (without pointing to an individual). I wrote a letter to the department of education and received no response. When I complained to a state senator ("Everyone is just cheating on these tests.") she replied, "Oh, we know that."

I was just informed a few minutes ago that testing procedures have tightened up considerably in my old district. Teachers now have to sign for their tests, which must be counted carefully. The tests have to be delivered to the office before recess and picked up after recess. They cannot be kept in the room overnight. However, my friend said that some principals are still "very lax" about these procedures. According to my friend, though, the tests are still pretty much the same from year to year. "For ten years, I've drilled the kids on the word 'hyperbole' and for the last two years, the word was on the test!"

So it does look as though things are getting better! I hope I didn't give the impression that I approve of testing "irregularities." Actually they make me very angry because I believe that misguided policy has been made on the promise of "miracle" test scores. Not only have children been harmed by this, but hard-working and honest teachers at small, isolated schools have been made out to be "bad" teachers while their less upstanding colleagues have been lauded for smoke and mirror "miracles."

I am glad that procedures have been tightened. And sorry that I jumped on you--I have generally regarded you as a good educator with a lot to contribute. I agree with you that the biggest losers when the adults cheat are the kids. There is so much misguided effort put into faulty "test prep" efforts--not to mention slick ways to cheat and convince one's self that "everyone does it," or that it's required by the system. In the end, I don't know that it even helps to pump up the scores in any meaningful way. I have watched schools in our local district slide into home via "safe harbor"--just barely making it out of SI status. It simply doesn't last. The kinds of reform that are really needed and would result in lasting improvement (things like early screening and detection of learning difficulties, followed by effective intervention, attention to the social-emotional environment, comprehensive learning efforts) just never happen because all of the energy goes into single-focus, everybody work harder kinds of efforts. The kid whose teacher was "reminding" him of what he know while he was being tested in third grade arrives in fourth with undetected difficulties, further behind than if someone had put equal effort into seeing that he had some major stumbling block in reading, or hated coming to school because he was getting beaten up on the playground every day, or spent too much time out sick because his parents couldn't always afford his asthma medication. These are fixable things that get overlooked when the belief is that everything is as good as it can be and the only way to move the scores up is to cheat (just a little bit).

Certainly there is some period of acclimatization to any test--as I recall my daughter's teachers telling me years ago when they were explaining her unexpectedly low scores on a new standardized test that the district was using. This might be expected to have a small initial impact. But, there are things that improve the state of education--in ways that are measureable, and these tend to be longer-lasting, possibly more profound in impact, and far better for children.

We are both passionate about education but we have different views on how to obtain the "best" for all children. You continue to demand that teachers be better than they are (and I guess I agree with that but I'm a little cynical as well) whereas I place my hopes in parents like you. As you know from my other posts, I see the parent as the primary educator with the peer group in second place, and the school/teacher in third place.

When I was a graduate school at Ohio State, the renowned educator Edgar Dale asked us to differenciate between "schooling" and "education." At the time, I thought he was just playing with semantics but his explanation made a lot of sense to me and I've never forgotten his words. Basically he explained that there are (were) many superbly educated people who had very little schooling (Lincoln, Edison)whereas there are many people with years of education who are poorly educated. In my career as a mother and teacher, I've seen that the parents are clearly in first place as educators of children.

That said, I basically agree with you that teachers could, and should, do a better job, but I guess I'm not that optimistic. Just as you can't depend on "the government" to do a great job with your child's dental or health care (even if it's "free") so we can't depend on the school to do the thorough job that we would like. But we can still strive for it.