Thursday, January 15, 2015

Brookings Fails to Make its Case for Annual Testing

I kind of love the guys at Brookings. They are such a reliable source of
earnest amateur writing about education. They're slick, polished, and
professional, and they rarely know what they're talking about when it
comes to education.

Like most everybody paying attention, they see the writing on the wall
for an ESEA rewrite by the GOP Congress, and the four (!) authors of
this piece would like to put their oar in for maintaining the regimen of
annual testing."The Case for Annual Testing," by Grover J. "Russ" Whitehurst, Martin R. West, Matthew M. Chingos and Mark Dynarski
of the Brown Center on Educational Policy, presents an argument that
they contend is composed of four part. And not one of them is correct.
The central foundation of the structure is that testing, standards and
accountability are discrete and totally separable. So we're in trouble
already with this argument. But let's go ahead and look at the four legs
of this stool.

Federal control of standards and accountability is unnecessary, but
the provision of valid and actionable information on school performance
is a uniquely federal responsibility.

Information on school performance in education is a public good,
meaning that individuals cannot be effectively excluded from using the
information once it exists. Because it is impossible to prevent
consumers who have not paid for the information from consuming it, far
too little evidence will be produced if it is not required by the
federal government.

IOW, local districts won't produce information because they are afraid
that someone will see it, so only the federal government can force the
production. And, the authors continue, only the feds can produce the
high-grade top-quality stuff. The argument is some combination of
"nobody else as good as the feds" and "others can do it, but they won't
unless the feds make them.

The states, they argue, are perfectly capable of setting standards and
holding schools accountable. But somehow, only the feds can get good
information. How does that even make sense? States are perfectly capable
of making a good pancake and telling if it's any good, but only the
feds can go to the store for the ingredients? How would states set
standards or hold schools accountable if they couldn't also come up with
the information implicit in each of those activities?

Nevertheless, Brookings says sternly, "If the federal government doesn't support it, it will not happen."

Note: they have made another bad assumption here, but I'll wait a bit to bring it up.

Student learning impacts long-term outcomes that everyone should value, and test scores are valid indicators of such learning.

Neither half of this sentence is correct.
The first half of the sentence is supported entirely and only in the article by the work of Chetty, Friedman andRockoff.
This is the infamous study asserting that a good teacher in elementary
school will make a difference of $250,000ish dollars in future earnings.
Disproving the study is a popular activity, made extra popular because
much of the proof is right there in the original study's own data set.
If you'd like to read a scholarly takedown, try this. If you'd like one with plain English and a Phineas and Ferb reference, try this. Either way, the study is bunk.

But while the first half is substantially wrong, but still kind of right
(yes, student learning results in stuff that people should care about),
the last half is just silly.

The authors try to shoe-horn some Chetty et al in to prove the second as
well, but it doesn't. This whole argument boils down to, "There's one
paper that shows some teeny tiny correlation between test scores and
doing well later in life."

But in terms of offering support for the assertion that test scores are a
valid measure of important learning, they offer nothing at all.
Nothing. At. All.

And here's the other thing-- even if they were a valid measure, so what?
What is the purpose of knowing before the fact which students are
headed for greater success as adults?

Many school management and improvement functions depend on annual measures of student growth.

The functions they're talking about include marketing charter schoolsand
"differentiating" teachers. They assert, with a straight face, that you
can't run VAM systems without test data, which they suggest is
important by alluding again to Chetty, thereby managing to cram two
discredited and debunked pieces of work into a single paragraph.

They also assert that test results are needed to evaluate policies that
are foisted on schools (because, I guess, the schools themselves don't
know or won't say). And they are looking out for the schools, which
won't be credited for their success (credited? by whom? who is out there
giving schools credits for doing a good job?).

Finally, you can't disaggregate data for subgroups if you don't have data.

Most of the opponents of federally imposed standards, testing, and
accountability should be in favor of federally imposed annual testing
shorn of standards and accountability.

Brookings' fourth and final point is that everybody really ought to love
annual testing once you remove accountability and standards from the
mix (if I could insert a Jon Stewart "Do tell" gif here, I surely
would).

Conservatives should love it because testing data can be used to feed
school choice. And to assuage their fears of federal oversight, the
writers offer this astonishing assertion:

And it doesn’t have to be the same test across the nation to provide
this information, or even a single end-of-the-year test as opposed to a
series of tests given across the year that can be rolled-up into an
estimate of annual growth. All that is required is something that tests
what a school intends to teach and is normed to a state or national
population.

I have no words. Apparently this entire article is a waste of time
because when they say they're in favor of annual testing, they just mean
that at least once a year teachers should give some sort of test. Well,
hey! Done!! I will leave it to you guys to figure out how those tens of
thousands of tests will be normed up so that all of those schools doing
testing a completely different way can somehow be legitimately
compared. Get back to me when you sort that one out, in a decade or two.

For progressives, we offer the argument that disagregated test data is a
useful tool for lobbying on behalf of whatever subgroup you're
concerned about. I've contemplated this argument before,
and while I understand the appeal of keeping groups from disappearing, I
have serious ethical issues with using students as tools to generate
talking points. If your argument for testing is, "Well, no, it doesn't
really serve the kids. It might even be damaging for the kids. But it
generates some real good lobbying material for advocates," I think
you're on shaky ground, indeed.

And parents? Well, there's this:

Surely, such parents no more want to be in the dark about a K-12
school’s academic performance than they would want to ignore the quality
of the college to which their child will eventually seek enrollment.

Because, of course, all students will eventually seek enrollment in a
college. Beyond that, I'm wondering as always-- where is this great mass
of parents clamoring for and demanding federal testing? Where are all
these parents who have no idea how well their child's school is doing
and so are desperately demanding federal test results so they will know?

Brookings finally notes that teachers unions might be a lost cause on
this issue because 9and they use very nice fancy language to say this)
teachers are all afraid of being evaluated and punished for the results.
But teachers should be practical enough to see the value in trading an
end to test-linked evaluations in exchange for keeping the annual tests
themselves.To wrap up

As always, Brookings really captures the point of view of economists who
haven't an actual clue about what goes in actual schools.

The biggest gaping hole in their proposal is an unfounded belief in the
validity of The Big Test. They believe that The Big Test is a valid
measure of learning, and that is an assumption that nobody, anywhere has
backed up. The closest these guys come is throwing around the infamous
Chetty results, and all that Chetty shows is that there is a slight
correlation between test scores and later financial success (thereby
creating supremely narrow definitions of learning and success). For
their purposes, that means nothing. I'll bet you that there's a
correlation between how nice a student's shoes are and how successful
that student is later in life, but that doesn't mean buying nice shoes
for every student would make the student successful later in life.

But every piece of the Brookings argument rests on that foundation--
that a narrow bubble test with some questions about math and some
reading questions somehow measures the full depth and breadth of a
student's education. Brookings assumes that people are just upset about
the High Stakes part of High Stakes Testing; they fail to grasp that a
major reason for being upset about the High Stakes portion is that the
Testing is crap. You can play with the data from the crap test all day,
but at teh end of the day, you'll just have crap data in a shiny report.

Final verdict? Brookings has completely failed to make a case for annual testing.

KSN&C

KSN&C

KSN&C is intended to be a place for well-reasoned civil discourse...not to suggest that we don’t appreciate the witty retort or pithy observation. Have at it. But we do not invite the anonymous flaming too often found in social media these days. This is a destination for folks to state your name and speak your piece.

It is important to note that, while the Moderator serves as Faculty Regent for Eastern Kentucky University, all comments offered by the Moderator on KSN&C are his own opinions and do not necessarily represent the views of the Board of Regents, the university administration, faculty, or any members of the university community.

On KSN&C, all authors are responsible for their own comments. See full disclaimer at the bottom of the page.

Why This Blog?

So far as we know, we only get one lifetime. So, when I "retired" in 2004, after 31-years in public education I wanted to do something different. I wanted to teach, write and become a student again. I have since spent a decade in higher ed.

I have listened to so many commentaries over the years about what should be done to improve Kentucky's schools - written largely by folks who have never tried to manage a classroom, run a school, or close an achievement gap. I came to believe that I might have something to offer.

I moved, in 1985, from suburban northern Kentucky to what was then the state’s flagship district - Fayette County. I have had a unique set of experiences to accompany my journey through KERA’s implementation. I have seen children grow to graduate and lead successful lives. I have seen them go to jail and I have seen them die. I have been amazed by brilliant teachers, dismayed by impassive bureaucrats, disappointed by politicians and uplifted by some of Kentucky’s finest school children. When I am not complaining about it, I will attest that public school administration is critically important work.

Democracy is run by those who show up. In our system of government every citizen has a voice, but only if they choose to use it.

This blog is totally independent; not supported or sponsored by any institution or political organization. I will make every effort to fully cite (or link to) my sources. Please address any concerns to the author.

On the campaign trail...with my wife Rita

An action shot: The Principal...as a much younger man.

Faculty Senate Chair

Serving as Mace Bearer during the Inauguration of Michael T. Benson as EKU's 12th president.

Teaching

EDF 203 in EKU's one-room schoolhouse.

Professin'

Lecturing on the history of Berea College to Berea faculty and staff, 2014.

Faculty Regent

One in a long series of meetings. 2016

KSN&C StatCounter

Disclaimer:

By accessing this website (http://theprincipal.blogspot.com) Kentucky School News and Commentary (hereafter KSN&C), a web browser (hereafter user) consents that she or he is familiar with, understands and absolutely accepts the following weblog disclaimer:

The views expressed by the authors and contributors on this website do not necessarily reflect the views of Kentucky School News and Commentary, those who link to this website, the author’s employers, mothers, fathers, sisters, brothers, other ancestors, blood-relatives, progeny, this website’s web host, moderator, designer, or any other organization in any way connected with this website.

While I presently serve as Chair of the Eastern Kentucky University Faculty Senate (August 2014-May 2016), none of the Moderator's comments are official statements attributable to EKU, its Faculty Senate or any of the institution's entities.

In all cases, comments are the personal views of the author. No individual contributor, author or commenter is paid for their opinion or beholden to a particular point of view. All contributors write in the English language and cannot be held responsible for unfortunate translations that may occur in other languages. KSN&C is not responsible for human errors involving grammar and punctuation.

Comments on this website are the sole responsibility of the author. The author assumes full responsibility, liability, and blame for any libel or litigation that may result from something written in or as a direct result of something written in a comment. The accuracy, completeness, veracity, honesty, exactitude, factuality and politeness of comments are not guaranteed. The content on the blog is not intended to malign any religious, ethnic group, club, organization, company or individual. Readers are advised to employ a healthy dose of rationality. Furthermore, information is always in transition. Web links change, and content published today may be out-of-date next week.

Readers are advised that some images used on the site are not the property of KSN&C but are reduced in size and used under fair-use. The same is true of certain copyrighted material. Any concerns should be addressed to the moderator. Due to the episodic nature of the blog, errors, when pointed out, may not be immediately corrected.

All trademarks, service marks, copyrights, registered names, mottos, logos, insignias and marks used or cited by this website are the property of their respective owners and this website in no way accepts any responsibility for an infringement on any of the above.

Despite any claims to the contrary, nothing on this website should be construed as professional advice. The information provided on this website is of a general, wide-ranging nature and cannot substitute for the advice of a licensed legal professional, physician, psychiatrist or member of the clergy. A competent authority with specialized knowledge operating within the Kentucky Department of Education, local public school district, church school, independent private school, home school, or in the journalistic, law enforcement or legal community is the only one who can address or comment on the specific circumstances covered in the news and commented upon herein. For personal advice, please contact your mother, father, BFF, local bar association, local bar tender, law society, medical board, county hospital, pastor, teacher, phone book, online directory, local emergency number in your jurisdiction, or Google to find a or obtain a referral to a competent professional.

This website has no control over the information you access via outbound link(s) in the post text, sidebar, header, footer or comment sections. This website does not endorse linked websites and cannot guarantee the accuracy of any information found by following said links or the correctness of any analysis found therein and should not be held responsible for it or the consequences of a user’s use of that information. In fact, we’re pretty sure we link to falsehoods perpetrated by others with some frequency. Be warned. Twistifications of supposed facts, biased reporting, and bad analysis is de rigueur for some of the sites we link.

This website may inadvertently link to content that is vacuous, obscene, venomous, frivolous, rotten, antagonistic, harsh, rancorous, acrimonious or repetitive. This website in no way condones, endorses or takes responsibility for such content. Please report anything really ugly to KSN&C’s Moderator.

This website publishes content regularly and said content is maintained in reference to the protections afforded it under local, state, martial, federal, international and school yard law. Publication of information found on this website may be in violation of the laws of the city, county, state, country or other jurisdiction from where you are viewing this website’s content and laws in your jurisdiction may not protect or allow the same kinds of speech or distribution. In the case that the laws of the jurisdiction where this website's content is maintained and those of yours conflict, this website does not encourage, condone, facilitate, recommend or protect the violation of any laws and cannot be responsible for any violations of such laws. We do condone lawful efforts to extend free speech protections to all parts of the world.

Because the World Wide Web is an integrated net of communication, discussion and litigation, this website encourages the distribution of its content. Cross, reciprocal or just plain friendly hyper-linking is consistent with this information sharing and this disclaimer should not be construed as a condemnation of any linking practices. That said, any reproduction of this website’s content must credit the website by name and Uniform Resource Locator (URL). Should you link to this domain or use, reproduce, republish, reiterate, imitate, or duplicate the information contained on this website, you alone are responsible for that action and should, under threat of litigation, credit this website by name and URL. In addition, any user who learns of information from this site, but traces back to our attributed sources in an effort to forego proper mention of KSN&C should seek therapy.

This website is not recommended for inmates, ingrates, illiterates, or anyone professing an irrational fear of CATS or any other mammal, or those who have a penchant for bullying or self- aggrandizement. Women who are pregnant or may become pregnant or are nursing are advised to consult their physician before reading this website. Eating before reading may result in indigestion. This website contains small pieces and is not recommended for children under the age of 4.