Bruno: Both Sides Mis-Calculate On "Out-Of School" Factors

@mrpabruno Last weekend Ken Libby took to Twitter to ask the masses, "what would it take to draw up a 'peace deal' among 'reformers' and 'anti-reformers'?" Many of the immediate responses were unintentionally ironic in a discouraging way and demonstrated the considerable distrust between the two sides. And much of the subsequent discussion has focused on the often hostile "tone" of the debate.

But my sense is that the tone is as likely to be a symptom of the distrust as a cause. As I detail below, I think that reformers need to worry less about whether a problem is "in-school" or "out-of-school" and just try to figure out what's really best for kids generally. Anti-reformers need to avoid making too much of the in-school/out-of-school distinction to excessively minimize the importance of in-school factors.

To elaborate a bit on my initial Tweet response to Ken, I think a major cause of acrimony is that reformers and anti-reformers each in their own way make to much of the distinction between the "in-school" and "out-of-school" factors of student success. Their respective interpretations of that distinction are different enough that each side concludes that the other side is acting in bad faith.

Reformers, for example, seem to focus almost exclusively on changing students' in-school environments. Rarely, if ever, do you see the most prominent education reform organizations throwing their considerable power behind improving students' out-of-school lives. Anti-reformers see this focus on in-school factors and interpret it as evidence of malice and secret agendas. After all, if reformers really cared first and foremost about helping students, wouldn't they dedicate themselves to helping them outside of school as well?

I think that interpretation of reformers' motives is mostly incorrect, and that reformers have just made the calculation that it's easier to change schools than to address society-wide poverty and inequality in a comprehensive way.

Reformers seem to think, in other words, that one of the important things about the "in-school/out-of-school" distinction is that the in-school stuff is feasible and the out-of-school stuff isn't. But if the education reform wars have taught us anything, though, I think it's that addressing in-school factors is probably not that much easier. And I think addressing out-of-school issues would go a long way toward demonstrating to anti-reformers that reformers' intentions are pure.

The flip side of this is that anti-reformers often make too much of the in-school/out-of-school distinction, too. They'll often correctly point out that out-of-school factors are far more important to student achievement than in-school factors, but then use that fact to excessively minimize the importance of in-school factors. (My experience is that many anti-reformers won't even discuss in-school factors without bringing up out-of-school factors, which is an especially annoying non-sequitur.)

A recent article in Education, Finance, and Policy explains why the dominance of out-of-school factors doesn't justify dismissing in-school factors: in-school factors may be easier to change, and the effects of in-school reforms may still be quite large. (If the article is gated, there's a shorter version of the argument here.) The problem is that when anti-reformers misjudge the potential impact of in-school reforms, this will often look to reformers as stubborn loyalty to the "status quo".

So while reformers may misestimate the potential for out-of-school reform, anti-reformers often misestimate the potential for in-school reform. Each side sees the in-school/out-of-school distinction as more important than it probably is, and so appear to the other side to be operating in bad faith, and thus deserving of neither cooperation nor a civil tone. It's not really clear to me that the in-school/out-of-school distinction is very helpful to the debate, all things considered. - PB (@MrPABruno) (image source)

You can follow this conversation by subscribing to the comment feed for this post.

There are no "anti-reformers," thus strawman. But educators who recognize the influence of out-of-school factors (67%-86% of outcomes) are both asking that we address OOS factors AND reconsider how we recognize in-school quality. Test scores are disproportionately linked to OOS factors, thus continuing to beat that drum is misleading.

I oppose the new reformers, and have been an advocate for major education reform since the first day I entered the field 30 years ago. Alas, as I am the person with experience and expertise, no one has yet allowed me to do the reforms needed to truly improve in-school results...

I just maintained Libby's nomenclature for convenience. I don't think it matters to the substance of the issue. The point is that, as the EFP paper explains, while OOS factors might "explain" more of results, that doesn't mean that in-school reforms can't have very large effects, and social context reformers often neglect that fact.

The first step is stopping the use of the perjorative and inaccurate term of anti-reforms. Two different types of reforms are being presented, and reformers should have to argue the merit of theirs, not just spin a simplistic message.

Labels do matter, especially when they misrepresent. Calling the "reformers" such gives them credibility they don't deserve, and calling educators "anti-reformers" is just inaccurate and marginalizing, and thus condoning the "reformers."

Further, please show me just ONE example of "social context reformers often neglect that fact." To state something doesn't make it true.

Social Context Reformers acknowledge both social and educational inequity and are calling for direct reform of both, thus acknowledging that what happens in schools can and will have large effects (especially if we do not focus excessively on test scores). My work repeatedly calls for ending inequitable IN-SCHOOL practices such as how we assign teachers, tracking, and excessive testing and test-prep.

I think it's a fine criticism that the labels "reformer" and "anti-reformer" are loaded or otherwise non-ideal, but note that they are basically the convention and the alternatives I often hear - like "social context reformer" - are not uniformly used or standard and therefore less useful still. I think your proposed nomenclature is better than the norm in principle, but I'm not going to adopt it until it's more of a standard formulation.

As for social context reformers neglecting in-school factors, again, the EFP paper provides a couple of examples, but really any time somebody says "this in-school factor accounts for/explains only X% of student achievement", they are committing the fallacy. I won't belabor the point more, since I've provided the links already.

Your link on no excuses reformers and OOS factors subtly misrepresents that WaPo editorial and so does not demonstrate that they believe oos factors don't matter, only that they underestimate them. That was my point.

Of course there’s a blatant tone difference between anti- and pro-reformers, it’s how politics in general has been for years. The problem is that, ultimately, it’s easier and more powerful to call someone wrong and immoral and invalid and incapable of trust than it is to intelligently debate.

Sarah - I think you're spot on about the easiness of villainizing those with whom we have a conflict or disagreement. Reformers focusing too much on VAM? They must just hate teachers and not care about kids! Reform critics not taking teacher quality seriously enough? They must want to preserve the status quo for their own benefit! It just so happens that the interpretation I prefer is the one that makes my opponents look the worst!

In everybody's defense, I think the inclination to villainize is pretty deeply ingrained psychologically and very difficult to resist. It sure does make reasonable debate harder, though. And it sure is obnoxious.