Category Archives: Data

[They] are like poets, you know, like Shelley or Byron, or people like that. The two totally distinct types of visionaries, it’s like fire and ice, and I feel my role in the band is to be kind of the middle of that, kind of like lukewarm water.

Disappointed Idealist writes that he loathes what he sees as the current government’s obsession with drawing a line in the sand and declaring those on one side winners and the other as losers. He writes movingly of the experiences of his three daughters:

They just called my daughters “mediocre failures” . . . Like most clever people who don’t have difficulty with language or maths or spatial awareness, or other academic activities, I fundamentally find it impossible to truly understand why they can’t, despite endless practice, remember how to spell basic words, or how to do basic sums. The school have tried all sorts of different methods of teaching it, and so have we at home, but one day it’s there, and the next it’s gone. Some things stick for a while, some things don’t stick at all . . . At home, they are delightful, loving, awkward, stroppy, generous, always hungry, funny and, above all, happy. But they won’t “pass” their Y6 SATs.

I am sure most teachers are familiar with that “one day it’s there, the next it’s not” sensation when teaching SEN students (I wrote a post about it a while back). In my experience, patience and kindness and persistence are the order of the day in this scenario (not that anybody says it’s not.) My experience also tells me that sometimes this works, and sometimes it doesn’t.

Horatio Speaks makes the case that recent scientific research shows that all children can be taught to read, write and do mathematics effectively, bar a very few severely disabled individuals.

To which I say: good! Like most teachers, show me a better way to teach and I am all over it. Horatio Speaks goes on to say:

I applaud the passion of the Disappointed Idealist . . . But I would be happier if he – and the thousands who cheered him on – were directing their anger at the education establishment’s assumption that we will always have children who fail. It’s a false assumption, as is the emotional caricature that those advocating for more accountability for children’s progress care less about the children. I have worked with SEN long enough to know that the most deadly poison is sympathy. It kills by paralysis.

Over the years and from time to time, sadly, I have seen some bad SEN: “death by word search”, for example. And Horatio Speaks is right, bad SEN can kill by paralysis; or, more probably, boredom. But, obviously, not all SEN is bad SEN.

The nub of the disagreement between Horatio Speaks and Disappointed Idealist, I believe, lies in the use of the phrase “children who fail”.

Horatio Speaks rails against an educational establishment that assumes that we will “always have children who fail”. In my view, he is referring to the fact that some children leave school without basic literacy and maths skills.

Disappointed Idealist rails against a system that wants to label children as “mediocre failures”. In my view, he is lamenting the fact that, according to a politically imposed and essentially arbitrary standard, some children will be labeled as “failures” through no fault of their own and that this is, frankly, unhelpful.

My own view is that both of them have valid points. While it is undeniable that some children will do less well than others, by whatever measure is taken, the question is: what should the education system do with this information?

I suspect that both Disappointed Idealist and Horatio Speaks would argue for a diagnostic rather than a judgemental approach as far as each individual student is concerned.

Kings made tombs more splendid than houses of the living and counted old names in the rolls of their descent dearer than the names of sons. Childless lords sat in aged halls musing on heraldry.

— J. R. R. Tolkein, The Two Towers

If there’s anything that makes me lose the will to live, it is being in the same room as an educational Player of Games. I’m sure everyone is reasonably familiar with the type: “I want every intervention from now until the end of term focused on improving the A*-C pass rate for left-handed Y11 students whose birthday month has an R in it.”

Yes, it might help, marginally, in some sense. On such massaging of the margins are modern educational careers and reputations built.

Personally, such considerations leave me cold. Such teachers, it seems to me, hold their statistics in higher esteem than their students. The percentage is adjudged to be the outcome, rather than merely an indicator of a number of successful outcomes.

Sometimes, when I try and express this, people look at me as if I had twelve heads. It is a nuanced and subtle difference of emphasis, admittedly, but I think it’s a valid one. As an analogy, imagine a doctor who focuses on (say) a patient’s temperature to the exclusion of all else: “Doctor, I think I’ve broken my leg.”

“Well, your body temperature is back to normal now. That means that we now have the officially mandated number of ‘healthy’ patients as per Ofdoc guidelines.”

“But what about my bloody BROKEN LEG?”

“My work here is done. Next patient please!”

The other note of caution that needs to be sounded more loudly in the education world is awareness of what is known as the Halo Effect.

I learned about this in Duncan Watts’ excellent book Everything Is Obvious (When You Know The Right Answer) in which he summaries the work of Phil Rosenzweig:

Firms that are successful are consistently rated as having visionary strategies. strong leadership, and sound execution, while firms that are performing badly are described as suffering from misguided strategy, poor leadership or shoddy execution. But, as Rosenzweig shows, firms that exhibit large swings in performance over time attract equally divergent ratings, even when they have pursued exactly the same strategy, executed in the same way, under the same leadership all along. Remember that Cisco Systems went from being the poster child of the Internet era to a cautionary tale in a matter of a few years . . . Rosenzweig’s conclusion is that in all these cases, the way firms are rated has more to do with whether they are perceived as succeeding than what they are actually doing.

— Watts, p.197 [emphasis added]

In one early experiment, several teams were asked to analyse the finances of a fictitious firm. Each team was rated on their performance and then asked to evaluate their team in terms of teamwork, communication and motivation. The high scoring teams assessed themselves very highly on these metrics compared with the low scoring teams, as you might expect. However, the kick was that performance scores had been allocated at random — there was no real difference between the teams’ performance at all. The conclusion is that the appearance of superior outcomes produced an illusion of superior functionality.

Watts argues persuasively that we tend to massively underestimate the role of plain, dumb luck in achieving success. He cites the case of Bill Miller, the legendary mutual fund manager who did something no other mutual fund manager has ever achieved: he beat the S&P 500 for fifteen straight years. Watts notes that this seems a classic case of talent trumping luck. However:

. . . right after his record streak ended, Miller’s performance was bad enough to reverse a large chunk of his previous gains, dragging his ten-year average below that of the S&P. So was he a brilliant investor who simply had some bad luck, or was he instead the opposite: a relatively ordinary investor whose ultimately flawed strategy just happened to work for a long time? The problem is that judging from his investing record alone, it’s probably not possible to say. [p.201]

I trust that I do not have to draw too many lines to highlight the relevance of these points to the education world. Outcomes, in the sense of exam grades, are currently the be-all and the end-all of education. But the Halo Effect makes it clear that a simplistic reading of successful outcomes can be highly misleading.

Negating the Halo Effect is difficult, because if one cannot rely on the outcome to evaluate a process then it is no longer clear what to use. The problem, in fact, is not that there is anything wrong with evaluating processes in terms of outcomes — just that it is unreliable to evaluate them in terms of any single outcome. [p.198]