There is an incredibly irritating 'Bad teacher' factoid which keeps cropping up in education. A factoid is not a fact. It is ‘an item of unreliable information that is reported and repeated so often that it becomes accepted as fact.’ The factoid to which I am referring is roughly as follows:

‘Teachers who work in a given school, and therefore teach students with similar demographic characteristics, can be responsible for increases in math and reading levels that range from a low of one-half year to a high of one and a half years of learning each academic year.’

‘A good teacher can get 1.5 years of learning growth; a bad teacher gets half a year of learning growth.’

You can find examples of this factoid everywhere, but the thing which has finally made me write this was seeing a head teacher – someone who, I have no doubt, has the best of intentions – making statements based on the factoid:

Headteacher Lynne Gavin, of Pakeman Primary School, says (between 0:20 and 0:36), "We looked at the Sutton Trust and we came across a piece of research which showed the impact of poor teaching on an average child and the devastating impact poor teaching can have on a disadvantaged child." This is based on a factoid which is simply not true. Here’s why.

The ‘Bad Teacher’ factoid first appeared in 1992, in a paper by Erik Hanushek, who is responsible for the two quotes above. The paper is actually about ‘The Trade-off between Child Quality and Quantity’ – whether the number of children in a family has any relationship to the ‘performance’ of a given child. The data used came from a sample of ‘low income black students from Gary, Indiana’ in 1972-1975 which, the author notes, ‘is clearly not representative of the entire population.’

In his conclusion, Hanushek blithely states that, ‘there is no doubt that teachers vary dramatically in effectiveness. The difference in student performance in a single academic year from having a good as opposed to a bad teacher can be more than one full year of standardised achievement.’

This damaging nonsense is a classic error of the Value Added delusion, which defines a ‘good’ teacher (or school) as one with ‘good’ results and a ‘bad’ teacher (or school) as one with ‘bad’ results. Additionally, the Value Added delusion ignores the crucial fact that a closed system ensures that it is impossible for all teachers or schools to ‘add value’ – it’s essentially a zero sum game. Furthermore, the average ‘student performance’ of different cohorts will naturally vary considerably because of underlying distribution of test scores, with some gaining ‘more’ and some gaining ‘less’. Finally, the delusion cannot separate correlation and causation, and simply uses Halo Effects to attribute the features of ‘success’ or ‘failure’ to ‘good’ and ‘bad’ based on badly-measured outcomes which can only be allocated from a limited - and closed – range of possible outcomes.

Hanushek works in the USA, but his incorrect claim has been taken up by those in England who want to believe that ‘bad teachers’ negatively impact on a pupil’s ‘performance’. The worst offenders are the Sutton Trust, who are quoted repeatedly by policy makers and politicians, and cited by the head teacher above, who appears to be referring to the Sutton Trust’s 2013 report ‘Testing Teachers’ which references Hanushek on page 7:

An earlier report by the Sutton Trust, ‘Improving the impact of teachers on pupil achievement in the UK – interim finding’ actually had Hanushek as an advisor. This report repeated the factoid, stating:

Whilst the Sutton Trust has become a trusted source of advice for many in teaching, it is worth remembering that it is a non-government organisation with its own distinct agenda.

The factoid is frequently used in educational research. The Institute of Education’s recent ‘Primary and secondary education and poverty review’ uses it to make assumptions about ‘Teacher Quality’. The Centre for Market and Public Organisation at Bristol University’s report ‘Do teachers matter? Measuring the variation in teacher effectiveness in England’ leads back to the factoid too. Politicians often quote the factoid as fact. The Social Mobility and Child Poverty Commission’s report Cracking the Code, quotes the Sutton Trust’s ‘Improving the impact of teachers on pupil achievement in the UK’, which uses Hanushek’s factoid as detailed above. There are countless other reports which all lead back to Hanushek: The Telegraph in 2009, and the same report rewritten for 2012The government on teacher’s pay.The Spectator teacher bashing.The Independent claiming children are being let down by ‘poor teaching’.And from the horse’s mouth. The ridiculous thing about all this is that even Hanushek knows it’s nonsense. He's been trying to find evidence for his factoid since the late 1960s. His research over the past 45 years has not managed to prove his hypothesis, and he can find no link between anything which would identify an effective teacher other than the defined/circular 'good test scores = good teacher' delusion: “Observed teacher characteristics do not represent teacher quality. The consistent finding over four decades has been that the most commonly used indicators of quality differences are not closely related to achievement gain, leading some to question whether teacher quality really matters (see the review in Eric A. Hanushek and Steven G. Rivkin 2006).” (Generalizations about Using Value-Added Measures of Teacher Quality, Hanushek & Rivkin, 2010) Essentially, the factoid – deliberately or otherwise - mistakes correlation for causation, and no-one has yet found any way whatsoever of identifying any underlying factor which links those who are identified as ‘good’ or ‘bad’ teachers by delusional Value Added number crunching. Some classes may make a year more progress on average than others, but this has nothing whatsoever to do with any underlying common teacher characteristics.There’s gold in them thar factoids So why on earth has this factoid been repeated over and over again? The answer, as with many things, is to follow the money. Those who want to believe that teachers and schools can be divided into Good/Bad are usually selling something, trying to get money from philanthropists, businesses or governments, or simply justifying their own existence. The Gates Foundation is a big believer in Hanushek’s factoid, even though their multi-million dollar Measures of Effective Teacher project was a monumental waste of money, misinterpreted its own data and couldn’t find any way to identify an effective teacher. And what they came up with would cost $5 billion a year in the USA alone. That’s why people want to believe the factoid. In this country there are any number of people and organisations trying to make a living by selling assessments of ‘student performance’, teaching training and continuing professional development, ‘progress tracking’ systems and all manner or other gubbins. Ofsted survives by attributing Halo Effects to delusional Value Added calculations. Many of these people are well served by a narrative of Good/Bad teachers and schools. But we need to stop believing things which are simply not true.

Teachers have succeeded in driving the charlatans behind Brain Gym and Learning Styles out of the mainstream discourse about education. Despite thousands of people being hoodwinked by the delusion of observing pupil progress in real time, we have stopped believing this to be true, by showing the underlying assumptions on which the delusion was based to be false. Even the drifting tanker that is Ofsted has stopped writing handbooks in which a preferred teaching style was proscribed despite ample evidence of its efficacy. We have accepted the errors which have been made and moved on. Just Stop it I work with primary school children. Being young, many mess about a bit in class, or attempt to when they think I’m not aware of what they are doing. Nearly every time I see a child who is messing about, I have a small voice in my head which is yelling, “Just stop it. Now. Just. Stop it.” I feel the same about those who believe the Hanushek 'Bad teacher' factoid. You have been had, either by choice or your own gullibility, or because no one has pointed out that it simply is not true. As I keep saying, teachers and schools need support, not condemnation for factors outside of their control. In this case, teachers should not have to bear the brunt of bad ideas based on something which is simply not true. By all means aim to help teachers to get better at what they do. But stop believing that some people are ‘good’ and some people are ‘bad’. Stop repeating nonsense about 'bad teachers', and saying that 'good teachers' are responsible for higher 'learning growth'. Education is more complicated than that, and you need to stop believing things which are not true. Just stop it. Now. Just. Stop it.

You make some fair points about misusing data, but I am not sure about your conclusions. In twenty years of teaching I have seen lots of teachers - good teachers care - they are professional, they plan lessons carefully, mark books, go the extra mile, reflect upon their practice; bad teachers - not so much. And... Good teachers end up getting better results than bad teachers

Reply

Jack Marwood

11/11/2014 03:55:30 pm

Thanks Mike - I appreciate your comment. My conclusions are based specifically on the 'Bad teacher' factoid, which makes an extraordinary claim which is simply not borne out by the facts. Clearly some teachers are more effective than others - it would be extremely odd if this was not the case.

Clearly some teachers are less conscientious than others.
If, by saying than some teachers 'end up getting better results' than others, you mean get consistently better test scores, I'm afraid that even Eric Hanushek doesn't think that is always true.

The language of 'good' and 'bad' teachers is extremely unhelpful and does our profession no favours. By all means celebrate success and support those who need it, but 'Bad teacher' is an untrue factoid and we should expose it as such.

Reply

chemistrypoet

11/11/2014 02:43:37 pm

I kinda agree with Mike....in that, I strongly suspect that there are some teachers who are more effective than other ones. I am not sure how much an affect they have...(how bad must a teacher be to have a very significantly negative effect?)...but there must be some effect? On the other hand, I agree that it's all much more nuanced than the factoid supposes. We want effective teachers, right? What is an effective teacher; what are their characteristics?

(Although, in the current high stakes environment I don't think we want to have this debate out loud).

Reply

Jack Marwood

11/11/2014 04:02:23 pm

I've had good look at the research into 'teacher effectiveness', and I'll be writing about it soon. Lots of people have tried to identify the characteristics of effective teachers, and it's much, much harder than one might think. I will also say that the difference between an 'effective' and 'ineffective' teacher is much, much less than the 'Bad Teacher' factoid suggests.

If my thoughts here make people pause when they consider repeating 'Bad teacher', I'll be happy. As with most things in education, it's is more complicated than that, and we need to stop believing things which are not true, particularly things based on flawed, non-replicated research.

Reply

Teacher 60152

12/11/2014 08:51:46 am

I agree with Jack.

While at the very extremes there may be poor teachers who may have some deleterious effect on their charges, most teacher effects ("good" and "bad") will be strongly masked by a multitude of other variables.

The parroting of the factoid by people who should certainly know better, highlights the standard of research and understanding going on in certain organisations. To use such a poor piece of work / data as a basis for anything, is certainly not good practice and is almost negligent in terms of research.

The more I hear of things like this, the more I despair at the state of education research. There are a few high profile names who seem totally clueless when it comes to professional standards of research and understanding what data actually shows or means. My previous field was science before becoming a teacher so perhaps I am better at sniffing out and understanding this sort of nonsense. I have come across lots of similar rubbish in mainstream, but have always done my best to ignore it, as I knew of no-one else who thought like me or who could see the bogus nature of the figures being used.

It's great to see this sort of stuff finally getting to a wider audience.

Keep it up Jack.

Reply

Simon Hepburn

21/11/2014 02:18:53 pm

Another really sensible article. In answer to Mike I'd agree that there are probably good and bad teachers - but the criteria you think might define them don't work. Some of the teachers I've worked with who get great results and are highly rated by students seemingly do little preparation or marking, and have a wide range of approaches to classrooom and behaviour management. Others follow the letter of every handbook and work themselves dizzy but achieve less. Some get great results at specific stages or with different groups or individuals.

Reply

Andrew Lowery

27/9/2015 09:42:53 am

How does this tally with Dylan Wiliam's oft-repeated work on teacher effects vs. school effects? Surely the quality of the 'teaching' (not the 'teacher') has an observable impact on achievement? Or am I missing a link in the logic?

Reply

Your comment will be posted after it is approved.

Leave a Reply.

Author

Me?
I work in primary education and have done for ten years. I also have children
in primary school. I love teaching, but I think that school is a thin layer of icing on top of a very big cake, and that the misunderstanding of test scores is killing the love of teaching and learning.