Exam head says assessment system encouraged teachers to boost marks

Andrew Hall said a system in which teachers were accountable for pupils’ results and also controlled 60% of marks was behind the furore. Photograph: Chris Radburn/PA

_

The controversial GCSE grade boundary changes of 2012 are back in the news with the appearance of AQA’s chief executive, Andrew Hall, before the education select committee. He appears to blame the system but it is really a veiled accusation that teachers are to blame for the fiasco that has damaged thousands of students’ futures when we all know the fault lies at the door of the DfE.

_

The head of one of the leading exam boards has told MPs investigating last summer’s GCSE English furore he believes teachers have been encouraged to boost marks by government measures that hold them to account based on their pupils’ results.

Andrew Hall, the chief executive of the AQA exam board told the House of Commons education select committee he did not believe teachers were cheating in the way they marked controlled assessment modules, but he said they were in a position where “their judgments were influenced by the pressures of the accountability system”.

He said this, combined with a qualification design that gave teachers control over 60% of marks, had been at the root of the disappointment caused when thousands of pupils last summer received D grades in GCSE English after exam boards moved a grade boundary to toughen up the exam.

The parliamentary hearing followed a high court ruling last month against an alliance of pupils, unions, schools and councils who alleged that the government’s exam regulator, Ofqual, and the exam boards Edexcel and AQA had unfairly moved the boundary, in a last-minute “statistical fix” to counter exam grade inflation.

The bar was raised higher than for pupils who submitted papers in the earlier January marking round and some pupils claim they missed out on sixth-form places because of the change.

Hall told the MPs his exam board’s data revealed peaks and troughs of marking around grade boundaries and that indicated teachers involved in internal controlled assessment of GCSE candidates’ work were engaged in “fine judgments”.

Hall agreed with the hypothesis of the committee chairman, Graham Stuart, that once teachers knew “all they had to do was find two more marks and magically a D would become a C” there was a temptation to overmark.

Ziggy Liaquat, the managing director of the exam board Edexcel, also said his exam board, which accounted for 10% of English GCSEs assessed last year, had observed inaccurate marking by teachers.

“We adjusted downwards 8% and we adjusted upwards 5% so there was inaccurate marking both ways,” he said. He added the evidence did not yet show teachers had pushed marks deliberately to cross grade boundaries.

Mark Dawe, the chief executive of the exam board OCR, told the committee it had not found evidence of overmarking of controlled assessment modules.

Liaquat apologised for the “distress to children and parents” that had been caused by the move to the grade boundary between the January marking cycle and the summer marking.

“We should be relentless in communicating that grade boundaries can constantly move,” he said. “We really need to educate teachers, parents and pupils in how the process works.”

Hall admitted to MPs there had been “a loss of trust” over the marking of last summer’s GCSE English. He said he had continued worries about the changes to standards in GCSE science, which is “one of the most sensitive things we are doing”, and stressed the need for work to communicate that to students, teachers and parents. “It is in the worry mix, of course it is,” he said.

Education in brief: are GCSEs the new O-levels?

guardian.co.uk |July 9, 2012

Warwick Mansell

Exams: should they be GCSEs or O-levels? Photograph: Jim Wileman/Alamy

GCSEs: the new O-levels?

Michael Gove’s leaked plans to reintroduce O-levels to schools, seemingly inspired by the success of an “international” version of the exam operated by one of England’s big three exam boards and taken by teenagers in Singapore, rightly made headlines last month.

But less noticed has been a move by another of the boards, Edexcel, quietly to scrap its own version of the exam three years ago.

Edexcel, owned by Pearson, replaced its International O-levels with its existing International GCSE brand. Intriguingly, a 2009 document for teachers explaining the move described the IGCSE as “the most up-to-date qualification from the UK” and “the same [as O-level] but with modern references”.

How very off-message. Speed Read wonders what Mr Gove thinks. A Pearson spokeswoman says: “The demand internationally is for qualifications which reflect the UK curriculum. With the introduction of the GCSE, the demand shifted to IGCSE, rather than international versions of an old qualification.”

Cheats’ charter

Confirmation came last week, in Peter Wilby’s interview in these pages with Ofqual’s chief executive, Glenys Stacey, that exam board seminars in which senior examiners give teachers advice on how to boost their pupils’ grades are being banned. These advice sessions were, of course, the backdrop to a series of undercover scoops in the Daily Telegraph last December. But is this the end of the matter?

In 2009, BBC Radio Five Live reported on controversial advice being given to teachers at a seminar run not by a board, but privately, by a former languages examiner who guided his attendees on how to “script” pupils’ answers in the oral section of French GCSE.

Would such seminars be banned? Ofqual’s powers are limited, it seems; it says it only has powers to regulate the work of “awarding organisations”, or the boards themselves. So while “face-to-face seminars that relate directly to specific, named qualifications” and are run by the boards themselves will cease from next year, there is no such stipulation on those hosted by private organisations. A loophole, perhaps?

A positive outlook

A fascinating insight into the darker arts of education public relations is provided on the website of the firm Communitas. The company, based in Battersea, south London, sets out how it has secured positive news coverage for its clients, many of them academies.

West London academy, which opened in 2003, had “significant reputation and messaging challenges to overcome in the local community”, Communitas tells readers, not least after Ofsted expressed serious concerns about management and pupil behaviour there two years later.

The company therefore launched a strategy to “limit the damage from the worst critical comments in the report”, and proceeded to “work the media”.

At Eastbourne academy in Darlington, where it created a new “brand identity” for the school, Communitas says “early challenges were around staff management issues that needed delicate and skilled management to avoid unwelcome press coverage”, while the section on Shirebrook academy in Derbyshire says Communitas’s emphasis was to make the consultation process as “easy as possible”, as “creating this ease is particularly vital for communications with any vocal minority who may be unsure about the … founding of an academy”.

Is this a good use of public money? Maybe Speed Read needs a good “working” before we are convinced.

Michael Rosen’s letter from a curious parent

guardian.co.uk |July 2, 2012

Does the education secretary, Michael Gove, have any evidence that making exams harder makes students better at anything? Photograph: Chris Radburn/PA

As you probably know, many people wonder how our children’s education is being run at the moment. Up until the last couple of weeks, I thought the one person who would know the answer to this would be you. Now I have my doubts. Can I run past you the chronology on how we parents heard about something that will fundamentally affect the education of all our children – in my case, my two youngest?

You’ll remember that on 20 June, Tim Shipman of the Daily Mail landed a sensational scoop: precise details of how GCSEs were to be scrapped from September 2014 to be replaced by O-levels, set and examined by a single exam board, alongside “simpler exams, similar to the old CSEs” for “less intelligent pupils”.

What we don’t know here is whose words are in quotation marks. Are they yours? One of your officials’? Or Shipman’s? All we can say is that it must be very convenient for you that we don’t know. That way, you can always lay claim to the words that people praise and disown the ones that people dislike.

When the story broke, we noticed, first, that you didn’t deny it, and secondly, it appeared as if you were the only person in the world who had heard of this plan. Will you ever clear that matter up? The reason I ask is that there is a feeling in and around schools and universities that education is too important and too complex to be left to one person, his pencil and the back of an envelope – even someone as wise and thoughtful as yourself.

What happened next wasn’t the most successful day in your career, with hardly a voice anywhere congratulating you on what was clearly a two-tier system, which would entail streaming pupils from the age of 12 or 13. With the rage and contempt you brought on yourself, you might just as well have been talking about bringing back dip-pens and ink-wells. (Now there’s an idea for you.)

A few days later, the BBC website told us of a speech you gave at a Spectator conference. Now we learned that it most certainly wasn’t going to be a two-tier system: everyone was going to take the new O-levels. In other words, it was going to be the GCSE but harder. Do you have any evidence that making exams harder makes students better at anything? I’m sure you could put yourself in charge of raising the high-jump bar in the Olympics, but that would ensure that fewer high jumpers could clear it. In so far as anything resembling a policy is emerging here, that’s about the only one I can discern: make the exams harder in order to get more students failing.

Then, on 28 June, the seemingly well-informed Shipman was back with confirmation that neither the prime minister nor the Lib Dems had known anything about your original announcement. He had something else up his sleeve: “Mr Gove made the case that he can tear up the exam system and bring back O-levels with the stroke of a pen, and since no legislation is required Mr Clegg can be ignored.”

Do you know, that’s precisely what is worrying many of us? It’s the image of you roaming round the Department for Education working out where you’re going to deliver your pen-stroke next. Meanwhile, we know that though this flourish of the pen will affect our children’s education, the matter need not pass through the mechanisms of government. It’s Govement, not government.

You’ve let it be known your inspiration for this harder exam is Singapore. The great advantage in invoking other countries is that few of us are well enough informed to question whether you’re having us on or not. But some people are. On 25 June, David Price OBE (for services to education), director of learning for the Liverpool Institute for Performing Arts, told us on his blog: “Three weeks ago I was in Singapore, invited by the ministry of education … to share the innovation of the educational projects I’ve led. While Gove’s proposed reforms are set to follow Singapore’s exam system, their aspirations have already moved on. Singapore’s minister of education has given officials 18 months to rebuild the system so that it can produce students who can create, collaborate, think critically and compete globally in our unpredictable future. Among many other initiatives, they have instigated a pilot programme based on my work” – Price’s specialism being “re-engaging learners weary of the exam-factory culture”.

From inside Westminster, and indeed inside your brain, it may seem as if you move like lightning: scrapping one exam, inventing another, getting a story out, then another, but the substance of what you have in mind is yesterday’s dinner. As an experienced professional like Price is telling you, the world is moving on. Singapore has noticed but you haven’t. Why not do us all a favour and forget all about that silly “stroke of a pen” stuff?

Yours, Michael Rosen

PS, I don’t suppose you read Private Eye, but would you like to comment on the story in the latest issue, which claimed “educational publishers are … wondering at the conflict of interest” in the roles played by Ruth Miskin, who is reported as being a) the only primary literacy expert on the government’s committee overseeing the national curriculum review, b) the creator of a reading scheme that is government approved and c) whose publishers are in receipt of up to £3,000 of government match-funding each time a school buys Miskin’s reading scheme?

Exam regulator in the front line

guardian.co.uk |July 2, 2012

Glenys Stacey, chief executive of Ofqual, says: ‘If we moved to a single exam board, the transitional risks would be very significant’. Photograph: Andrew Fox

When foot and mouth disease struck Britain in 2007, Glenys Stacey was in the front line. As chief executive of Animal Health (formerly the State Veterinary Service), her job was to ensure it didn’t spread. In that, as she proudly explains, she was successful, averting the crisis in British agriculture and tourism caused by the outbreak of 2001. “We had about a dozen outbreaks and we contained them all,” she says. We may have her to thank that meat supplies weren’t wiped out and that, when the virulent H5N1 virus was found among Suffolk turkeys, we didn’t all fall victim to avian flu.

But can she now cope with what some would describe as an outbreak of Mad Gove Disease? In March last year, she became chief executive of Ofqual, the examinations regulator. If the education secretary goes ahead with plans to scrap the GCSE, bring back O-levels and put them under a single exam board, she will have to make them work. When we meet at the Commonwealth Club in London, I ask if she knew about them before they were leaked (by Gove himself, according to some accounts)?

“We’ve known for months,” she says, “that ministers were concerned about the quality of GCSE qualifications. Our job is to give ministers wise and timely advice about how policy aims should be met and how transition can be managed without putting standards at risk. We’ve had opportunities to provide that advice.”

Which I take to be a “yes”. So what was that advice? “Significant change takes very careful planning indeed and, in that time, children are still taking exams and still need viable qualifications. We have to advise the minister on what is an achievable timetable. It depends on the details. His immediate interests are in English, maths and science. It makes sense to focus on some subjects, not all, and to choose the hard-hitting ones that affect people’s life chances.” The implication – though she won’t confirm it – is that Gove may have to rethink his ambitious timetable to scrap GCSEs by 2014.

Ofqual is answerable to parliament, not to Gove, but John Bangs, a former National Union of Teachers official, now a visiting professor at London University’s Institute of Education, says “it acts as if accountable to the secretary of state”. That view was shared by others I spoke to, but Alan Smithers, director of the centre for education at Buckingham University, was more generous: “She has brought a sharp brain and considerable political skills to the job. She briefs herself very thoroughly. She knows the world in which she has to operate. She navigates the political landscape, rather than confronting it.” The political reality, he said, is that “she may be at arm’s length from Gove, but it’s not a very long arm”.

Stacey herself says: “It’s a relationship. Government can take a policy decision on qualifications but, if it tries to change the nature of assessment, and we believe that will affect standards, it’s our obligation to say no.”

So far, there has been no overt clash. On the contrary, Stacey shares many of Gove’s concerns on GCSE and has striven “to strengthen the qualification, to get it to where it should be”. When she arrived at Ofqual, she dismissed grade inflation – the suggestion that “dumbed down” papers and more lenient marking made it easier to get top grades – as a “not very helpful expression” and argued that it could be attributable to “young people being taught well and working hard”. A year later, after looking at data that “just didn’t make sense” and discussing it with “real experts in assessment”, she said there had been “persistent grade inflation … over at least a decade” at both GCSE and A-level.

She ordered that some GCSE syllabuses be rewritten to ensure candidates cover wider ground; agreed to Gove’s demands that marks be awarded for grammar, spelling and punctuation, albeit so far only in four subjects; launched a review of “controlled assessment” (a new version of “continuous assessment”), arguing that it eats too much into teaching time; and announced a review of how GCSE results are graded. When we first talked at Ofqual’s headquarters in Coventry, two weeks before Gove’s plans were revealed, she echoed another of the minister’s concerns: the growth of non-academic GCSE courses. “Should there be a GCSE in flower arranging?” she asked. “Where is the line drawn? I don’t have a predetermined view, but at the moment the line isn’t drawn.”

On one matter on the Gove agenda – having a single exam board to set and mark GCSEs – she has seemed consistently cool. “If we moved to a single exam board,” she says, “the transitional risks would be very significant.”

For now, she says, her job is to ensure that the competitive market at GCSE and A-level operates in a “healthy” manner. Each board has an incentive to bump up pass rates so that it gets a higher share of the hefty exam entry fees. This, Gove believes, is partly responsible for grade inflation. Stacey has already banned seminars in which board representatives “coached” teachers in how to get their pupils through the exams – sometimes giving strong hints about that year’s questions – and has now begun “a very close look” at whether boards should continue publishing textbooks, which schools are naturally tempted to buy and follow slavishly. She has a new power to fine boards up to 10% of turnover for misbehaviour. Has she used it yet? “Give me a chance, we’ve only just got the power.” Has she issued any warnings? “I wouldn’t be warning them, I’d be giving notice of an intention to fine. I’m not in that position now, but I am ready.”

Stacey has a soft voice and great charm, but there is also an underlying sternness and, if I were Gove, I would be wary. She has no background in education – beyond two children and 25 years of marriage to a teacher – though she is taking an MA in educational assessment. She has spent most of her working life in important public-service jobs, yet is unknown to the public and not well-known even to practitioners in areas where she played a leading role. That may be because she has never worked in London.

Now 58, she is a living embodiment (her words) of mid-20th century social mobility. Her father was a painter and decorator, her mother a factory worker. She went to grammar school near Walsall and got “seven or eight” O-levels, but left school at 16 because that’s what people of her background did. Armed with a couple of science O-levels and (she thinks) a short skirt, she landed a job in a Royal Ordnance laboratory, quality assessing explosives. “I realised I wasn’t going to be a Nobel prizewinner, so I went to work for a local law firm, and fell into something I really enjoyed.” After taking A-levels at evening classes, she read law at Kent University before qualifying as a solicitor and working first in private practice and then for the Legal Aid Board, where she became an area manager.

Her first big job, at 43, was to set up the Criminal Cases Review Commission, created after the belated acquittals of the Birmingham Six and the Guildford Four on IRA bombing charges. She moved on to run the magistrates courts in Greater Manchester and, after Animal Health – the job which, to judge by the enthusiasm with which she recalls it, gave her greatest pleasure and success – she became chief executive of Standards for England, responsible for dealing with misdemeanours in local government. The latter body disappeared in the coalition government’s quango cull.

Her work there was largely about stamping out corruption. Now she is dealing with a system that Mick Waters, a former official at Ofqual’s predecessor, has described as “almost corrupt” and which has never before been properly regulated. It involves 25m exam scripts a year, 2,500 different papers, 15,000 qualifications (most of them vocational) and 180 awarding bodies. She has taken a more proactive approach to this daunting portfolio than many expected. For example, she commissioned a study of how other countries assess exams at the equivalent stage to A-level, concluding that they make far more use of teacher assessment and multiple-choice tests. On the latter front, we can expect activity.

“I have never sought a public profile,” she tells me. But before the GCSE saga is over, she will surely have one. It is impossible to believe she won’t cope.

Exam boards should not set their own syllabuses, say MPs

The chairman of the education select committee said confidence in exams had been eroded by ‘grade inflation’. Photograph: Ben Birchall/PA

MPs have acknowledged the problem of “grade inflation” and recommended stripping exam boards of the right to set their own syllabuses.

In a report on the future of exams published on Tuesday, the education select committee proposes a single syllabus for each subject with the aim of restoring confidence in the system by removing the “significant pressures” to drive standards down.

But the report comes out against moving to a single national exam board, which the MPs say would be a disruptive change that would hamper innovation and make it harder to control costs. “There could be a competition to decide which exam board would design the syllabus for a particular subject which would then be accredited by the regulator, Ofqual. After that any board could set an exam for that syllabus and compete on innovation, efficiency, service and support,” the report suggests.

Graham Stuart MP, Tory chairman of the select committee, said public confidence had been eroded by grade inflation.

“There has been grade inflation. There has been a denial of that going on. I think a recognition of where we’re at will help restore confidence. If you see the denial of obvious truths, that people see in their own lives, they will lose confidence in those who are vouching for that system.”

Grade inflation has been a problem for decades; analysis by an expert at Durham University has found that candidates of the same ability have been awarded A-levels a 10th of a grade higher every year since 1988.

The MPs’ report acknowledges the challenge of maintaining standards over time, as increasing numbers of children sit exams.

“A-levels cater for a broader ability range, with larger numbers going to university, then they did 30 years ago,” the report says.

The MPs advocate “explicit recalibration” of grading standards, with ministers and the exams regulator openly saying what the consequences will be for exam candidates, rather than a “slow creep”.

The education secretary, Michael Gove, is phasing out modular GCSEs from this September and plans further radical changes to the exam system.

Leaked proposals to scrap GCSEs in favour of a system modelled on O-levels and CSEs provoked a row within the coalition last month. Under the plans, children would sit new exams in English, maths and sciences in summer 2016.

This timetable was described as “reckless” by the chairman of the education select committee on Monday.

“This is not an area for anyone who has got the urgency bug,” Stuart said.

The select committee’s report is critical of school league tables based on GCSEs. The inquiry heard evidence about the resources schools invest in getting students across the “C/D” grade boundary so their results count towards the standard measure of five A*-C grades including English and maths.

The MPs praise the idea of “sample testing” schools to gauge information about standards without judging individual pupils or schools.

The MPs’ report raises concern that allowing examiners to write textbooks creates a potential conflict of interest. It warns exam boards against marketing text books as narrow guides to passing exams, with descriptions such as: “All you need for your course.”

The exams regulator has announced a ban on examiners conducting face-to-face seminars with teachers, which comes into force in August 2013, after finding evidence of “serious malpractice”.

But the MPs’ report finds that while exam boards charge for attendance, they make a loss on training courses.

Andrew Hall, chief executive of the AQA exam board, said: “AQA has never competed by lowering exam standards, although I accept this may have been the case elsewhere in the market in the past. We have been pressing for stronger regulation of standards between awarding bodies for some time and have been pleased to see that the regulator has addressed many of our concerns over the last year.”

Sats test scoring angers school head teachers

Head teachers in England have criticised arrangements for scoring this year’s Sats tests taken by 11-year-olds.

The National Association of Head Teachers says this year’s primary school league tables will be based on “a flawed statistical model”.

The government says this year’s results will be robust but are only an interim arrangement ahead of wider changes.

It is changing the Sats system after a review by Lord Bew.

Last week, 10- and 11-year-olds in England took Sats papers in English and maths, the results of which are used to compile the primary school league tables.

For the first time, the writing element of the English paper is being assessed by teachers throughout the year – and not just on the basis of a standard test.

This was a change recommended by Lord Bew and supported by teachers and heads.

The government has just announced the details of how the English Sats will be scored for this year and published guidanceon this.

In the past, parents have been given an overall grade – or level – for their children’s English Sats. The national target is for children to reach Level Four.

But this year, because of the changes coming in, parents will be given separate ratings for their child’s written work and their ability in reading, but not an overall grade combing the two scores.

They will be given these ratings by schools in July.

‘Disappointing’

What head teachers are unhappy about is that for this year’s league tables, due later in the year, the government will add together the two scores to give an overall grade for a child.

In essence, they say it is not statistically sound to do this because one score will be based on something that has been measured precisely (the reading test) and something that has been assessed“qualitatively” – the written work done throughout the year.

Kathryn James, director of policy for the NAHT, said: “To say that the NAHT is concerned and disappointed with the education department’s guidance on how the overall English score will be calculated this year is something of an understatement.

“We believe this guidance is built on a flawed statistical model which we have flagged up to the government. It is also disappointing that it has taken the department almost a year to produce its guidance, despite reminders from the NAHT.”

Many of the association’s heads joined teachers in a boycott of the Sats tests two years ago, complaining that they were an unreliable indicator of children’s performance and that the league tables damaged learning by encouraging teachers to follow a narrow curriculum by “teaching to the test”.

On coming to power, the coalition government commissioned a review of the tests by Lord Bew, and last year agreed to bring in his recommendations.

These included greater use of teacher assessment and the introduction of a new distinct test of spelling and grammar which is being piloted this year.

The changes should be in place for next year.

‘Useful information’

The government says the method being used to compile this year’s league tables is the best that can be achieved and is an interim measure.

It says the methodology will go through “a further quality assurance process” once the test results have been published in July and before the publication of national results and school performance tables.

A spokesman for the Department for Education said schools were being given guidance and factsheets to help them explain the changes to parents.

He said: “We asked Lord Bew to review the Key Stage 2 testing, assessment and accountability system and make recommendations on how it could be improved, particularly in light of the concerns of head teachers and teachers. We remain absolutely committed to taking those recommendations forward.

“An overall English result has been part of the school level accountability system for many years and Lord Bew underlined its importance.

“For this year’s interim assessment arrangements, we are using the most appropriate methodology available to calculate an overall English result. Parents will receive separate reading and writing results for their child so they have the most useful information.”

Overhaul of GCSE results could mean fewer grades

GCSE results could be overhauled with a cut in the number of grades available suggests the exams regulator for England, Ofqual.

GCSE grades range from A* to G but a new report from the regulator questions whether this will remain the best structure in future.

The government wants a new National Curriculum with new GCSEs by 2015.

An Ofqual spokesman said the curriculum review was a timely opportunity to ask if the grading system met its purpose.

Ofqual’s new corporate plan which sets out its aims for the next three years states: “Before we implement new GCSEs to match the new National Curriculum, we will review the way in which GCSE results are reported …

“The grading structure stretches from A* to G and it is time to look now at whether this is how it should be.”

Prof Dylan William of University of London’s Institute of Education said reducing the number of grades would be a mistake.

“What would be more appropriate to have is a percentage score with a measurement error,” he said.

For example, he suggested, a candidate might score 60% plus or minus 15% for marking errors.

“The problem is, we are not honest about the inaccuracy of assessment,” he added.

Prof William said the wide range of GCSE grades was a throw-back to the days of the two tier O’ Level and CSE exam system.

GCSEs ‘irrelevant’The two grade systems overlapped with a top CSE grade being equivalent to a C grade at O’level.

Professor William also predicted that GCSEs would become increasingly irrelevant as more pupils remained in education until 19.

Sue Kirkham of the Association of School and College Leaders said the number of grades should depend on who the exams are aimed at: “If it’s for the whole cohort, then you need a wide range.

“If you want to have different qualifications for different groups of people you don’t need so many grades.”

Ms Kirkham added that reform of GCSEs should be the subject of a major consultation involving teachers, pupils and employers.