Gaming the system: NCEA and the Record of Achievement

Five ‘Big Opportunities’ have been identified and tendered for public discussion as part of the Government’s ongoing review of the NCEA system. ‘Big Opportunity 5’ focuses on the Record of Achievement – the distillation of a student’s journey through NCEA – and how it might better reflect what a student has actually learned (and what they haven’t).

Universities in particular have said that the Record of Achievement is very opaque and often masks shortcomings in a student’s learning history that can prove to make all the difference when they’re confronted with the more demanding world of academia.

While the job of Victoria University Dean of Engineering Dale Carnegie may not be to win friends and influence people (with apologies to him, undoubtedly not for the first time), part of it is to ensure that students entering this demanding course of study are equipped with everything they need to hit their potential. Carnegie says that he gets no help in this from the NCEA system and its distillation, the Record of Achievement.

While he is quick to point out that NCEA has in one aspect succeeded – in its mission to keep kids at school longer – he says that’s of no consolation when he finds himself faced with a classroom in which a decent proportion of students are clearly in over their heads, despite what their Record of Achievement may say.

“What’s really clear, especially in engineering and the other physical sciences, is that students are coming through woefully underprepared – not just in terms of content knowledge, but also in study skills and approach to study. The standards and the ability of the students have plummeted to the point that we almost write off the first trimester now.”

Carnegie believes that the Record of Achievement is designed to make the best of a student’s learning record, which isn’t a lot of help in assessing whether they have what it takes to get through their first year. There is a reason why New Zealand is experiencing a chronic shortage of engineering graduates, and no amount of window dressing can change that.

Is ‘achieved’ a predictor of success?

Carnegie and colleagues have run some analysis to examine whether an ‘achieved’ NCEA grade is a predictor of success in engineering, taking in data from most of New Zealand’s major universities. Spoiler: it is not.

“The ‘achieved’ grade is an incredibly poor indicator. I believe that the achieved grade can span anything from the historical 40 percent right through to about 75 percent. We don’t even trust what an achieved grade means anymore. Whereas before, when a student came through with the old bursary results – while that was a flawed system also – we could actually rely on that as being an indicator of student ability. We simply can’t with NCEA.”

Carnegie says that he would go back to the old system “in a heartbeat”, and although he acknowledges it had some serious flaws “at least students were learning a subject in its entirety, and they were getting a numeric grade that was at least a reasonable indicator of ability, whereas that’s lost now.”

Carnegie believes that, in effect, NCEA serves to teach students how to play their cards right.

“Here I’m talking about students of around median ability, shall we say. The high-flyers, they’re fine, there’s no problem at all. But what NCEA is training kids to do is to game play. What we see is that students come through with this expectation that because they’ve been learning in modules, and not learning a subject in its entirety, they think they can just pick and choose what they’re going to learn. Some of them think that they can do re-sits; they can take alternative modules. There are all these ways they can game the system to get past.

“It’s not unusual for us – because we track students’ progress from the time they arrive – at around week four, five or six, to be having conversations with students and saying, ‘Hey, we’re getting really worried that you’re getting behind’. They’ll say things like, ‘Don’t worry, I was behind at school too. I caught up then, I’ll catch up now’. In their minds, failure is nothing to worry about because they’re used to a system in which they can just pick up other modules.

“What we never used to have to do was to teach the students how to study. Now we do.”

A glossy record

Bronwyn Wood is a senior lecturer at Victoria University’s School of Education. She believes that the NCEA Record of Achievement is set up to enhance a student’s post-secondary school opportunities, which is a good thing in some respects, but that this approach can create a less-than-honest portrait of how they’ve conducted their learning.

“Firstly it shows what they’ve achieved, but it doesn’t show what they’ve attempted but didn’t achieve. So it’s very rewarding to the student who has this glossy record of achievement, but at no stage would you know that they’ve actually previously failed courses that they’ve achieved on a subsequent attempt, or that they’ve failed to even attempt stuff that would have been a good idea.”

Wood’s other major criticism of NCEA as a whole, reflected by the Record of Achievement, is that it disincentivises students, who need to learn how to extend themselves before being confronted with the rude awakening that is university.

“There’s absolutely no incentive to do anything challenging or difficult. This is reflected in the Record of Achievement. There’s simply no incentive to do a difficult credit when you could just do a lightweight one as an alternative.

“We’ve got evidence from a study we’re doing for students arriving at university having completed NCEA that shows that 27 percent of them admitted that they avoided completing achievement standards because they didn’t have time in the exam so they just focused on the ones that they wanted to do, or that they didn’t feel like doing it, and wanted to do something else.

“The beauty of NCEA is its flexibility, but that’s also its greatest flaw – the flexibility means that there’s no requirement to do anything that’s difficult at all.”