Grammar and punctuation at centre of NAPLAN controversy

By Jordan Baker & Henrietta Cook

10 August 2018 — 12:00am

Poorer performances in grammar and punctuation by students who sat NAPLAN online are among the problems worrying education officials as they desperately try to resolve a stand-off that is delaying the release of this year's results.

The agency in charge of NAPLAN, the Australian Curriculum, Assessment and Reporting Authority (ACARA), held an emergency meeting on Thursday ahead of another discussion between state education officials via teleconference on Friday aimed at finding a resolution.

Loading

Almost 200,000 Australian students sat NAPLAN online this year and the rest did a pen-and-paper version, but state education ministers and directors general are concerned the two sets of results are not statistically comparable.

Ministers in two states said they were considering withdrawing from NAPLAN online until their confidence was restored.

Advertisement

If they decide results from the online and written tests can't be compared, there will be implications for the controversial My School website, which contrasts each school's performance against an average of similar schools, and for the national snapshot of student performances.

The Herald understands the main problem with differing results relates to the grammar and punctuation test.

One of the innovations of NAPLAN online is that the test adapts to the child's ability. If the students get the first set of answers correct, the questions get harder. These tests give a more accurate diagnosis of strengths and weaknesses.

But this year, strong performers in the reading test were given difficult questions from the beginning of the grammar and punctuation test. They did not get the so-called "confidence building" questions, a key part of test design.

The students who sat the written version did have those confidence-builders. As a result, the top-performing students in schools that ran the test online did not perform as well as the students who sat the written version.

Because the online version is more accurate, it also more effectively separates the very top and bottom-performing students across all the tests, so some of the highest performers might appear to have not performed as well as they did last time.

“On the pen-and-paper test there are very few questions for the upper and lower levels, which means you have less information about the student’s ability and what they are ready to learn,” said University of Melbourne emeritus professor Patrick Griffin.

Loading

It was “almost impossible” for very bright students to hit the ceiling with the adaptive, online test, he said.

Education officials are also concerned about publishing a national data snapshot when each state had different levels of involvement in the online test. In Canberra, every school did NAPLAN online; in the Northern Territory, none did.

In Queensland, 20 schools participated. In NSW, there were more than 500.

"Unless we can have confidence in NAPLAN online I have serious concerns about continuing the transfer from paper tests to online tests," said Victorian Education Minister James Merlino. He called for an urgent review into the comparability of the figures.

Western Australia, which had many schools running the new version of the test, threatened to pull out of the three-year transition to NAPLAN online until the system was ready. Education Minister Sue Ellery described the discrepancies as "incredibly frustrating”.

ACARA has long reassured the states that a scaling system could equate the different test formats. But state education officials are understood to be reluctant to allow too much massaging of the data.

The preliminary national results had been due to be released this week, with the schools and students to be given their results in mid-August (that will still happen by the end of the month, ACARA said on Wednesday).

At Friday's phone hook-up, state and federal education chiefs will debate if and how to release the national performance report. The most likely outcome is a release of data with caveats explaining which states and schools did the online version.