Online NAPLAN test results called into question

For the first time in 10 years, the MySchool website might not be able to compare 2018 NAPLAN scores between schools or with previous years amid concerns that the results from the online and pen-and-paper versions of the tests are too different.

State education department heads have questioned whether the two sets of results are statistically comparable after seeing the test scores ahead of the release of a summary of preliminary results, which had been due on Wednesday.

There are concerns that the 2018 NAPLAN results could be invalid.

At a meeting of department heads and the Australian Curriculum, Assessment and Reporting Authority (ACARA) last week, three options were canvassed: reporting preliminary results as planned, issuing two separate reports, or not reporting them at all, the ABC reported.

Victorian Education Minister James Merlino slammed ACARA over its management of the NAPLAN online on Wednesday, saying it had been appalling.

Advertisement

"I am extremely concerned," he said. "We are now in a position where the data may not be comparable. It is simple not good enough.

Loading

"ACARA has management of this, this is their responsibility and they have not performed well. From the very moment NAPLAN online started, we have had issues."

Education department secretaries are in Canberra today for a regular meeting at which the issue is expected to be discussed. Mark Scott, the secretary of the NSW Department of Education, said the tests would still give parents an insight into their own child's capabilities.

"What we might not be able to do is compare, for example, system-wide performance from this year to last year or previous years," he told ABC radio. "To compare those who have done the online results to the pen-and-paper results, that might be a bit hard.

"The reports as far as [individual] students are concerned will be accurate and we will certainly be making the case today that these reports should be released as soon as possible."

However, ACARA chief executive Robert Randall said results were not delayed and would be released "within coming days". He said extra care had been taken to ensure the data was compatible with previous years.

"ACARA’s data analysts and measurement advisory experts have advised the data is valid and comparable," he said in a statement.

Fears of incompatibility were raised by principals, parents and teachers before students sat their NAPLAN tests in May.

Chris Presland from the NSW Secondary Principals Council said on Wednesday the 2018 NAPLAN results were always going to be compromised.

"We have stated many times since the decision to continue with NAPLAN online even this year, that as far as we were concerned the 2018 comparable data would be meaningless to schools because of the fact that there are so many variables involved," he said.

"Not only were some kids doing pen and paper, and some doing online. Even in the online environment, there are issues associated with running the test, even assessing writing skills or keyboard skills.

"The minute we had some kids doing it one way and some kids doing it another, it was not like with like."

The online version of the test, sat at about 500 schools in NSW, was tailored to the students' ability. If they answered the first stage easily, the questions would become more difficult; if not, the questions would get easier.

ACARA tried to allay fears of incomparable results before the tests, saying a scaling system would be used to bring tests into line. At the meeting last week, the authority said it was getting more advice on this and acknowledged that, even if it did compare the results, there could be a lack of public confidence, the ABC reported.

The authorities carrying out the online test said it went smoothly, although one official from the NSW Education Standards Authority, which administers the test in NSW, told the EduTECH conference in June that the process highlighted the fact that the digital divide was "real".

The government's MySchool website uses NAPLAN results to compare school performance to the national average and to similar schools.

Outside the exam room, debate rages over NAPLAN's future after teachers and Labor backed NSW Education Minister Rob Stokes' criticism in May of NAPLAN as outdated and misleading.

This year was the first step in a three-year process to move NAPLAN online.

ACARA and the NSW Department of Education have been approached for comment.