Education experts last night called for a more balanced view of Wales’ schools after the validity of an international benchmark was called into question.

Compiled every three years, the Programme for International Student Assessment (Pisa) is the world’s biggest education survey and involves 67 member countries from across the world.

It is designed to study how different countries’ education systems are performing against one another and was last published in December 2010.

Wales’ dismal performance in the latest round of tests, which found Welsh students languishing well behind their UK peers, prompted widespread criticism.

A series of measures have been introduced to raise the bar but while the need to improve is widely held, not everyone gives Pisa and its findings the same weight.

A new study from the Institute of Education claims that England – which performs far better than Wales by comparison – has been harshly treated and evidence of a decline in standards cannot be justified.

In his analysis of English pupils’ scores, Dr John Jerrim unearthed problems with missing data, survey procedures and the target population.

He said the fact there are more participating countries now than when Pisa launched in 2000 could have something to do with England’s slide down the rankings.

Dr Jerrim said the introduction of poorer performing nations into the fray – like Wales, which joined Pisa in 2006 – would inevitably impact on scores.

Welsh pupils scored an average of 472 on the Pisa 2009 maths test compared to 492 for those from England and Dr Jerrim said: “The fact that Welsh schools did not take part in the Pisa 2000 study hence means that the average Pisa maths test score for England in that year is likely to be higher than in the other survey waves.”

Next page: Different agencies

Dr Jerrim also points out that Pisa has been conducted by different agencies – which employ contrasting methods – in England.

“Perhaps the most important is the month when children sat the Pisa test moved from between March and May (in Pisa 2000 and 2003) to roughly five month earlier in Pisa 2006 and 2009,” he said.

“England had special dispensation to make this change and although it was for good reason (Pisa clashed with GCSE exam preparation) it may have had unintended consequences.”

Professor Gareth Rees, an expert on Welsh education, said he has his own reservations.

The academic, based at Cardiff University, said Wales was right to join Pisa but there are fundamental questions about how data is collated.

“Pisa is an important study but as with any set of results, we’ve got to interpret them sensitively,” he said.

“The notion is that, between 2006 and 2009, there has been a deterioration but you can’t conclude that from those results because each measurement is based on different pupils.

“We shouldn’t conclude that because the Pisa results have gone down that everything is terrible in the Welsh education system and that the system is deteriorating.

“The countries which do well in Pisa have a school curriculum that fits with what Pisa measures.”

Professor David Egan, who was a government advisor when Wales first took part in Pisa, called for a more “rounded” view of the education system.

But Rex Phillips, NASUWT Wales organiser, accused the Welsh Government of “misusing” Pisa data for its own purposes.

He said: “The minister’s 20-points were a cynical reaction to Pisa in order to push forward his agenda for change in Wales. He should take stock of the Jerrim report and reflect on its content.”

A spokesman for the Welsh Government said: “Pisa is not the only piece of evidence that points towards a need to improve in Wales, as the minister pointed out in his speech last February, but it is an important external benchmark that we would be remiss to ignore.

“There can be little doubt that recent results, including Pisa and other examination and assessment data, were a wake-up call for the education system in Wales.”