The United Kingdom government's plan to import Shanghai-style "math mastery" teaching to England has shown little signs of success in pupils' results, according to a new research report.

The Department of Education's study, which compared test scores to similar English schools that have not implemented the mastery method, found there had been no change in the test results of 11-year-olds.

For seven-year-olds, there was a small change but nothing that would suggest a massive improvement in scores from the new mastery method.

The initiative came after international tests showed that Shanghai students ranked top for math in the Program for International Student Assessment, also known as PISA, in 2012, and that 15-year-olds in China are three years ahead of their English peers in their ability to solve math problems.

Around that time, the UK, whose students ranked 26th that year, decided to adopt some Shanghai math mastery teaching techniques into the British curriculum, hoping that it would push England up the international league table.

Every year since 2014, groups of British and Chinese teachers have visited each other's schools to share expertise and discuss effective methods.

So far, nearly 600 teachers, from both England and China, and mainly from primary schools, have engaged in the exchange, and approximately 5,000 of Britain's 16,000 primary schools have adopted the Shanghai math teaching method.

But, the results from the report have not been encouraging.

The research, which looked only at those schools that had implemented Shanghai math for at least two years with pupils taking the 2017 tests, found taking part in the exchange did not lead to better outcomes at either Key Stage 1 (usually referring to pupils aged between 5 to 7 years) or at Key Stage 2 (pupils aged between 7 and 11 years).

"Analyses of impact of participation in the math teacher exchange alone do not indicate any effect on attainment, either at KS1 or KS2. There is no quantifiable evidence from this evaluation that the math teacher exchange or implementation of East Asian informed teaching alone is leading to improvements in pupil attainment in mathematics at KS2 in comparison with other schools," the report conducted by the Sheffield Hallam University found.

Mark Boylan, principal researcher for the report, said: "For 11-year-olds we found no significant difference once other factors were taken into account. For seven-year-olds we found a small positive effect.

"The KS1 outcome is good news but to put it into perspective, it is the sort of gain in math that could be expected if a school started Philosophy for Children," said Boylan, who pointed out that the findings are inconclusive and do not mean there will not be improvements in the future.

The ineffective outcomes in the findings raised questions about whether the 74 million pound ($97 million) investment on the Teaching for Mastery approach is worth continuing.

Supporters for the scheme, however, argue there is much in the report that is highly positive.

Charlie Stripp, director at the National Centre for Excellence in the Teaching of Mathematics, or NCETM, said: "Teachers in the exchange schools report, in overwhelming numbers, that the opportunity to take part in the exchange and work alongside Shanghai teachers is a transformational professional experience."

English teachers who participated in the exchange program expressed in large numbers the conviction that the changes in math teaching methods that they are implementing will have lasting impact on pupils, in terms of depth of mathematical understanding.

Debbie Morgan, director for primary schools at NCETM, said: "We have never seen 'Teaching for Mastery' as a quick fix and recognize that it will take time to embed into our system.

"I think it will prove effective over time and already teachers are reporting that children's learning has improved, they are becoming more fluent and their understanding is better," she said. "This does not translate immediately into improved tests scores – we are not teaching children more, just teaching it better."

While the Shanghai math adopted in England has not yet had the effect on results that was hoped for, Boylan said that perhaps the teaching method was not the only reason for China's success.

"In some ways, Shanghai math hasn't yet had a fair test in England. UK teachers have been expected to adopt Shanghai approaches, but have not had the same amount of time for professional development or to plan lessons," he added.

In Shanghai primary schools, children are generally taught by the same teacher for two to three years, so the teacher gets to know the pupils very well. Adult attention from parents and grandparents could also help their development.

Boylan said if East Asian methods do not import easily because of other factors, then perhaps UK politicians should broaden where they look for ideas in teaching math. "Switzerland, for example, was the top European performer in math in PISA in 2015 – not far behind East Asian countries. Canada also did significantly better than England."

He noted all high-performing education systems have some common features, which include: supporting teachers with good pay, conditions and status.

"Teachers that are given the freedom to decide what and how to teach with support from researchers but without political interference, also seems to be another contributing factor," Boylan said. "As does having an emphasis on good outcomes for everyone – not just the highest performing children – most high performing countries do not group children by ability from a young age, as often happens in England."