Fighting for Equity in Education

The struggle is long but hope is longer

Pyne’s School Autonomy Myth

Friday April 11, 2014

John F. Kennedy once said that “the great enemy of truth is very often not the lie…but the myth” because belief in myths allows “the comfort of opinion without the discomfort of thought” [Kennedy 1962]. Both the Coalition Government and the Labor opposition are enjoying the comfort of the myth that school autonomy in budgeting and staffing leads to better student outcomes while ignoring the discomfort of mounting evidence against it. It is a textbook example of pure ideology in public policy.

The Federal Education Minister, Christopher Pyne, recently announced a $70 million program to turn 1500 public schools into independent public schools by 2017. Independent public schools will have more power to select staff and allocate funding, but will have to teach the national curriculum and pay teachers the same as in other schools. State education departments will decide which schools are given independent status. Each school will receive additional funding to help with the transition to the new governance structure.

Pyne claims that “...the more autonomous a school, the better the outcomes for students” [Pyne 2014a]. The Federal Department of Education website claims that: “The evidence shows, and overseas experience highlights, that increasing school autonomy can help lift student outcomes…” The Coalition’s election policy on schools states that: “A programme to implement Independent Public Schools will lead to higher productivity, better quality education outcomes for students [Federal Coalition 2013: 10].

The reference point for the Coalition’s program is Independent Public Schools (IPS) in Western Australia. Principals in these schools have greater autonomy with respect to budgeting, staff recruitment and selection, the management of teachers and other staff, and maintenance of buildings and facilities. Greater control for principals over budgets and staff is the priority for Pyne. He has emphasised that principals will have more capacity to choose their own staff and decide how the school budget is allocated.

The Labor Government also had its own school autonomy plan called Empowering Local Schools to enable principals to hire teachers and control their own school budget. A former education minister, Peter Garrett, justified it on the same grounds as Pyne claiming that “greater school autonomy is linked with improved student results, behaviour and attendance” [Garrett & Piccoli 2012]. As education minister, Labor Opposition Leader, Bill Shorten, claimed that “schools that enjoy greater local school authority in relation to resource allocation tend to show better student performance than those with less autonomy” [Shorten 2013].

Pyne has made grandiose claims about the research evidence on school autonomy. For example, he has said that “all the studies indicate that the more autonomous a school, the better the outcomes for students” [Pyne 2014b]. He has told the Parliament:

I can point out that the government’s policy on implementing an independent public schools program across Australia will have a major impact on outcomes and results for students. [Hansard, 5 December 2013]

At the launch of the Coalition’s plan, he released a document containing a series of quotes from several studies which he said supported school autonomy [Department of Education 2014]. It cites a range of sources including the OECD’s Programme for International Student Assessment (PISA), the Melbourne Graduate School of Education, the Productivity Commission, the Grattan Institute, the Victorian Competition and Efficiency Commission, and the Gonski report. The Federal Department of Education has also cited World Bank research.

The evidence presented by the Minister is highly selective and misleading.

The last two OECD PISA studies have found that greater school autonomy in curriculum and assessment tends to improve outcomes, but greater school autonomy in budgeting and staffing has little to no effect. Yet, the focus of both the Coalition and Labor school autonomy plans is providing greater autonomy for principals over budgeting and staffing decisions. If anything, curriculum and assessment are becoming more centralised.

The evidence from the 2012 PISA study is unequivocal and compelling:

PISA shows that school systems that grant more autonomy to schools to define and elaborate their curricula and assessments tend to perform better than systems that don’t grant such autonomy….In contrast, greater responsibility in managing resources appears to be unrelated to a school system’s overall performance. [OECD 2013: 52]

The report found no statistically significant effect of greater principal autonomy over budgeting and staffing on student achievement.

It also did a more in-depth investigation of the impact of the two forms of school autonomy in different governance contexts. One analysis examined the interaction between school autonomy and accountability arrangements (such as the publication of school results) on student achievement.

This analysis also showed that school autonomy in curricula and assessment has a significant effect but that school autonomy in managing resources has very little effect. It found that in school systems where all schools publish achievement data, a student who attends a school with greater autonomy in curricula and assessment scores seven points higher in mathematics than a student who attends a school with less autonomy. To put this into perspective, one school year is equivalent to 41 points across OECD countries on the PISA mathematics scale.

In contrast, the gain for school autonomy in staffing and budgeting amounted to about two points and was not statistically significant. The report concludes that “the performance advantage for schools with greater autonomy in this regard is relatively small” [p. 53]. In other words, staffing and budgeting autonomy in the context of public posting of school results has very little impact on student performance.

Examination of the top performing countries in PISA shows the same pattern. Only Netherlands out of the top twelve countries in reading and mathematics has a high degree of autonomy in budgeting and staffing in public schools, with the rest having relatively high centralisation. In contrast, eight countries have relatively high autonomy in curriculum and assessment. Shanghai, the top performing region, has a centralised public school system for budgeting, staffing, curriculum and assessment.

In summary, the new PISA results show that greater school autonomy for curricula and assessment has a positive and significant effect on student performance, but greater school autonomy in staffing and budgeting has little to no effect. This suggests that the IPS model is completely wrong-headed. It extends school autonomy in an area that will have, at best, minimal effect and ignores a form of autonomy that offers more potential for improvement.

The results from the OECD’s 2009 PISA study also found that more autonomy in curriculum and assessment is linked to better student achievement, but autonomy in budgeting and staffing is not. It found that “in countries where schools have greater autonomy over what is taught and how students are assessed, students tend to perform better” and “the prevalence of schools’ autonomy to define and elaborate their curricula and assessments relates positively to the performance of school systems” [OECD 2010: 14, 41].

Interestingly, the first quote is cited in Pyne’s list of research evidence to support the new funding program for independent public schools. Despite the fact that the quote explicitly refers to curriculum and assessment, not budgeting and staffing, Pyne uses it as support for greater autonomy in budgeting and staffing. It is a case of clear misrepresentation of the evidence by the Minister. In misrepresenting this finding of the OECD, he ignored the specific finding of the study relating to autonomy over budgeting and staffing which says:

While there is a clear relationship between the degree of curricular autonomy a school system offers its schools and the system’s performance, this relationship is less clear when the degree of autonomy in allocating resources is analysed through measures such as: selecting teachers for hire, dismissing teachers, establishing teachers’ starting salaries, determining teachers’ salary increases, formulating the school budget, and deciding on budget allocations within the school. [41]

Pyne similarly misrepresents another OECD report, The Lessons from PISA for the United States [OECD 2011], as supporting independent public schools. However, the citation he uses from the report refers to autonomy in instructional policies and practices. All public schools in Australia have a significant degree of autonomy in teaching practices and, in any case, this is not the focus of the independent public schools plan.

The 2009 PISA study found that in the vast majority of 64 countries participating in PISA 2009, including in Australia, there was no relationship between student achievement in schools and the degree of autonomy in hiring teachers and over the school budget. It concluded emphatically that “...greater responsibility in managing resources appears to be unrelated to a school system’s overall student performance” and that “...school autonomy in resource allocation is not related to performance at the system level” [41, 86 (note 7)].

The one qualification is that “...where schools are held to account for their results through posting achievement data publicly, schools that enjoy greater autonomy in resource allocation tend to do better than those with less autonomy” [14]. However, the impact is trivial. Students in higher autonomy schools achieve only 2.6 points higher in reading on the PISA scale than those in an average autonomy school. To put this in perspective, increased learning over the school year amounts to an average of about 35-40 points on the PISA scale. This is hardly compelling evidence.

The national report on Australia’s 2009 PISA results shows virtually no difference in the correlation between school autonomy and student achievement in NSW, with the lowest degree of autonomy of any jurisdiction, and Victoria which has a high degree of autonomy [Thomson et.al. 2010: 274]. Moreover, it found no significant relationship between student performance and school autonomy in budgeting and staffing in any school sector – government, Catholic or Independent – even though government schools overall have significantly less autonomy than Independent schools.

Pyne also selectively and misleadingly cites several Australian reports. He cites a finding of an evaluation of IPS in Western Australia by the Melbourne Graduate School of Education [2013] that principals support the program and believe it will lead to increased student outcomes. However, he ignores the many statements in the report that there has been no improvement in student outcomes:

In this early phase of the IPS development there is little evidence of changes to student outcomes such as enrolment or student achievement. [8]
....there was no evidence of substantial differences in outcomes between schools that were selected into IPS and those that were not. [9]
Analysis of the secondary data shows that IPS were generally high-performing before transition, and there has been no substantive increase in student achievement after becoming IPS. [36]
“Similarly to student achievement data, analysis of available data on student enrolment and behaviour across all public schools showed no change for IPS. [38]
....the secondary data shows no substantial change in staffing, student behaviour, attendance or performance between IPS and other public schools. [56]

World renowned education academic, Professor John Hattie, who assisted in the WA evaluation told the ABC’s FactCheck [2013] that before joining the initiative the schools involved had better academic results than other public schools, but they made “no improvement under the ‘independent’ model” .

Pyne’s citation of support by the Productivity Commission ignores its conclusion that the evidence on school autonomy is mixed. The Commission’s report on the schools workforce noted that past studies “have found mixed impacts from delegating decision-making to schools” [Productivity Commission, 2012: 246]. It also said that “...allowing schools greater autonomy has the potential to exacerbate inequalities” [44] and that “increased autonomy could, in several respects, work against the interests of disadvantaged students” [277].

Pyne also cites a report by the Grattan Institute, The Myth of Markets in School Education, as supporting school autonomy. Yet, its conclusion is in stark contrast to Pyne’s claim:

On autonomy, Australia and other countries have the wrong strategy. The world’s best systems have varying levels of autonomy. But it is not central to their reforms. Instead, they articulate the best ways to teach and learn, then implement reform through high-quality systems of teacher development, appraisal and feedback, among other policies. Autonomous schools in Australia and other countries are no better at implementing these programs than are centralised schools. [1]

The school education program director at the Grattan Institute, Ben Jensen, totally contradicted Pyne recently in saying that the evidence that independent public schools produce better student outcomes was “very mixed” and could even leave students worse off. He noted:

Victoria has gone very far down the autonomous path – it was one of the first in the world to do so – while NSW has had a very centralised system and they have, essentially, the same results. [cited in Knott 2014]

As Pyne says, a report by the Victorian Competition and Efficiency Commission, Making the Grade: Autonomy and Accountability in Victorian Schools, does support greater school autonomy in budgeting and staffing. However, the Commission’s support is based on faith rather than evidence and it was forced to acknowledge the lack of evidence for its faith:

The existing body of empirical evidence on the impacts of school autonomy is by no means definitive. The international evidence is mixed regarding its impacts on student achievement and educational opportunity. It is also not conclusive about which areas or domains might benefit from autonomy. [47]

Ironically, Pyne cites the Gonski report on school funding as supporting greater school autonomy. However, the report failed to do a comprehensive review of the evidence. It cited only one study amongst a vast literature on the subject [Woessmann et.al. 2007]. The study was based on a cross-country study using the 2003 PISA data. Pyne has also cited a review of several studies using international test data from the early 2000s [Woessmann 2007]. The findings of these studies have been contradicted since by the 2009 and 2012 PISA studies referred to above.

The Federal Department of Education has cited World Bank research as supporting greater school autonomy. An Associate Secretary of the Department, Tony Cook, told Senate Estimates that:

The World Bank has done research that would indicate that in those countries where there is greater autonomy in their schools, the performance of those countries in terms of international assessments is higher…. [Parliament of Australia 2013: 23]

This claim is false. A recent World Bank review of research studies says that there is no convincing evidence of the effects of school autonomy in Australia, New Zealand and the UK on student achievement [Bruns et. al. 2011: 11]. The review focuses on studies of school autonomy in developing countries and notes that there are few rigorous studies available and that the evidence on impact on student test scores is mixed [12, 103, 106, 131].

Thus, the evidence presented by Pyne in support of independent public schools fails to sustain his case. His evidence is highly misleading and relies on selective citation. As one education academic observed, Pyne’s claim is based on “a range of cherry picked quotes” [Savage 2014]. It has to be said that one of Pyne’s Labor predecessors, Peter Garrett, also adopted the same practice, but at least he admitted that “...there is little, if any, evidence to suggest overseas experiments like charter schools or student vouchers would lead to better education results” [Australian Financial Review, 1 March 2013].

The list of research studies that Pyne presented in support of his claims is highly selective. It ignores many international studies of a variety of types of school autonomy around the world that show the evidence is not compelling. While some studies show positive effects, the mass of evidence from recent research studies in several countries is that it has little impact on student achievement [for a detailed review see Save Our Schools 2013].

New Zealand has the most decentralized school system in the western world. It is unique in that government schools are stand alone schools. Yet, a major study by the head of research at the New Zealand Council for Educational Research found that there has not been any significant gains in student achievement, new approaches to learning, or greater equality of educational opportunity since school autonomy was introduced in 1989 [Wylie 2012]. She said that New Zealand should return to more central and regional support for schools

Charter schools in the United States are another form of school autonomy. They are independent public schools. The weight of evidence from the most sophisticated studies of charter schools is that there is no difference in results between charter schools and traditional public schools [Save Our Schools 2013]. Indeed, some studies show that charter schools do worse.

A new large national study of charter schools found that three-quarters of all charter schools are doing no better than traditional schools in reading and 70 per cent are doing no better in mathematics [CREDO 2013]. The study concluded that the greater school autonomy granted to charter schools had little effect on student achievement over time.

Sweden’s free schools are privately-operated schools which receive the same level of government funding as municipal schools. They have been operating since 1992 and many are run by for-profit companies. The research evidence on these schools is mixed – some showing better performance by free schools and some showing better performance by municipal public schools. A recent review of studies published by the Institute of Education at the University of London found that the benefits were small, largely concentrated on children from highly educated families and do not persist through to the end of school [Allen 2010].

Academies and foundation schools in England are publicly-funded schools that have greater freedom over how to allocate their budgets and over staffing than more traditionally-governed state schools. Research evidence on foundation schools shows no increase in student achievement while some studies of academies show improvement and others no improvement [Save Our Schools 2013]. A major review of academies concluded that “it is increasingly clear that academy status alone is not a panacea for improvement. [Academies Commission 2013: 4].

The evidence that school autonomy has little effect on student achievement continues to accumulate. For example, a recently published evaluation of an autonomous school management program in Chicago’s public schools found that greater autonomy had no statistically significant impact on average reading and mathematics results [Steinberg 2014].

Pyne’s claims about the success of independent public schools are even rejected by his own colleagues. For example, a bi-partisan report by the Senate Education Committee stated that “...it is unclear whether school autonomy ultimately improves student outcomes.” [Senate 2013: 47]. The NSW Minister for Education, Adrian Piccoli, has said: “We don’t believe the research supports creating either charter schools or fully independent public schools” [Sydney Morning Herald, 5 February 2014; see also 20 July 2013].

All this shows that Pyne and Labor have got it drastically wrong on school autonomy in budgeting and staffing. Their claims of success are greatly exaggerated and based on highly selective and misleading evidence. The evidence shows that promoting independent public schools as a centrepiece of education policy to lift student results is ill-conceived and misplaced.

In his address referred to above, President Kennedy exhorted everyone to “move on from the reassuring repetition of stale phrases to a new, difficult, but essential confrontation with reality”. It is clear the Coalition is not up to this and there is considerable doubt about whether Labor is up to it. We can only hope that some politicians may have the decency and the integrity to confront the reality of the evidence on school autonomy and revise their position.

Trevor Cobbold

This article was originally published in the Autumn/Winter 2014 issue of Dissent Magazine.