In English education, we are living in extraordinary times. A striking example is the huge growth of academy schools, particularly in the secondary sector, a growth that has happened without key questions about effectiveness, sustainability or value for money being addressed. It is a vast structural experiment, in scale going far beyond what has been tried in any comparable country. However, Unleashing Greatness, a new 130-page report by the high-powered Academies Commission has investigated the issues in some depth. Its report makes extremely uncomfortable reading.

They provide some useful data that will dispel common misconceptions. Although most comment is about sponsored academies, the great majority, more than three-quarters, are now "converter" academies – schools judged outstanding or good by Ofsted that have chosen to become academies since 2010. The other new types of academy which get a lot of attention – "free" schools, university technical colleges and studio schools – account for just 4% of the total.

Two features stand out from the report's penetrating analysis. The first is the gross exaggeration in both the present and the previous governments' claims for the success of this far-reaching and costly policy. Looking at a range of evidence on the last government's academies, the commission concludes that, while there have been examples of stunning success, this is not common. Indeed, many schools in disadvantaged areas that had been performing poorly "have done just as well as those which embarked on the academy route". Even though a lot of money and attention were directed towards around 200 schools, they didn't perform noticeably better than similar schools. Nor did the commission find evidence of widespread innovation across the sector – one of the key arguments for the policy of enhancing autonomy. They think that "arguments for the importance of autonomy may be overplayed … academy status means more autonomy for some and less for others".

An even more disturbing issue is that the report's analysis repeatedly uncovers major holes in the new framework. In a range of areas – including school admissions, governance, accountability, financial oversight and system improvement – serious gaps are appearing which raise profound questions about the policy in terms of effectiveness, equity and sustainability. Only a few examples can be given here.

When "outstanding" and "good" schools (Ofsted judgements) were invited in 2010 to convert to academy status, they had to say how they would support a struggling school nearby. It appears from the evidence the commission received that, under the pressure to show good results and a favourable Ofsted rating, these schools are mostly breaking that promise. Yet at the RSA-Pearson report's launch, Andreas Schleicher, deputy director for education at the OECD, said international research showed that a strong collaborative culture was vital for success. The poorest performing school systems were ones with high autonomy for institutions but low inter-connections between them and within the system generally.

The commission found that, with the move to academy status making many more schools effectively their own admissions authority, the system was becoming unacceptably complex for parents to navigate. This had serious implications for genuine parental choice, social justice and system-wide improvement.

The position of local authorities was increasingly anomalous in that they still have extensive statutory responsibilities, but they have the power to fulfil them only in respect of some schools. The commission regards this position as unsustainable, a judgement it would also apply to the system of individual funding agreements, which breeds confusion as well as a lack of openness and transparency. With no local mechanisms for community and stakeholder engagement, whistleblowing or parental complaints (especially important in an autonomous system), there is a serious democratic deficit.

Perhaps of even greater significance are the commission's warnings about the likelihood of finding enough governors with the time, expertise and qualities to carry out their wide responsibilities as company directors, especially since the requirements for community and stakeholder representation are much more limited for academies than for maintained schools. This raises the question of sustainability even more acutely, as arguably it does, too, in relation to headteachers, an issue the commission doesn't address. However, a recent major review by London's Institute of Education for the National College for School Leadership shows that school leadership has become much more complex, and this puts severe demands on capacity. The report says that the governance of individual academies is a "key risk for the quality of the system overall", and this surely must apply to leadership, too.

The commission sets out sensible recommendations to try to deal with these and other defects in the framework, but it is highly questionable whether most of them could be satisfactorily remedied without changing the whole vision. The report reveals a structure that is fundamentally flawed and that will significantly damage education and worsen inequities.

The academies policy was always a political and not an educational one, and the Department for Education's response suggests it is in no mood to listen, even to closely argued and evidence-based proposals: "The report rightly acknowledges the overwhelming success of the expanding academies programme in driving up standards for hundreds of thousands more pupils. So we make no apologies for accelerating the programme," says the department.

The RSA-Pearson report ends by asserting that "all publicly funded schools should be placed within a common administrative and legal framework". That would be a starting-point towards building a strong and successful school system.

Ron Glatter is emeritus professor of educational administration and management at The Open University.

• This article was amended on 25 January 2013 to change the photograph and caption; the previous photograph was meant only as a general illustration of an academy school and was not to meant to suggest that the article was about the particular school pictured.