Raising the standard of teachers is a greater priority than reducing class sizes because poor teachers are harming the chances of many children, the shadow schools secretary, Michael Gove, said this morning.

In a speech outlining a series of ideas to improve the quality of teachers, Gove rejected a Liberal Democrat proposal that would focus attention on cutting class sizes.

Gove praised many of Clegg's ideas on education, including a plan to direct resources at disadvantaged pupils through a "pupil premium", which is similar to a Tory idea.

But Gove said that reducing class sizes would not automatically drive up standards. "In order to reduce class sizes you inevitably need to employ more; you employ more and teaching practice doesn't necessarily rise to the same extent," he told a Tory education seminar.

The shadow schools secretary cited the experience of the US state of California, which failed to improve its position as the second-worst US state for education despite directing resources to cutting class sizes.

Gove said: "California spent an enormous amount of money on reducing its class sizes. At the end of that process they remained 48th in America. There were some measurable improvements in some schools, but these were restricted to schools where the children were all from middle-class backgrounds, and then things settled down after a while.

"That is not to say that [cutting class sizes] doesn't help in all circumstances. It increases the personal contact between a brilliant teacher and children. It is manifestly the case that if you have children who have special educational needs, or are currently struggling, closer attention to their needs is manifestly a good thing.

"[But] ... the key thing is giving schools the freedom to allocate their resources so that strong leaders can choose smaller class sizes in these circumstances [with] a brilliant and inspiring figure, who is going to give more too."

Gove outlined a twin-track approach to raise the standards of teachers: immediate measures to tackle the fear of challenging pupils teachers may experience in the classroom and a longer term plan to attract a higher grade of graduate into the teaching profession.

On the immediate measures, Gove said: "We would change the rules on detention to make it easier to enforce. We would change the rules on confiscating the sorts of devices – from mobile phones to iPods – which disrupt learning to remove the fear of challenge from teachers.

"We would give teachers better protection from vexatious complaints – including anonymity when allegations of abuse are made and a time limit to ensure these allegations are either brought to court or dropped – so that careers aren't unnecessarily blighted by students playing the system."

On the longer term measures to attract high-flying graduates into the profession, Gove is looking at Singapore, where only the top 30% of graduates can become teachers, and Finland, where teaching is restricted to the top 10%.

"The sort of young people who compete for high status jobs here – such as fast-track entry to the civil service, bar pupillages, news traineeships with the BBC, places on the accountancy or management-consultancy milk round – compete to become teachers in these countries."

Gove agreed with Clegg that the freedoms enjoyed by city academies ought to be extended to all schools. But he said that they were being curtailed by internal rows among ministers.

"The freedoms which existing academies have used to drive up standards are now being curtailed. Freedoms over the curriculum, over staffing, freedoms over building and IT, freedom from local authority bureaucracy and central government control, have all been restricted.

"Sadly, education reform has become a casualty of internal Labour politics – specifically the ongoing leadership struggle being fought within the cabinet. In the war of position between Ed Balls and James Purnell, James has positioned himself as the reformer, the heir to Blair, and Ed has positioned himself as the opposite.

"Ed's judgment may be right when it comes to garnering support within the Labour party's electoral college – I defer to his judgment on that subject – but I fear it's wrong for the country. Just when we need to make our education system more flexible, more open to innovation, more dynamic and more influenced by the best international experience, the government are going backwards."