In an audacious raid on the political territory Gordon Brown has long considered his own, the Tories will make support for the most disadvantaged children an education priority in the first year of a David Cameron government.

Legislation will be included in the first Queen's Speech to remove up to 640 of the worst secondary schools in England from town hall control in a move that will be fiercely opposed by Labour. Most of the schools are in its northern and metropolitan heartlands.

Control of the schools would be switched to city academies, charitable trusts, and parent co-operatives in the biggest shake-up in secondary education since the introduction of league tables in 1988.

The Prime Minister has made it his "national challenge" to help the 638 worst schools - one in five - where fewer than 30 per cent of pupils receive five GCSE passes at grades A to C.

Almost 550,000 children are taught at the schools, but Mr Brown has given the local authorities -which have presided over the failure - another four years to turn the schools round.

However, the Tories believe action should be taken immediately and that Labour has not come down hard enough on failing schools.

Michael Gove, the shadow schools secretary, will say in a speech today that time has run out for local authorities.

"We will help new schools set up in areas where local authorities have let parents down," Mr Gove told The Daily Telegraph.

"But, in areas where the same party has been in power for too long, and where standards remain poor, we will have the most failing schools transferred to academy sponsors and others who have a proven record of improving education for the poorest. We will bring forward legislation to do so in the first Queen's Speech."

The Gove plan is modelled on the Swedish experience in encouraging a wave of taxpayer-funded, independently run schools, which were backed by Blairite modernisers but opposed by Mr Brown.

Under the Tory strategy, city academies and other groups could take over local authority schools. They would receive funding directly from central government, but would be free - as academies are - to raise their own additional funds.

They would be required to spend at least the same amount on each pupil as councils do. The schools would then operate independently.

The Conservatives believe neighbouring state schools will not want to lose pupils to the new rivals so will be forced to raise their standards.

The decision by the Tories to remove failing schools from local authority control was given added momentum last year when councillors in the deprived London borough of Tower Hamlets rejected an offer by Goldman Sachs to open a city academy.

The party believes this demonstrated the resistance of many councils to independent sector involvement.

The Tory announcement comes after the most recent Programme for International Student Assessment surveys - the international league tables of education standards - showed that English schools fell from fourth to 14th in science, seventh to 17th in literacy, and eighth to 24th in maths.

Mr Gove said: "The areas of greatest failure can be clearly identified. There are more than 630 schools across England which can't even generate five decent GCSE passes for a third of their students.

"The Government has defined these as the schools where special attention needs to be focused. They are right. It's appalling that you can educate children for 11 years, with the state devoting more than £5,000 every year to every child, and then have schools where more than two thirds of children cannot secure a basic portfolio of qualifications.

"While ministers tell us, like the board of Northern Rock, that their figures can be trusted and their phenomenal achievements are soundly based, the external audits tell a very different story. When we judge our performance against the reality of what other nations are achieving we're falling behind."

The speech, to the Centre-forum think-tank, will highlight the fact that the gap between good schools and failing ones is growing under Labour, blowing a hole in the Government's claims of helping the poorest.

Official figures show that 1.75 million pupils, more than half, are in comprehensives where fewer than 50 per cent leave with acceptable GCSEs. An estimated 25,000 pupils will drop out of secondary schools this year without a single qualification.

In 2006, only 25.3 per cent of pupils in the most deprived areas gained five A*-C grades including English and maths, compared with 68.4 per cent in the least deprived areas.

Mr Gove said this was an unacceptable social divide. "The buck has to stop with the local authorities in whose areas failure has been concentrated. In Manchester; in Sunderland, in Knowsley. All run by Labour - and in most cases run for a generation by Labour."