Why, as revealed this week, will more than 20% of parents applying for places in secondary schools in England this September fail to get their first choice? Easy, say politicians and leader writers. There aren't enough good schools, or enough places in the ones that exist.

So we must create more good schools with more places, while closing the bad schools or beating up their teachers until they improve and become good schools. Then all will be well.

Over the last decade, we've heard this story as often we've heard Gordon Brown say he was a thoroughly prudent chancellor and, if things went belly-up, it was somebody else's fault. But what do we mean by a good school? Examination results are the only criteria we accept, and politicians imply these have something to do with quality of teaching, sound leadership, strong discipline, clarity of aims and so on. And these can all make a difference. The brutal truth, however, is that the surest way to turn a bad school into a good one is to change the pupils who attend it.

As research from the London School of Economics has shown, the differences between intakes even among community comprehensives is enormous. "The average ability of pupils going into the 'best' comprehensive schools is way above average ability in the worst," says a summary of the research, published in the LSE's magazine CentrePiece. The scores of the average pupil in the "worst" comprehensive fall into the bottom third in tests at 11, while the average in the "best" come in the top third. If schools that have greater freedom to select pupils are included - church schools and academies mostly - the gap is even greater.

Test scores at 11 aren't infallible predictors of performance at GCSE and A-level, but they are a good guide. Parents, particularly middle-class parents, look at exam results and choose accordingly.

Some don't bother with the results; they see well-scrubbed, nicely dressed and well-behaved pupils, and say that's the one for their child. They are right: home background, as you'd expect when you think how much more time children spend at home than at school, is another guide to attainment. The government tries to make allowances for differing intakes by publishing "value-added" scores in the annual league tables of results. These supposedly measure how far a school improves the children it recruited, with a score of 1,000 being the average.

If parents took this seriously - and even statisticians mostly don't - they would clamour to send their children into the London boroughs of Islington and Lambeth where all schools, with two exceptions, record scores well above 1,000. In fact, parents will do anything, including sending their children across London on public transport, to avoid those schools. Likewise, if "value-added" meant anything, Kent's grammar schools would have closed long ago, since exactly a third record scores below 1,000.

In effect, then, schools are judged on their intake. A school can change the head, sack teachers, crack down on truancy and bad behaviour, draft a new "mission statement". These things may make the school happier, more peaceful, more businesslike. They may even improve exam results. They will make no long-term difference at all unless the intake changes. Rebranding the school, usually as an academy with a new building and a rich sponsor, may make a difference, attracting aspirational parents for a year or two. A bright new wrapper always helps, for schools as well as chocolate bars. Schools may also manipulate admission arrangements to favour more able children. That is easier for church schools and academies because they, not local councils, control admissions. Despite government attempts to tighten the admissions code - by forbidding interviews with parents, for instance - such schools still find ways round it, as more LSE research showed this week.

But if one school achieves a better intake, the less able children blight another nearby school, which then becomes "bad". So after 30 years of giving parents more freedom to choose schools - which was supposed to transform standards by putting customers in charge - we are still left, by the government's own estimation, with hundreds of "bad schools", along with frustrated and angry parents who've been offered choice but don't get what they choose.

Even when the school system as a whole improves, raising exam results year by year, the goalposts are shifted. In 2007, Downing Street boasted that the number of "failing" secondary schools was down "from over 500 to barely 200". A year later, Ed Balls, the children's secretary, demanded "action plans" for 638 low-performing secondaries. Had there been a sudden deterioration in results? Not at all. The criterion for failure used to be fewer than 15% of pupils getting five or more GCSE A-C grades. Then ministers raised it to 30%, with the added requirement that the five must include English and maths. If every school got over the 30% hurdle, would all be well and parents happy to accept a school that managed 31%? Of course not. As long as schools have differing pupil intakes, some will be deemed "good" and some "bad".

Is there an answer? Is it possible to give every school a roughly similar intake, or at least a balanced one so that none has a preponderance of low achievers or an overwhelming proportion of alienated pupils from disadvantaged backgrounds? The latest big idea is to introduce random allocation (crudely, lotteries) to decide which children get places when a school is oversubscribed. This creates an admissions system that is simple and transparent, rather than one that involves complex criteria for who gets preference, with admission forms that run to several pages. Potentially, it replaces a "postcode lottery", whereby children attend schools according to where they live, with a genuine lottery that gives children from deprived areas as much chance of entering a favoured school as those from affluent homes.

But the idea is fraught with practical difficulties, as responses to its tentative introduction show. Many middle-class parents, who have bought expensive houses close to "good schools" (ie, schools with favoured intakes), believe that to deny them places is a form of expropriation, not least because their house values are likely to fall. It is hard, in any case, to argue that 11-year-olds, even if they are the beneficiaries of privilege, should be denied school places within walking distance of home. If success in a lottery is the sole criterion for admission, many children will be parted from primary school or neighbourhood friends, even from siblings.

We need to face the truth. Britain has a problem with school admissions because it is a grossly unequal society, in which rich and poor are segregated. Schools wouldn't have such grossly dissimilar intakes - and such dramatically contrasting results - if extremes of wealth and disadvantage, which Labour has reduced only marginally if at all since 1997, were not so great. The never-ending debate about pupil admissions and bad schools is a diversion. We have a social and economic problem, and we have to tackle that before we have a hope of curing educational problems.

• This article was amended on Monday 9 March 2009. We previously said more than 20% of parents applying for places in secondary schools this September will fail to get their first choice. That figure related to schools in England only. This has been corrected