Darren is 15, and has just gone back to school after missing 14 months. He is not a truant, was not unwell; he was a top-set student, and he has never been permanently excluded from school. His story encapsulates what experts and campaigners say is a quiet scandal in the education system, with a growing number of children being illegally denied an education through unofficial exclusions and local authority failings.

Darren's mother, Denise, says she was told by Brent local authority, in north-west London, that he had just slipped through a few nets. "That's all the council could tell me," she says. "But that's not good enough. I want to know, how can a child slip through so many nets?"

Darren admits he had behaviour issues – according to his mother, these ranged from the trivial, such as asking why the school wasn't marking anti-slavery day, to an argument with a female student that ended up in a shoving match. These led to a series of internal exclusions, where he and other students in trouble were taught separately. Eventually he was offered a "managed move" to another north London school.

Managed moves were introduced under the Labour government to offer a fresh chance to children at risk of permanent exclusion. They are supposed to be agreed with all parties involved, but are open to abuse, according to experts.

Local authority education welfare officers, speaking anonymously to the Guardian, said managed moves were one of the ways some schools were unofficially excluding pupils, by "offering" them under threat of a permanent exclusion. Denise claims this is exactly what happened to her son.

"I felt bullied into it," Darren says, "and I felt like I would lose all my friends." The move broke down and the new school suggested Denise take her son home to consider her options and write back to them. Neither school informed the council of the breakdown of the managed move, and Darren found himself at home and unable to get back into education for another 14 months.

The Guardian has followed Darren's case for half a year as his mother tried to get him back into school. She applied for places and chased the local authority with letters and phone calls. Eventually the case was investigated by the local government ombudsman, who has just issued a private report.

That evidence, seen by the Guardian, suggests the schools did not follow the managed move protocol correctly when it broke down in July 2011, and that the council was possibly unaware of his status until January 2012. Even then, failings by the council led to him receiving no education until May 2012, when he was offered just three 45-minute home tutoring sessions a week.

By then, Darren had become introverted and had lost interest in studying. "I used to be bright, motivated, more living life … I should be studying for GCSEs, but I'm too far behind now," he said in April. By July, he was not interested in talking about school at all. His mother was alarmed by the change in her son: "He's gone from that, to barely getting a word out of him, miserable. His self-esteem has been severely knocked. It's the complete transformation of a child.

"He's been failed, not by me as a parent but by the system. I've done everything I can … He's been left, effectively, to roam the streets. He could easily be in jail by now, at 15."

When a pupil is excluded officially, the local authority manages the transition to alternative provision or a new school. By law, this should happen within 15 days. When a student is excluded unofficially, according to a report by the Centre for Social Justice (CSJ - pdf), parents do not have the same legal rights, and the position goes unchallenged. "Many pupils are falling through the net – lost to the education system and, in some cases, to society … it's invariably vulnerable children," the report notes.

Managed moves, it adds, are "subject to no regulation and barely any statutory guidance". The process is seen as a "creative and 'increasingly popular' way for headteachers to avoid permanent exclusions, which are recorded, and to 'massage' their figures".

"There are many other tricks of the trade that schools use," said a local authority officer. Children can be sent home to cool off, or on enforced study leave. They can be marked on the register with a "b-code", he says, "a powerful tool" because it says the child is being educated offsite and "doesn't even show up on the records as an absence". He cites a recent case where a headteacher told a parent to home-educate her child and gave her a pre-written letter to send to the council. Another school using such unlawful methods "probably got rid of 100 children".

While official exclusion numbers fell from almost 10,000 in 2003-4 to 5,740 in 2009-10, evidence has been mounting of the two related problems of illegal exclusions and children missing education. In August 2010, Ofsted warned that local authorities were unable to keep track of missing pupils. In the same month, the government cancelled a £235m database called ContactPoint, set up in the aftermath of the Victoria Climbié and Khyra Ishaq neglect cases.

Following the CSJ report, in March 2012 the children's commissioner for England uncovered further evidence in a report that included striking testimony from headteachers. And numbers released under the Freedom of Information Act last year revealed that 12,000 children were registered in local council databases as "missing from education" with 1,500 of these "untraceable".

But the real picture is likely to be worse. Darren was registered on the second school's roll for half a year while not attending the school, and so would not have shown up on any database. "Whereas we used to have some kind of idea, nowadays we don't really know," another senior education welfare officer at a local authority told the Guardian.

"We don't have the same kind of units we had to keep track of attendance, exclusions, children missing in education. It becomes blurred. I could say: 'X number of people are missing education,' but they're just the ones I know about. I would say double it, but trying to put a national figure on it is impossible."

A mix of budget cuts, merging of services at local authority level, and greater independence for schools with the rise of academies and free schools, he says, mean "the job [of an education welfare officer] has become impossible".

David Simmonds, of the Local Government Association, estimates there have been 25% cuts to education welfare and admissions services.

"Despite a 25% reduction in government funding there has been no reduction in the statutory services we have to provide," he says. "If anything, there is an increase as competing schools play pass the parcel with difficult children, with councils trying to ensure they get the education they deserve, which remains a core legal duty."

Despite the warnings from Ofsted, and the investigations by the CSJ and children's commissioner, the Guardian has learned that the Department for Education (DfE) does not monitor the performance of local authorities in tracking children missing from education, despite clarifying the law last September requiring local authorities to provide full-time education.

It has also refused to share what information it held on unofficial exclusion, on the grounds it would be detrimental to "policy formulation".

In a statement, the DfE said: "Unofficial exclusions are unlawful. All schools must follow the legal exclusion process. Schools must ensure pupils on their register are attending. If a pupil is not on a school register, the local authority has a duty to ensure that suitable, full-time education is arranged."

Sir Alan Steer, a former headteacher and "behaviour tsar" for the Labour government, said it was "absolutely neglectful" of the government not to monitor local authorities figures and ensure compliance with the law.

"It comes from a government obsessed with having no regulation … this is irresponsible. You have laws and regulations to protect the vulnerable. They've sacrificed some of this, and at the end of the line is some poor kid who ends up in crime, or drugs, or abuse."

The problem, he says, is not a new one, but one that will get worse with the fragmentation of the education system, and weakening local authority powers and resources. "We often look at the (bad) behaviour of schools, and that's right. But if you look above that level, to local authorities and the Department for Education itself, they are also behaving wrongly. It's a taboo subject, everybody knows the situation but it's a truth that dare not speak its name."

Brent council would say only, on behalf of itself and the schools involved in Darren's case: "Brent council has agreed to take action in response to the local government ombudsman."

That action came in the form of a £2,000 cheque for Darren's mother to compensate for the missed school time.

Darren is now attending "PRU" – a pupil referral unit, for those who have been excluded. When asked if he can make up the lost time, Darren says: "I don't think I can do it, I'm out of it." His mother interrupts: "Don't say that, you have to." They both look away and fall silent.

Some names have been changed

Sign up for the Guardian Today

Our editors' picks for the day's top news and commentary delivered to your inbox each morning.

More on this story

Meet Darren, one of a growing number of vulnerable children illegally denied an education through 'unofficial exclusions' and local authority failings

GCSE schools guide

Interactive: This guide is designed to help parents find and research local schools in England. Search by postcode to find which schools offer individual subjects, and compare how they have performed in GCSE exams