If all the places are taken up, non-participation might dip from 14% to around 10%. And yet, as many as 100,000 16- and 17-year-olds currently in employment (with or without training) would still be at risk from the recession.

This isn't the first time youth unemployment has seen a worrying bulge. Since the early 1970s, policymakers have tried over 30 different schemes. So what worked, and what didn't? And have we learned anything, from the millions spent, about what actually gets young people off benefits and into work or full-time education? Our study of past schemes highlights ways of dealing with a very current problem.

The context, of course, has changed. In the 1980s, most 16- to 17-year-olds were in work of some kind, with just 39% in full-time education in 1987. This figure had risen to 73% by 2007. Currently, participation in full-time education falls by around 11% between the ages of 16 and 17. Faced with limited job opportunities, the choice for many 17-year-olds is between remaining in education – and unemployment. Many will, hopefully, stay on in full- or part-time education. But some might reject both, unless labour market interventions can increase the number of jobs with training, and especially apprenticeships.

The research shows that financial incentives for employers haven't worked – they will employ young people if they need them, with or without financial reward. In a minority of cases, employers used financial incentives to replace existing full-time staff with cheaper alternatives.

The main problem has been that wages in youth schemes have been too low compared with market rates. Employers saw themselves being perceived as "mean", and young people were turned off the whole idea. In addition, the training provided by employers has sometimes been poor (for example, in one study, 90% of employers said they had provided training, but only 70% of recruits thought they had actually received any). At the same time, employer thinking is important, and schemes based on consultation with employers at the design stage were found to be more successful.

Financial payments to young people in general are, inevitably, a working incentive. Historically, there was a split between allowances paid at 16 and those paid at 17. This principle has been forgotten in current financial support policy. Policymakers need to consider a higher rate of financial support to encourage 17-year-olds either to stay on in full-time education or enter unwaged training such as programme-led apprenticeships.

The most effective programmes included a level of personalisation. One-to-one support from individual mentors works, as do individual plans with incremental targets that build confidence and allow progress to be monitored. National programmes need flexibility to meet different local needs and participants' expectations need to be managed – inaccurate information and unrealistic expectations have previously been the root cause of disillusionment and dropping out.

• Tony McAleavy is director of education at the CfBT Education Trust. Lessons from History: Increasing the Number of 16- and 17-year-olds in Education and Training is available to download from www.cfbt.com

Sign up for the Guardian Today

Our editors' picks for the day's top news and commentary delivered to your inbox each morning.