The justice secretary, Ken Clarke, will today launch a scathing attack on the Victorian "bang 'em up" prison culture of the past 20 years.

His speech to the Centre for Crime and Justice Studies in London marks a major assault on the "prison works" orthodoxy launched by former Tory home secretary Michael Howard – and is believed to be causing nervousness in Downing Street.

Clarke will warn that simply "banging up more and more people for longer" is actually making some criminals worse, without protecting the public.

"In our worst prisons it produces tougher criminals. Many a man has gone into prison without a drug problem and come out drug dependent. And petty prisoners can meet up with some new hardened criminal friends," says advance extracts of his speech.

Clarke faces mounting pressure to halt the £4bn prison building programme – the largest in Europe – and his speech will fuel expectations that he intends to divert thousands of offenders away from short-term prison sentences when the government's review of sentencing is published in the autumn.

The justice secretary faces a battle if he is to stabilise the growth of the prison population, which is forecast to rise to 94,000 before the next general election.

Clarke was last in charge of prisons when he was home secretary between 1992 and 1993, when the prison population in England and Wales stood at 44,628. He says today that the current population of 85,000 is "an astonishing number which I would have dismissed as an impossible and ridiculous prediction if it had been put to me in a forecast in 1992."

He says that "for as long as I can remember" the political debate on law and order has been reduced to a competition over whether a government has spent more public money and locked up more people for longer than its predecessor. It now costs more to put someone in prison – £38,000 – than it does to send a boy to Eton.

He said: "The consequence is that more and more offenders have been warehoused in outdated facilities and we spend vast amounts of public money on prison. But no proper thought has been given to whether this is really the best and most effective way of protecting the public against crime."

Clarke will point out that prison is the necessary punishment for many offenders, but he questions whether "ever more prison for ever more offenders" always produces better results for the public. He provides his own answer by observing that the record prison population and the crime rate in England and Wales are now among the highest in Western Europe.

He says that just locking people up without actively seeking to change them is "what you would expect of Victorian England" and he notes that reoffending rates among the 60,000 prisoners given short sentences has reached 60% and rising.

"This does not surprise me. It is virtually impossible to do anything productive with offenders on short sentences. And many of them end up losing their jobs, their homes and their families during their short time inside," says Clarke.

The justice secretary's speech will fuel expectations among prison reform groups that the sentencing review will lead to a drive to divert short-sentence inmates away from prison.

But Clarke himself is careful not to spell out that solution in today's speech. He says that a "far more constructive approach" is to make prisons places of education, hard work and change, and to provide rigorous enforced community sentences that get offenders off drugs and alcohol and into jobs.

In doing so he puts his weight behind "the most radical" Conservative plans for a "rehabilitation revolution," involving the voluntary and private sectors in programmes to change offenders inside and outside prison, and paying them by results.

"They would have clear financial incentives to keep offenders away from crime. And success would be measured by whether or not they are reconvicted within the first few years of leaving prison," he will say.

Howard's "prison works" approach was outlined in October 1993 and has held sway ever since.

Clarke's speech marks a return to the language of former home secretary Douglas Hurd's 1991 white paper, which said that "prison was an expensive way of making bad people worse" – and in those days the prison population stood at only 42,000.