A rift between Kenneth Clarke and Downing Street opened today when government sources indicated that one of his ministers would be sacked at the next reshuffle after he announced a relaxation of rules governing prison parties.

No 10 took the rare step of issuing a formal instruction censuring Crispin Blunt, the justice minister, after he said he was lifting a "deleterious" instruction banning prison parties. Downing Street also slapped down his suggestion that the government may abolish indeterminate sentences. "No 10 has instructed the Ministry of Justice to make it very clear that there will be no prison parties," a Downing Street spokeswoman said. "I understand the Ministry of Justice guidance to prison governors doesn't quite give carte blanche to such parties, but we just want to make it clear to the public there will be no such parties."

David Cameron was said to have been annoyed when Blunt also took a swipe at his membership of the Oxford Bullingdon Club in his speech. Downing Street sources said the prime minister would not tolerate "freelancing" by ministers. The public slapdown comes amid tensions between No 10 and Clarke, the justice secretary. Sources in Clarke's department said they saw little wrong with Blunt's speech which cited Winston Churchill's call in 1910 for humanity towards offenders. The ministry believed Blunt's speech was in the spirit of a recent speech by Clarke in which he criticised the "bang 'em up" prison culture of the past 20 years. But there was an acknowledgment that Blunt could have chosen his words more carefully when he said he had marked the anniversary of Churchill's speech to rescind an instruction, imposed by Jack Straw in 2008, to restrict prison parties and comedy shows. Blunt said Straw had acted under pressure from the tabloids.

"As a measure it was typical of the last administration's flakiness under pressure," Blunt said. "At the slightest whiff of criticism from the popular press, policy tended to get changed and the consequence of an absurd over-reaction to offenders being exposed to comedy in prison was this deleterious, damaging and daft instruction."

Sources said that Blunt was technically incorrect because the Straw instruction had lapsed naturally. It was then replaced by a new one last Tuesday which gave greater discretion to prison governors on parties.

There was some amusement that No 10 swung into action when the very same tabloids criticised by Blunt turned against him. Today's Daily Mail front page headline read: "Now you pay for prison parties."

Cameron was said to have been annoyed with Blunt's suggestion that members of the Bullingdon Club, who are famous for trashing Oxford restaurants, were able to commit offences, unlike less well-off contemporaries. Blunt quoted Churchill who said privileged young men commit offences "in boisterous and exuberant moments, whether at Oxford or anywhere else" and escape prison.

Blunt said: "There may be connections between the nearest prison to Oxford these days, HMP Bullingdon, and other institutions of that name. I have yet to investigate these and, on reflection, probably won't."

One government source said: "Crispin should probably be looking to join his niece [Emily Blunt] in her acting career."