Michael Gove has rejected the suggestion he is soft on crime, telling a Conservative MP that as a Christian he believes in "redemption".

The justice secretary was today accused by Philip Davies of having "gone native" at his department as he was too willing to listen to the likes of the Howard League for Penal Reform.

"When will the secretary of state get back his mojo and actually put the victims of crime at the heart of what he is doing," he said.

Speaking in the Commons, Davies, who is on the right of the party, accused Gove of being more liberal on criminal justice than one of his more liberal predecessors. "Come back Ken Clarke all is forgiven," he said. It was not a compliment.

Gove disagreed. "I am not sure that MPs on the Opposite [Labour] benches would agree I've become a sandal-wearing muesli-munching vegan vaguester. I think they would probably say I'm the red in tooth-and-claw blue Tory I have always been," he told Davies.

"It's because I am a Conservative I believe in the rule of law as the foundation stone of our civilisation. It's because I'm a Conservative I believe that evil must be punished. But it's also because I'm a Conservative and a Christian I believe in redemption."

Gove added that he belived the purpose of the British prison system was to "keep people safe by making people better".

Since being appointed justice secretary following May's general election, Gove has set about reversing the policies put in place by his predecessor Chris Grayling.

He has ditched Grayling's ban on prisoners receiving books in jail and also abandoned a Ministry of Justice contract that would have seen the UK government involved in the construction of prisons in Saudi Arabia.