I thought I knew Iain Duncan Smith

I no longer recognise the Iain Duncan Smith with whom I have had a cross-party friendship for eight years. In 2002, as the Conservative party leader, he visited the young people's project I helped to found in Easterhouse, Glasgow. I was impressed by his willingness to take local residents seriously. He has described the visit as a kind of epiphany: "I saw the poverty among a swath of forgotten people. I felt I had to do something and came away a changed man."

After losing the party leadership he founded the Centre for Social Justice, which has published some insightful reports. But since becoming work and pensions secretary this year he appears to have accepted old Tory policies on every crucial issue. Let's look at a couple of them.

Poverty. In 2003, Duncan Smith said: "I want to be the party for the poor." He bravely spoke at a 2005 Labour party conference fringe meeting, saying Labour's definition of poverty was too limited: it is "not just about a lack of basics but a lack of sufficient resources to participate in the life of the community". What has happened now? At a time of rising food prices, the freezing of benefits and tax credits makes the poor even poorer. The new system of local housing allowances will mean 774,000 households losing an average of £9 a week.

Communities. Duncan Smith has said again and again that stable communities and families are the core of a good society. And yet the cap on housing benefits will force thousands of families to move to cheaper areas. Simultaneously, deprived communities to which they are drafted will find it difficult to accept more pressure. One of Duncan Smith's great achievements has been to highlight the capacity of locally run community groups to improve local life. He promised his party would legislate to ensure that "public money would flow to more diverse, innovative and locally based projects". But the cuts imposed by the coalition government on councils have resulted in grants to many local groups being axed; and many are closing down entirely.

My long experience in deprived areas tells me that the number who make a rational decision to live on benefits is tiny. I know others who cannot face working: those with mental health problems, for instance, or severe behavioural difficulties. These are the very people that the small voluntary projects, highlighted by the Centre for Social Justice, can help. But this involves building relationships and providing support, not compulsion.

Most jobless people do not need this treatment. I have a friend, a single man with no qualifications. He has made numerous job applications but is still long-term unemployed. Recently he got as far as an interview. He was one applicant among 20. He failed again. Last year he wanted to help as a volunteer for our annual camp. The week coincided with his signing on, and officials warned that if he went his money would be docked.

It would make more sense if local groups like ours were financed to employ people like him. Why didn't Duncan Smith propose this?

The IDS I knew was a politician who almost wept at the plight of the poor. My guess is that, in order to reach his costly goal of a universal credit scheme, he has had to mollify the chancellor, George Osborne – and that can only be done by being like those Tories who take pleasure in punishing the poor.

There is an alternative. I have observed his rare gift of being able to listen to and communicate with people crushed by social deprivation. I believe he should leave the cabinet and devote himself to the cause of those at the hard end. He cannot create compassionate Conservatism alongside Osborne and Cameron; the danger is that they will change him instead.