David Cameron's government is accused by Labour of imposing a 'tax on strivers' during prime minister's questions. Photograph: PA

David Cameron has defended the Tories as "the party for people who work" despite admitting that the latest round of benefit cuts unveiled by the chancellor will affect workers who receive tax credits.

The prime minister defended the need to reduce the welfare bill further as he clashed with the Labour leader, Ed Miliband, over government plans announced by George Osborne in his autumn statement last week to limit most working-age benefit rises to 1%, less than half the rate of inflation.

Miliband told Cameron the chancellor was engaged in a game of "divide and rule" by seeking to portray those on benefits as "scroungers", and that the government was imposing a "tax on strivers".

"Despite the impression given by the chancellor of the exchequer, over 60% of those affected are in work," Miliband said in the Commons. "It's the factory worker on the night shift, it's the carer who looks after elderly people around the clock and it's the cleaner who cleans the chancellor's office while his curtains are still drawn and he's still in bed. The chancellor calls them scroungers; what does the prime minister call them?"

Cameron said welfare needed to be controlled as he confirmed during prime minister's questions that everyone in receipt of tax credits would be affected by the changes. "We have to get on top of the welfare bill," he said. "That is why we are restricting the increase on out-of-work benefits and it is also the reason why we are restricting in-work benefits."

But Cameron, pointing to the increase in the personal income tax allowance, added: "We believe in cutting people's taxes when they are in work."

He said a person working full-time on the minimum wage would see their income tax bill halved under the coalition government. "We are saying to working people you can earn another £3,000 before you even start paying income tax. This is why we are taking 2 million people out of tax altogether … because this is the party for people who work; his is the party for unlimited welfare."

Miliband seized on the cut in the top rate of income tax from 50p to 45p, which comes into effect in April, to accuse the Conservatives of being a party that looks after its donors and the people on their Christmas card list while hitting people "they never meet and whose lives they will never understand".

Cameron retorted that Labour's union donors had put Miliband in his post and "determine his policies", adding: "They [Labour] are not serious about welfare, they are not serious about the deficit, they are not a serious party and everyone can see it."

The prime minister hailed the latest job figures that showed a record number of people in work and the largest quarterly fall in youth unemployment yet, but warned there was no room for complacency as he conceded that long-term unemployment remained stubbornly high.

"The good news about the figures today is that long-term youth unemployment is actually down by 10,000 this quarter, so that is encouraging," he said.

The government's work programme would play an important part but there was "more to do".

During a heated interchange, Cameron turned on the shadow chancellor, Ed Balls, who was heckling from the opposition bench. In reference to Balls's poor response to the autumn statement, which subsequently led the shadow chancellor to admit to the difficulties caused by his stammer, Cameron said: "We learned last week, like bullies all over the world, he can dish it out but he can't take it."

Miliband defended Balls, saying: "I've heard everything when the boy from the Bullingdon Club lectures people on bullying. Have you wrecked a restaurant recently?"