Prime Minister Tony Abbott has refused to back comment by his Treasurer Joe Hockey that the poor "don't have cars or actually don't drive very far".

Mr Abbott, speaking in Canberra on Friday, insisted Mr Hockey still had his support after the disastrous comments, which have left the Treasurer's colleagues and political allies questioning his judgment and the quality of advice he is receiving.

The Prime Minister said Mr Hockey was the only person with a plan to make life better for Australians, but distanced himself from the Treasurer's claim that poor people would be less affected by an increase in fuel taxes because they either don't have cars or don't drive very far.

Asked if Mr Hockey had an image problem and appeared out of touch with ordinary voters, Mr Abbott responded that he was ''proud of the budget that the Treasurer and I have brought down''.

"I want to focus on the fundamental challenges facing our country and the fundamental challenge that the Treasurer and I are totally focused on is how do we manage Labor's debt and deficit disaster, how do we get our budget back under control, so we can make our economy stronger so that every Australian can be better off," he said.

Opposition Leader Bill Shorten said on Friday that Mr Abbott was indirectly criticising his Treasurer by saying he would not have made similar comments.

''It's clear to me that this friendless budget, this unfair friendless budget now has a friendless Treasurer championing the budget. This is a friendless budget with a friendless Treasurer,'' Mr Shorten said.

Mr Shorten said the Prime Minister now had ''a big problem on his hands'' that he suggested be dealt by either dumping the budget or Mr Hockey.

''It's probably now time that the Prime Minister considers if the Treasurer can't sell the budget, you either dump the unfair budget or dump the Treasurer,'' he said.

The Prime Minister's comments came after senior minister Christopher Pyne declined six times in an interview on Friday to say he supported Mr Hockey's assertions about poorer Australians.Mr Pyne said Mr Hockey had been ''standing up for low-income workers; and middle-income workers''.

"The simple fact is that this is the guy who has driven the abolition of the carbon tax, the cost of which fell heaviest on low-income households,'' he said, adding that Mr Hockey had also opposed Labor's move to scrap a self-education tax deduction and the changes to fringe benefit tax changes for cars," Mr Pyne told the Nine Network.

"What he is talking about is the fuel excise changes, which are a 40 cents a week impost on households on average."

But Labor transport spokesman Anthony Albanese swiftly pointed out that Mr Pyne had "had six opportunities to support what he said and you refuse to do so".

"What he said was rubbish, if you are in an outer suburb or a regional community you have no choice but to drive and this is the guy who cut all funding for public transport in the budget,'' Mr Albanese said on Friday.