One of Australia's top retail bosses says the unemployment rate is putting a dampener on his sector and has made a controversial appeal for the new federal government to address pay rates for retail staff.

"The increases in penalty rates that we've suffered under both through general retail award and the Fair Work Act have been substantial," he said.

"They've made us uncompetitive on a world scale and our view is that we're not looking for a winding back of those penalty rates.

"What we are looking for is stopping the continued escalation of them, particularly on Saturdays and Sundays and after hours during the week."

The increases in penalty rates that we've suffered under, both through general retail award and the Fair Work Act, have been substantial

Bernie Brookes

However, the Australian Council of Trade Unions (ACTU) says the comments confirm the union movement's fears that the new government will face pressure to get rid of penalty rates.

"Business will put enormous pressure on the Abbott government to get rid of penalty rates," ACTU president Ged Kearney said.

"We will fight rigorously against a reining in of penalty rates. We will prosecute our argument at the Productivity Commission, we will campaign heavily around it. We will, if we have to, mobilise the workers who depend on penalty rates.

"To say you can take $150 a week out of a lowly paid workers' pockets, or their household budget, could be devastating to those workers."

Ms Kearney says there is no evidence that current penalty rates should be changed.

"There has been no increase. I mean, you have to go through a whole process to have penalty rates increased," she said.

"They are based in awards and they are based in bargaining processes. There's really nothing to indicate that there is a problem."