Plans by Benetton to turn a historic palazzo on Venice's Grand Canal into a shopping centre have met with fierce resistance from Italian heritage experts.

The fashion retailer with headquarters in Treviso has agreed with the city to convert the Fondaco dei Tedeschi, which sits metres from the Rialto bridge, into a bustling collection of shops, complete with roof terrace and escalator spanning the internal courtyard.

The company has guaranteed a €6m (£5m) contribution to the city's depleted coffers if all building permits are handed over by the end of 2012.

But the heritage group Italia Nostra has mounted a legal challenge to the plan, alleging "serious damage to the building's physical integrity and historical identity".Leading art historian Salvatore Settis, in a front page article in Monday's La Repubblica, condemned the idea of carving out a terrace for tourists on the roof and accused Benetton of using its financial clout to buy "full and fast obedience" from the council.

The dispute is the latest clash between conservationists who oppose the creeping commercialisation that tourism brings and developers who believe that without shops and services, Venice risks becoming a museum without inhabitants.

"A city with just museums will die," said Benetton's spokesman, Federico Sartor. "There is lots of culture in Venice but you cannot find a sandwich."

Sartor said the escalator, which will sit in the courtyard under a glass roof, was a must for modern shopping complexes, "because people won't go up to the higher floors if forced to use the stairs", but stressed that the palazzo had already been tinkered with since it was built in 1506. "The frescoes came out 100 years ago and concrete ceilings went in during the 1930s," he said.

Architect Ippolito Pestellini, who is working with the Dutch architect Rem Koolhaas on the conversion, said the roof terrace would be only be revealed on special occasions by sliding panels and would not be visible from below.

Built as a trading post for German merchants in Venice, the palazzo was turned into a customs house by Napoleon before being converted to a post office in the 1930s until Benetton bought it for €53m in 2008. "It started life as commercial centre and we are bringing it back to that function, not to mention bringing 400 jobs to the city," said Sartor. "We will also be restoring a palazzo that has been falling to pieces for 50 years."

One local activist alleged that Benetton's millions had prompted the town hall to drop the palazzo's legal status as a public space.

"We Venetians don't want to turn back the clock and we do want to rethink the use of the city, but public space is more valuable than fashion store chains," said Michela Scibilia, of Venice activists' group 40xVenezia. "And shop assistants on short-term contracts will commute out from the mainland because they cannot afford to live here," she added.

Venice's mayor Giorgio Orsoni has complained that the millions of euros Venice once received from the Italian government to pay for the upkeep of the fragile city have all but dried up.

Sartor denied Benetton had bought the city's goodwill. "It's common practice that if you stand to derive an economic benefit from a project you share part of the benefit with the city," he said.