WorkCover New South Wales has dismissed union concerns that 400 workers have been exposed to asbestos while renovating a Sydney museum.

The Construction, Forestry, Mining and Energy Union says workers have come into contact with the carcinogenic fibres at the Museum of Contemporary Art, which is undergoing a $50-million refit and extensions.

The ABC has been told workers were not told of the risk and have been cutting into asbestos without wearing masks.

The union says the site contractor has risked the health of hundreds of 400 workers and their families, and even people passing by one of Australia's busiest tourist zones.

But WorkCover's acting chief executive, John Watson, says the CFMEU has not told the authority about its exposure concerns.

"If they're saying that they need to come and talk to us, because I don't think that's what they've said to us previously," Mr Watson said.

"They've said that they're concerned about the asbestos management of the site. They haven't said that 400 people have been exposed to asbestos.

"That would be somewhat of an alarmist declaration, given the nature of the work that's been carried out at the site and the level of controls that we found were in place."

Mr Watson says it is no surprise that asbestos exists in the museum, given the age of the building.