"All judicial and administrative duties that were previously assigned to the magistrate have been withdrawn and the magistrate will not be assigned any such duties until the charges are resolved," Mr Lauritsen said in a statement on Thursday.

"It would be improper for the chief magistrate to make any further comment while the proceeding is before the courts."

A spokesperson for Attorney-General Robert Clark said given the nature of the charges, the magistrate had been immediately suspended from all duties.

The man has been bailed to appear in Melbourne Magistrates Court on Friday.

Victoria Police said the investigation was continuing and further details of the charges could not be released.