The palace was built halfway the 18th century as a citypalace for the merchatnt Joan Alexander van Susteren (1728-1764) and was designed by architectliet het bouwen door de Antwerpse architect Jan Pieter Van Baurscheit. Due to the favourable location the palace attracted the attention of Emperor Napoleon, he bought it in 1811. He redecorated the salons and added empire furnishings to the building. After that the palace was property of the Dutch monarch, and later of the Belgian ones. Leopold II built the new hall of mirrors and king Albert I stayed here for a short time during WWI. King Boudouin donated the palace to the people on 19 December 1969.

This is no longer in royal hands. It belongs to "Erfgoed Vlaanderen", the Flemish Heritage Foundation. It used to be the site of the city of Antwerp's film museum. Nowadays it's being restaured. In 2010 it will be opened to the public again.

There was discussion about the original furniture. Prince Philippe had "borrowed" some of it, and refused to give it back to the new owners - the Flemish Heritage. It even became a political debate. In the end the prince wisely decided to give the furniture back.