Originally the residence of the Venetian doge Antonio Grimani, it was rebuilt in 1532-1569 by his heirs Vittore, procuratore generale of the city, and Giovanni Grimani, cardinal and Patriarch of Aquileia. The latter allegedly collaborated with Michele Sanmicheli, who had been usually credited as the designer of the whole construction. The palace was completed in 1575 by Giovanni Rusconi. Alessandro Vittoria was responsible for the ornamentation of the doorway.

The palace has a three parts composition with a small backyard. The façade, sporting characteristical massive windows arches, is decorated with polychrome marble. The most striking feature of the interior is the Sala di Psiche (c. 1540), with frescoes by Francesco Menzocchi, Camillo Mantovano and Francesco Salviati. Other artists who worked to the palace's decoration include Taddeo Zuccari and Giovanni da Udine.