The original sculptures and paintings in the Borghese Gallery date back to Cardinal Scipione's collection, the son of Ortensia Borghese - Paolo V's sister - and of Francesco Caffarelli, though subsequent events over the next three centuries entailing both losses and acquisition have left their mark.

Cardinal Scipion was drawn to any works of ancient, Renaissance and contemporary art which might re-evoke a new golden age. He was not particularly interested in medieval art, but passionately sought to acquire antique sculpture. But Cardinal Scipione was so ambitious that he promoted the creation of new sculptures and especially marble groups to rival antique works.

The statue of Pauline Bonaparte, executed by Canova between 1805 and 1808, has been in the villa since 1838. In 1807, Camillo Borghese sold Napoleon 154 statues, 160 busts, 170 bas-reliefs, 30 columns and various vases, which constitue the Borghese Collection in the Louvre. But already by the 1830s these gaps seem to have been filled by new finds from recent excavations and works recuperated from the cellars and various other Borghese residences.

Cardinal Scipione's collection of paintings was remarkable and was poetically described as early as 1613 by Scipione Francucci. In 1607, the Pope gave the Cardinal 107 paintings which had been confiscated from the painter Giuseppe Cesari, called the Cavalier d'Arpino. In the following year, Raphael's Deposition was secretely removed from the Baglioni Chapel in the church of S.Francesco in Perugia and transported to Rome. It was given to the Cardinal Scipione through a papal motu proprio.

In 1682, part of Olimpia Aldobrandini's inheritance entered the Borghese collection; it included works from the collections of Cardinal Salviati and Lucrezia d'Este.