Urbino was at that time the most refined and elegant of the Italian courts, a cultural center ably directed and managed by the Duchess Elisabetta Gonzaga and her sister-in-law Emilia Pia, whose portraits, along with those of many of their guests, were painted by Raphael, a native of Urbino. Regular guests included: Pietro Bembo; Ludovico da Canossa; Giuliano de' Medici; Cardinal Bibbiena; the brothers Ottaviano and Federigo Fregoso from the Republic of Genoa.;[5]Francesco Maria della Rovere (nephew and heir of Duke and Duchess of Urbino); and Cesare Gonzaga, a cousin of both Castiglione and the Duke. The hosts and guests organized intellectual contests, pageants, dances, concerts, recitations, plays, and other cultural activities, producing brilliant literary works.[6] Elisabetta's virtue and abilities inspired Castiglione to compose a series of Platonic love songs and sonnets in her honor. She deeply loved her husband though his invalid state meant they could never have children.

In 1506 Castiglione wrote (and acted in) a pastoral play, his eclogueTirsi, in which he depicted the court of Urbino allegorically through the figures of three shepherds. The work contains echoes of both ancient and contemporary poetry, recalling Poliziano and Sannazzaro as well as Virgil.

Castiglione wrote about his works and of those of other guests in letters to other princes, maintaining an activity very near to diplomacy, though in a literary form, as in his correspondence with his friend and kinsman, Ludovico da Canossa (later Bishop of Bayeux).

Francesco Maria della Rovere succeeded as Duke of Urbino at Guidobaldo's death, and Castiglione remained at his court. He and the new Duke of Urbino took part in Pope Julius II's expedition against Venice, an episode in the Italian Wars. For this the Duke conferred on Castiglione the title of Count of Novilara, a fortified hill town near Pesaro. When Pope Leo X was elected in 1512, Castiglione was sent to Rome as ambassador from Urbino. There he was friendly with many artists and writers; including Raphael, whom he already knew from Urbino, and who frequently sought his advice. In tribute to their friendship, Raphael painted his famous portrait of Castiglione, now at the Louvre.[7]

In 1516 Castiglione was back in Mantua, where he married a very young Ippolita Torelli, descendant of another noble Mantuan family. That Castiglione's love for Ippolita was of a very different nature from his former platonic attachment to Elisabetta Gonzaga is evidenced by the two deeply passionate letters he wrote to her that have survived. Sadly, Ippolita died a mere four years after their marriage, while Castiglione was away in Rome as ambassador for the Duke of Mantua. In 1521 Pope Leo X conceded to him the tonsura (first sacerdotal ceremony) and thereupon began Castiglione's second, ecclesiastical career.

In 1524 Pope Clement VII sent Castiglione to Spain as Apostolic nuncio (ambassador of the Holy See) in Madrid, and in this role he followed court of Emperor Charles V to Toledo, Seville and Granada. At the time of the Sack of Rome (1527) Pope Clement VII suspected Castiglione of having harbored a "special friendship" for the Spanish emperor: Castiglione, the pope believed, should have informed the Holy See of Charles V's intentions, for it was his duty to investigate what Spain was planning against the Eternal City. On the other hand, Alonso de Valdés, twin brother of the humanist Juan de Valdés and secretary of the emperor, publicly declared the sack to have been a divine punishment for the sinfulness of the clergy.

Castiglione answered both the pope and Valdés in two famous letters from Burgos. He took Valdés to task, severely and at length, in his response to the latter's comments about the Sack of Rome. While in his letter to the pope (dated December 10, 1527), he had the audacity to criticize Vatican policies, asserting that its own inconsistencies and vacillations had undermined its stated aim of pursuing a fair agreement with the emperor and had provoked Charles V to attack.

Against all expectations, Castiglione received the pope's apologies and the emperor honored him with the offer of the position of Bishop of Avila. Historians today believe that Castiglione had carried out his ambassadorial duties to Spain in an honorable manner and bore no responsibility for the sack of Rome. He died of the plague in Toledo in 1529.

After his death in 1529 a monument was erected to him in the sanctuary of Sta Maria delle Grazie, outside of his birthplace of Mantua. It was designed by the mannerist painter and architect Giulio Romano, a pupil of Raphael, and inscribed with the following words:

Baldassare Castiglione of Mantua, endowed by nature with every gift and the knowledge of many disciplines, learned in Greek and Latin literature, and a poet in the Italian (Tuscan) language, was given a castle in Pesaro on account of his military prowess, after he had conducted embassies to both great Britain and Rome. While he was working at the Spanish court on behalf of Clement VII, he drew up the Book of the Courtier for the education of the nobility; and in short, after Emperor Charles V had elected him Bishop of Avila, he died at Toledo, much honored by all the people. He lived fifty years, two months, and a day. His mother, Luigia Gonzaga, who to her own sorrow outlived her son, placed this memorial to him in 1529.[8]