In the election which followed Henry's early death the next year, Baldwin lent his support to Louis the Bavarian. He later repudiated the Bavarian, however, and moved to support his great-nephew Charles IV in 1346. He was Charles's greatest supporter in the west of the realm.

From 1324 to 1326, he was one of the four lords in the Four Lords' War against Metz. In 1328, he was lured into Starkenburg castle on the Moselle by Loretta von Sponheim. He was only released after a ransom was paid and concessions made. Likewise in that year, he was put forward by his own cathedral chapter as a candidate for the vacant Mainz archdiocese, but the Pope appointed Henry III of Virneburg. Baldwin did govern the archbishopric until the so-called "Mainz Schism" was terminated by the Avignoncuria in 1336. During that time period, from 1331 to 1337, Baldwin had as well endeavoured to control the dioceses of Worms and Speyer.

Baldwin, like most medieval bishops, was a patron of his see. He built the Balduinbrücke (Baldwin's Bridge) in Koblenz and repaired the old Roman bridge in Trier. He reformed the administration of the archdiocese and preserved official documents. He had many copies made and four manuscript copies of the archives of the diocese survive in the main national archives in Koblenz. Baldwin also strove to connect the various regions of his diocese (Trier and Koblenz). He did not shy away from military methods.