I think they ditched the White caps after one road trip. The dirt showed too much.

No, the Yankees complained that the white under sleeves and white cap on a pitcher was distracting, the umpire made the pitcher take off his white undershirt and the league later banned the White Sox from wearing all white caps.