The following season Caffery signed with the Whalers and met with his greatest success as a pro, scoring 100 points to finish in the WHA's top ten in the loop's inaugural season and proving a key component in the Whalers' league championship that year. He was awarded the Lou Kaplan Trophy as the league's rookie of the year. However, he suffered a knee injury late that year that, despite him starring in the playoffs, forced him to miss the entire 1974 season. He returned to play for the Whalers the year after that, but remained seriously impaired, and retired during the 1976 season.

His brother was Jack Caffery, a two-sport player who also played baseball with the Milwaukee Braves and was credited with inventing the backward grip for faceoffs.