He was the only player in the major leagues who was both active in 1934 when Babe Ruth was still playing for the New York Yankees, and also active in 1954 when Hank Aaron started. Oddly enough, he wasn't one of the five oldest players in the league when he played that season.

In his first full season with the Cubs in 1935, he played alongside the 36-year-old Kiki Cuyler and the 34-year-old Gabby Hartnett on a Cubs team that won the 1935 pennant with 100 victories. Eighteen years later, in his last year with the Cubs in 1953, he managed and also appeared in 27 games on a team that also featured the 22-year-old rookie Ernie Banks. When he finished out his career with the 1955 Chicago White Sox, his teammates included the 27-year-old Nellie Fox and the 32-year-old George Kell.

Cavarretta was later a coach with the Detroit Tigers from 1961 to 1963 and was a member of the New York Mets staff from 1975 to 1978. Starting in 1973, he had been the Mets' full-time hitting instructor, roving the minor leagues after finishing spring training with the major league club.