His younger brother, Ken Taylor also played football for Huddersfield. Ken was also a professional cricket, playing three Tests for England and first-class cricket for Yorkshire; his son (and Jeff's nephew), Nick Taylor also played cricket for Yorkshire.