Floyd Smith began his hockey career in the OHA with the Galt Black Hawks in 1952-53 season. He signed with the Boston Bruins and made his NHL debut in the 1954-55 season playing three games. Smith spent the next season with the team's AHL affiliate Hershey Bears and then split the next season in the AHL and NHL after being traded to the Rangers.

After three seasons in the minors, Smith was called up to Broadway to play 29 games for the Rangers, but he was still sent back to the AHL after this stint. He was claimed by the Re Wings in the Intra-League Draft of 1962 but once again split time in both leagues before playing his first full season in the NHL in 1964-65. Smith spent two more seasons in Detroit before being traded to the Leafs during the 1967-68 season. Two more seasons would go by before he was dealt one last time, to the Sabres in 1970. He finished his playing career in the next season.

Smith took over Buffalo's coaching duties from Punch Imlach in the 1971-72 season for one game before taking over the coaching position of their minor league team in Cincinnati. He made his way back to the Sabres' bench in the 1974-75 season, guiding the team to the Stanley Cup finals. The Flyers' netminder Bernie Parent put an end to their run as Philadelphia took the Cup from Buffalo. Smith was the head coach until 1977 when he moved on to Toronto to coach the Leafs for the 1979-80 season. He then went on to become a scout for the team, then chief scout and eventually the general manager from 1989 to 1991.