Born in Swindon, Smart was a hard-working forward player, who worked his way up through the youth ranks - playing for the schoolboy team before making his 'A' team debut in September 1959, and then the reserve team the following April, having just turned seventeen.

Smart signed professional forms with the club in May 1960, but he had to wait almost two years for his first team debut, deputising at wing-half for the injured Keith Morgan in a 1-1 draw at Reading, on March 2nd, 1962. The following season, Smart was pushed forward into a more-familiar inside forward position, and he scored his first goal for the club on September 1st, 1962, opening the scoring in a 2-2 draw with Queens Park Rangers. He was still only used sparingly - after making seven consecutive appearances at the start of the season, he was dropped, before a run of seventeen goals in ten reserve and third team games won him a recall for the final three games of the season. It was a masterstroke - Smart scoring three goals - including both goals in a 2-0 win at Colchester, and an 88th minute winner against Shrewsbury which confirmed the Town's promotion to the Second Division for the first time in their history.

Over the next few seasons, Smart became a first-team regular - missing just four league games between the 1965/66 and 1969/70 seasons - his fitness, endeavour and commitment making him the ideal foil for Don Rogers, who was allowed to play in a more free role. Smart's best form came in 1968/69, when he scored fifteen goals - six of which came in the League Cup campaign, including the opening goal in the final, when he benefitted from a mistake in the Arsenal defence, before bundling the ball into the net. He was also a part of the Anglo-Italian Cup and Cup-Winners'-Cup winning sides.

In the twilight of his Town career, Smart moved back into a wing-half or midfield position. After making just over four hundred appearances over twelve seasons for his hometown club, he left Swindon at the end of the 1972/73 season, and joined Charlton on a free transfer. He returned to the area just a year later, and went on to run the Plough Inn in Old Town.