This is the first time Sinatra worked with arranger/composer Neal Hefti on an album project, following a single-only session that took place in April 1962. For Sinatra and Swingin' Brass, the singer re-recorded a number of songs he had previously recorded for Capitol. "They Can't Take That Away from Me" and "I Get a Kick Out of You" had been recorded eight years earlier for Songs for Young Lovers, "You Brought a New Kind of Love to Me" had been recorded six years prior for Songs for Swingin' Lovers, and "At Long Last Love" had been recorded five years earlier for A Swingin' Affair!.