scalawag "disreputable fellow," 1848, Amer.Eng., originally in trade union jargon, of uncertain origin, perhaps an alteration of Scottish scallag "farm servant, rustic" (by influence of wag "habitual joker"). An early recorded sense was "undersized or worthless animal" (1854), which suggests an alteration of Scalloway, one of the Shetland Islands, in allusion to little Shetland ponies. In U.S. history, used from 1862 of anti-Confederate native white Southerners.

I hadn't remembered the "anti-Confederate Southerner" connotations! And I think I always spelled it (in my mind) 'scallywag'...