Apparently, the SKTextureAtlas cropping method used when generating a physics body has issues figuring out exactly what to crop, so setting the alphaThreshold explicitly seemed to fix it.

Obviously, that value will rely on your own image and what kind of transparency you have. But if you have hard edges like I did, you should be able to set that to some arbitrary value and be good to go.