Swipe to delete only appears to happen in the 'Songs' tab of the Music app. To remove songs from a playlist I only get the option of hitting Edit at the top of the playlist, then using the red icon on the left of the song name. - iOS 5.1