The problem may in fact be with libao: I can speak directly to the sound card via mplayer, and through esd to the sound card via mplayer. Using libao results in, at best, really screwy sound (I can't quite describe it, but a given track will complete playing much faster than it should).