After playing some more UT, I found that going to Advanced Options -> Audio and setting UseDigitalMusic to False gives me a pretty good framerate boost and it also enables UT to close down properly. Under Audio there is also an option Use3dHardware that you can set to true if you have a 3D sound card.

Also, if I start a game, then hit esc and go to Advanced Options, the game crashes, apparently because it cannot go into windowed mode anymore.

Update: Setting UseDigitalMusic to False does allow UT to shut down and return you to the desktop, but it remains crashed (the Task List says it's not responding). You need to kill the task manually to get it to go away.