If I remember well, there is a command in the menus to uninstall palettes in the version 5.
For the version 6, I guess the best way is to go in your folders with all your palettes and remove or copy your palette somewhere else. As a result, after reopening Mathematica, it won't be in the list anymore.
There might be a proper way, but I don't know it.
Cheers,