Setting Pale Moon as default browser creates the file "~/.local/share/applications/userapp-Pale Moon-G2GJRZ.desktop" and adds numerous references to the file in ~/.config/mimeapps.list. (The "G2GJRZ" part of the file name seems to be randomly-generated.)

Removing the space in the file name and correcting the corresponding entries in ~/.config/mimeapps.list fixes the problem.I tested twice, one with the file name changed it to "~/.local/share/applications/userapp-PaleMoon-G2GJRZ.desktop", the other to "~/.local/share/applications/userapp-Pale_Moon-G2GJRZ.desktop".