Short description of error
When using Blender on Ubuntu / Xubuntu 20.04, whenever you switch to or away from something GPU Cycles related, Blender pins one CPU thread at 100% and the system hangs for a few seconds.
Some examples of this are:

Switching from Solid to Rendered (and vice versa) in the Viewport

Switching to the Render Properties when Cycles and GPU are set

Starting a render with Cycles GPU

It seems that this occurs when Blender / Cycles needs to "probe" the gpu(s) in the system.
At least that's the best way I can describe it.
This behavior is absent on Xubuntu 18.04 on the same hardware.

In 2.80 the Blender UI merely freezes a few seconds, from 2.81 onward the entire desktop freezes and hitches.

For reference, I'm running a 1st gen Threadripper CPU with 16 cores, which might be why we're experiencing similar behavior.
I'll try the same on Pop_OS and see it it persists, so we can get some more info, although with it being based on Ubuntu I'm not expecting massively different results.

Did some additional testing with different linux distros to complete the picture.
Same results occur with Pop!_OS, Solus & Fedora 32, all using Nvidia driver version 440.82, with the same hardware as the original report.
At least we know it's not something OS specific, might even be hardware related.

If there's anything else I can test, just let me know, happy to help figure it all out.