Correct me if I'm wrong, do these compute shaders make it possible for GPU applications to let the GPU write things to CPU memory?

Well, that's not directly what compute shaders are for, but you can use them to compute something and then use the rest of the GL API to get the results back to the CPU.

Compute shaders are really about exposing the shader hardware in a way that isn't tied in to the old graphics pipeline (which goes through tess, geo, vert, frag). Once you compute something, it's just there in the GPU and you can manipulate it however you want.