That low-pitched buzz is obviously a cat growling at the mouse whenever it moves. If the tape-like tweeting is a canary then it's probably a Sylvester, otherwise a Tom.

More seriously, it sounds like old-fashioned analogue interference. The audio pitch of the interference matches the mouse signalling data rate - movement has higher resolution than scrolling. The mouse circuitry is leaking the signal as electrical noise and the audio analogue output circuit is picking it up. Hard to say whether the mouse circuit is leaking more noise than it should or whether the audio is more sensitive than it should be.

Do your headphones plug directly into the PC or a separate audio system?

Typical causes include: an earthing problem, otherwise known as the 0 V line. Could also be a common +V supply rail. One of the connectors might be loose, dirty or faulty, or even the mouse itself. Do the audio and mouse sockets share a common wiring connection back to the motherboard? Do you have a spare mouse and/or headphone set with connector that you can try?

If all that checks out, there's probably a fault on the motherboard (or in the separate audio box, if present) - a failed decoupling capacitor, a dodgy soldered joint, a failing chip, something like that.

"Klinger, do you know how many zoots were killed to make that one suit?" — BJ Hunnicutt, 4077 M*A*S*H

Pretty sure they don't share a connection. I have 2 sets of headphone sockets. One is on the back near the top and is hardwired to the motherboard. The one I'm using now is on the front of the PC on a box with headphone, mic and 2 USB 2 sockets.

By "on the front of the PC on a box", do you mean one of those toys fitted inside a spare disk drive bay?

Try these in turn, checking to see if each helps:
Step 1: Swap to the audio connector round the back.
Step 2: swap to a different USB socket, say one on the front.
Step 3: Power down, unplug from the mains, open the case and check all relevant connectors are fully home, clean and secure.

Oh and has the PC always done this or did it start suddenly one day?

"Klinger, do you know how many zoots were killed to make that one suit?" — BJ Hunnicutt, 4077 M*A*S*H