I'm not a fan of structured wiring... I'd suggest disconnecting as much of it as you can, then slowly add back jacks until you start to encounter the problem. I wouldn't be surprsied if there was a voltage filter to prevent back-feeding of the signal, or some other strange thing inside the panel.