Strange problem with a ProCurve network

Hi,

While troubleshooting network problems this week using WireShark I noticed some odd behaviour on our network, which consists entirely of HP ProCurve switches with several VLANs.

Sniffing on a standard untagged port of a switched network (even in promiscuous mode) I would not expect to see traffic other than that which is destined for my host, broadcast traffic, or multicast traffic. However, I was seeing certain (but not all) traffic destined for other hosts on the network - hosts that in most cases were on a separate physical switch, but the same VLAN - the oddest of which was syslog traffic.

We have a few SonicWALL firewalls on site that file their logs to a ViewPoint host which runs on a VMware box, and I was seeing short (up to approx. 10 minutes) bursts of syslog traffic coming from the SonicWALLs and destined for the ViewPoint host which contained real-time data leak onto the network, and then stop, and it was doing this every 20-30 minutes. When I mirrored the firewall's interface and sniffed on the mirror port I could see that there was a continuous stream of syslog data destined for the ViewPoint host, and it seems that at a certain threshold it would leak out onto the network and then eventually correct itself and stop leaking, and continue streaming its data direct to the ViewPoint host like you would expect... and then repeat itself 20-30 minutes later.

The switch the firewall is connected to is a 5300xl running the E.11.21 firmware, and the traffic seems to be leaking out onto the whole VLAN as I've observed the problem when sniffing from other switches on the network.

Does anyone have any ideas what this might be? Or have any tips on how best to debug it and figure out the cause of the problem? It seems like a CAM table overflow, but not *all* traffic appears to be getting repeated across the network, and it does seem to correct itself eventually... but is a CAM table overflow still possible with modern firmware?

Re: Strange problem with a ProCurve network

If you are using the VM as purely a sink for syslog traffic, perhaps it's so quiet it ages out of the MAC table.

If you completely stealthed the ViewPoint host, then you would expect traffic for it to flood the network.

Possibly what you are seeing is the occasional ARP request/response bringing it back into the MAC table, then it aging out again. That can be verified by looking at packets with the source address of the ViewPoint host VM.