Update: Maybe related, we are using net.ipv4.tcp_congestion_control = cubic. Kernel version is 2.6.28 with Gentoo hardening patches.

Update: This could probably really be related to hardware/driver issues, as we are seeing a "hickup" on the switches sometimes: The port goes down and comes up again. Only lately we discovered the following dmesg output related to these port hickups: