I observe millions of Exception Ingress packets on all 4 up-links of my single vDS switch and probably tens of millions Dropped Egress packets on all ports of vDS regardless of a port-group they belong.

What I have looked into:

- I can see these drops on a VM-level statistics (Performance>Network) as well, but only for RX.

- I don't experience any connectivity problems at application level although (no packet loss in guest OS logs or during ping).

This issue occurs when packets filtered by the IO chain are incorrectly recorded as dropped packets. This is a reporting issue, the packets are not dropped, hence they cannot be seen using esxtop or other network monitoring tools.

As I mentioned in previous post the only dropped packets I see for the moment are ethertype 0x8922 packets

The 0x8922 ethertype broadcasts are used not only for beacon probing, but also for the new 5.1 distributed vSwitch network health check feature. Have you enabled that?

The source MAC of these frames is encoded in the format 00:50:56:5[random Value]:[Last 2 Byte of the physical vmnic MAC].

This is exactly what I see. Many thanks for the lead. Disabled Health Check - no more broadcasts

The only thing left is when Health Check is ON again all my Ubuntu (13.04) VMs are reporting dropped packets which completely correspond to ethertype 0x8922 broadcasts captured by tcpdump from another VM.

user01@ubuntu01:~$ ifconfig eth0

eth0 Link encap:Ethernet HWaddr 00:50:56:bf:7c:48

inet addr:10.113.0.177 Bcast:10.113.0.255 Mask:255.255.255.0

inet6 addr: fe80::250:56ff:febf:7c48/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:3250670 errors:0 dropped:91575 overruns:0 frame:0

TX packets:4468580 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:2785809569 (2.7 GB) TX bytes:2847014532 (2.8 GB)

I presume Ubuntu is not able to recognize them. Tried different nic types. vmtools are the latest. Is there any solution to that?

If it really just drops the 0x8922 frames because it doesn't recognize this ethertype, then this can be safely ignored, even though it may look a bit eyebrow-raising.

I tested a bit with CentOS 6.3 VMs but I wasn't able to reproduce the dropped packet count increase you're seeing on your Ubuntu VMs.

Did you increase the MTU on the dvSwitch? The size of the MTU Health Check 0x8922 frames should correspond to the MTU you've set on the dvSwitch. I thought maybe your VM drops them because they are too large.

Does the dropped frame counter increase in a VM if if you're running tcpdump on it (running it should "accept" all frames)?

I also tried crafting custom layer 2 frames with scapy, using the same or other random ethertypes and larger frames sizes, but I never got the dropped packet counter to increase. I'm not sure when the kernel would decide to drop frames, except maybe for wrong checksums or such.

Check if you can reproduce the drops with scapy or some other packet crafting tool like this. But like I said above, it shouldn't be something to worry about if your Ubuntu is just dropping ethertypes it doesn't know.