Oracle VM VirtualBox: Ticket #3783: OSX Bridged Networking unreliable due to incorrect MTU setting! => Fixed in SVNhttps://www.virtualbox.org/ticket/3783
<p>
I setup a clean install of XP SP2 on <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> on my new MacPro. I setup the network adapter (Intel Pro 1000 T Server) with Bridged mode. After getting into XP, I was getting an IP address from my router and a few websites would load, but others would not. For example google.com was OK but mozilla.com and microsoft.com would not load. Some experimenting pointed to an MTU issue.
</p>
<p>
I determined this WAS due to the MTU setting. My testing showed that when using Bridged Mode and performing "ping -f" to my home network router, I could see that setting the ping packet size (-l flag) larger than 1467 caused the packets to timeout, and a size over 1472 to get the expected "Packet needs to be fragmented but DF set." When using an adaptor in NAT mode, this problem does not occur, and pings up to 1472 in size are allowed as expected. Setting the MTU lower in Windows XP registry and rebooting provides a workaround, and allows all websites and pages to load OK again.
</p>
<p>
It's clear something about using Bridged mode is "padding" the packets with an additional 4 or 5 bits of data...
</p>
<p>
For reference Windows XP's registry key for MTU size:
</p>
<pre class="wiki">System Key: [HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\[Adapter ID]]
Value Name: MTU
Data Type: REG_DWORD (DWORD Value)
</pre>en-usOracle VM VirtualBox/graphics/vbox_logo2_gradient.pnghttps://www.virtualbox.org/ticket/3783
Trac 0.12DrewMerkleFri, 08 May 2009 23:08:30 GMThttps://www.virtualbox.org/ticket/3783#comment:1
https://www.virtualbox.org/ticket/3783#comment:1
<p>
FWIW, the same thing happens with Ubuntu guest VM using <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> 2.2.2 on a Mac OS X Server host. So far I have tested several flavors of Ubuntu for guest, as well as Windows XP Pro as guest. In my case, a ping of 1469 or larger will not work, but smaller does (consistent between Ubuntu and Windows XP guests). Not sure why it's a different number from the initial report... NAT mode works as expected. Changing the MTU seems to be a successful workaround (thank you thank you thank you!).
</p>
<p>
I'm using <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> 2.2.2 <a class="changeset" href="https://www.virtualbox.org/changeset/46594/vbox" title="VMM/HMSVMR0: AMD-V bits.
">r46594</a> on a Mac OS X Server 10.5.6 host.
</p>
Ticketmsully725Sun, 17 May 2009 20:00:36 GMThttps://www.virtualbox.org/ticket/3783#comment:2
https://www.virtualbox.org/ticket/3783#comment:2
<p>
I can confirm this same issue, and that changing MTU resolved the issue.<br />
<a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> 2.2.2 <a class="changeset" href="https://www.virtualbox.org/changeset/46594/vbox" title="VMM/HMSVMR0: AMD-V bits.
">r46594</a> <br />
Host: Ubuntu 8.10 bridged on eth0<br />
Guest: Win XP SP3 using Intel PRO/1000 MT Desktop
</p>
<p>
After adding MTU of 1469 to above registry entry I was able to navigate to any site. Before that I could navigate to some sites, but others would never load (I could ping them, but nothing over port 80. telnet to port 80 would never connect). Problem only occurred when bridged on eth0, wlan0 worked with no issues.
</p>
Ticketmsully725Sun, 17 May 2009 20:02:17 GMThttps://www.virtualbox.org/ticket/3783#comment:3
https://www.virtualbox.org/ticket/3783#comment:3
<p>
Forgot to mention the host is Ubuntu 8.10 64 bit, guest is 32 bit.
</p>
TicketDrewMerkleSun, 17 May 2009 23:24:00 GMThttps://www.virtualbox.org/ticket/3783#comment:4
https://www.virtualbox.org/ticket/3783#comment:4
<p>
For what it's worth, I've now observed similar behaviors with various hosts/guests, as well as with other virtualization software. In particular, I've recently managed to troubleshoot issues using Parallels where the host is Mac OS X 10.5.x and the guest is Windows XP Pro by changing the MTU in the Windows registry. Without knowing more,
</p>
<p>
I can only speculate, but I wonder if something about the process a guest OS uses to discover a maximum transmission unit (MTU) may be problematic in general that is resolved by setting it manually. I discovered RFC 2923 which is a memo discussing "TCP Problems with Path MTU Discovery" (<a class="ext-link" href="http://tools.ietf.org/html/rfc2923"><span class="icon"> </span>http://tools.ietf.org/html/rfc2923</a>); sections of that read exactly like my experiences with this issue.
</p>
<p>
Hope this is helpful.
</p>
TicketIscreaManWed, 10 Jun 2009 21:22:47 GMThttps://www.virtualbox.org/ticket/3783#comment:5
https://www.virtualbox.org/ticket/3783#comment:5
<p>
I can confirm this issue with Mac OS X 1.5.7 host, <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> 2.2.4 <a class="changeset" href="https://www.virtualbox.org/changeset/47978/vbox" title="Changelog.">r47978</a> and Windows Xp and Ubuntu 9.04 guests. This bug just prevents many sites from loading, as well as other web-based services like Windows Update and apt-get (argghhh!!! ;)
</p>
<p>
But i am happy that i found why this happens. So many lost hours... Google will rule the world! Even with this MTU bug, it still works :)
</p>
TicketslebbonWed, 10 Jun 2009 23:25:59 GMThttps://www.virtualbox.org/ticket/3783#comment:6
https://www.virtualbox.org/ticket/3783#comment:6
<p>
I wanted to update as well, in my further experience with Windows 7 RC, when using bridged mode, not only does the internet not work, but attempting to use the internet completely freezes windows in the VM requiring the Virtual Machine be hard powered off and restarted. When this happens the VM uses 100% cpu. With NAT mode it runs fine, except I can't assign the VM it's own IP address on my network.
</p>
TicketstephenjuMon, 29 Mar 2010 14:22:21 GMThttps://www.virtualbox.org/ticket/3783#comment:7
https://www.virtualbox.org/ticket/3783#comment:7
<p>
I just updated to <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> 3.1.6 <a class="missing changeset" title="No changeset 59351 in the repository">r59351</a> and this problem still exists. Any MTU value above 1468 hangs Windows 7 upon login. After a few minutes it then indicates no network connection.
</p>
<p>
This is on OS X 10.5.8.
</p>
TicketbrianmMon, 10 May 2010 21:11:39 GMThttps://www.virtualbox.org/ticket/3783#comment:8
https://www.virtualbox.org/ticket/3783#comment:8
<p>
I'm also having this same problem with a Windows 7 guest on an OS X 10.6.3 host using Virtualbox version 3.1.6
</p>
TicketbuffygMon, 14 Jun 2010 10:39:53 GMThttps://www.virtualbox.org/ticket/3783#comment:9
https://www.virtualbox.org/ticket/3783#comment:9
<p>
I've also seen problems consistently with Windows 7 guests on OS X using bridged networking, which are fixed by the MTU change. I'm also running 10.6.3 (server), but this is also consistent with problems I saw under 10.5.9.
</p>
TicketcrileyThu, 02 Sep 2010 14:08:25 GMThttps://www.virtualbox.org/ticket/3783#comment:10
https://www.virtualbox.org/ticket/3783#comment:10
<p>
I may have a similar problem.
</p>
<p>
Yesterday I installed 3.2.8 on my OS X 10.6.4 server and installed XP Professional as my guest. I thought all was well until this morning I noticed that I could not log onto certain sites like Google.com among others from my client machines in the office. The OS X server is providing the DNS for the office machines. I restarted the server and all was fine only because I did not run VB. The guest is set up to run in bridge mode. So I am wondering how to correct this. I do not use the Guest OS for surfing the internet, just to run a few programs that need to talk with the host.
</p>
<p>
Any help would be greatly appreciated.
</p>
TicketbuffygTue, 26 Oct 2010 14:44:53 GMThttps://www.virtualbox.org/ticket/3783#comment:11
https://www.virtualbox.org/ticket/3783#comment:11
<p>
I've also seen this problem with OpenSolaris and OpenIndiana, where reducing the MTU fixes the problem. There's no record of anyone from <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> reviewing the ticket. Any chance that someone could at least review the issue for reproducibility?
</p>
TicketfrankThu, 10 Feb 2011 13:41:43 GMTcomponent changedhttps://www.virtualbox.org/ticket/3783#comment:12
https://www.virtualbox.org/ticket/3783#comment:12
<ul>
<li><strong>component</strong>
changed from <em>network</em> to <em>network/hostif</em>
</li>
</ul>
TicketalekseyMon, 21 Feb 2011 10:48:26 GMThttps://www.virtualbox.org/ticket/3783#comment:13
https://www.virtualbox.org/ticket/3783#comment:13
<p>
Can anybody confirm the problem is still present in 4.0.4 and provide VBox.log?
</p>
TicketvmannFri, 25 Feb 2011 05:08:27 GMTattachment sethttps://www.virtualbox.org/ticket/3783
https://www.virtualbox.org/ticket/3783
<ul>
<li><strong>attachment</strong>
set to <em>Windows XP Test-2011-02-24-21-06-04.log</em>
</li>
</ul>
<p>
VBox.log of WinXP guest, Mac 10.6.6 host, <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> 4.0.4 replicating defect <a class="closed ticket" href="https://www.virtualbox.org/ticket/3783" title="defect: OSX Bridged Networking unreliable due to incorrect MTU setting! =&gt; Fixed ... (closed: fixed)">#3783</a>
</p>
TicketfrankSun, 27 Feb 2011 09:29:29 GMThttps://www.virtualbox.org/ticket/3783#comment:14
https://www.virtualbox.org/ticket/3783#comment:14
<p>
Log was attached.
</p>
TicketalekseyThu, 03 Mar 2011 14:56:50 GMThttps://www.virtualbox.org/ticket/3783#comment:15
https://www.virtualbox.org/ticket/3783#comment:15
<p>
vmann,
</p>
<p>
Using the same host OS (32-bit, 10.6.6), the same <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> version (4.0.4) and Windows XP SP2 as a guest with Intel PRO/1000 T Server adapter I see no symptoms at all: 'ping -f -l 1472 &lt;router_ip&gt;' works properly. Could you check MTU size on the host adapter you bridge to (en0)? If it is 1500, could you enable internal packet capture with:
</p>
<pre class="wiki">VBoxManage modifyvm &lt;your-vm&gt; --nictrace&lt;adapter-number&gt; on --nictracefile&lt;adapter-number&gt; file.pcap
</pre><p>
then start VM and do a couple of pings with 1472-byte payload? You can mail the resulting file or its tcpdump printout to me at aleksey dot ilyushin at oracle dot com.
</p>
TicketbuffygThu, 07 Apr 2011 16:57:22 GMThttps://www.virtualbox.org/ticket/3783#comment:16
https://www.virtualbox.org/ticket/3783#comment:16
<p>
I did further testing, and it looks like the MTU can't be increased above 1496. The packets appear in the trace, but the OS doesn't get them. I'll e-mail the relevant output to you shortly.
</p>
TicketbuffygSun, 05 Jun 2011 09:07:01 GMThttps://www.virtualbox.org/ticket/3783#comment:17
https://www.virtualbox.org/ticket/3783#comment:17
<p>
Debug was sent about two months ago, and I've not heard anything back. I'll check in with #vbox-dev shortly.
</p>
TicketalekseyMon, 06 Jun 2011 14:28:37 GMThttps://www.virtualbox.org/ticket/3783#comment:18
https://www.virtualbox.org/ticket/3783#comment:18
<p>
Sorry for such delay in communication, I got carried away with other stuff. In the packet capture file you've sent me I see both requests and replies. How did you obtain this file? Using internal capture as described in my comment above or by running tcpdump on the host (guest)?
</p>
TicketbhamailMon, 21 Nov 2011 05:41:51 GMThttps://www.virtualbox.org/ticket/3783#comment:19
https://www.virtualbox.org/ticket/3783#comment:19
<p>
I just got bitten by this bug when I moved WinXP and Win7 VMs from VBox on linux to VBox on MacOSX.
Here's the link to the forum thread detailing the issue: <a class="ext-link" href="https://forums.virtualbox.org/viewtopic.php?f=8&amp;t=46111&amp;p=208586#p208586"><span class="icon"> </span>https://forums.virtualbox.org/viewtopic.php?f=8&amp;t=46111&amp;p=208586#p208586</a>
</p>
<p>
Is there any way to get this bug looked into, and hopefully fixed! The Win Registry workaround is nasty and I imagine a number of people give up before finding that workaround. Ideally, Bridged networking should just work...
</p>
<p>
Anything I can do to help?
</p>
Ticketppinter1Tue, 06 Dec 2011 03:37:03 GMThttps://www.virtualbox.org/ticket/3783#comment:20
https://www.virtualbox.org/ticket/3783#comment:20
<p>
I can't believe I blew most of the day on this. Only by the sheerest fluke did I think to try NAT, which lead me here.
</p>
<p>
If I had the skill, I'd fix it myself, but I don't so add me to the queue of those eager to see this ancient bug squashed.
</p>
TicketkarlwSun, 05 Feb 2012 20:28:31 GMThttps://www.virtualbox.org/ticket/3783#comment:21
https://www.virtualbox.org/ticket/3783#comment:21
<p>
same problem here on VB 4.1.8 <a class="missing changeset" title="No changeset 75467 in the repository">r75467</a> on MC OSX 10.7.3 as host using guests Win xp or Win 7 or Win7 64 bit
( no problems with NAT)
</p>
<p>
With bridged network Using the same guest vdi
win xx guest on a mac book pro 2011 works fine.
win xx guest on MacPro 2008 does not work
</p>
<blockquote>
<p>
It works only when
</p>
</blockquote>
<ul><li>pick PCnet-PVI II (Am79C970A) ethernet card. No other ethernet card works
</li></ul><blockquote>
<p>
with this card I can ping my router etc, but chrome, IE etc etc do not work properly
</p>
</blockquote>
<ul><li>set MTU to &lt;1500 (1485 works) as described with regedit , then everything works fine
</li></ul>
TicketJessePetersonWed, 22 Feb 2012 20:47:10 GMThttps://www.virtualbox.org/ticket/3783#comment:22
https://www.virtualbox.org/ticket/3783#comment:22
<p>
Problem confirmed here as well. Mac OS X Server 10.6.8 (v1.1) host running <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> 4.1.8 with a Windows XP SP3 guest using bridged networking using a PCnet III adapter. Adjusted MTU to 1468 in registry and networking started working properly.
</p>
TicketJessePetersonTue, 20 Mar 2012 23:13:43 GMThttps://www.virtualbox.org/ticket/3783#comment:23
https://www.virtualbox.org/ticket/3783#comment:23
<p>
Further testing of this problem appears to indicate an issue regardless of the MTU size. Rather it is a problem with sending and receiving packets that are larger than 4 bytes less than the MTU. For example, and under a default MTU of 1500, if I send a packet that is 1497 bytes (a 1469 payload ICMP ping packet) to a <a class="wiki" href="https://www.virtualbox.org/wiki/VirtualBox">VirtualBox</a> host then the packet is reported as an oversized frame. However if one uses a payload size of 1468 (1496 byte packet which is 4 bytes less than the MTU) then it passes the interface without any issue. This case is exactly the same even when using lower than 1500 MTU. It might be characterized like this:
</p>
<p>
IF PKTSZ &gt; (MTU - 4) THEN drop packet oversize (PKTSZ + 18) ELSE pass packet
</p>
<p>
The additional 18 bytes I'm not sure off, it relates to NetBSD's reporting by the wm driver (the Intel 1000 adapter). But it is odd that any oversized packet is reported so by 18 bytes larger than the packet.
</p>
TicketJessePetersonThu, 22 Mar 2012 23:17:38 GMThttps://www.virtualbox.org/ticket/3783#comment:24
https://www.virtualbox.org/ticket/3783#comment:24
<p>
More testing on different hardware and host OSes has complicated this issue.
</p>
<table class="wiki">
<tr><th> HW </th><th> SW </th><th> Ethernet </th><th> Result
</th></tr><tr><td> MacBookPro1,1 </td><td> Mac OS X 10.6.8 </td><td> Marvell Yukon Gigabit Adapter 88E85053 ? </td><td> No issue
</td></tr><tr><td> MacBookPro5,4 </td><td> Mac OS X 10.7.3 </td><td> Nvidia MCP79 Ethernet </td><td> No issue
</td></tr><tr><td> MacBookPro5,4 </td><td> Mac OS X 10.7.3 </td><td> Apple USB Ethernet 10/100 </td><td> No issue
</td></tr><tr><td> MacMini1,1 </td><td> Mac OS X Server 10.6.8 </td><td> Marvell Yukon Gigabit Adapter 88E85053 </td><td> No issue
</td></tr><tr><td> Xserve2,1 </td><td> Mac OS X Server 10.6.8 </td><td> Intel 80003ES2LAN Gigabit Ethernet Controller (Copper) </td><td> Problematic!
</td></tr><tr><td> Xserve2,1 </td><td> Mac OS X Server 10.7.3 </td><td> Intel 80003ES2LAN Gigabit Ethernet Controller (Copper) </td><td> Problematic!
</td></tr></table>
<p>
Lights-Out Management has been disabled on this Xserve's interfaces, and there is no VLAN configurations (ideas to track down the 4-byte off issue). Also tried to track down whether the Ethernet controller chip had any issue. But so far it's just the Xserve that is displaying issues for my.
</p>
<p>
And one last note is that all of this is (mostly) tested under a NetBSD 5.1.2 guest OS. There is limited testing under Windows XP Pro SP3.
</p>
TicketalekseyFri, 30 Mar 2012 18:03:19 GMTdescription changedhttps://www.virtualbox.org/ticket/3783#comment:25
https://www.virtualbox.org/ticket/3783#comment:25
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/3783?action=diff&amp;version=25">diff</a>)
</li>
</ul>
<p>
I've just tried debian-based vyatta guest with PCnet III adapter bridged to Intel 80003ES2LAN Gigabit Ethernet Controller (Copper) on OS X Server 10.6.8 host. I can successfully ping the host with ICMP payload sizes from 1468 to 1472, fragmentation disabled.
</p>
TicketalekseyTue, 03 Apr 2012 12:12:59 GMThttps://www.virtualbox.org/ticket/3783#comment:26
https://www.virtualbox.org/ticket/3783#comment:26
<p>
I was able to reproduce the problem with hardware mentioned above, pinging external host. Thanks a lot, Jesse!
</p>
TicketJessePetersonWed, 04 Apr 2012 05:28:51 GMThttps://www.virtualbox.org/ticket/3783#comment:27
https://www.virtualbox.org/ticket/3783#comment:27
<p>
Fantastic. What was the difference between your initial test and the test that reproduced the problem? Just the hardware? How can I help?
</p>
TicketalekseyWed, 04 Apr 2012 07:07:15 GMThttps://www.virtualbox.org/ticket/3783#comment:28
https://www.virtualbox.org/ticket/3783#comment:28
<p>
It was just the hardware. It turns out that Intel 80003ES2LAN does not strip FCS before passing the packet up the stack. So the packets coming from the wire were rejected by our PCNet as oversized. You have already helped immensely by pinpointing the hardware. You can also help with fix verification. The fix will be included into the next maintenance release.
</p>
TicketJessePetersonThu, 05 Apr 2012 03:14:37 GMThttps://www.virtualbox.org/ticket/3783#comment:29
https://www.virtualbox.org/ticket/3783#comment:29
<p>
Firstly thanks so much for looking at and resolving this issue! Can you link to the source commit? E.g. If I wanted to implement this change now how might I do that? It looks like maintenance releases are released every month or so but 4.1.12 just came out - is there a timeline for the next release? Thanks again!
</p>
TicketalekseyThu, 05 Apr 2012 10:34:34 GMTsummary changedhttps://www.virtualbox.org/ticket/3783#comment:30
https://www.virtualbox.org/ticket/3783#comment:30
<ul>
<li><strong>summary</strong>
changed from <em>OSX Bridged Networking unreliable due to incorrect MTU setting!</em> to <em>OSX Bridged Networking unreliable due to incorrect MTU setting! => Fixed in SVN</em>
</li>
</ul>
TicketalekseyThu, 05 Apr 2012 12:39:30 GMThttps://www.virtualbox.org/ticket/3783#comment:31
https://www.virtualbox.org/ticket/3783#comment:31
<p>
You can download test build from <a class="ext-link" href="http://www.virtualbox.org/download/testcase/VirtualBox-4.1.13-77269-OSX.dmg"><span class="icon"> </span>http://www.virtualbox.org/download/testcase/VirtualBox-4.1.13-77269-OSX.dmg</a> .
If you want to build OSE version yourself here is the link to the revision containing the fix: <a href="https://www.virtualbox.org/changeset/40764/vbox">https://www.virtualbox.org/changeset/40764/vbox</a> .
</p>
TicketJessePetersonThu, 05 Apr 2012 20:37:53 GMThttps://www.virtualbox.org/ticket/3783#comment:32
https://www.virtualbox.org/ticket/3783#comment:32
<p>
Installed 4.1.13-77269 from above link and indeed WinXP Pro SP3 guest using a PCnet-FAST III adapter seems fixed. Along the original report webpages load without issue anymore. The same is true with a NetBSD guest using the PCnet adapter, too. However:
</p>
<p>
Using the Intel PRO/1000 adapter continues to have the same issues using a NetBSD guest and the above hardware. Perhaps the Intel drivers need to be adjusted as well?
</p>
TicketalekseyTue, 10 Apr 2012 07:44:11 GMThttps://www.virtualbox.org/ticket/3783#comment:33
https://www.virtualbox.org/ticket/3783#comment:33
<p>
Intel PRO/1000 issue should be fixed in <a class="ext-link" href="http://www.virtualbox.org/download/testcase/VirtualBox-4.1.13-77358-OSX.dmg"><span class="icon"> </span>this</a> test build. Could you give it a try?
</p>
TicketJessePetersonMon, 16 Apr 2012 19:13:21 GMThttps://www.virtualbox.org/ticket/3783#comment:34
https://www.virtualbox.org/ticket/3783#comment:34
<p>
Issue appears to be fixed! I think this is <a class="changeset" href="https://www.virtualbox.org/changeset/40799/vbox" title="e1000: Handle packets with unstripped Ethernet CRC (#6152)">r40799</a> . Thank you very much!
</p>
TicketfrankThu, 26 Apr 2012 16:07:20 GMTstatus changed; resolution sethttps://www.virtualbox.org/ticket/3783#comment:35
https://www.virtualbox.org/ticket/3783#comment:35
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>closed</em>
</li>
<li><strong>resolution</strong>
set to <em>fixed</em>
</li>
</ul>
<p>
Fix included in VBox 4.1.14.
</p>
Ticket