OpenBSD PPPoE and RFC 4638

I upgraded my Internet connection from ADSL 2+ to FTTC a while ago. I’m with Eclipse as an ISP, but it’s basically the same product as BT Infinity, right down to the Openreach-branded modem, (a Huawei Echolife HG612 to be exact).

With this modem, you need to use a router or some software that can do RFC 2516PPPoE so I simply kept using OpenBSD on my trusty Soekrisnet4501 and set up a pppoe(4) interface, job done. However what became apparent is the 133 MHz AMD Elan CPU couldn’t fully utilise the 40 Mb/s bandwidth I now had, at best I could get 16-20 Mb/s with a favourable wind. An upgrade was needed.

Given I’d had around 8 years of flawless service from the net4501, another Soekris board was the way to go. Enter the net6501 with comparatively loads more CPU grunt, RAM and interestingly Gigabit NIC chips; not necessarily for the faster speed, but because they can naturally handle a larger MTU.

The reason for this was that I had read that the Huawei modem and BT FTTC network fully supported RFC 4638, which means you can have an MTU of 1,500 bytes on your PPPoE connection which matches what you’ll have on your internal network. Traditionally a PPPoE connection only allowed 1,492 bytes on account of the overhead of 8 bytes of PPPoE headers in every Ethernet frame payload. Because of this it was almost mandatory to perform MSS clamping on traffic to prevent problems. So having an MTU of 1,500 bytes should avoid the need for any clamping trickery, but means your Ethernet interface needs to cope with an MTU of 1,508 bytes, hence the Gigabit NIC (which can accommodate an MTU of 9,000 bytes with no problems).

One small problem remained, pppoe(4) on OpenBSD 5.0 didn’t support RFC 4638. While I sat down and started to add support I noticed someone had added this to the NetBSD driver already, (which is where the OpenBSD driver originated from), so based on their changes I created a similar patch and with some necessary improvements based on feedback from OpenBSD developers it has now been committed to CVS in time for the 5.1 release.

To make use of the larger MTU is fairly obvious, simply set the MTU explicitly on both the Ethernet and PPPoE interfaces to 8 bytes higher than their default. As an example, my /etc/hostname.em0 now contains:

I also added support to tcpdump(8) to display the additional PPPoE tag used to negotiate the larger MTU, so when you bring the interface up, watch for PPP-Max-Payload tags going back and forth during the discovery phase.

With that done the remaining step is to remove any scrub max-mss rules in pf.conf(5) as with any luck they should no longer be required.

I haven’t tried it with NetBSD, but I imagine it can’t be too different given the similar codebase, I would suggest configuring both the parent Ethernet interface and the PPPoE virtual interface with the increased MTU before attaching them together.

I have the net6501-70 model with the 1.6 GHz Atom CPU, and it copes perfectly fine at 40 Mb/s and I wouldn’t expect any difficulty with 80 Mb/s speeds either. I expect the lowest spec 600 MHz net6501-30 model would also work fine in this scenario, I just chose to future-proof as evidence suggests I hold onto my Soekris machines for some years 😉

Do you know if this patch is still included? Just before I go and start trawling the code. 🙂

Have just moved from an old FitPC that I was using as a router (only had 2 NICs), to a 6501-70. The only difference I can see is that I’m using a tagged VLAN interface for the PPPoE connection, legacy config really from my FitPC although it does seem silly to use an entire physical interface if I don’t need to. Set an MTU of 1508 on the physical interface and the VLAN interface, then 1500 on the PPPoE interface however it still connected at 1492.

Am running OpenBSD 5.4, Flashrd. Will try in the next day or so removing the tagged VLAN interface from the equation and see what happens.

Must have been having a stupid moment earlier, set mtu on physical interface of 1512 (4 bytes for dot1q), 1508 on the vlan tagged interface, then 1500 on the PPPoE interface – just incase anyone else attempts this 😉

After a couple of reboots it seems a bit hit and miss with the VLAN tag interface, I’m unclear why it works sometimes and not others I would expect it to be one or the other, tcpdump-ing the VLAN interface that PPPoE is running on doesn’t show me the Max Payload tags allof the time.

It seems 802.1Q has a maximum frame size of 1518 (1500 MTU + 18 ethernet header), it was expanded by 4 bytes to include QinQ (802.3ac) so a maximum of 1522. By my calculations for this the switch would need to be 1508 + 18, for standard 802.1Q which puts me outside the maximum of 1522?

On a side note I don’t need to increase the 802.1q interface in OpenBSD by 4 bytes, I suspect this was just one of the times it worked and then didn’t subsequently, as the em driver should handle this automatically. Dedicated interface it is it seems. Unless anyone reading this comes up with anything and can show me otherwise?

On a side note, my eth3 on the 6501 wont seem to get layer 1 connectivity – suspect it might be faulty, but will install an Ubuntu setup just to see if it works there. I assume all 4 interfaces on yours just worked out of the box with OpenBSD, no tweaking?

I presume you’ve set up a handful of hostname.if(5) files? I did find if you attach the pppoe(4) interface to a normal-sized interface and then tweaked the MTU afterwards it would not automatically increase its own MTU. I wonder if there’s some timing nit somewhere in the boot process that means you’re not getting the same emX -> vlanX -> pppoeX order every time. Once the Soekris has booted can you manually fix it?

The switch shouldn’t stop you configuring the interface as there’s no MTU negotiation as such, although if it does indeed have a problem then you would notice that as soon as you pass full frames of traffic. What switch are you using? I have my Soekris attached directly to the Huawei modem, but my switches are Gigabit-capable so they should be able to do jumbo frames. IIRC part of the RFC 4638 specification does say to try and send some maximum MTU-sized packets as part of the negotiation to make sure there’s no intermediate device in the way that can’t handle them, but it’s not present in pppoe(4).

For your faulty eth3 I’m only currently using the first two interfaces but I’ve just done a quick test and I get connectivity on the other two interfaces so it possibly is faulty.

Thanks for the reply, I need to have a play around and see if I can work out what is going on – however the physical and VLAN interface seem to take the MTU fine, but the PPPoE wouldn’t start if the tagged VLAN interface wasn’t up? But I agree something timing wise sounds most probable.

For now am waiting a new board for my device from Soekris, pretty good service so far, will have a play with everything once it arrives.

I’m sure there’s a few different 100 Mb/s NIC chipsets that have undocumented higher MTU sizes just due to how they’re designed. After all, it only has to allow an additional 8 bytes rather than support ~ 9,000 byte frames.

I’ve never tried to get console access on my HG612, all I know is it will happily negotiate the larger PPPoE payload. I can’t remember from testing what happens if I try to negotiate an MTU larger than 1500.