All test below were from the ftp -4a command you suggested. I tried different sites (first with the bsd and when that was done i downloaded the exact same file with my laptop). All the test were done through the dlink so I didnt have to deal with pppoe authentication.

i tried several sites with similar results. openbsd got no greater then 300k...windows consistantly got 1+ meg

Then I took a old 3com 100mb card and created /etc/hostname.xl0 with dhcp in the file. I never could get a file to download. I kept getting watchdog messages.

Then I formatted and put windows vista on the atom machine.(flame suit on) After installing the network card drivers I got 1.7meg from the same file. I got the same results from both network cards.

Then I reformatted openbsd with i386 with no changes except for /etc/hostname.re0 == dhcp
Top speed on the same file was 300k.
I was able to get a different 3com card to work on this box but i only got about 200k though the pci slot.

I have saved my dmesg and hopefully that will help diagnose the problem

So this seems to be the optimal setting for getting 1.6 MB downloads, when the Atom board has been connected directly to your D-Link router, and doesn't have to do the PPPoE stuff.

The bad news is that increasing this value on OBSD as an router, has no influence whatsoever
It only has effect on a communication end-point, or a box which has setup a network or internet socket. A router only passes packets in and out and doesn't use network sockets, whose speed of operation can be optimized by increasing buffer space.

RE: the netstat diagnostics

The netstat -ss doesn't show any excessive retransmissions or other errors, so that is ok. Same applies to the netstat -in output.

Strategy to use OBSD on the Atom board as router

One simple way is to enable pf and instruct it to do NAT.

Why? Your external interface re0 gets a 192.168.0.0/24 address from the D-link, while your internal network, your laptop, is connected to re1, which has a 10.0.0.1/24 address.

The D-link will have no problem NATting the 10.0.0.0/24 address of the laptop to it's public address. When a return packet arrives for a connection initiated by the 10.0.0.0 network, the D-link will have no problem in properly converting, deNATting, the packet back to the initiating 10.0.0.0/24 address.

It has only one slight problem, it doesn't know where the hell to send this packet to. It only knows its internal 192.168.0.0/24 network.

It wouldn't have this problem if it could be told to route reply packets for the 10.0.0.0/24 network to 192.168.0.1 address. In other words for the D-link 192.168.0.1 is the gateway for the 10.0.0.0/24 network.

If the D-link doesn't allow to add this static route, you can circumvent it by doing NAT on the OBSD box.

My assignment to you:

Refrain from doing PPPoE on the OBSD box for now,

Create a simple pf which just does NAT and only allows traffic out, which has been initiated by your 10.0.0.0/24 and 192.168.0.0/24 network.

Because pf makes these rules stateful, the return traffic will have allowed in without any problem

Find out whether the D-link can be configured to accept static routes.

If that is possible pf doesn't have to do NAT

Re: amd64 versus i386 speed

Because I wanted this thread to focus on the network speed issue, and not drift away to a why "OBSD i386 would be faster than amd64 " discussion I refrained from speaking out till now.
.
I have read several times on misc where Henning Brauer suggested to use OpenBSD i386 on AMD boxes, instead of the amd64 version if speed for a pf box was a concern. I only didn't manage to locate those messages in the misc mailing list archives

Feel free to open another thread for this issue, but in this thread let us focus on the network speed problem. Thanks

__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

I will just move from 10.0.0.X to 192.168.0.XXX to make things simpler for now.

I will keep the dlink doing pppoe and use re0 to interface with the router via dhcp like you suggested.
Then I will use these rules (for now)

Code:

ext_if="re0"
int_if="re1"
set block-policy drop
match in all scrub (no-df)
nat on $ext_if from !($ext_if) -> ($ext_if:0)
block in
block out
pass out keep state
antispoof quick for { lo $int_if }
pass out quick on $int_if from any to any
pass in quick on $int_if from any to any
pass out quick on $ext_if from any to any

If increasing the receive space doesnt affect the router then what can I do to increase the throughput? I know its the box itself that is the bottle neck so there has to be something that can be done to make this thing route fast?

You cannot move the re1 interface from 10.0.0.0/24 to 192.168.0.0/24 when re0 is already using 192.168.0.0/24. That doesn't work in the BSDs

Both interfaces, re0 and re1 have to be on different networks. If the D-link hands out a 192.168.0.0/24 address by DHCP to re0 then using a 10.0.0.0 network for re1is a good choice.

Re: throughput

I suspect the userland PPPoE you used was the culprit. Just try this setup first and check which download speed your laptop gets now.
We still have other possibilities to explore, but first check whether my recommendation works. Then we always can fall back on it, when the other alternatives like kernel PPPoE don't work out

RE: your pf

J65nko's first rule for a pf ruleset:

Start with block log all as the default policy. This will block both in and outgoing traffic.
In a second terminal as root do:

Code:

tcpdump -eni pflog0

Any blocked packets will show up in the tcpdump output, which is a great help in fixing your ruleset.

__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

I have read several times on misc where Henning Brauer suggested to use OpenBSD i386 on AMD boxes, instead of the amd64 version if speed for a pf box was a concern. I only didn't manage to locate those messages in the misc mailing list archives

Henning has told people not to use MP systems as routers because the kernel and pf do not take advantage of them.. but AFAIK he hasn't said anything about the i386 port being faster than the amd64 port.

As for cranking the {recv,send}space knobs, they typically don't recommend tweaking those either.. 65535/65536 are probably the highest you should set them if you ever do, any higher and you're wasting memory and bandwidth on every connection.

Your Atom router should be more then capable achieving full bandwidth if you generate enough traffic on your LAN systems, if you find the global TCP/UDP window sizes on your BSD systems too small then you're free to tweak them if you have the need.. the need for speed.

Sorry for going off-topic J65nko, but personally I would recommend against using SOHO router devices.. OpenBSD has 2 PPPoE clients, kernel and userland.

Henning has told people not to use MP systems as routers because the kernel and pf do not take advantage of them.. but AFAIK he hasn't said anything about the i386 port being faster than the amd64 port.

As for cranking the {recv,send}space knobs, they typically don't recommend tweaking those either.. 65535/65536 are probably the highest you should set them if you ever do, any higher and you're wasting memory and bandwidth on every connection.

If using 2x65535 speed gives him the equivalent download speed of his laptop directly connected to his D-link router, that proves, it was neither a hardware issue nor an OpenBSD issue.

This was the only point of the exercise, because the size of the receive space will have nada effect when the Atom board will be used as a router.

Quote:

Sorry for going off-topic J65nko, but personally I would recommend against using SOHO router devices.. OpenBSD has 2 PPPoE clients, kernel and userland.

I mentioned in my previous post that kernel PPPoE would be the second option we would explore. But we first want to make sure his board performs well as a router using the most simple configuration.

Please remember that the OP figuratively speaking , just walked out of Uncle Bill Gates' kindergarden, so we go step by step

__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

just for grins....i decide to put my old ppp.conf back in, change rc.conf and pf.conf to reflect tun0

The speed went right down to 300k

What doesnt make sence is the fact that this is the same ppp.conf i have used for over 2 years with no problems in speed. After I upgrade the hardware (and the os to 4.6) all of a sudden I have these problems. (maybe the ISP changed their settings?)

OK, so we now have confirmation that the hardware is OK and that OpenBSD is not the bottleneck in routing and filtering packets to your laptop.
One problem solved

There are 2 implementation of pppoe in OpenBSD, one is part of the kernel and the other one is in userland.
The userland one has the advantage that it is somewhat easier to debug, but it is slower because data needs to be copied from/to kernel space to/from userland.

The kernel pppoe is described in the pppoe(4) man page. It is quite readable and even gives an example configuration.

Give it a try. Just make sure you know how to revert your setup in case things don't work out

One warning, I don't use PPPoE so any efforts of my side will be completely theoretical. Or maybe somebody with kernel pppoe experience can give a helping hand.

BTW My daughter lives in Eindhoven, her appartment block has 100mbit up/down. Unbelievable fast.
I live about 25 km more to the southeast, 1 km from the Belgian border. The town government is trying to get people interested in a glass fiber connection here too.

__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump