KPTree Router Setup

Work in progress

Router Background

Well here it goes. I have had the router hardware for a few months now. I am proposing to setup up primarily as a home router on baremetal, and with DNS, DHCP and VPN probably running in VMs or containers. Further to this I will attempt to use NFTables instead of IPTables to setup the router. It is a bit to bite off, but I have the time now. It has taken a lot of time to read, research, and test the configuration. I have yet to fully implement. Further to this separate configuration files are required, as the exact specific configuration would be a security risk to make public.

A number of online references were used to assist with planning and configuring this router. These can be found in the Reference (Related Links) section and were particularly relevant are also repeated in the section text. No individual online reference was exclusivly used.

Router Hardware

I looked at the various options for the router hardware:

A small ARM based machine, e.g. Raspberry Pi. However these machines are generally limited in a number of way, including by definition not x86 based. Many do not have more than one NIC and the NIC are often not full Gigabit. The main upside is that they are small, low power and relatively cheap. Those with only one NIC need to be setup with USB NIC adaptors, that further complicates setup, performance and reliability. Although better spec'ed machines, e.g. with multiple gigabit NICs, start getting more pricey too. I suppose you get what you pay for....

An older x86 based machine. The main downside to these is poor power consumption and large size, even an old server tends to use more than 30W at the wall. Also the board I had only had one built in NIC, so I would need a PCIe NIC card. There is also the issue of reliability and performance for the older hardware, although it is probably good enough in this respect.

At the moment there are a lot of Intel Celeron J1900 based units with 4 NICs around. The J1900 is an older CPU, 4 cores, 2.0-2.42 GHz. Also in many cases the NIC hardware is older, particularly on the cheaper units, so care must be taken if you want to ensure more up to date hardware. These machines are a good option, low power (~8 - 10W), small size. They come with 2 SATA ports and mini PCI-E slots. By the time you fit them out they cost out USD250 - 350, with 4-8GB RAM and 120GB mSata drive. The cheaper options are as noted above usually with older NIC hardware and lower memory and HD size and can be had at even lower prices.

I decided to get a Supermicro SYS-E200-9B Intel Pentium N3700 system with 4 x Intel i211-AT GbE LAN. I got with maximum 8GB RAM and 120GB mSata HD. The N3700 CPU is more modern than the J1900 and includes AES instruction that the J1900 does not have. The AES CPU instruction help improve encryption performance significantly, hand for SSL / VPN. Otherwise the overall performance is slightly better (1.6-2.4GHz) and power lower than the J1900. This unit also comes with a dedicated IPMI LAN Port, allowing full remote KVM operation on the network. A downside of the IPMI is that it uses another 3.5W of power. My home server is also a Supermicro based unit with dedicated IPMI LAN Port and has given me a good 2 years of service to date. Downside is mainly the price, USD490 + delivery, as these units are not sold locally I purchase in USA and had it mailed at USD75. In any case this hardware should allow for a router with great performance for some years to come. Again you get what you paid for.....

I dont see the point installing a 64bit OS on systems with less than 4GB of RAM. A 32bit OS can only natively access up 4 GB RAM, but should give better compromise with such limited RAM.

IPMI KVM Problems

The remote KVM and IPMI, BMC are not used often, however they negate the need for the use of separate keyboards and monitors to setup and maintain these machines and allow true convenient headless setup, maintenance and use.

Basically after setting up Ubuntu 16.04 amd64 server edition on the router hardware I noticed a problem with the IPMI KVM terminal. During the Ubuntu startup the KVM screen would just go blank. However login into a SSH session on the main board NIC was working normally. After a bit of head scratching and investigation I worked out the problem to be related to the design of Intel N3700 with the built graphics processor that was conflicting with the BMC built into the motherboard a Supermicro X11SBA-LN4F in the also Supermicro SYS-E200-9B. So the solution is to ensure that Ubuntu does not load any "special" main board (Celeron N3700) CPU graphic drivers. For Debian and Ubuntu this is done by setting the "noomodest" option into the grub bootloader. this can be done by editing the grub bootloader during boot up, a one off solution and by making permanent by editing the grub configuration file. The reliablesite.net give a good explanation in their article How to set 'Nomodeset' into the grub bootloader. For the permanent solution basically edit etc/default/grub, adding nomodeset such that GRUB_CMDLINE_LINUX_DEFAULT= "nomodeset" and then execute sudo update-grub.

My home server already in service over 2 years has a Supermicro motherboard with Intel Atom C2750 CPU A1SAi-2750F also with IPMI, BMC & KVM and did not display this problem. This makes sense as the Atom C2750 CPU does not have a internal graphic capacity, so the only graphics capacity was on the BMC video controller. The Ubuntu drivers defaulted to this basic BMC graphics display system.

Router Linux Network Setup

The hardware comes with 4 dedicated NIC controllers. NIC0 is on a dedicated PCIe lane, whereas NIC1 to 3 use a multiplexer to share another PCIe lane. The PCIe lane with the 3 shared NIC controllers have enough bandwidth to handle maximum combined throughput of the 3 NICs, however the multiplex does add a minor processing delay, although better than an additional external switch.

I plan to dedicate NIC0 to the WAN and bridge NICs 1-3 to the LAN. Also the bridged LAN network will used for the VMs with dedicated IP addresses on the LAN. The main NFTables based router will run on baremetal and a number of VMs used for DNS, DHCP, VPN and logger.

Router Basic Ubuntu Server Setup

Download the latest Ubuntu Server amd64 ISO file from the Ubuntu website. I setup the IPMI KVM to provide virtual storage to the Ubuntu ISO file and started up the Router. The Ubuntu software loads up of the ISO across the KVM virtual storage setup and can be then setup as normal. When setting up Ubuntu below are some the of keypoints:

I setup Ubuntu to use LAN port 2, as I want to use LAN port 1 as the Router WAN port.

I do not encrypt the home directory. (See How to install Ubuntu Server - Xenial Xerus 16.04LTS for an explanation.)

I just use standard setup for one main partition, which basically gives one large data partition using all the disk space, save that allowed for the SWAP partition. The SWAP partition is automatically sized based upon detected memory. (I have never been one for multiple partitions.)

Ubuntu Network Setup

The units 4 main port will be setup to look like a router with 1 WAN port and 3 LAN port. The WAN port will be setup on NIC0 with the LAN ports 0-2 will be on bridged NIC1-3.

# The gateway directive is not required as any traffic to 192.168.a.1 not on subnet /24 will be Netfiltered and if accepted passed to WAN.

# gateway 192.168.a.1

# auto eth1 and iface eth1 inet manual are not required as as iface inet br0 will bring up the components assigned to it.

# iface eno2 inet manual

# iface eno3 inet manual

# iface eno4 inet manual

# The use of allow-hotplug eth1 is not used as normally these interfaces should be running. The br0 interface will be used for virtual machine access and must be running to allow coorect VM startup. During inital boot this will cause delays as attempts are made to find network devices, particularly any not used. These delays are necessary for reliable start-up and operation.

# allow-hotplug eno2

# iface eno2 inet manual

# pre-up ifconfig $IFACE up

# pre-down ifconfig $IFACE down

System Forwarding Enable

To allow the router to forward packet the Linux kernal must be setup to allow this. This is not necessarily a standard option.

NFTables Configuration

Under construction

If is difficult to find good simple comprehensive information for NTFilter at this time, perhaps it is too new. There is alot of information on iptables. One of the best references I found Wolfhechel github nftables router. The reference at stosb is good, but not for a router Explaining My Configs: nftables. Create the following file called: "router.nft".

To add elements to nat tcp_nat_map: "sudo nft add element nat tcp_nat_map { 81 : 192.168.1.100, 8080 : 192.168.1.101 }". (For some reason the version of nft I have will not read in the elements via a nft -f command.)

DNS Setup

Next check the named.conf configuration file, "less /etc/bind/named.conf". This can remain as default as below. However the configuration files noted there in will need to be set up. We will copy the existing files to default:

The forwarders section contains the DNS servers to be checked if this DNS does not have the record. I have been using OpenDNS to allow some free security screening, IP 208.67.222.222 and 208.67.220.220. Another common option is to use Google DNS at 8.8.8.8 and 8.8.4.4

Next create a cryptographic key file using "Sudo /usr/sbin/rndc-confgen -a", note that this command can take quite some time to complete, a number of minutes. The command produces a key file "/etc/bind/rndc.key".

# This line indicates that the object we're configuring below (in this case,

# kptree.net) has its origin at the "." domain. "." is the root domain

# from which all the TLDs branch.

$ORIGIN .

# Next line defines the DNS time-to-live setting

$TTL 907200 ; 1 week 3 days 12 hours

# The next set of lines are the "Start of Authority" record and define important

# info about the domain. In my case, we're defining kptree.net and saying

# that router.kptree.net is its source host, and webmaster@kptree.net

# is the domain maintainer. For the e-mail address, we use a dot instead of an @.

# The lines after that define the zone serial number, which is used to keep track

# of when the zone file was modified, and then some interval definitions which

# you can leave as default.

kptree.net IN SOA router.kptree.net. admin.kptree.net. (

234284 ; serial

10800 ; refresh (3 hours)

3600 ; retry (1 hour)

604800 ; expire (1 week)

38400 ; minimum (10 hours 40 minutes)

)

# Next, we define the hosts necessary to make the domain function. First, we add

NS router.kptree.net.

# ...then an "A Record" for the domain server's IP address...

A 192.168.0.1

# ...and finally "MX Records" so that e-mail for the domain's e-mail addresses

# goes to the right place.

MX 10 mail.kptree.net.

#

# NOTE THE TRAILING PERIODS. THEY ARE EXTREMELY IMPORTANT.

#

#

# Now we're ready to begin adding hosts, but first we need another origin

# statement to indicate that the hosts added below originate not from ".", like

# the domain itself; rather, they originate from "kptree.net".

#

$ORIGIN kptree.net.

#

# Again, NOTE THE TRAILING PERIOD.

# Now we add A records for the non-DHCP hosts in the domain:

kptreeserver A 192.168.0.2

switch A 192.168.0.3

wwwserver A 192.168.0.4

mailserver A 192.168.0.5

Define the reverse zone, "sudo vim /var/lib/bind/192.168.1.rev"

# Again, we have an origin record and a TTL entry...

$ORIGIN .

$TTL 907200 ; 1 week 3 days 12 hours

# note the name of the reverse domain: "0.168.192.in-addr-arpa". This

# is a special name format used only by reverse lookup domains.

0.168.192.in-addr.arpa IN SOA router.kptree.net. admin.kptree.net. (

12 ; serial

10800 ; refresh (3 hours)

3600 ; retry (1 hour)

604800 ; expire (1 week)

38400 ; minimum (10 hours 40 minutes)

)

NS router.kptree.net.

#

# Just like above, we now set our origin away from "." to the actual domain

# name, which is "0.168.192.in-addr-arpa", and then we add records. However,

# this time, we're adding "PTR records", or pointer records.

$ORIGIN 1.168.192.in-addr.arpa.

2 PTR switch.kptree.net.

3 PTR kptreeserver.kptree.net.

4 PTR wwwserver.kptree.net.

5 PTR mailserver.kptree.net.

If and of the above files are changed the serial number should be incremented up before updating the the DNS service, "sudo systemctl restart bind9". A common technique is to use the date followed by a small single or double digit number, e.g. 2017072101.

RFC 3232 replaced RFC 1700. RFC 3232 states that RFC 1700 has been replaced by an online database, see link given above. RFC 6335 also has information on Port Number Registry and the associated database.

Most Unix like operating systems have a service name database file: "/etc/services". It is assumed that nft uses the /etc/services database for named ports definition.

IPv4 uses some of these special addresses for private LANs (Local Area Network)s with NAT (Network Address Translation) used to connect the LANs to the WAN (Wide/World Area Network) via a router. This was required to compensate for the limited address space in IPv4. IPv4 NAT also provide some security benefits by obscuring the private LAN addresses from the public WAN.

IPv6 does not use NAT as its native address space is suffiently large never to require in the forseeable future.

Ubuntu Network Setup Links

Links relating to bridged and bonded Networking

A bridged network allows different networks to be connected, both physical, like NICs or Wifi and virtual, allowing virtual machine to connect to a physical network and even be assigned a LAN IP address. Bonding allows physical networking devices such as NICs or Wifi to be bonded to allow increased bandwidth or redundancy. Sadly there seems to be alot of information out there that isceither for older version of software or other purposing.

Disclaimer: All data and information provided on this site is for informational purposes only. kptree.net makes no representations as to accuracy, completeness, currency, suitability, or validity of any information on this site and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis. kptree.net does not collect any personal information about its visitors.