Linux on The Gig of Hamhttps://www.gigofham.com/categories/linux/index.xml
Recent content in Linux on The Gig of HamHugo -- gohugo.ioen-usThis work is licensed under a Creative Commons Attribution 4.0 International License.Something better than a Rasberry Pi 2Q-2018 Editionhttps://www.gigofham.com/post/2018/05/23-better-than-RPi-2Q2018/
Wed, 23 May 2018 22:16:20 +0000https://www.gigofham.com/post/2018/05/23-better-than-RPi-2Q2018/<p>Ah, the ubiquitous Raspberry Pi. Loved and hated by many. I fall in both camps. I have at least 20 of them currently, and several are in production use. I also spend a lot of time with the larger ARM community, including the <a href="https://www.worksonarm.com/">Works on arm</a> project, and <a href="https://www.96boareds.org/openhours/">96boards Open Hours</a>. Something that comes up a lot is people wanting something &ldquo;better&rdquo; than a Raspberry Pi, and I finally decided to start writing up a comparison of boards I have used and like, that are generally available, and how they are better. The goal is to update this on a regular basis, at least once a year. There is some great stuff coming down the pipeline. But for now, let&rsquo;s jump in.</p>
<p></p>
<h1 id="raspberry-pi-s-current-state-of-the-art-the-3b">Raspberry Pi&rsquo;s current state of the art: The 3B+</h1>
<p>Before we jump into &ldquo;better&rdquo; let&rsquo;s talk about the gold standard from which everything is compared: The Raspberry Pi. The current generation is the <a href="https://www.raspberrypi.org/magpi/raspberry-pi-specs-benchmarks/">3B+</a>, which is a minor revision to the <a href="https://www.raspberrypi.org/magpi/raspberry-pi-3-specs-benchmarks/">3B</a> (which almost everyone refers to as the 3). The 3B and 3B+ are based on a quad-core Broadcom SoC that implements the ARMv8 instruction set using the Cortex-A53 IP. It also has 1GB of RAM, SD Storage, and USB based Ethernet. Both the 3B and the 3B+ also implement WiFi connectivity, but the WiFi on the 3B+ is the major improvement for that release. It&rsquo;s gone from 2.5GHz only to dual band, and updates the speed from the N standards into the AC standards.</p>
<h1 id="defining-better">Defining &ldquo;better&rdquo;</h1>
<p>So, now that we know what the state of the art with the Raspberry Pi is, let&rsquo;s talk about it&rsquo;s shortcomings. Where I think the RPi falls short is RAM and Ethernet. The 1GB of RAM is very limiting, and the USB based Ethernet has a myriad of limitations when it comes to throughput and latency. The WiFi is nice, but when running things like small clusters it&rsquo;s more of a burden than an asset. Another major issue, especially on the newer boards, is power delivery. The MicroUSB power connector is a major limiting factor as it&rsquo;s limited to about 2.5A@5VDC and the 3B+ can easily use more than this. Finding good power supplies and cables can also be an issue here. Ideally you could power the board off the 5VDC pins on the GPIO connector, but it turns out doing this bypasses all the voltage regulation and power conditioning circuitry, so now that&rsquo;s a problem you have to deal with as well.</p>
<p>So, how I am defining better is:</p>
<ul>
<li>Better power connector</li>
<li>Native Ethernet</li>
<li>More RAM</li>
</ul>
<p>Minimum requirements are:</p>
<ul>
<li>Must be ARMv8 based, minimum quad core</li>
<li>Must run Linux at a minimum</li>
<li>Must be &ldquo;available to purchase&rdquo; and not a pre-order, coming-soon, or discontinued.</li>
</ul>
<p>Not a terribly long list, but this is only my first go at this comparison. I&rsquo;m sure more requirements will be added as time goes on, please feel free to leave your thoughts in the comments at the bottom of the article.</p>
<h1 id="the-contenders">The Contenders</h1>
<p>Here are the contenders and the related specs in no particular order:</p>
<table>
<thead>
<tr>
<th>Name</th>
<th>Cores</th>
<th>RAM</th>
<th>Network</th>
<th>Storage</th>
<th>OS</th>
<th>Form Factor</th>
<th>Case</th>
<th>Approx Cost</th>
<th>Notes</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G145457216438">ODroid-C2</a></td>
<td>4xCortex-A53 (AMLogic S905)</td>
<td>2GB</td>
<td>1GbE Ethernet</td>
<td>MicroSD or eMMC</td>
<td>Vendor Ubuntu, AOSP, Armbian</td>
<td>RPi Compatible</td>
<td>Yes</td>
<td>$46</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.pine64.org/?page_id=7147">ROCK64</a></td>
<td>4xCortex-A53 (RockChip RK3328)</td>
<td>2/4GB</td>
<td>1GbE Ethernet</td>
<td>MicroSD, eMMC, SPI</td>
<td>Vendor Ubuntu</td>
<td>RPi Compatible</td>
<td>Yes</td>
<td>$35/$45</td>
<td></td>
</tr>
<tr>
<td><a href="http://en.t-firefly.com/product/Rk3399/index.html">Firefly-RK3399</a></td>
<td>4xCortex-A53,2xCortex-A72 (RockChip RK3399)</td>
<td>2/4GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD</td>
<td>Vendor Ubuntu</td>
<td>Custom</td>
<td>Yes</td>
<td>$199/$209</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.solid-run.com/marvell-armada-family/macchiatobin/">MACCHIATObin</a></td>
<td>4xCortex-A72 (Marvell ARMADA 8040)</td>
<td>1xDDR4 DIMM</td>
<td>2x10GbE, 1x2.5GbE, 1x1GbE</td>
<td>MicroSD, eMMC, SATA</td>
<td>Vendor SuSE, any UEFI Linux Installer</td>
<td>Custom</td>
<td>Yes</td>
<td>$269 and up</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.96boards.org/product/poplar-hoperun/">Poplar</a></td>
<td>4xCortex-A53 (HiSilicon Hi3798CV200)</td>
<td>2GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD</td>
<td>Any UEFI Linux Installer</td>
<td>96Boards-EE</td>
<td>????</td>
<td>$99</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.96boards.org/product/hikey970/">HiKey970</a></td>
<td>4xCortex-A73, 4xCortex-A53 (HiSilicon Kirin 970)</td>
<td>6GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD</td>
<td>Vendor Debian</td>
<td>96Boards-CE</td>
<td>????</td>
<td>$299</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.96boards.org/product/dragonboard820c/">DragonBoard 820c</a></td>
<td>4xKryo (Qualcomm Snapdragon 820E)</td>
<td>3GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD</td>
<td>Vendor Debian</td>
<td>96Boards-CE</td>
<td>????</td>
<td>$199</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.nvidia.com/en-us/autonomous-machines/embedded-systems-dev-kits-modules/">Jetson TX1 Kit</a></td>
<td>4xCortex-A57</td>
<td>4GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD, SATA</td>
<td>Vendor Ubuntu</td>
<td>Custom</td>
<td>????</td>
<td>$477</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.nvidia.com/en-us/autonomous-machines/embedded-systems-dev-kits-modules/">Jetson TX2 Kit</a></td>
<td>4xCortex-A57</td>
<td>8GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD, SATA</td>
<td>Vendor Ubuntu</td>
<td>Custom</td>
<td>????</td>
<td>$570</td>
<td></td>
</tr>
</tbody>
</table>
<h1 id="those-who-did-not-make-the-cut-and-why">Those who did not make the cut, and why</h1>
<ul>
<li><a href="http://www.lenovator.com/product/103.html">LeMaker Cello</a> : No longer available</li>
<li><a href="https://www.pine64.org/?page_id=1194">Pine64</a> : Gigabit Ethernet doesn&rsquo;t work due to design flaw in board.</li>
</ul>
<h1 id="next-steps">Next steps</h1>
<p>So, as I said above, I would love to hear from your additional suggestions meeting the criteria listed above. Please comment below or reach out to me on Twitter. The more feedback I get, the easier it will be to update this as time goes on!</p>Bring up of the LeMaker Cellohttps://www.gigofham.com/post/2018/02/06-cello/
Tue, 06 Feb 2018 22:16:20 +0000https://www.gigofham.com/post/2018/02/06-cello/<p>Many, many moons ago I pre-ordered the <a href="http://www.lenovator.com/product/103.html">LeMaker Cello</a>. It&rsquo;s a 96boards Enterprise Edition Single Board Computer based on the AMD &ldquo;Seattle&rdquo; Opteron A1100 ARM Processor. I was very excited, because unlike many of the boards I already had it sported three key features: Native SATA ports, a full PCI Express 16x mechanical expansion port, and 2x SO-DIMM slots for memory. After many months of delays, we got some bad news: AMD had discontinued the processor and the PCI Express port didn&rsquo;t work. LeMaker offered those of us who pre-ordered to swap our pre-orders for a <a href="http://www.lenovator.com/product/80.html">HiKey 960</a> board instead based on this news, since that had a PCI Express expansion port. Unfortunately for me, the <a href="http://www.lenovator.com/product/80.html">HiKey 960</a> doesn&rsquo;t have gigabit ethernet or expandable RAM, so I stuck with my pre-order. I had planned to use them as a pair of Database servers for my hosting company, and the lack of PCI Express expansion wasn&rsquo;t a big deal. Several months later, the boards shipped and then my adventure began.
</p>
<h1 id="getting-the-boards">Getting the boards</h1>
<p>The boards arrived in terrible condition physically. I had ordered two, and they were bubble wrapped and then stacked on top of each other, placed in a makeshift cardboard box that barely contained them, and then handed to China Post. They suffered the damage one would expect when something like this is shipped with insufficient packaging (eBay buyers of electronics around the world know this pain). One of the boards appeared fine, the other had a corner chipped off and the heatsink/fan was ripped off the board in transit. I was able to find the <a href="https://www.96boards.org/documentation/enterprise/cello/quickstart/">quick start guide on the 96boards site</a> and get some of the specifications of components I needed to continue.</p>
<p>I knew a needed a power supply, some RAM, and possibly a few other accessories. Sadly, the power supplies linked from the quick start guide were not available to me or super expensive, so I went digging for some alternatives. Some searching on <a href="https://www.digikey.com/">DigiKey</a> yielded a <a href="https://www.digikey.com/products/en?keywords=62-1186-ND">Volgen KTPS90-1207</a> which was not an exact match, but was reasonably priced and so far has worked without issue. I&rsquo;m also running the board rather lightly loaded, just some RAM and a single SATA SSD. I was able to get the SATA power adapter from Amazon, and then went looking for RAM. Ideally I would have got a 4pin DIN power supply, but those were going to cost as much as the boards, so this 7A barrel connector power supply would have to do as a starting place.</p>
<p>Initially I foolishly bought some regular old SO-DIMM modules. They didn&rsquo;t appear to work, so I contacted LeMaker support. They were using the <a href="https://www.micron.com/parts/modules/ddr3-sdram/mt18ksf1g72hz-1g6">MT18KSF1G72HZ-1G6E2</a> modules from <a href="https://www.micron.com">Micron</a>. Turns out these are ECC, and specifically ECCx72 (which is not common). Some digging later and I found some <a href="http://www.memory4less.com/micron-8gb-sodimm-pc10600-mt18ksf1g72hz-1g4d1ze">8GB Micron modules from memory4less.com</a> that are compatible. I bought two of them, and waited for all the accessories to arrive.</p>
<h1 id="more-bad-news-from-lemaker">More bad news from LeMaker</h1>
<p>It was at this point that I got some more bad news from LeMaker: the boards may not boot. They believe the boards were shipped to me without the necessary firmware installed. The provided two options: I could upload the firmware my self, or send the boards back to LeMaker for repair. I had planned on flashing the board that appeared to be intact, and return the second.</p>
<h2 id="attempting-to-flash-the-firmware-on-the-board">Attempting to flash the firmware on the board</h2>
<p>LeMaker provided the necessary <a href="https://edolnx-public.objects-us-east-1.dream.io/cello/cello_platform_firmware.hex">firmware</a>, the target part (an Atmel AT24C512B at I2C Address 0x54), and this image of the I2C connector:</p>
<p><img src="https://edolnx-public.objects-us-east-1.dream.io/cello/8723352C%4009753F7%2805-09-14-29-52%29.jpg" alt="I2C pins next to the SATA power connector" /></p>
<p>I attempted to flash the part using a RaspberryPi, but was unable as I think the voltages were different or possibly pull-up resistors were necessary. Either way, I gave up and shipped both boards back and eagerly awaited their return.</p>
<h1 id="the-boards-arrive-part-2">The boards arrive, part 2</h1>
<p>A few weeks pass, and I now have known good boards. I get home, hook them up, and get: a spinning fan and LED7 turning on when I apply power but that is it. No output from the serial console device. I reach out to LeMaker and Lenovator for support, and get back that a wiki page is forthcoming. However, the board has been marked as sold out on their site and the word on the street is that AMD has killed the Seattle project and discontinued the processors.</p>
<p>Time passes.</p>
<p>I ask for updates, and reach out to see if anyone has got these boards to work on twitter and the Phoronix forums.</p>
<p>Nothing happens.</p>
<p>The boards sit in a drawer, essentially very expensive paper weights. I&rsquo;ve mostly lost hope that they will ever be more that museum artifacts.</p>
<h1 id="office-hours">Office Hours</h1>
<p>Several months, I get involved with the <a href="http://www.worksonarm.com/">Works on ARM</a> community, and ask there for help. They point me to the <a href="https://www.96boards.org/openhours/">96boards Open Hours</a> and after a couple of false starts there I&rsquo;m able to reach out and get some help. Robert from Open Hours gets me in touch with Ard and Ricardo who were engineers on the Cello project. I explain my delemma and learn an important step: on my Rev 003 boards there is another serial port on the board which tells you if the memory can be configured, and there is a power button (YMMV on other board revisions).</p>
<h1 id="the-scp-serial-port">The SCP Serial Port</h1>
<p>Turns out there is a microcontroller on the board that is responsible for setup of the DRAM and other components. It has a read only 6 pin FTDI serial header next to the PCI Express port, and it outputs lots of useful debugging information at 38400bps. I don&rsquo;t have an 6 pin FTDI cable lying around, but I have a lot of serial adapters and some jumpers, so I find the port and connect:</p>
<p><img src="https://edolnx-public.objects-us-east-1.dream.io/cello/IMG_20180207_125551_r.jpg" alt="6 pin FTDI header next to PCI Express connected to serial adapter" /></p>
<p>The green cable is ground and connected to the labeled pin 1. The yellow cable is connected to the RX pin on my serial adapter. I plug this into a Linux host, fire up <code>picocom</code>, and get the following output upon power on of the Cello board:</p>
<pre><code>SCP Bootrom version : 1.1004
SCP Bootloader version : 2.1001
BL2 POST code : 0x00000020
BL2 POST code : 0x00000021
BL2 POST code : 0x00000022
BL2 POST code : 0x00000025
BL2 POST code : 0x00000026
BL2 POST code : 0x00000023
BL2 POST code : 0x00000024
BL2 POST code : 0x00000027
........
System Control Processor Firmware
Version: 1.0.0.2 (ROD1002C_0493 R)
Advanced Micro Devices, Inc.
All rights reserved.
[INFO] Seattle Model B SOC detected. Now initializing.
[INFO] Interpreting EEPROM settings...
[INFO] EEPROM root table - ID: 0000000000000000, Major: 0, Minor: 152.
[INFO] EEPROM area - Sign: PFDT, Rev: 0, Offset: 0x00000070, Len: 19.
[INFO] EEPROM area - Sign: PMAT, Rev: 0, Offset: 0x00000000, Len: 0.
[INFO] EEPROM area - Sign: USAT, Rev: 0, Offset: 0x00000000, Len: 0.
[INFO] EEPROM area - Sign: EATL, Rev: 0, Offset: 0x00000000, Len: 0.
[INFO] EEPROM area tables found - 4.
[INFO] Area table: Platform and feature definition signature.
[INFO] Clock generator device set to I2C0/0x68, SpreadSpectrum=1.
[INFO] VRM device set to I2C0/0x40.
[INFO] EEPROM device set to I2C0/0x54, 64kB, Page size 128 bytes.
[INFO] Flash device set to SPI0, 16MB, Page size 256 bytes.
[INFO] MAC address 0 added: AA:BB:CC:DD:EE:01
[INFO] MAC address 1 added: AA:BB:CC:DD:EE:02
[INFO] RTC device set to I2C0/0x6f, format 1.
[WARN] Unrecognized table entry type: 0, sub type: 0.
[INFO] Area table: Platform management signature.
[INFO] Area table: UEFI/SCP shared area signature.
[INFO] Area table: End of area tables signature.
[INFO] Management device unconfigured, set to listen on I2C2/0x70.
[INFO] SOC executed from cold reset.
[INFO] ISCP Task started
[INFO] Default value of ACLOCK is 1700MHz
</code></pre>
<p>I cannot tell if this is good or bad, but it&rsquo;s the first sign of life I have seen on this board. Ard informs me that now all I need to do is press the &ldquo;power button&rdquo;. Which makes me say &ldquo;wait, what?&rdquo;</p>
<p>Turns out, these two buttons on the board are &ldquo;reset&rdquo; and (for lack of a better label) &ldquo;boot&rdquo;:</p>
<p><img src="https://edolnx-public.objects-us-east-1.dream.io/cello/IMG_20180207_125608_r.jpg" alt="Power button is closest to the USB connector, Reset closer towards the power connectors" /></p>
<p>The reset button will cold boot the board and cause LED7 to blink. The other button seemed to do nothing, but that starts the boot sequence. The SCP serial port starts showing me lots of secrets, and informs me that I cannot count:</p>
<pre><code>Memory Training Firmware
Version: 1.0.0.2
Memory Training for DDR3, one DIMM/channel
Memory Training ... start
[INFO] UEFI memory Setup fail safe counter = 10
********** Load default memory setup settings **********
SPD Channel 0 Dimm 0 SmbusAddr A0, DIMM absent
SPD Channel 1 Dimm 0 SmbusAddr A4, 18KSF1G72HZ-1G4D1 UDIMM ECC 2Rx8
92 11 0B 08 04 21 02 09 0B 11 01 08 0C 00 7E 00
69 78 69 30 69 11 20 89 20 08 3C 3C 00 F0 83 05
80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 0F 11 03 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 80 2C 01 12 26 B1 F7 CD B3 B1 93
31 38 4B 53 46 31 47 37 32 48 5A 2D 31 47 34 44
31 20 44 31 80 2C 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
********** Both channels must have the same memory size **********
******************************************
* *
* Memory Training Failed *
* *
******************************************
</code></pre>
<p>It turns out that you need 2 identical SO-DIMM modules installed on the board, of the same size, to boot. Not the end of the world, as I had two but it means I will need to order two more for the other board. So, I drop in the other module, re-apply power, press the boot button and wait&hellip;</p>
<p>You see, it takes about 3min for these boards to boot. The whole time it&rsquo;s spewing lots of interesting data over the SCP serial port, but it&rsquo;s obvious that microcontroller is running debug code and taking it&rsquo;s time. Eventually, I get a UEFI boot prompt on the console&rsquo;s USB serial port at 115200bps, and can start the GRUB application from the USB CDROM:</p>
<pre><code>UEFI Interactive Shell v2.1
EDK II
UEFI v2.60 (AMD Seattle, 0x00010000)
Mapping table
FS0: Alias(s):CD0d0a:;BLK1:
PcieRoot(0x0)/Pci(0x2,0x2)/Pci(0x0,0x0)/USB(0x3,0x0)/CDROM(0x0)
BLK2: Alias(s):
VenHw(0D51905B-B77E-452A-A2C0-ECA0CC8D514A,000030E00000000000)/Sata(0x
1,0xFFFF,0x0)
BLK0: Alias(s):
PcieRoot(0x0)/Pci(0x2,0x2)/Pci(0x0,0x0)/USB(0x3,0x0)
Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell&gt; FS0:
FS0:\&gt; cd efi
FS0:\efi\&gt; cd boot
FS0:\efi\boot\&gt; dir
Directory of: FS0:\efi\boot\
12/09/2017 13:56 &lt;DIR&gt; 2,048 .
12/09/2017 13:56 &lt;DIR&gt; 2,048 ..
12/09/2017 13:56 401,408 bootaa64.efi
1 File(s) 401,408 bytes
2 Dir(s)
FS0:\efi\boot\&gt; bootaa64.efi
</code></pre>
<h1 id="installing-an-os">Installing an OS</h1>
<p>After I stop jumping for joy, I realize I need an installer. I have this lovely Zalman ZM-VE200 (which is similar to <a href="http://zalman.com/contents/products/view.html?no=20">this device</a>), which lets me load an ISO on an SSD/HDD and present it as an USB CDROM. It&rsquo;s a lifesaver. So, I drop the stock <a href="https://cdimage.debian.org/debian-cd/current/arm64/iso-dvd/">Debian Stretch ARM64 DVD</a> on and plug it in. Then I tell the UEFI shell to reboot and&hellip; it works. The Debian installer comes up no problem. So, I power off the board, attach Ethernet and a brand new SATA SSD, and boot again.</p>
<p>This time, things are not as smooth. I use <code>picocom</code> as my default serial console of choice, and UEFI doesn&rsquo;t seem to like it. Ironically, <code>minicom</code> lets you get through the UEFI menus and prompts without issue, but then the Debian installer is terrible. A little more testing, and I figure out that the UEFI really wants an ANSI serial terminal, and GRUB and Debian prefers a linux/xterm/vt102 terminal. This is a simple fix for picocom where I can start with <code>TERM=ANSI picocom -b 115200 /dev/ttyUSB0</code> and later restart without the <code>TERM=ANSI</code> and all is well. I had very strange issues with <code>minicom</code>, which should come as a surprise to no one.</p>
<p>The Debian installer goes along without issue, finds the NIC, but can&rsquo;t DHCP. My instinct to grab the DVD instead of a netboot ISO pays off! I tell it to ignore the ethernet and figure I&rsquo;ll look at it later. SATA is detected without issue, install completes without issue. Rebooting however&hellip;.</p>
<h1 id="lack-of-console">Lack of console</h1>
<p>The Debian installer and/or GRUB2 figure out that the console is a serial port, and you can see the GRUB2 menu without issue. However, the Linux kernel seems to ignore the serial console. Ard tells me that the console should be <code>ttyAMA0</code>, so I edit the kernel command line in GRUB to remove the stupid <code>quiet</code> flag and add <code>console=ttyAMA0,115200n8</code>. This is when I learned that GRUB, too, wants an ANSI terminal. Now I can see the kernel messages, and then Debian starts up and gives me a login prompt! Holy cow! I have a fully installed and working Debian environment on the Cello. To make those changes permanent, you will need to modify the <code>/etc/defaults/grub</code> file and edit the line that starts with <code>GRUB_CMDLINE_LINUX_DEFAULT</code></p>
<h1 id="lack-of-ethernet">Lack of ethernet</h1>
<p>On to that pesky ethernet. The ethernet device is detected as enp2s0, but a keen reader may discover an issue here:</p>
<pre><code>cello2:~$ ip link list
1: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp2s0: &lt;BROADCAST,MULTICAST&gt; mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
</code></pre>
<p>Turns out the Ethernet MAC is missing it&rsquo;s address. This explains why the installer couldn&rsquo;t use DHCP. No the end of the world, an address can be specified in the <code>/etc/network/interfaces</code> file like so:</p>
<pre><code>auto enp2s0
iface enp2s0 inet dhcp
hwaddress ether 00:00:1A:00:1A:01
iface enp2s0 inet6 auto
</code></pre>
<p>Note that if you have multiple boards, those MAC addresses must be unique on each system. I used the AMD vendor code, and then made up the rest. You can use whatever you like.</p>
<p>I reached out to ask Ard about this, and he informed me that the tool Realtek (who makes the RTL8169 PHY device used on this board) only provides an x86 UEFI application to set these addresses. At the time LeMaker had no way of doing this, but now there is apparently an emulation layer that can be loaded into UEFI which allows running x86 PE/COFF code for tools like this. This isn&rsquo;t a big deal once the OS is installed, but can be a problem if you want to netboot or do a netinstall. In those cases, you would need a workaround:</p>
<pre><code>So we don't have the means to set the MAC address persistently,
unfortunately. What we do have is a firmware based driver that sets it
from the firmware at boot.
Please try the attached driver. You need to drop it into the ESP (the
FAT partition that has /efi/debian/grubaa64.efi in it), and enter the
following commands from the shell:
setvar -guid 8d97e056-777c-4850-ab61-8166b1777f2d MacOverride -nv -bs
=112233445566
where 112233445566 is the MAC address you want to assign (note that
copy paste may not work very well in the UEFI shell so if it doesn't
seem to work, please double check whether you got all the GUID digits
right). Then you can load the driver
load fs0:Realtek8169MacOverride.efi
and check whether it prints something like 'using MAC override value
11:22:33:44:55:66'
If that works, you can register the driver to be loaded automatically at boot
bcfg driver add 0 fs0:Realtek8169MacOverride.efi &quot;R8169MacOverride&quot;
and it should set the MAC address in the PCI config space at each boot.
</code></pre>
<p>Totally not ideal, but here is <a href="https://edolnx-public.objects-us-east-1.dream.io/cello/Realtek8169MacOverride.efi">the tool if anyone is interested</a>.</p>
<h1 id="next-steps">Next Steps</h1>
<p>So, at this point, everything is working. I had planned to deploy these to a data center to be database servers, but it&rsquo;s obvious that would be a terrible idea. These are very much development boards, and when something goes wrong will need button presses to fix. So, they are going to stay at my home lab instead. I&rsquo;m working on building an Open Source ARM CI pipeline where community members can get various software built on various armel, armhf, and arm64 platforms and distributions. Since these boards have Cortex-A57 cores, they can run ARMv6, ARMv7, and ARMv8 code. So my goal is to make them compile boxes, and then I have a suite of other machines to test on. But, before any of this some changes are going to be needed.</p>
<p>First, I need a case. While there are many 96boards CE cases around, I have yet to find an EE one. So, I&rsquo;m probably going to design a one that borrows heavily from <a href="http://wiki.t-firefly.com/index.php/Firefly-RK3399/Peripherals_%E5%A4%96%E5%A3%B3/en">The Firefly-RK3399 case</a> which is just some acrylic and stand-offs. It gets the job done. I&rsquo;m probably going to make one edge open so that if you have something like the <a href="https://www.96boards.org/product/poplar/">Poplar</a> board, you can use the PCI Express slot. Not so important for this board since the PCI Express slot is non-functional.</p>
<p>The second, the CPU fan on this board is small and very loud. My goal is with this case, I&rsquo;m going to remove that small fan and instead place a nice big 120mm standard PC case fan on top and have it cool the CPU and other components. The CPU cooler uses a standard PC Case 3pin fan connector, so this is a simple change once there is a place to mount it on the case.</p>
<p>Once I have the case design done, I will post here for others. It will be CC0 licensed.</p>
<h1 id="summary">Summary</h1>
<p>If you skipped all the text above, here is what you need to know:</p>
<ul>
<li>You will need an IDENTICAL PAIR of ECCx72 SO-DIMM modules, I&rsquo;m using <a href="http://www.memory4less.com/micron-8gb-sodimm-pc10600-mt18ksf1g72hz-1g4d1ze">8GB modules from memory4less.com</a></li>
<li>You will need a USB-TTL serial adapter to connect to the SCP FTDI header next to the PCI Express slot in order to see if the SCP manages to configure the memory and other hardware correctly. That serial port operates at 38400bps.</li>
<li>If you don&rsquo;t get output from the SCP, you may need to apply firmware to the board. See above, good luck, and let me know if you succeed.</li>
<li>If you do get output, then press the &ldquo;Boot&rdquo; button. You should get lots more output on the SCP and if memory configuration succeeds, it will boot and in several minutes you can interact with the UEFI console on the integrated USB serial port at 115200bps.</li>
</ul>
<p>Hope that helps others, feel free to leave comments below if you have questions or concerns. Thanks a bunch to Ard, Robert, and Ricardo from 96boards for their help getting my Cello boards up and running!</p>Quick update: Sub $200 Stratum 1 NTP Server with ODROID and Adafruit partshttps://www.gigofham.com/archives/73
Sat, 11 Jun 2016 22:19:34 +0000https://www.gigofham.com/archives/73<p>I was building another one of these, and found some bugs in my previous post. I&rsquo;ve corrected the post, the biggest being that the kernels in Armbian 5.10 and later include a 1-Wire interface in the DTB which conflicts with the 1PPS pin used by the GPS module. The solution is to either comment out the <code>w1</code> stanza in the device tree, or move it to another pin. I moved it to <code>GPIOY_7</code> which is two pins down on the same row of the connector. The DTS and DTB links in the previous post have been corrected, along with the new SHA256 checksum of the DTB itself. I also tried to build on an ODROID-C2 platform, but it&rsquo;s missing the GPIO 1PPS client in the kernel, so I&rsquo;ve gone back to just using ODROID-C1+ for now. I also found that if you do not have the I2C display on top of the GPS module, but the module is near a window and pointing up the GPS will lock. Pretty cool. Onward to deployment!</p>
Sub $200 Stratum 1 NTP Server with ODROID and Adafruit partshttps://www.gigofham.com/archives/70
Thu, 02 Jun 2016 01:23:20 +0000https://www.gigofham.com/archives/70
<p>Time is important. In modern computing, I would say time is second in importance only to good entropy. Unfortunately, getting accurate, reliable time is getting harder. The global NTP pool is under attack, and worse it&rsquo;s also being used as an amplification vector for DDOS attacks. Primarily because of those two reasons, I&rsquo;ve decided to block NTP traffic at the border of all Data Center. But this leave a problem: how to get time. The easy answer is an off-the-shelf NTP server that uses CDMA or GPS. The easiest is CDMA because it requires no external antenna. The problem is the major CDMA carriers have committed to turning off the CDMA networks in the next few years. So that leaves GPS. No big deal, until you try to get pricing information for a standalone GPS backed NTP server. Cheapest I could find was only rated at about 10 clients and costs well over three thousand dollars. So I kept digging.</p>
<p>The commercial products have some fancy features: Oven controlled oscillators, 10Mpps outputs, sometimes even rubidium oscillators. I don&rsquo;t really need any of that, I just need a descent NTP source that supports the 1PPS signal for synchronization. The 1PPS signal is important because you get messages from the GPS receiver once every second, including time. That message is long, and it&rsquo;s over a 9600bps serial link. That means it can have a variance well in the hundreds of miliseconds. That&rsquo;s too much. The 1PPS system in the Linux kernel, tied to the 1PPS signal from a GPS receiver gets you down into the 5 microsecond range for variance. Much better.</p>
<p>It turns out, gpsd does almost everything you need to be a stratum 1 NTP server. Pair that with NTPd or chrony and you&rsquo;re in great shape. Several people have done this with Raspberry Pi systems, but networking on the Pi is miserable since it&rsquo;s over a USB interface and now most of that work you did to eliminate variance is gone. Some people have also tried to do this with a BeagleBone based system, which has worked but is a bit hacky because the BeagleBone was never really as popular as the RPi. But, I was going in the right direction. Cheap board, simple GPS module, done and done.</p>
<p>Adafruit stocks and ships a number of lovely modules with a 1PPS out, and I started with their <a href="https://www.adafruit.com/products/746">Ultimate GPS breakout board</a> because it&rsquo;s the cheapest and most flexible. I then tried to tie that to various systems. First I got the module working and talking to gpsd over a USB Serial interface. Worked great. The serial interface devices I have lying around are not spectacular so I had no way to test the 1PPS. No big deal. I did have several <a href="http://www.linksprite.com/linksprite-pcduino3-nano/">pcDuino 3 Nano</a> boards, and Adafruit also ships a <a href="https://www.adafruit.com/products/1272">GPS module for the arduino family</a>. Cool. The levels are wrong, because an Arduino is 5V logic, and the pcDuino is 3.3 but you can <a href="http://store.linksprite.com/t-board-to-bridge-arduino-shield-to-pcduino-with-level-shifter/">get adapter boards</a>. I also wanted a <a href="https://www.adafruit.com/products/772">simple display</a> to show status so it could run headless (and also because the serial port would be consumed by the GPS module). So get all those parts in, assemble, and&hellip;nothing. Turns out the serial port on the pcDuino 3 Nano is a non-starter. Not wired correctly or something. OK, back to the drawing board.</p>
<p>While the RPi is out, there are a lot of boards that are &ldquo;RPi compatible&rdquo; in terms of form factor and the expansion ports. One such is the ODROID series from <a href="http://www.hardkernel.com">HardKernel</a>. I had a couple <a href="http://odroid.com/dokuwiki/doku.php?id=en:odroid-c2">ODROID-C2</a> boards, so I grabbed the Adafruit <a href="https://www.adafruit.com/products/2324">Ultimate GPS Hat</a> and a <a href="https://www.adafruit.com/products/1115">simple display hat</a> and tied them to the C2. Initially, there was much better success than with the pcDuino 3 nano. The C2 also has two serial ports, one for a consle and a second on the RPi connector. It could talk to the GPS reciever no problem, and gpsd got a lock and time. Next was 1PPS, and that&rsquo;s where I hit a brick wall. At the time, the C2 kernel did not support an IRQ for the GPIO pins (I&rsquo;m told it does now) so there was no way to use the pps-gpio driver in the kernel. Drat. However, the <a href="http://odroid.com/dokuwiki/doku.php?id=en:odroid-c1">ODROID-C1+</a> didn&rsquo;t have that problem so I ordered a pair of those and waited.</p>
<p>Finally got the boards in, tied everything together and had&hellip;minimal success. It took weeks to get the boards, so this project started as a October timeframe &ldquo;Hey! I have an idea!&rdquo; and I just got it working last week the way I want. The first issue was the image I was using, odrobian, didn&rsquo;t have the gpio-pps module and had problems booting on every MicroSD card I tried. Turns out, this is mostly a kernel and u-boot issue on this board. Switching to <a href="http://www.armbian.com">armbian</a> fixed all those (and it ships with the needed module).</p>
<p>The serial port on the C1+ has an interesting bug, in that it won&rsquo;t work unless you run a specific command:</p>
<p><code>stty -F /dev/ttyS2 9600</code></p>
<p>This forces the port to 9600bps and then everything is fine. Other tools can later change the baud rate, but for whatever reason gpsd cannot. Adding that line to /etc/rc.local solved the problem permanently.</p>
<p>The next issue is the 1PPS signal. The pps-gpio module uses the device tree to find which port the 1PPS signal should monitor. After a lot of research I found that the correct port for the Adafruit hat is GPIOY_3. This means you need to add the following stanza to the dts:</p>
<pre style="padding-left: 30px;">pps {
compatible = "pps-gpio";
gpios = "GPIOY_3";
};</pre>
<p>This conflicts with the 1-Wire interface defined earlier in the file (search for <code>w1</code> and comment out the whole stanza, or change the pin used to a different value like <code>GPIOY_8</code>). Or.. you can just download my compiled <a href="http://edolnx-public.objects-us-east-1.dream.io/ODROID-C1/3.10.101-odroidc1-meson8b_odroidc.dtb">dtb</a> that&rsquo;s a drop in replacement. (SHA256 sum: 21a16397fcd1248f2fa79ce94f567cd1b230bae4926c1b83bdaaad12bde82b6c ) I also have the source <a href="http://edolnx-public.objects-us-east-1.dream.io/ODROID-C1/3.10.101-odroidc1.dts">dts</a> available if you want to compare.</p>
<p>A few software tweaks, and it all works. I haven&rsquo;t finished the software for the display yet, but you can at least get time. Here&rsquo;s the HOWTO:</p>
<h1 id="building-a-odroid-c1-based-ntp-server-with-adafruit-ultimate-gps-hat-v3">Building a ODROID-C1+ based NTP server with Adafruit Ultimate GPS Hat v3</h1>
<h2 id="parts-list">Parts List:</h2>
<ul>
<li><a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G143703355573">ODROID-C1+</a> (an ODROID-C2 may work now, I haven&rsquo;t tested)</li>
<li>MicroSD card (I&rsquo;m using a Samsung EVO PRO 16GB, you don&rsquo;t need a lot of space) and USB adapter for it</li>
<li><a href="https://www.adafruit.com/products/2324">Adafruit Ultimate GPS Hat</a></li>
<li>CR1220 Battery for GPS Hat (allows GPS to retain state data for faster startup, buy locally)</li>
<li><a href="http://www.adafruit.com/products/851">Adafruit uFL GPS cable</a> (optional for external antenna)</li>
<li><a href="http://www.adafruit.com/products/960">Adafruit external GPS antenna</a> (optional, but highly recommended)</li>
<li><a href="https://www.adafruit.com/products/1115">Adafruit Blue&amp;White 16x2 LCD+Keypad Kit for Raspberry Pi</a> (optional)</li>
<li><a href="http://www.adafruit.com/products/2336">Adafruit brass standoffs for the hats</a> (optional, <a href="https://www.adafruit.com/products/2337">16mm variants</a> are great for GPS hat)</li>
<li><a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G134111972476">ODROID Power Adapter/Cable</a> (min 2A, don&rsquo;t use MicroUSB for power)</li>
<li><a href="https://www.adafruit.com/products/1979">Adafruit Raspberry Pi Extra Tall Stacking Header</a></li>
<li>Case (You can use the <a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G143730080763">ODROID one</a>, or modify <a href="https://www.adafruit.com/products/2256">this one from Adafruit</a> slightly for the power connector to work, or 3D print your own)</li>
<li><a href="http://armbian.com/">Armbian</a> Image</li>
<li><a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G134111883934">ODROID Serial Header and USB module</a> (you can&rsquo;t debug boot error messages on anything but the serial console)</li>
</ul>
<h2 id="software-used">Software used:</h2>
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/ODROID-C1/Armbian_5.10_Odroidc1_Debian_jessie_3.10.101.7z">Armbian 5.10</a> (beware, 5.11 has a busted kernel, but 5.12 seems to be fine)</li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/ODROID-C1/3.10.101-odroidc1-meson8b_odroidc.dtb">dtb</a> with the pps stanza for the 1PPS line from the GPS module</li>
<li>chrony (it&rsquo;s easier to integrate with gpsd than ntpd, more secure, and will keep things moving along during bad weather when you may not have GPS lock. It&rsquo;s also compatible with regular ntp clients.)</li>
<li>gpsd from Debian Testing (the version in Jessie has broken 1PPS support)</li>
</ul>
<h2 id="hardware-notes">Hardware notes:</h2>
<ul>
<li>The GPS antenna needs a good view of the sky. I&rsquo;ve had good luck just placing the external antenna on a window sill.</li>
<li>If you are going to add the I2C Display module, the external GPS antenna is a must because the onboard antenna will be obstructed</li>
<li>You may not want to put the whole device on a window, it could overheat with the sun shining on it directly. Use the external antenna instead and keep the rest of it cool and safe.</li>
<li>If your antenna is going to be outdoors, don&rsquo;t forget GPS compatible lightning arrestors!</li>
<li>The ODROID boards have a lovely heatsink on them, so the header bundled with the GPS hat will not work. Use the extra tall or stacking header and make sure there is clearance between the board and the heatsink. I placed the stacking header on the ODROID, and some business cards between the GPS Hat and the heatsink, then tacked both sides of the header with my soldering iron to ensure a gap. YMMV.</li>
</ul>
<h2 id="configuration">Configuration:</h2>
<ul>
<li>Flash the Armbian image onto the MicroSD card from a Linux box using a USB adapter (using dd)</li>
<li>Boot the device. Note the messages about a reboot, wait 5min for the reboot to occur.
<ul>
<li>If the reboot does not occur:
<ul>
<li>Login with the default root password of &ldquo;1234&rdquo; and follow the prompts to change the password and add a user account. Then, shutdown the system.</li>
<li>Place the MicroSD card some other Linux box over a USB device and run e2fsck and then resize2fs on the 2nd partition (ex: <code>sudo e2fsck -f /dev/sdb2 &amp;&amp; sudo resize2fs /dev/sdb2</code> )</li>
<li>Remove the card from the USB adapter, and place back into the ODROID and power up.</li>
<li>The system should boot and the root filesystem should be almost the same size of the MicroSD card used.</li>
</ul></li>
</ul></li>
<li>Login with the default root password of &ldquo;1234&rdquo; and follow the prompts to change the password and add a user account (if not already done)</li>
<li>Backup the existing dtb file: <code>sudo cp /boot/dtb/meson8b_odroidc.dtb /boot/dtb/meson8b_odroidc.dtb.orig</code></li>
<li>Download the custom dtb file above, and then place on the system as <code>/boot/dtb/meson8b_odroidc.dtb</code></li>
<li>Edit the file <code>/etc/rc.local</code> and add the line <code>stty -F /dev/ttyS2 9600</code> before the line <code>exit 0</code></li>
<li>Reboot</li>
<li>Login with the new user account from now on (it&rsquo;s good practice). You can also login over SSH for easier cut and paste.</li>
<li>Create a new file <code>/etc/apt/preferences.d/testing_unstable</code> with the following contents:</li>
</ul>
<pre style="padding-left: 30px;">Package: *
Pin: release a=stable
Pin-Priority: 700
Package: *
Pin: release a=testing
Pin-Priority: 650
Package: *
Pin: release a=unstable
Pin-Priority: 600</pre>
<ul>
<li>Create a new file <code>/etc/apt/sources.list.d/testing_unstable.list</code> with the following contents:</li>
</ul>
<pre style="padding-left: 30px;">deb http://httpredir.debian.org/debian testing main contrib non-free
deb-src http://httpredir.debian.org/debian testing main contrib non-free
deb http://httpredir.debian.org/debian/ unstable main contrib non-free
deb-src http://httpredir.debian.org/debian/ unstable main contrib non-free</pre>
<ul>
<li>Pin the current kernel image (it works, and the one currently available from armbian did not for me at all, causing a kernel panic at boot. You can try now, if it fails just repeat up to this step) with the following command: <code>sudo apt-mark hold linux-image-odroidc1</code></li>
<li>Get the new package list and perform a security update: `sudo apt-get update &amp;&amp; sudo apt-get dist-upgrade -y`</li>
<li>Install chrony and the prerequisites for gpsd (we have to do it this way to not confuse the package manager when we pull the newer gpsd): <code>sudo apt-get install chrony python-gtk2 python-cairo pps-tools</code></li>
<li>Install gpsd from Debian Testing: <code>apt-get install gpsd gpsd-clients -t testing</code></li>
<li>Stop the gpsd service: <code>sudo service gpsd stop</code></li>
<li><p>Edit the <code>/etc/default/gpsd</code> file changing the following values:</p>
<ul>
<li><pre>USBAUTO="false"</pre></li>
<li><pre>DEVICES="/dev/ttyS2 /dev/pps0"</pre></li>
<li><pre>GPSD_OPTIONS="-n"</pre></li>
</ul></li>
<li><p>Test the 1PPS link from the gps reciever using the command <code>sudo ppstest /dev/pps0</code></p></li>
<li><p>Modify the <code>/etc/chrony/chrony.conf</code> file to contain the following:</p></li>
</ul>
<pre style="padding-left: 30px;">refclock SHM 0 refid GPS precision 1e-1 offset 0.280 delay 0.2
refclock SOCK /var/run/chrony.pps0.sock refid PPS
keyfile /etc/chrony/chrony.keys
commandkey 1
driftfile /var/lib/chrony/chrony.drift
log tracking measurements statistics
logdir /var/log/chrony
maxupdateskew 100.0
dumponexit
dumpdir /var/lib/chrony
local stratum 10
allow 10/8
allow 192.168/16
allow 172.16/12
logchange 0.5
rtconutc</pre>
<ul>
<li>Restart chrony: <code>sudo service chrony restart</code></li>
<li>Next, we are going to pre-load the latest almanac into the GPS module so it will lock on MUCH quicker (for details, see the <a href="https://github.com/f5eng/mt3339-utils">github repo for the tools</a>). Run the following commands:</li>
</ul>
<pre style="padding-left: 30px;">wget https://github.com/f5eng/mt3339-utils/archive/gps.zip
unzip gps.zip
cd mt3339-utils-gps
eporetrieve MTK14.EPO /tmp/MTK14.EPO
epoinfo /tmp/MTK14.EPO
(You should get some formatted output and not an error)
epoloader @epoloader.conf /tmp/MTK14.EPO /dev/ttyS2</pre>
<ul>
<li>Edit the gpsd init script located at <code>/etc/init.d/gpsd</code>, and add the word &#8216;chrony&rsquo; to the Should-Start line.</li>
<li>Start the gpsd service</li>
<li>Check the gps status using gpsmon, it should lock within a few moments and show a time and the PPS block should have numbers. Type <code>quit</code> to exit.</li>
<li>Now you can check the chrony service to see if it&rsquo;s getting GPS information using <code>chronyc sources</code></li>
<li>That&rsquo;s it! You can adjust the allow lines in chrony.conf and deploy. Don&rsquo;t forget to give the device a static IP as well.</li>
</ul>
My new ARM goal: hostinghttps://www.gigofham.com/archives/64
Thu, 24 Mar 2016 22:19:48 +0000https://www.gigofham.com/archives/64
<p>I&rsquo;ve been doing a lot of work on various ARM things. A lot of it has been in support of making community builds of Chef for other folks, but I have a new goal: providing ARM nodes to the Internet at large as a hosting option.</p>
<p>I&rsquo;m not going to go into a lot of details here yet, as I&rsquo;m saving a lot of that content for my hosting company&rsquo;s blog. I&rsquo;ll announce it here when it goes live. There are a few things in the way, but A lot of the deep dive technical stuff will be posted here. So, now that I&rsquo;ve gotten that out of the way let&rsquo;s get into the usual ARM update:</p>
<h2 id="rk3368-8211-it-s-been-fun">RK3368 &#8211; It&rsquo;s been fun</h2>
<p>First the bad news: I&rsquo;m giving up on the RK3368 as an ARMv8 platform for now. Part of this is the kernel mess (but things are looking better for 4.6), but most of it is availability of hardware (low), the (relative) high cost of the hardware, the focus of Android on this SOC, and the common implementation of the serial port connected to the MMC/SD/TF slot. Finding RK3368 hardware is somewhat difficult, and when I do it&rsquo;s always an Android Media Player/STB. Getting these to run anything else is..really hard. Rockchip doesn&rsquo;t support upstream das u-boot, and while the #linux-rockchip community on freenode is making strides, they are focused on the much more available ARMv7 variants instead of the ARMv8 RK3368. It&rsquo;s understandable, and I&rsquo;m not at all upset. The reality of the situation is that the RK3368 based systems I can find are much more expensive than other ARMv8 solutions. That then just leaves the Serial Port on MMC/SD/TF slot issue, which is a real deal breaker for me. In a 100% remote environment, OOB console is required and for most things that means IPMI or Serial. IPMI is out on most ARM devices so that means serial. USB isn&rsquo;t great for primary storage (for a lot of reasons), and having the storage card share pins with the serial console just means it&rsquo;s a disaster waiting to happen. I had high hopes for the <a href="http://www.geekbox.tv/">Geekbox</a>, but even that turned out to be a real disappointment. The focus for this SOC is obviously Android, and that&rsquo;s fine, but it doesn&rsquo;t suit my needs at all.</p>
<h3 id="on-the-geekbox-http-www-geekbox-tv">On the <a href="http://www.geekbox.tv/">GeekBox</a></h3>
<p>I don&rsquo;t want to rag on them too hard, it&rsquo;s a lovely piece of kit. But it suffers from some shortcomings. Stock and availability of components being one of them. I&rsquo;ve placed several orders for them, and they never ship on the advertised time frame. The folks at Geek Buying are very nice, and very transparent about supply issues, but it&rsquo;s still been rather frustrating. Then there is the odd selection of accessories. The docking station is mostly a USB hub and easier exposure of the edge connector functions, but there is no case for it. The CPU fan accessory does nothing and doesn&rsquo;t have a reasonable mount even if you want to use it (you do, I&rsquo;ve melted the case by accident on one of mine doing compiles). The lack of case when the board is in the docking station caused me to loose another board due to static discharge. Then there is the software: android runs fine. No major issues there. The Linux side however, is a bit of a mess. It&rsquo;s a customized version of Ubuntu 14.04&rsquo;s unsupported arm64 port. Most vendors seem to be taking this route, and I don&rsquo;t understand why. The real problem: the video is not stable. The signal is solid (it&rsquo;s digital after all), but the picture has waves going up and down the screen on every monitor I&rsquo;ve tried. Android is fine, it has something to do with the Linux kernel they provide. Makes it very hard to even get the DHCP IP address and then drop out to a console. In fact, a FB console is not even supported. X or nothing. I chose nothing.</p>
<p>Again, this isn&rsquo;t a bad device, just doesn&rsquo;t suit my needs.</p>
<h2 id="the-odroid-c2-http-www-hardkernel-com-main-products-prdt-info-php-g-code-g145457216438">The <a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G145457216438">ODROID-C2</a></h2>
<p>This device has been a breath of fresh air. Also ARMv8 (based on an amlogic 905S SoC) but only 4 cores (compared to the RK3368&rsquo;s 8 cores). No onboard storage, you have to add a MicroSD/TF card. The best part: broad community support. This SoC just landed in 4.6 and the makers of the board (<a href="http://www.hardkernel.com/main/main.php">HardKernel</a> out of Republic of Korea) are working with Amlogic to get full support upstream in mainline. Right now, mine are still running their heavily patched 3.14 kernel &#8211; but the source and configs are readily available and adding support for things like LXC was very simple. I&rsquo;m running an &ldquo;odrobian&rdquo; install, which is a minimal Debain Jessie root on an ext4 filesystem and some kernel config tweaks for Docker support. I&rsquo;ve rolled my own kernel to add the rest of the configuration options needed for LXC and I&rsquo;m running that for my build and test nodes.</p>
<p>I also just deployed four of these into production: two for recursive name servers, and two for authoritative name servers. They are cheap ($40 from <a href="http://www.hardkernel.com/main/main.php">HardKernel</a>, and there are worldwide resellers) and they are case compatible with RPi B rev 2. They also appear to sip power. I picked up a <a href="https://www.adafruit.com/products/2690">USB Meter from Adafruit</a>, and I can&rsquo;t get them above 0.3A at 5V for most of my testing. Now, I don&rsquo;t entirely believe those numbers (this device has a very low sample rate), so I&rsquo;m going to do some more testing with a NI VirtualBench at TechShop this weekend to see if I can learn more. I&rsquo;ll post more details on that once I have them. For now, I&rsquo;m being ultra conservative and deploying with a <a href="http://smile.amazon.com/dp/B011LSGMT2">PSU that can support six devices up to 2A@5v each</a>.</p>
<p>So far, these boards are great. Biggest thing to be aware of: when you buy one, get their power cord. It&rsquo;s a 2.5mm barrel connector (HardKernel makes and sells <a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G138960965859">pigtails</a> and <a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G141637559827">USB adapters</a>). Those connectors are apparently common everywhere but here.</p>
<h3 id="on-power">On Power</h3>
<p>The crappy cell phone chargers we use for MicroUSB power everywhere are usually not enough for most of these devices (I&rsquo;ve killed more than a couple RPi boards this way), so get a good solid power supply for more than $3. The ones sold by <a href="https://www.sparkfun.com/categories/28">SparkFun</a> and <a href="https://www.adafruit.com/category/44">Adafruit</a> are well tested and good if you need basic MicroUSB power, they also offer good all around 5V supplies for general use. If you have several devices, <a href="http://www.ikea.com/us/en/catalog/products/10291881/">IKEA makes an inexpensive 3 port USB charge</a>r that I&rsquo;ve been using without issue, or the <a href="http://smile.amazon.com/dp/B011LSGMT2">Anear power adapter</a> I&rsquo;m using in a few places with up to six devices pulling 2.4A@5v each seems to be working very well also.</p>
<h2 id="pcdunio3-nano-http-www-linksprite-com-linksprite-pcduino3-nano"><a href="http://www.linksprite.com/linksprite-pcduino3-nano/">PcDunio3 Nano</a></h2>
<p>I&rsquo;ve recently pulled mine out of mothballs for a few reasons:</p>
<ol>
<li>They run a mainline kernel now, so they are great for a serial console server which cheap USB-Serial adapters</li>
<li>I need an inexpensive but reasonably accurate NTP server fed from GPS</li>
</ol>
<p>These are descent little boards, with a Dual Core ARMv7 Allwinner A20 proc and 1GB of RAM. They also have onboard eMMC, but the TF/MicroSD is better supported. They also have native 1Gb Ethernet (instead of a USB interface like the RPi products).</p>
<p><a href="http://armbian.com/">The Armbian project</a> supports these (along with most commonly available Allwinner based boards), with a descent Jessie rootfs and either a recompiled vendor kernel or a vanilla upstream kernel. I&rsquo;m using the vanilla for my console server, and the vendor kernel for the GPS stuff for now. I want to move to the vanilla kernel for that as well, but there are Device Tree issues to work out.</p>
<p>These are also neat because they have Arduino compatible headers, but at a different voltage (3.3v vs the normal Ardunio of 5v). The good news is that there is a <a href="http://store.cutedigi.com/t-board-to-bridge-arduino-shield-to-pcduino-v1-with-level-shifter/">level conversion shield</a>, the bad news is that it doesn&rsquo;t quite fit on the nano (the Ethernet jack is in the way), so I&rsquo;ve needed to get some header extensions to plug it in completely. Once I get the headers extensions, I plan on stacking an <a href="http://adafru.it/1272">Adafruit GPS Logger Shield</a> w/ Module and then a <a href="http://adafru.it/715">Adafruit I2C Controlled Keypad and display</a> shield on top. I&rsquo;ve already tested that the GPS module is compatible with gpsd, and with a vanilla kernel I should be able to get the 1PPS off the shield and into gpsd and ntpd for an inexpensive Stratum 1 time source. I&rsquo;ll update here will the full build out and configuration once I&rsquo;ve got it all/mostly working.</p>
<h2 id="pine64">Pine64</h2>
<p>I don&rsquo;t have any yet, but I&rsquo;m excited. Quad Core ARMv8, 2GB RAM, under $30. I ordered 22 of these through their Kickstarter. I hope to have them by the end of the month, and they will be the basis for Gen 1 of my hosting product. I&rsquo;ll be talking a lot more about these once they arrive and I start testing and design for support hardware.</p>
<h2 id="huskyboard-lemaker-cello">Huskyboard / LeMaker Cello</h2>
<p>I&rsquo;m really hoping that 2016 is the year of AMD&rsquo;s comeback for many reasons. Either way, the A series Opteron has always caught my eye. The <a href="http://www.96boards.org/products/ee/huskyboard/">Huskyboard</a> is still &ldquo;Soon&rdquo; but the <a href="http://www.lenovator.com/product/103.html">LeMaker Cello</a> is shipping in the next couple months, looks to be nearly identical specs wise, and comes in at $299. I&rsquo;ve ordered two for testing and might use one to be my new mail server since I can add 2 SSDs over SATA to them. Expect more on these in the future as well. I just wish either of them shipped with headers for the second onboard gigabit NIC.</p>
<h2 id="on-sd-cards">On SD Cards</h2>
<p>Typically, I&rsquo;ve been buying the cheapest 16G SD cards I can get my hands on from Fry&rsquo;s. These are usually name brand (PNY, Toshiba, Patriot, etc) and they have been &ldquo;reasonable&rdquo;. With the goal of making a commercial hosting product with these, &ldquo;reasonable&rdquo; probably won&rsquo;t fly with paying customers. So, I did some research and everything was pointing in one direction: Samsung EVO. So, I went back to Fry&rsquo;s and run some tests with iozone3. I&rsquo;m including the results below:</p>
<ul>
<li>Toshiba 16GB on PcDuino 3 Nano Armbian 5.0
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/a20-toshiba16.xls">iozone3 xls</a> (was done over SSH, may need to redo)</li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/a20-toshiba16.log">iozone3 log</a> (was done over SSH, may need to redo)</li>
</ul></li>
<li>Samsung EVO 64GB on PcDuino 3 Nano Armbian 5.0
<ul>
<li>missing files, will re-run and update once I get another EVO 64GB</li>
</ul></li>
<li>Samsung EVO+ 32GB on PcDuino 3 Nano Armbian 5.0
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/a20-evoplus32.xls">iozone3 xls</a></li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/a20-evoplus32.log">iozone3 log</a></li>
</ul></li>
<li>Toshiba 16GB on ODROIC-C2 odrobian Jessie (kernel3.14.29-14-odrobian+)
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-toshiba16.xls">iozone3 xls</a></li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-toshiba16.log">iozone3 log</a></li>
</ul></li>
<li>Samsung EVO 64GB on ODROIC-C2 odrobian Jessie (kernel3.14.29-14-odrobian+)
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-evo64.xls">iozone3 xls</a></li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-evo64.log">iozone3 log</a></li>
</ul></li>
<li>Samsung EVO+ 32GB on ODROIC-C2 odrobian Jessie (kernel3.14.29-14-odrobian+)
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-evoplus32.xls">iozone3 xls</a></li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-evoplus32.log">iozone3 log</a></li>
</ul></li>
</ul>
<p>All of the tests were run on the device while connected over a serial console, using the default EXT4 filesystem on the SD card directly. No other activity was being run on these devices at the time, but they were connected to a local LAN with (no WiFi). Here is the exact iozone3 command used:</p>
<pre style="padding-left: 30px;">iozone -az -i 0 -i 1 -e -R -+T -b ~/output.xls | tee ~/output.log</pre>
<p>Power wise, it looks like the Samsung EVO is the big winner. It appeared to consume less power than the other cards. Again, my power measurements are inaccurate and I hope to address that this weekend. Beyond that, the Toshiba is the clear looser but there doesn&rsquo;t seem to be much difference between the EVO and EVO+ on these boards (probably due to limitations of the MMC interface of the SOC). So, I&rsquo;m sticking with the EVO cards when I care about performance. I also don&rsquo;t have a 32GB EVO card or a 32GB Toshiba to do a direct 1:1 comparison. I hope to fix that before I get the Pine64 boards. The price/performance curve definitely has the EVO in it&rsquo;s favor, where the bulk of the 64GB MicroSD cards on Amazon are (as of 24MAR2016) around $15, the <a href="http://smile.amazon.com/dp/B00IVPU7AO">EVO is $20</a> and the <a href="http://smile.amazon.com/dp/B01273JZMG">EVO+ is $25</a>. These tests show that the EVO+ doesn&rsquo;t appear to be worth the extra 20% price increase for these devices.</p>
<h2 id="phew">Phew</h2>
<p>I think that&rsquo;s a good update for now, I&rsquo;ll post more as I have it!</p>
Hacking the Uppel/Sunchip CX-R8, Rikomagic MK68, and Tegra Jetson TK1https://www.gigofham.com/archives/51
Sat, 05 Dec 2015 21:48:36 +0000https://www.gigofham.com/archives/51
<p>Long overdue follow-up from my previous post. Let me dive in by topic:</p>
<h2 id="uppel-sunchip-cx-r8-http-edolnx-public-objects-us-east-1-dream-io-cx-r8-index-html"><a href="http://edolnx-public.objects-us-east-1.dream.io/CX-R8/index.html">Uppel/Sunchip CX-R8</a></h2>
<p>After a lot of roadblocks, I&rsquo;ve given up on this particular platform. The primary reason for this was getting the MK68 boxes and being able to compare the two. The CX-R8 is pretty hacker un-friendly. The bootloader does some form of signing the boot, recovery, and kernel images. All of the existing Rockchip tools I could find to make these images say the ones from the device have bad checksums, and the images it creates the device reports as having bad checksums. I wound up putting the stock image back on and giving this device to a friend as a media player.</p>
<h2 id="rikomagic-mk68-http-edolnx-public-objects-us-east-1-dream-io-rkm-mk68-index-html"><a href="http://edolnx-public.objects-us-east-1.dream.io/RKM-MK68/index.html">Rikomagic MK68</a></h2>
<p>Out of the box, this device is much more hacker friendly. To start with SuperSU is pre-installed in the factory image, so the device is pre-rooted. Very handy for getting files on and off. On the hardware front, there is some bad news. Looking at the top of the board there is a set of holes that look like the serial port on the CX-R8. Sadly, they are not. Looking at the &ldquo;<a href="http://edolnx-public.objects-us-east-1.dream.io/RKM-MK68/DSC_9582.JPG">Top of board&rdquo; image I posted</a>, the port is on the far left hand side next to the processor heatsink. The pin on the left edge of the board is Ground, and the pin closest to the heatsink is +3.3v. The two in the middle? No idea, and I don&rsquo;t have a scope to probe them properly. However, the UART2 serial port is shared with the SD card, so I was able to find that the RX pin is the 5th pad from the bottom and the TX pin is the 3rd pad from the top on this image. Also interesting: unlike the CX-R8, the OTG port is the USB port on the back of the device instead of the one next to the MicroSD slot. Also unlike the CX-R8, this is clearly stated in the manual (!!), and it came with a USB A-A cable for flashing (!!). The rkflashtool programs worked as expected, and I was able to put a stock Debian 8 initrd with no kernel modules in it, adjust the params file to my USB root filesystem, and watch it boot into Debian 8. I was able to make a USB drive with a arm64 root filesystem using the <a href="https://wiki.ubuntu.com/ARM/RootfsFromScratch/QemuDebootstrap">new debootstrap with qemu-static support</a>. Very handy that.</p>
<p>However, now the bad news. Since I was using the kernel that ships with the device (in order to ensure the hardware is correctly supported) and that doesn&rsquo;t have the correct hooks for f**king systemd. So, systemd fails to find the Ethernet device or the serial ports and thus no way to login to the device. I can think of two ways of working around this: build a new kernel or move off of systemd. Moving off of systemd on Debian 8 doesn&rsquo;t look like an option, and Debian 7 didn&rsquo;t ship with arm64 packages. There may be an arm64 Debian 7 repo around somewhere, but I haven&rsquo;t found it yet (I also haven&rsquo;t looked very hard). The second option is to build a new kernel. This leads to a couple of interesting issues. I haven&rsquo;t found the RK3386 patched 3.10 kernel to just make some quick changes to the existing config, and it doesn&rsquo;t look like they left the /proc/config.gz option on. The latest 4.3 kernel does look like it has enough support for the SoC and the Ethernet in order to get that all working, which is all I really need (no video is expected and that&rsquo;s fine). However, building a kernel is not something I have done in a very long time and cross-compiling a kernel is something that I haven&rsquo;t done in over a decade. Needless to say, that hasn&rsquo;t gone anywhere very fast. Cross-compiling an arm64 kernel has been a complete disaster for me.</p>
<p>In brighter news: one of the nice folks in the #linux-rockchip channel (naobsd as I recall) pointed me at the <a href="http://www.geekbuying.com/item/GeekBox-Open-Source-Cross-TV-BOX-Android-Ubuntu-Dual-Boot-4K-RK3368-Octa-Core-2G-16G-AC-WIFI-1000M-LAN-BT4-1-HDMI2-0-OTG-358067.html">GeekBox</a>. It&rsquo;s another RK3368 based system, but it comes with Ubuntu on it as a boot option and has a docking board with a SATA port! One of those, with the docking board, the UART adapter, and a real CPU heatsink with fan I was able to get for about $150. So I ordered two. They are currently very backordered, but the nice folks who will be shipping it to me from China assure me they will fulfill my order as soon as they are back in stock. I&rsquo;m going to wait for one of those to show up, and build a new kernel on that for the MK68 and possibly just use that as-is with some containers or chroots on it for build targets on Debian 8 and Ubuntu. I&rsquo;ll update again once those arrive for testing &amp; compiling.</p>
<h2 id="nvidia-jetson-tk1">nVidia Jetson TK1</h2>
<p>In the mean time, the <a href="http://cubietruck.com/collections/frontpage/products/cubieboard4-cc-a80-high-performance-mini-pc-development-board">Cubieboard 4</a> I ordered seems to be lost in the great black hole that is international shipping and customs without a tracking number. I was hoping to use this for all the ARMv6 and ARMv7 builds, but I&rsquo;ve not seen it and the shipper is non-responsive to my inquires of a tracking or customs document number. Another kind soul on the #linux-rockchip channel suggested getting a Jetson TK1 board since it has a native SATA port (turns out very few ARM boards have native SATA, and that Geekbox docking board is just a SATA-USB interface as well). I figured this would be much easier to get something working with. So, I grabbed one via a <a href="https://developer.nvidia.com/embedded/makejtk1">Make: promotion</a> for only $99. It arrived a few days ago.</p>
<p>This device comes running Ubuntu 14.04, which is very nice. The downside is that all the current gen ARM boards seem to target a heavily patched 3.10 kernel because that&rsquo;s what Android requires. But, major MAJOR bonus: nVidia has found a great workaround for the bootloader issue that I&rsquo;m going to steal for other boards that don&rsquo;t implement this. They also use u-boot to bring up the board, but then embed EXTLINUX (from the SYSLINUX) project as a second stage bootloader. This means there is a text file on the system that tells it what kernel to use, and can even present a menu to choose what you want to boot from over the serial port. This is so much nicer than having to re-flash the parameters &ldquo;file&rdquo; from a host device while in recovery mode to adjust kernel command line options or try a new kernel!</p>
<p>With the Jetson having a SATA port and the Geekboards also having a SATA port I went and grabbed three 240GB SSDs from Fry&rsquo;s post black friday for under $80/ea. The Patriot Blaze SSD I grabbed for the Tegra is giving lots of SCSI errors, but it also turns out that the board has an older software loadout on it&rsquo;s internal flash. So, I went and got the latest build of their JetPack tool to install, and it&rsquo;s been a parade of failures ever since.</p>
<p>The JetPack installer refuses to function on any host machine that is not x86_64 running Ubuntu 14.04. So, I put a LiveCD of 14.04 in a system with a 32GB ext4 formatted USB drive in to put the stupid installer on. It then goes and downloads a bunch of junk from the Internet, builds a new image, and slowly re-flashes all 16GB of flash on the TK1 over USB2. Then, it fails to boot. No idea why. The process is very unclear of what is supposed to happen, it may need to make a TFTP boot but while it can configure a DHCP server correctly, it&rsquo;s not running it&rsquo;s own TFTP server or putting the TFTP files in the right place, so I have no idea what it&rsquo;s trying to do. I&rsquo;m sure I&rsquo;m missing some package it wants, but it&rsquo;s not telling me that either. Grumble. Going to have to troll the nVidia developer forums next week to figure out what&rsquo;s wrong with that. But soon, I hope I can get that working again and at least get the 32bit ARM build environments setup there.</p>
<hr />
<p>So that&rsquo;s where all the ARM stuff stands for now. Not a lot of successes to report, but some forward progress. More updates next weekend, I&rsquo;m sure.</p>
SSL Labs A+ Certification for Apache 2.4https://www.gigofham.com/archives/32
Tue, 16 Dec 2014 04:44:18 +0000https://www.gigofham.com/archives/32<p>After gaining some insight from <a href="https://sethvargo.com/getting-an-a-plus-on-qualys-ssl-labs-tester/">Seth Vargo&rsquo;s excellent post on doing the same things with nginx</a>, I decided to tweak my Apache 2.4 config to get the same results:</p>
<p><a href="http://gigofham.com/wp-content/uploads/2014/12/Screenshot-12152014-101830-PM.png"><img class="alignnone size-full wp-image-33" src="http://gigofham.com/wp-content/uploads/2014/12/Screenshot-12152014-101830-PM.png" alt="Screenshot - 12152014 - 10:18:30 PM" width="916" height="508" srcset="http://gigofham.com/wp-content/uploads/2014/12/Screenshot-12152014-101830-PM.png 916w, http://gigofham.com/wp-content/uploads/2014/12/Screenshot-12152014-101830-PM-300x166.png 300w, http://gigofham.com/wp-content/uploads/2014/12/Screenshot-12152014-101830-PM-239x133.png 239w" sizes="(max-width: 916px) 100vw, 916px" /></a></p>
<p>At first, I got everything but the TLS_FALLBACK_SCSV support. I did some digging and discovered that I had missed and OpenSSL update. Applying that and then restarting apache did the trick. Here are the relevant security announcements with the required versions of OpenSSL for <a href="http://www.ubuntu.com/usn/usn-2385-1/">ubuntu</a>, <a href="https://www.debian.org/security/2014/dsa-3053">debian</a>, and <a href="https://rhn.redhat.com/errata/RHSA-2014-1652.html">RHEL/CentOS</a>. The server I configured is running Ubuntu 14.04 LTS.</p>
<p>As Seth mentions, you will need an SSL certificate. I get mine for free from <a href="http://www.startssl.com/">StartSSL</a>. No really, they are free and they are recognized by all modern browsers and phones. They even sign all their certs using SHA256 and provide a full <a href="https://www.startssl.com/certs/class1/sha2/pem/">SHA256 chain</a> (if you know where to look). <a href="http://googleonlinesecurity.blogspot.com/2014/09/gradually-sunsetting-sha-1.html">This is important if any of your users are using Chrome or Chromium</a>. (UPDATE: <a href="https://blog.mozilla.org/security/2014/09/23/phasing-out-certificates-with-sha-1-based-signature-algorithms/">Mozilla Firefox too</a>.) You can pay for some upgrades, like organizational verification (which I do), for an annual fee and then you can still get as many certificates as you like. Same model for their EV certs as well, just be aware that you may need to stack other certification levels on your account before adding EV support. Still, once you pay those fees: all your certificates are EV. Kind of a sweet deal, I just wish they had an API.</p>
<p>Now, on to apache. The configuration is a bit more complex than nginx, I break it into two parts: one per site (which I break out to a file and include in all my sites for easier maintenance), and one for the mod_ssl module itself. Let&rsquo;s start with the mod_ssl config (which must be defined OUTSIDE a VirtualHost block):</p>
<pre style="padding-left: 30px;"># OCSP Stapling
SSLUseStapling on
SSLStaplingResponderTimeout 5
SSLStaplingReturnResponderErrors off
SSLStaplingCache "shmcb:logs/stapling_cache(128000)"</pre>
<p>The first few lines are self explanatory, the last one needs some discussion. The <a href="http://httpd.apache.org/docs/2.4/mod/mod_ssl.html#sslstaplingcache">OCSP stapling cache can be configured many ways</a>, in this case I decided on a <a href="http://httpd.apache.org/docs/2.4/mod/mod_ssl.html#sslsessioncache">shared memory cyclic buffer</a> with a reasonable size. There are other options defined in the docs, but this one is reasonably performant and doesn&rsquo;t have a major cost or configuration impact.</p>
<p>Now comes the fun part, which is the per site configuration. Apache 2.4 supports <a href="http://en.wikipedia.org/wiki/Server_Name_Indication">Server Name Indication (or SNI)</a>, so you can run multiple SSL sites on a single IP address. This is also supported by most modern browsers and phones (the Wikipedia article linked as a very through list). Because I run multiple sites on a single IP, I didn&rsquo;t want to keep the same huge block of configuration information in each site, so I break out the common bits to a file like this:</p>
<pre style="padding-left: 30px;">&lt;IfModule mod_ssl.c&gt;
# SSL Engine Switch:
# Enable/Disable SSL for this virtual host.
SSLEngine on
# SSL Engine Options:
# Set various options for the SSL engine.
# o FakeBasicAuth:
# Translate the client X.509 into a Basic Authorisation. This means that
# the standard Auth/DBMAuth methods can be used for access control. The
# user name is the `one line' version of the client's X.509 certificate.
# Note that no password is obtained from the user. Every entry in the user
# file needs this password: `xxj31ZMTZzkVA'.
# o ExportCertData:
# This exports two additional environment variables: SSL_CLIENT_CERT and
# SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
# server (always existing) and the client (only existing when client
# authentication is used). This can be used to import the certificates
# into CGI scripts.
# o StdEnvVars:
# This exports the standard SSL/TLS related `SSL_*' environment variables.
# Per default this exportation is switched off for performance reasons,
# because the extraction step is an expensive operation and is usually
# useless for serving static content. So one usually enables the
# exportation for CGI and SSI requests only.
# o StrictRequire:
# This denies access when "SSLRequireSSL" or "SSLRequire" applied even
# under a "Satisfy any" situation, i.e. when it applies access is denied
# and no other module can change it.
# o OptRenegotiate:
# This enables optimized SSL connection renegotiation handling when SSL
# directives are used in per-directory context.
#SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
&lt;FilesMatch "\.(cgi|shtml|phtml|php)$"&gt;
SSLOptions +StdEnvVars
&lt;/FilesMatch&gt;
# SSL Protocol Adjustments:
# The safe and default but still SSL/TLS standard compliant shutdown
# approach is that mod_ssl sends the close notify alert but doesn't wait for
# the close notify alert from client. When you need a different shutdown
# approach you can use one of the following variables:
# o ssl-unclean-shutdown:
# This forces an unclean shutdown when the connection is closed, i.e. no
# SSL close notify alert is send or allowed to received. This violates
# the SSL/TLS standard but is needed for some brain-dead browsers. Use
# this when you receive I/O errors because of the standard approach where
# mod_ssl sends the close notify alert.
# o ssl-accurate-shutdown:
# This forces an accurate shutdown when the connection is closed, i.e. a
# SSL close notify alert is send and mod_ssl waits for the close notify
# alert of the client. This is 100% SSL/TLS standard compliant, but in
# practice often causes hanging connections with brain-dead browsers. Use
# this only for browsers where you know that their SSL implementation
# works correctly.
# Notice: Most problems of broken clients are also related to the HTTP
# keep-alive facility, so you usually additionally want to disable
# keep-alive for those clients, too. Use variable "nokeepalive" for this.
# Similarly, one has to force some clients to use HTTP/1.0 to workaround
# their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
# "force-response-1.0" for this.
BrowserMatch "MSIE [2-6]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
# MSIE 7 and newer should be able to use keepalive
BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
# Disable insecure SSL versions, and enable secure ones in an explicit fashion
SSLProtocol -all -SSLv2 -SSLv3 +TLSv1 +TLSv1.1 +TLSv1.2
# Use the cipher list provided below, not the built-in default
SSLHonorCipherOrder on
# SSL compression is bad for attack vectors
SSLCompression off
# Modified from Mozilla, only support 256bit AES
SSLCipherSuite "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK"
# HSTS, so make sure you have the Header module enabled
Header set Strict-Transport-Security "max-age=31536000; includeSubDomains"
&lt;/IfModule&gt;</pre>
<p>I&rsquo;ve tried to be explicit as possible with comments from the Apache documentation as well as the other sources I&rsquo;ve pulled from. The Mozilla reference is to their <a href="https://wiki.mozilla.org/Security/Server_Side_TLS">Server Side TLS</a> settings recommendations. In order to get 100% on the SSL Labs test, you need to disable everything but the 256bit AES DHE and ECDHE ciphers. This is a explicit breakout of the ciphers referenced in Seth&rsquo;s configuration. With this all in a single file, configuring a virtual host is pretty simple:</p>
<pre style="padding-left: 30px;">&lt;IfModule mod_ssl.c&gt;
&lt;VirtualHost _default_:443&gt;
ServerAdmin webmaster@example.com
ServerName example.com
ServerAlias www.example.com
DocumentRoot /home/https/example.com/www/wordpress
&lt;Directory /home/https/example.com/www/wordpress&gt;
Options FollowSymLinks
AllowOverride None
Require all granted
Order allow,deny
allow from all
&lt;IfModule mod_rewrite.c&gt;
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
RewriteRule ^/wp-content/uploads/.*\.php$ - [F,NC]
&lt;/IfModule&gt;
&lt;/Directory&gt;
&lt;Directory /home/https/example.com/www/wordpress/.git&gt;
Order deny,allow
Deny from all
Require all denied
&lt;/Directory&gt;
ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9001/home/https/example.com/www/wordpress/$1
ErrorLog ${APACHE_LOG_DIR}/www.spherecube.io-error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/www.example.com-access.log combined
<span style="color: #ff6600;">Include /etc/apache2/sites-available/common-ssl.inc</span>
<span style="color: #ff6600;">SSLCertificateFile /etc/ssl/certs/www.example.com.crt</span>
<span style="color: #ff6600;">SSLCertificateKeyFile /etc/ssl/private/www.example.com.key</span>
<span style="color: #ff6600;">SSLCACertificateFile /etc/ssl/certs/startcom-ca.pem</span>
<span style="color: #ff6600;">SSLCertificateChainFile /etc/ssl/certs/startcom-sub.class2.server.ca.pem</span>
&lt;/VirtualHost&gt;
&lt;/IfModule&gt;</pre>
<p>Yeah, I like verbose apache configs. The relevant lines are highlighted. You basically pull in the config defined above, and then define the SSL certificate and key on a per site basis. Could I have added the CA and chain files to the global config? Yes, but then I would only be able to support a single CA. This is still a tiny amount of configuration information compared to the whole mess in the previous section and provides some flexibility. That&rsquo;s pretty much it, now go secure everything.</p>
Building an IPv6 Firewallhttps://www.gigofham.com/archives/12
Tue, 31 May 2011 03:53:56 +0000https://www.gigofham.com/archives/12
<p>As <a href="http://ipv6.he.net">As</a> will gladly tell you, we are rapidly running out of unique IP addresses on the Internet. The good news is we solved this problem in 1998. The bad news is that, after more than ten years, we still haven&rsquo;t widely implemented the new standard: IPv6. All of the big carriers support it, but getting access to the IPv6 Internet is still very tough. If you are a Comcast customer it is possible to get IPv6 access, and there are a few others who allow access as well. For everyone else, you either have to wait or you can get a tunnel (again from HE through their awesome <a href="http://tunnelbroker.net">Tunnel Broker</a> service).</p>
<p>Let me take a moment here and state that I really applaud HE&rsquo;s leadership in the transition to IPv6. They run an excellent tunnel broker service for end users, they run a very large transit network, they share a lot of information, and they were not afraid to jump in the pool. Without their efforts I don&rsquo;t think we would be as far along today as we are now, and I thank them deeply for their efforts to date.</p>
<p>Now, some may know the differences between IPv4 (the current Internet addressing scheme) and IPv6 (the new scheme), but some may not. There are a lot of excellent resources that go into much deeper detail than I will but here are the basics:</p>
<ul>
<li>The space for IP addresses has changed from 32bits to 128bits &#8211; an immensely huge number of addresses.</li>
<li>There are NO private address spaces in IPv6 (unlike IPv4 which has several: 10.0.0.0/8 192.168.0.0/16 and 172.16.0.0/12).</li>
<li>While there is Network Address Translation (NAT) in IPv6, it is 1-to-1 &#8211; as opposed to the typical 1-to-many used by almost every consumer/business router product on the market.</li>
<li>Unlike current IPv4 addressing where end users typically get a single IP address with a consumer connection, you get an entire 64bit subnet on a consumer connection with IPv6.</li>
<li>Unlike the current IPv4 Internet where many people are on the same &ldquo;private&rdquo; subnets, everyone on the IPv6 Internet is uniquely addressable. This has the benefit of P2P and gaming systems not having to use so much &ldquo;magic&rdquo; in order for connectivity.</li>
</ul>
<p>There are downsides to this: because everything is Internet addressable, the need of security in depth firewall solutions is much greater than it has been in the past. You can no longer depend on a big NAT firewall at the border of your network to protect you. The old, lazy M&amp;M security model (hard outer shell, soft and chewy on the inside) is dead. This also brings up some interesting concerns for &ldquo;consumer devices&rdquo; (such as game consoles, and other black-box network appliances) on your network as they could now be more of an attack vector into your network than they were before.</p>
<p>With all this in mind I got a HE tunnel and wanted to build a firewall for my stuff at home. Previously, I used a <a href="http://pfsense.org">pfSense</a> firewall on a <a href="http://soekris.com">Soekris NET4801</a> hardware platform. This was great, but pfSense does not support IPv6 on their General Availability release. Also the old NET4801 doesn&rsquo;t have enough horsepower to handle my forthcoming FiOS high speed connection (the combination of the NICs and the processor can only handle about 9mbps, where as the FiOS package I want is 25mbps). I also have a game console (PS3) which needs UPnP IGD service on my router in order for on-line gaming to work. There are other firewall solutions which do support IPv6, but none supported IPv6 and UPnP. So, I started to build one myself.</p>
<p>I decided to build the firewall on a Jetway VIA C7 platform using FreeBSD. Why FreeBSD? The miniupnpd server is ported to it, and it includes the pf firewall. Also, FreeBSD is the basis for pfSense so I could re-use my old IPv4 pf rules. I could have used Linux, but I woud have to compile miniupnpd myself and I haven&rsquo;t tested it on Linux.</p>
<p>Before making the firewall, I sat down and figured out what traffic I wanted my firewall to pass and what I wanted it to block for IPv6. Here are the goals I cam up with:</p>
<ul>
<li>Anyone can ping anyone on either side of the firewall</li>
<li>Anyone inside the firewall can connect to anyone on the outside of the firewall</li>
<li>No TCP/UDP connections be made from outside the firewall to hosts inside the firewall by default (specific ports can be opened later)</li>
<li>Router Advertisements from outside the firewall will NOT be passed inside the firewall</li>
<li>Router Solicitations from inside the firewall will NOT be passed to outside the firewall</li>
<li>Neighbor Solicitations will not pass through the firewall</li>
</ul>
<p>Obviously, more complicated that a traditional IPv4 firewall ruleset. So I started to build the rules in pf, and then ran into some problems because I forgot the rules of pf:</p>
<ol>
<li>LAST match wins</li>
<li>The &ldquo;quick&rdquo; keyword means stop processing rules and use that one</li>
</ol>
<p>With rules like that, there are two solutions for building your rulesets:</p>
<ol>
<li>First rules drop everything, then the subsequent rules allow traffic (with some short cuts using quick)</li>
<li>Last rules drop everything, all previous rules have the quick short cut</li>
</ol>
<p>The pf firewall rules, use method 2. I would have preferred method 1, but oh well. So, here is the ruleset I built:</p>
<p>`<br /></p>
<h1 id="system-aliases-br">System Aliases<br /></h1>
<p>loopback = &ldquo;{ lo0 }&rdquo;<br />
lan = &ldquo;{ re1 }&rdquo;<br />
wan = &ldquo;{ re0 }&rdquo;<br />
enc0 = &ldquo;{ enc0 }&rdquo;<br />
hurrelec=&rdquo;<span style="color: #ff0000;">[HE tunnel IPv4 endpoint]</span>&ldquo;`</p>
<p><code># Allow IPv6&lt;br /&gt;
pass out quick on gif0 inet6 label &quot;gif0 tunnel out allowed&quot;&lt;br /&gt;
pass in quick inet proto 41 from $hurrelec to $wan keep state label &quot;tunnel traffic from HE&quot;&lt;br /&gt;
pass out quick inet proto 41 from $wan to $hurrelec keep state label &quot;tunnel traffic to HE&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto tcp from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any keep state label &quot;IPv6 TCP from the LAN&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto udp from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any keep state label &quot;IPv6 UDP from the LAN&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 to any keep state label &quot;ICMPv6 from the LAN&quot;</code></p>
<p><code># Allow some ICMPv6 through&lt;br /&gt;
pass in quick log on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type echoreq label &quot;ICMPv6 echo request in lan&quot;&lt;br /&gt;
pass in quick log on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type echorep label &quot;ICMPv6 echo reply in lan&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from any to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type echoreq label &quot;ICMPv6 echo request out lan&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from any to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type echorep label &quot;ICMPv6 echo reply out lan&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type timex label &quot;ICMPv6 time exceeded&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type unreach label &quot;ICMPv6 destination unreachable&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type paramprob label &quot;ICMPv6 invalid header (parameter problem)&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type toobig label &quot;ICMPv6 too big&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type groupqry label &quot;ICMPv6 group membership query&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type listqry label &quot;ICMPv6 multicast listener query&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type grouprep label &quot;ICMPv6 group membership report&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type groupterm label &quot;ICMPv6 group membership termination&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type listendone label &quot;ICMPv6 multicast listener done&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 icmp6-type routersol label &quot;ICMPv6 lan router solicitations&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 icmp6-type neighbrsol label &quot;ICMPv6 lan neighbor solicitations&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 router]&lt;/span&gt; to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type routeradv label &quot;ICMPv6 lan router advertisements&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 router]&lt;/span&gt; to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type neighbradv label &quot;ICMPv6 lan neighbor advertisements&quot;</code></p>
<p>``</p>
<p><code># loopback&lt;br /&gt;
anchor &quot;loopback&quot;&lt;br /&gt;
pass in quick on $loopback inet6 all label &quot;pass in loopback IPv6&quot;&lt;br /&gt;
pass out quick on $loopback inet6 all label &quot;pass out loobkack IPv6&quot;</code></p>
<p>``</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
<p>```As <a href="http://ipv6.he.net">As</a> will gladly tell you, we are rapidly running out of unique IP addresses on the Internet. The good news is we solved this problem in 1998. The bad news is that, after more than ten years, we still haven&rsquo;t widely implemented the new standard: IPv6. All of the big carriers support it, but getting access to the IPv6 Internet is still very tough. If you are a Comcast customer it is possible to get IPv6 access, and there are a few others who allow access as well. For everyone else, you either have to wait or you can get a tunnel (again from HE through their awesome <a href="http://tunnelbroker.net">Tunnel Broker</a> service).</p>
<p>Let me take a moment here and state that I really applaud HE&rsquo;s leadership in the transition to IPv6. They run an excellent tunnel broker service for end users, they run a very large transit network, they share a lot of information, and they were not afraid to jump in the pool. Without their efforts I don&rsquo;t think we would be as far along today as we are now, and I thank them deeply for their efforts to date.</p>
<p>Now, some may know the differences between IPv4 (the current Internet addressing scheme) and IPv6 (the new scheme), but some may not. There are a lot of excellent resources that go into much deeper detail than I will but here are the basics:</p>
<ul>
<li>The space for IP addresses has changed from 32bits to 128bits &#8211; an immensely huge number of addresses.</li>
<li>There are NO private address spaces in IPv6 (unlike IPv4 which has several: 10.0.0.0/8 192.168.0.0/16 and 172.16.0.0/12).</li>
<li>While there is Network Address Translation (NAT) in IPv6, it is 1-to-1 &#8211; as opposed to the typical 1-to-many used by almost every consumer/business router product on the market.</li>
<li>Unlike current IPv4 addressing where end users typically get a single IP address with a consumer connection, you get an entire 64bit subnet on a consumer connection with IPv6.</li>
<li>Unlike the current IPv4 Internet where many people are on the same &ldquo;private&rdquo; subnets, everyone on the IPv6 Internet is uniquely addressable. This has the benefit of P2P and gaming systems not having to use so much &ldquo;magic&rdquo; in order for connectivity.</li>
</ul>
<p>There are downsides to this: because everything is Internet addressable, the need of security in depth firewall solutions is much greater than it has been in the past. You can no longer depend on a big NAT firewall at the border of your network to protect you. The old, lazy M&amp;M security model (hard outer shell, soft and chewy on the inside) is dead. This also brings up some interesting concerns for &ldquo;consumer devices&rdquo; (such as game consoles, and other black-box network appliances) on your network as they could now be more of an attack vector into your network than they were before.</p>
<p>With all this in mind I got a HE tunnel and wanted to build a firewall for my stuff at home. Previously, I used a <a href="http://pfsense.org">pfSense</a> firewall on a <a href="http://soekris.com">Soekris NET4801</a> hardware platform. This was great, but pfSense does not support IPv6 on their General Availability release. Also the old NET4801 doesn&rsquo;t have enough horsepower to handle my forthcoming FiOS high speed connection (the combination of the NICs and the processor can only handle about 9mbps, where as the FiOS package I want is 25mbps). I also have a game console (PS3) which needs UPnP IGD service on my router in order for on-line gaming to work. There are other firewall solutions which do support IPv6, but none supported IPv6 and UPnP. So, I started to build one myself.</p>
<p>I decided to build the firewall on a Jetway VIA C7 platform using FreeBSD. Why FreeBSD? The miniupnpd server is ported to it, and it includes the pf firewall. Also, FreeBSD is the basis for pfSense so I could re-use my old IPv4 pf rules. I could have used Linux, but I woud have to compile miniupnpd myself and I haven&rsquo;t tested it on Linux.</p>
<p>Before making the firewall, I sat down and figured out what traffic I wanted my firewall to pass and what I wanted it to block for IPv6. Here are the goals I cam up with:</p>
<ul>
<li>Anyone can ping anyone on either side of the firewall</li>
<li>Anyone inside the firewall can connect to anyone on the outside of the firewall</li>
<li>No TCP/UDP connections be made from outside the firewall to hosts inside the firewall by default (specific ports can be opened later)</li>
<li>Router Advertisements from outside the firewall will NOT be passed inside the firewall</li>
<li>Router Solicitations from inside the firewall will NOT be passed to outside the firewall</li>
<li>Neighbor Solicitations will not pass through the firewall</li>
</ul>
<p>Obviously, more complicated that a traditional IPv4 firewall ruleset. So I started to build the rules in pf, and then ran into some problems because I forgot the rules of pf:</p>
<ol>
<li>LAST match wins</li>
<li>The &ldquo;quick&rdquo; keyword means stop processing rules and use that one</li>
</ol>
<p>With rules like that, there are two solutions for building your rulesets:</p>
<ol>
<li>First rules drop everything, then the subsequent rules allow traffic (with some short cuts using quick)</li>
<li>Last rules drop everything, all previous rules have the quick short cut</li>
</ol>
<p>The pf firewall rules, use method 2. I would have preferred method 1, but oh well. So, here is the ruleset I built:</p>
<p>`<br /></p>
<h1 id="system-aliases-br-1">System Aliases<br /></h1>
<p>loopback = &ldquo;{ lo0 }&rdquo;<br />
lan = &ldquo;{ re1 }&rdquo;<br />
wan = &ldquo;{ re0 }&rdquo;<br />
enc0 = &ldquo;{ enc0 }&rdquo;<br />
hurrelec=&rdquo;<span style="color: #ff0000;">[HE tunnel IPv4 endpoint]</span>&ldquo;`</p>
<p><code># Allow IPv6&lt;br /&gt;
pass out quick on gif0 inet6 label &quot;gif0 tunnel out allowed&quot;&lt;br /&gt;
pass in quick inet proto 41 from $hurrelec to $wan keep state label &quot;tunnel traffic from HE&quot;&lt;br /&gt;
pass out quick inet proto 41 from $wan to $hurrelec keep state label &quot;tunnel traffic to HE&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto tcp from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any keep state label &quot;IPv6 TCP from the LAN&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto udp from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any keep state label &quot;IPv6 UDP from the LAN&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 to any keep state label &quot;ICMPv6 from the LAN&quot;</code></p>
<p><code># Allow some ICMPv6 through&lt;br /&gt;
pass in quick log on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type echoreq label &quot;ICMPv6 echo request in lan&quot;&lt;br /&gt;
pass in quick log on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type echorep label &quot;ICMPv6 echo reply in lan&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from any to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type echoreq label &quot;ICMPv6 echo request out lan&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from any to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type echorep label &quot;ICMPv6 echo reply out lan&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type timex label &quot;ICMPv6 time exceeded&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type unreach label &quot;ICMPv6 destination unreachable&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type paramprob label &quot;ICMPv6 invalid header (parameter problem)&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type toobig label &quot;ICMPv6 too big&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type groupqry label &quot;ICMPv6 group membership query&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type listqry label &quot;ICMPv6 multicast listener query&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type grouprep label &quot;ICMPv6 group membership report&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type groupterm label &quot;ICMPv6 group membership termination&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type listendone label &quot;ICMPv6 multicast listener done&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 icmp6-type routersol label &quot;ICMPv6 lan router solicitations&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 icmp6-type neighbrsol label &quot;ICMPv6 lan neighbor solicitations&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 router]&lt;/span&gt; to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type routeradv label &quot;ICMPv6 lan router advertisements&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 router]&lt;/span&gt; to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type neighbradv label &quot;ICMPv6 lan neighbor advertisements&quot;</code></p>
<p>``</p>
<p><code># loopback&lt;br /&gt;
anchor &quot;loopback&quot;&lt;br /&gt;
pass in quick on $loopback inet6 all label &quot;pass in loopback IPv6&quot;&lt;br /&gt;
pass out quick on $loopback inet6 all label &quot;pass out loobkack IPv6&quot;</code></p>
<p>``</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
<pre><code>
You also need a few lines in your /etc/rc.conf:
````As [As](http://ipv6.he.net) will gladly tell you, we are rapidly running out of unique IP addresses on the Internet. The good news is we solved this problem in 1998. The bad news is that, after more than ten years, we still haven't widely implemented the new standard: IPv6. All of the big carriers support it, but getting access to the IPv6 Internet is still very tough. If you are a Comcast customer it is possible to get IPv6 access, and there are a few others who allow access as well. For everyone else, you either have to wait or you can get a tunnel (again from HE through their awesome [Tunnel Broker](http://tunnelbroker.net) service).
Let me take a moment here and state that I really applaud HE's leadership in the transition to IPv6. They run an excellent tunnel broker service for end users, they run a very large transit network, they share a lot of information, and they were not afraid to jump in the pool. Without their efforts I don't think we would be as far along today as we are now, and I thank them deeply for their efforts to date.
Now, some may know the differences between IPv4 (the current Internet addressing scheme) and IPv6 (the new scheme), but some may not. There are a lot of excellent resources that go into much deeper detail than I will but here are the basics:
* The space for IP addresses has changed from 32bits to 128bits &amp;#8211; an immensely huge number of addresses.
* There are NO private address spaces in IPv6 (unlike IPv4 which has several: 10.0.0.0/8 192.168.0.0/16 and 172.16.0.0/12).
* While there is Network Address Translation (NAT) in IPv6, it is 1-to-1 &amp;#8211; as opposed to the typical 1-to-many used by almost every consumer/business router product on the market.
* Unlike current IPv4 addressing where end users typically get a single IP address with a consumer connection, you get an entire 64bit subnet on a consumer connection with IPv6.
* Unlike the current IPv4 Internet where many people are on the same &quot;private&quot; subnets, everyone on the IPv6 Internet is uniquely addressable. This has the benefit of P2P and gaming systems not having to use so much &quot;magic&quot; in order for connectivity.
There are downsides to this: because everything is Internet addressable, the need of security in depth firewall solutions is much greater than it has been in the past. You can no longer depend on a big NAT firewall at the border of your network to protect you. The old, lazy M&amp;M security model (hard outer shell, soft and chewy on the inside) is dead. This also brings up some interesting concerns for &quot;consumer devices&quot; (such as game consoles, and other black-box network appliances) on your network as they could now be more of an attack vector into your network than they were before.
With all this in mind I got a HE tunnel and wanted to build a firewall for my stuff at home. Previously, I used a [pfSense](http://pfsense.org) firewall on a [Soekris NET4801](http://soekris.com) hardware platform. This was great, but pfSense does not support IPv6 on their General Availability release. Also the old NET4801 doesn't have enough horsepower to handle my forthcoming FiOS high speed connection (the combination of the NICs and the processor can only handle about 9mbps, where as the FiOS package I want is 25mbps). I also have a game console (PS3) which needs UPnP IGD service on my router in order for on-line gaming to work. There are other firewall solutions which do support IPv6, but none supported IPv6 and UPnP. So, I started to build one myself.
I decided to build the firewall on a Jetway VIA C7 platform using FreeBSD. Why FreeBSD? The miniupnpd server is ported to it, and it includes the pf firewall. Also, FreeBSD is the basis for pfSense so I could re-use my old IPv4 pf rules. I could have used Linux, but I woud have to compile miniupnpd myself and I haven't tested it on Linux.
Before making the firewall, I sat down and figured out what traffic I wanted my firewall to pass and what I wanted it to block for IPv6. Here are the goals I cam up with:
* Anyone can ping anyone on either side of the firewall
* Anyone inside the firewall can connect to anyone on the outside of the firewall
* No TCP/UDP connections be made from outside the firewall to hosts inside the firewall by default (specific ports can be opened later)
* Router Advertisements from outside the firewall will NOT be passed inside the firewall
* Router Solicitations from inside the firewall will NOT be passed to outside the firewall
* Neighbor Solicitations will not pass through the firewall
Obviously, more complicated that a traditional IPv4 firewall ruleset. So I started to build the rules in pf, and then ran into some problems because I forgot the rules of pf:
1. LAST match wins
2. The &quot;quick&quot; keyword means stop processing rules and use that one
With rules like that, there are two solutions for building your rulesets:
1. First rules drop everything, then the subsequent rules allow traffic (with some short cuts using quick)
2. Last rules drop everything, all previous rules have the quick short cut
The pf firewall rules, use method 2. I would have preferred method 1, but oh well. So, here is the ruleset I built:
`&lt;br /&gt;
# System Aliases&lt;br /&gt;
loopback = &quot;{ lo0 }&quot;&lt;br /&gt;
lan = &quot;{ re1 }&quot;&lt;br /&gt;
wan = &quot;{ re0 }&quot;&lt;br /&gt;
enc0 = &quot;{ enc0 }&quot;&lt;br /&gt;
hurrelec=&quot;&lt;span style=&quot;color: #ff0000;&quot;&gt;[HE tunnel IPv4 endpoint]&lt;/span&gt;&quot;`
`# Allow IPv6&lt;br /&gt;
pass out quick on gif0 inet6 label &quot;gif0 tunnel out allowed&quot;&lt;br /&gt;
pass in quick inet proto 41 from $hurrelec to $wan keep state label &quot;tunnel traffic from HE&quot;&lt;br /&gt;
pass out quick inet proto 41 from $wan to $hurrelec keep state label &quot;tunnel traffic to HE&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto tcp from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any keep state label &quot;IPv6 TCP from the LAN&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto udp from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any keep state label &quot;IPv6 UDP from the LAN&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 to any keep state label &quot;ICMPv6 from the LAN&quot;`
`# Allow some ICMPv6 through&lt;br /&gt;
pass in quick log on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type echoreq label &quot;ICMPv6 echo request in lan&quot;&lt;br /&gt;
pass in quick log on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type echorep label &quot;ICMPv6 echo reply in lan&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from any to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type echoreq label &quot;ICMPv6 echo request out lan&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from any to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type echorep label &quot;ICMPv6 echo reply out lan&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type timex label &quot;ICMPv6 time exceeded&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type unreach label &quot;ICMPv6 destination unreachable&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type paramprob label &quot;ICMPv6 invalid header (parameter problem)&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type toobig label &quot;ICMPv6 too big&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type groupqry label &quot;ICMPv6 group membership query&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type listqry label &quot;ICMPv6 multicast listener query&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type grouprep label &quot;ICMPv6 group membership report&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type groupterm label &quot;ICMPv6 group membership termination&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type listendone label &quot;ICMPv6 multicast listener done&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 icmp6-type routersol label &quot;ICMPv6 lan router solicitations&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 icmp6-type neighbrsol label &quot;ICMPv6 lan neighbor solicitations&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 router]&lt;/span&gt; to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type routeradv label &quot;ICMPv6 lan router advertisements&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 router]&lt;/span&gt; to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type neighbradv label &quot;ICMPv6 lan neighbor advertisements&quot;`
``
`# loopback&lt;br /&gt;
anchor &quot;loopback&quot;&lt;br /&gt;
pass in quick on $loopback inet6 all label &quot;pass in loopback IPv6&quot;&lt;br /&gt;
pass out quick on $loopback inet6 all label &quot;pass out loobkack IPv6&quot;`
``
&amp;nbsp;
&amp;nbsp;
&amp;nbsp;
```As [As](http://ipv6.he.net) will gladly tell you, we are rapidly running out of unique IP addresses on the Internet. The good news is we solved this problem in 1998. The bad news is that, after more than ten years, we still haven't widely implemented the new standard: IPv6. All of the big carriers support it, but getting access to the IPv6 Internet is still very tough. If you are a Comcast customer it is possible to get IPv6 access, and there are a few others who allow access as well. For everyone else, you either have to wait or you can get a tunnel (again from HE through their awesome [Tunnel Broker](http://tunnelbroker.net) service).
Let me take a moment here and state that I really applaud HE's leadership in the transition to IPv6. They run an excellent tunnel broker service for end users, they run a very large transit network, they share a lot of information, and they were not afraid to jump in the pool. Without their efforts I don't think we would be as far along today as we are now, and I thank them deeply for their efforts to date.
Now, some may know the differences between IPv4 (the current Internet addressing scheme) and IPv6 (the new scheme), but some may not. There are a lot of excellent resources that go into much deeper detail than I will but here are the basics:
* The space for IP addresses has changed from 32bits to 128bits &amp;#8211; an immensely huge number of addresses.
* There are NO private address spaces in IPv6 (unlike IPv4 which has several: 10.0.0.0/8 192.168.0.0/16 and 172.16.0.0/12).
* While there is Network Address Translation (NAT) in IPv6, it is 1-to-1 &amp;#8211; as opposed to the typical 1-to-many used by almost every consumer/business router product on the market.
* Unlike current IPv4 addressing where end users typically get a single IP address with a consumer connection, you get an entire 64bit subnet on a consumer connection with IPv6.
* Unlike the current IPv4 Internet where many people are on the same &quot;private&quot; subnets, everyone on the IPv6 Internet is uniquely addressable. This has the benefit of P2P and gaming systems not having to use so much &quot;magic&quot; in order for connectivity.
There are downsides to this: because everything is Internet addressable, the need of security in depth firewall solutions is much greater than it has been in the past. You can no longer depend on a big NAT firewall at the border of your network to protect you. The old, lazy M&amp;M security model (hard outer shell, soft and chewy on the inside) is dead. This also brings up some interesting concerns for &quot;consumer devices&quot; (such as game consoles, and other black-box network appliances) on your network as they could now be more of an attack vector into your network than they were before.
With all this in mind I got a HE tunnel and wanted to build a firewall for my stuff at home. Previously, I used a [pfSense](http://pfsense.org) firewall on a [Soekris NET4801](http://soekris.com) hardware platform. This was great, but pfSense does not support IPv6 on their General Availability release. Also the old NET4801 doesn't have enough horsepower to handle my forthcoming FiOS high speed connection (the combination of the NICs and the processor can only handle about 9mbps, where as the FiOS package I want is 25mbps). I also have a game console (PS3) which needs UPnP IGD service on my router in order for on-line gaming to work. There are other firewall solutions which do support IPv6, but none supported IPv6 and UPnP. So, I started to build one myself.
I decided to build the firewall on a Jetway VIA C7 platform using FreeBSD. Why FreeBSD? The miniupnpd server is ported to it, and it includes the pf firewall. Also, FreeBSD is the basis for pfSense so I could re-use my old IPv4 pf rules. I could have used Linux, but I woud have to compile miniupnpd myself and I haven't tested it on Linux.
Before making the firewall, I sat down and figured out what traffic I wanted my firewall to pass and what I wanted it to block for IPv6. Here are the goals I cam up with:
* Anyone can ping anyone on either side of the firewall
* Anyone inside the firewall can connect to anyone on the outside of the firewall
* No TCP/UDP connections be made from outside the firewall to hosts inside the firewall by default (specific ports can be opened later)
* Router Advertisements from outside the firewall will NOT be passed inside the firewall
* Router Solicitations from inside the firewall will NOT be passed to outside the firewall
* Neighbor Solicitations will not pass through the firewall
Obviously, more complicated that a traditional IPv4 firewall ruleset. So I started to build the rules in pf, and then ran into some problems because I forgot the rules of pf:
1. LAST match wins
2. The &quot;quick&quot; keyword means stop processing rules and use that one
With rules like that, there are two solutions for building your rulesets:
1. First rules drop everything, then the subsequent rules allow traffic (with some short cuts using quick)
2. Last rules drop everything, all previous rules have the quick short cut
The pf firewall rules, use method 2. I would have preferred method 1, but oh well. So, here is the ruleset I built:
`&lt;br /&gt;
# System Aliases&lt;br /&gt;
loopback = &quot;{ lo0 }&quot;&lt;br /&gt;
lan = &quot;{ re1 }&quot;&lt;br /&gt;
wan = &quot;{ re0 }&quot;&lt;br /&gt;
enc0 = &quot;{ enc0 }&quot;&lt;br /&gt;
hurrelec=&quot;&lt;span style=&quot;color: #ff0000;&quot;&gt;[HE tunnel IPv4 endpoint]&lt;/span&gt;&quot;`
`# Allow IPv6&lt;br /&gt;
pass out quick on gif0 inet6 label &quot;gif0 tunnel out allowed&quot;&lt;br /&gt;
pass in quick inet proto 41 from $hurrelec to $wan keep state label &quot;tunnel traffic from HE&quot;&lt;br /&gt;
pass out quick inet proto 41 from $wan to $hurrelec keep state label &quot;tunnel traffic to HE&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto tcp from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any keep state label &quot;IPv6 TCP from the LAN&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto udp from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any keep state label &quot;IPv6 UDP from the LAN&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 to any keep state label &quot;ICMPv6 from the LAN&quot;`
`# Allow some ICMPv6 through&lt;br /&gt;
pass in quick log on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type echoreq label &quot;ICMPv6 echo request in lan&quot;&lt;br /&gt;
pass in quick log on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type echorep label &quot;ICMPv6 echo reply in lan&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from any to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type echoreq label &quot;ICMPv6 echo request out lan&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from any to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type echorep label &quot;ICMPv6 echo reply out lan&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type timex label &quot;ICMPv6 time exceeded&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type unreach label &quot;ICMPv6 destination unreachable&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type paramprob label &quot;ICMPv6 invalid header (parameter problem)&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type toobig label &quot;ICMPv6 too big&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type groupqry label &quot;ICMPv6 group membership query&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type listqry label &quot;ICMPv6 multicast listener query&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type grouprep label &quot;ICMPv6 group membership report&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type groupterm label &quot;ICMPv6 group membership termination&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 to any icmp6-type listendone label &quot;ICMPv6 multicast listener done&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 icmp6-type routersol label &quot;ICMPv6 lan router solicitations&quot;&lt;br /&gt;
pass in quick on $lan inet6 proto icmp6 icmp6-type neighbrsol label &quot;ICMPv6 lan neighbor solicitations&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 router]&lt;/span&gt; to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type routeradv label &quot;ICMPv6 lan router advertisements&quot;&lt;br /&gt;
pass out quick on $lan inet6 proto icmp6 from &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 router]&lt;/span&gt; to &lt;span style=&quot;color: #ff0000;&quot;&gt;[IPv6 prefix]&lt;/span&gt;/64 icmp6-type neighbradv label &quot;ICMPv6 lan neighbor advertisements&quot;`
``
`# loopback&lt;br /&gt;
anchor &quot;loopback&quot;&lt;br /&gt;
pass in quick on $loopback inet6 all label &quot;pass in loopback IPv6&quot;&lt;br /&gt;
pass out quick on $loopback inet6 all label &quot;pass out loobkack IPv6&quot;`
``
&amp;nbsp;
&amp;nbsp;
&amp;nbsp;
</code></pre>
<p>You also need a few lines in your /etc/rc.conf:</p>
<p>````</p>
<p>You also need the secret decoder ring:</p>
<ul>
<li><span style="color: #ff0000;">[IPv6 prefix]</span> &#8211; Your delegated netblock, typically ends in &ldquo;::/64&rdquo;</li>
<li><span style="color: #ff0000;">[IPv6 router]</span> &#8211; This is the address of your router in your netblock</li>
<li><span style="color: #ff0000;">[HE tunnel IPv4 endpoint]</span> &#8211; This information provided by HE for your tunnel</li>
<li><span style="color: #ff0000;">[provided by HE]</span> &#8211; This information is also provided by HE for your tunnel</li>
</ul>
<p>The rules work as I expected, after some debugging. They also complete all the goals above, I&rsquo;m sure I&rsquo;ll be adding specific rules for specific applications as time goes on. If I were going to start from scratch, I would reverse the rules in order and then remove all the quick keywords. But I&rsquo;m not..for now. Hope this helps others, please don&rsquo;t hesitate to contact me if you have questions.</p>
Please stop developing for CentOS 5https://www.gigofham.com/archives/8
Sun, 13 Mar 2011 05:37:14 +0000https://www.gigofham.com/archives/8<p>Good gravy, this has been the bane of my existence of recent. I get it, it&rsquo;s got &ldquo;Enterprise&rdquo; in the title and makes you compatible with RHEL. But, your kernel is SEVEN YEARS OLD. Hardware that was common place then is non-existent now and hardware that is common place now was poorly supported then (I&rsquo;m looking at you, bnx2). We have better options out there now: Ubuntu Server and Ubuntu Server LTS release on regular cycles and have much more recent kernels. The LTS versions even come with the same 5yr lifespan for security updates as well. I know RHEL6 just shipped a few months ago, but CentOS 6 has not and do you really want to be in the same boat again in a few years? On top of that, CentOS/RHEL 6 use a dead-end replacement for init that everyone else (save Fedora) has already moved away from. Don&rsquo;t get me wrong, I still like what RedHat as done and continues to do for the Linux community. I just wish they did it a little more frequently than every five years.</p>