Arm on The Gig of Hamhttps://www.gigofham.com/categories/arm/index.xml
Recent content in Arm on The Gig of HamHugo -- gohugo.ioen-usThis work is licensed under a Creative Commons Attribution 4.0 International License.Something better than a Rasberry Pi 2Q-2018 Editionhttps://www.gigofham.com/post/2018/05/23-better-than-RPi-2Q2018/
Wed, 23 May 2018 22:16:20 +0000https://www.gigofham.com/post/2018/05/23-better-than-RPi-2Q2018/<p>Ah, the ubiquitous Raspberry Pi. Loved and hated by many. I fall in both camps. I have at least 20 of them currently, and several are in production use. I also spend a lot of time with the larger ARM community, including the <a href="https://www.worksonarm.com/">Works on arm</a> project, and <a href="https://www.96boareds.org/openhours/">96boards Open Hours</a>. Something that comes up a lot is people wanting something &ldquo;better&rdquo; than a Raspberry Pi, and I finally decided to start writing up a comparison of boards I have used and like, that are generally available, and how they are better. The goal is to update this on a regular basis, at least once a year. There is some great stuff coming down the pipeline. But for now, let&rsquo;s jump in.</p>
<p></p>
<h1 id="raspberry-pi-s-current-state-of-the-art-the-3b">Raspberry Pi&rsquo;s current state of the art: The 3B+</h1>
<p>Before we jump into &ldquo;better&rdquo; let&rsquo;s talk about the gold standard from which everything is compared: The Raspberry Pi. The current generation is the <a href="https://www.raspberrypi.org/magpi/raspberry-pi-specs-benchmarks/">3B+</a>, which is a minor revision to the <a href="https://www.raspberrypi.org/magpi/raspberry-pi-3-specs-benchmarks/">3B</a> (which almost everyone refers to as the 3). The 3B and 3B+ are based on a quad-core Broadcom SoC that implements the ARMv8 instruction set using the Cortex-A53 IP. It also has 1GB of RAM, SD Storage, and USB based Ethernet. Both the 3B and the 3B+ also implement WiFi connectivity, but the WiFi on the 3B+ is the major improvement for that release. It&rsquo;s gone from 2.5GHz only to dual band, and updates the speed from the N standards into the AC standards.</p>
<h1 id="defining-better">Defining &ldquo;better&rdquo;</h1>
<p>So, now that we know what the state of the art with the Raspberry Pi is, let&rsquo;s talk about it&rsquo;s shortcomings. Where I think the RPi falls short is RAM and Ethernet. The 1GB of RAM is very limiting, and the USB based Ethernet has a myriad of limitations when it comes to throughput and latency. The WiFi is nice, but when running things like small clusters it&rsquo;s more of a burden than an asset. Another major issue, especially on the newer boards, is power delivery. The MicroUSB power connector is a major limiting factor as it&rsquo;s limited to about 2.5A@5VDC and the 3B+ can easily use more than this. Finding good power supplies and cables can also be an issue here. Ideally you could power the board off the 5VDC pins on the GPIO connector, but it turns out doing this bypasses all the voltage regulation and power conditioning circuitry, so now that&rsquo;s a problem you have to deal with as well.</p>
<p>So, how I am defining better is:</p>
<ul>
<li>Better power connector</li>
<li>Native Ethernet</li>
<li>More RAM</li>
</ul>
<p>Minimum requirements are:</p>
<ul>
<li>Must be ARMv8 based, minimum quad core</li>
<li>Must run Linux at a minimum</li>
<li>Must be &ldquo;available to purchase&rdquo; and not a pre-order, coming-soon, or discontinued.</li>
</ul>
<p>Not a terribly long list, but this is only my first go at this comparison. I&rsquo;m sure more requirements will be added as time goes on, please feel free to leave your thoughts in the comments at the bottom of the article.</p>
<h1 id="the-contenders">The Contenders</h1>
<p>Here are the contenders and the related specs in no particular order:</p>
<table>
<thead>
<tr>
<th>Name</th>
<th>Cores</th>
<th>RAM</th>
<th>Network</th>
<th>Storage</th>
<th>OS</th>
<th>Form Factor</th>
<th>Case</th>
<th>Approx Cost</th>
<th>Notes</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G145457216438">ODroid-C2</a></td>
<td>4xCortex-A53 (AMLogic S905)</td>
<td>2GB</td>
<td>1GbE Ethernet</td>
<td>MicroSD or eMMC</td>
<td>Vendor Ubuntu, AOSP, Armbian</td>
<td>RPi Compatible</td>
<td>Yes</td>
<td>$46</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.pine64.org/?page_id=7147">ROCK64</a></td>
<td>4xCortex-A53 (RockChip RK3328)</td>
<td>2/4GB</td>
<td>1GbE Ethernet</td>
<td>MicroSD, eMMC, SPI</td>
<td>Vendor Ubuntu</td>
<td>RPi Compatible</td>
<td>Yes</td>
<td>$35/$45</td>
<td></td>
</tr>
<tr>
<td><a href="http://en.t-firefly.com/product/Rk3399/index.html">Firefly-RK3399</a></td>
<td>4xCortex-A53,2xCortex-A72 (RockChip RK3399)</td>
<td>2/4GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD</td>
<td>Vendor Ubuntu</td>
<td>Custom</td>
<td>Yes</td>
<td>$199/$209</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.solid-run.com/marvell-armada-family/macchiatobin/">MACCHIATObin</a></td>
<td>4xCortex-A72 (Marvell ARMADA 8040)</td>
<td>1xDDR4 DIMM</td>
<td>2x10GbE, 1x2.5GbE, 1x1GbE</td>
<td>MicroSD, eMMC, SATA</td>
<td>Vendor SuSE, any UEFI Linux Installer</td>
<td>Custom</td>
<td>Yes</td>
<td>$269 and up</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.96boards.org/product/poplar-hoperun/">Poplar</a></td>
<td>4xCortex-A53 (HiSilicon Hi3798CV200)</td>
<td>2GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD</td>
<td>Any UEFI Linux Installer</td>
<td>96Boards-EE</td>
<td>????</td>
<td>$99</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.96boards.org/product/hikey970/">HiKey970</a></td>
<td>4xCortex-A73, 4xCortex-A53 (HiSilicon Kirin 970)</td>
<td>6GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD</td>
<td>Vendor Debian</td>
<td>96Boards-CE</td>
<td>????</td>
<td>$299</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.96boards.org/product/dragonboard820c/">DragonBoard 820c</a></td>
<td>4xKryo (Qualcomm Snapdragon 820E)</td>
<td>3GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD</td>
<td>Vendor Debian</td>
<td>96Boards-CE</td>
<td>????</td>
<td>$199</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.nvidia.com/en-us/autonomous-machines/embedded-systems-dev-kits-modules/">Jetson TX1 Kit</a></td>
<td>4xCortex-A57</td>
<td>4GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD, SATA</td>
<td>Vendor Ubuntu</td>
<td>Custom</td>
<td>????</td>
<td>$477</td>
<td></td>
</tr>
<tr>
<td><a href="https://www.nvidia.com/en-us/autonomous-machines/embedded-systems-dev-kits-modules/">Jetson TX2 Kit</a></td>
<td>4xCortex-A57</td>
<td>8GB</td>
<td>1GbE Ethernet</td>
<td>eMMC, MicroSD, SATA</td>
<td>Vendor Ubuntu</td>
<td>Custom</td>
<td>????</td>
<td>$570</td>
<td></td>
</tr>
</tbody>
</table>
<h1 id="those-who-did-not-make-the-cut-and-why">Those who did not make the cut, and why</h1>
<ul>
<li><a href="http://www.lenovator.com/product/103.html">LeMaker Cello</a> : No longer available</li>
<li><a href="https://www.pine64.org/?page_id=1194">Pine64</a> : Gigabit Ethernet doesn&rsquo;t work due to design flaw in board.</li>
</ul>
<h1 id="next-steps">Next steps</h1>
<p>So, as I said above, I would love to hear from your additional suggestions meeting the criteria listed above. Please comment below or reach out to me on Twitter. The more feedback I get, the easier it will be to update this as time goes on!</p>Bring up of the LeMaker Cellohttps://www.gigofham.com/post/2018/02/06-cello/
Tue, 06 Feb 2018 22:16:20 +0000https://www.gigofham.com/post/2018/02/06-cello/<p>Many, many moons ago I pre-ordered the <a href="http://www.lenovator.com/product/103.html">LeMaker Cello</a>. It&rsquo;s a 96boards Enterprise Edition Single Board Computer based on the AMD &ldquo;Seattle&rdquo; Opteron A1100 ARM Processor. I was very excited, because unlike many of the boards I already had it sported three key features: Native SATA ports, a full PCI Express 16x mechanical expansion port, and 2x SO-DIMM slots for memory. After many months of delays, we got some bad news: AMD had discontinued the processor and the PCI Express port didn&rsquo;t work. LeMaker offered those of us who pre-ordered to swap our pre-orders for a <a href="http://www.lenovator.com/product/80.html">HiKey 960</a> board instead based on this news, since that had a PCI Express expansion port. Unfortunately for me, the <a href="http://www.lenovator.com/product/80.html">HiKey 960</a> doesn&rsquo;t have gigabit ethernet or expandable RAM, so I stuck with my pre-order. I had planned to use them as a pair of Database servers for my hosting company, and the lack of PCI Express expansion wasn&rsquo;t a big deal. Several months later, the boards shipped and then my adventure began.
</p>
<h1 id="getting-the-boards">Getting the boards</h1>
<p>The boards arrived in terrible condition physically. I had ordered two, and they were bubble wrapped and then stacked on top of each other, placed in a makeshift cardboard box that barely contained them, and then handed to China Post. They suffered the damage one would expect when something like this is shipped with insufficient packaging (eBay buyers of electronics around the world know this pain). One of the boards appeared fine, the other had a corner chipped off and the heatsink/fan was ripped off the board in transit. I was able to find the <a href="https://www.96boards.org/documentation/enterprise/cello/quickstart/">quick start guide on the 96boards site</a> and get some of the specifications of components I needed to continue.</p>
<p>I knew a needed a power supply, some RAM, and possibly a few other accessories. Sadly, the power supplies linked from the quick start guide were not available to me or super expensive, so I went digging for some alternatives. Some searching on <a href="https://www.digikey.com/">DigiKey</a> yielded a <a href="https://www.digikey.com/products/en?keywords=62-1186-ND">Volgen KTPS90-1207</a> which was not an exact match, but was reasonably priced and so far has worked without issue. I&rsquo;m also running the board rather lightly loaded, just some RAM and a single SATA SSD. I was able to get the SATA power adapter from Amazon, and then went looking for RAM. Ideally I would have got a 4pin DIN power supply, but those were going to cost as much as the boards, so this 7A barrel connector power supply would have to do as a starting place.</p>
<p>Initially I foolishly bought some regular old SO-DIMM modules. They didn&rsquo;t appear to work, so I contacted LeMaker support. They were using the <a href="https://www.micron.com/parts/modules/ddr3-sdram/mt18ksf1g72hz-1g6">MT18KSF1G72HZ-1G6E2</a> modules from <a href="https://www.micron.com">Micron</a>. Turns out these are ECC, and specifically ECCx72 (which is not common). Some digging later and I found some <a href="http://www.memory4less.com/micron-8gb-sodimm-pc10600-mt18ksf1g72hz-1g4d1ze">8GB Micron modules from memory4less.com</a> that are compatible. I bought two of them, and waited for all the accessories to arrive.</p>
<h1 id="more-bad-news-from-lemaker">More bad news from LeMaker</h1>
<p>It was at this point that I got some more bad news from LeMaker: the boards may not boot. They believe the boards were shipped to me without the necessary firmware installed. The provided two options: I could upload the firmware my self, or send the boards back to LeMaker for repair. I had planned on flashing the board that appeared to be intact, and return the second.</p>
<h2 id="attempting-to-flash-the-firmware-on-the-board">Attempting to flash the firmware on the board</h2>
<p>LeMaker provided the necessary <a href="https://edolnx-public.objects-us-east-1.dream.io/cello/cello_platform_firmware.hex">firmware</a>, the target part (an Atmel AT24C512B at I2C Address 0x54), and this image of the I2C connector:</p>
<p><img src="https://edolnx-public.objects-us-east-1.dream.io/cello/8723352C%4009753F7%2805-09-14-29-52%29.jpg" alt="I2C pins next to the SATA power connector" /></p>
<p>I attempted to flash the part using a RaspberryPi, but was unable as I think the voltages were different or possibly pull-up resistors were necessary. Either way, I gave up and shipped both boards back and eagerly awaited their return.</p>
<h1 id="the-boards-arrive-part-2">The boards arrive, part 2</h1>
<p>A few weeks pass, and I now have known good boards. I get home, hook them up, and get: a spinning fan and LED7 turning on when I apply power but that is it. No output from the serial console device. I reach out to LeMaker and Lenovator for support, and get back that a wiki page is forthcoming. However, the board has been marked as sold out on their site and the word on the street is that AMD has killed the Seattle project and discontinued the processors.</p>
<p>Time passes.</p>
<p>I ask for updates, and reach out to see if anyone has got these boards to work on twitter and the Phoronix forums.</p>
<p>Nothing happens.</p>
<p>The boards sit in a drawer, essentially very expensive paper weights. I&rsquo;ve mostly lost hope that they will ever be more that museum artifacts.</p>
<h1 id="office-hours">Office Hours</h1>
<p>Several months, I get involved with the <a href="http://www.worksonarm.com/">Works on ARM</a> community, and ask there for help. They point me to the <a href="https://www.96boards.org/openhours/">96boards Open Hours</a> and after a couple of false starts there I&rsquo;m able to reach out and get some help. Robert from Open Hours gets me in touch with Ard and Ricardo who were engineers on the Cello project. I explain my delemma and learn an important step: on my Rev 003 boards there is another serial port on the board which tells you if the memory can be configured, and there is a power button (YMMV on other board revisions).</p>
<h1 id="the-scp-serial-port">The SCP Serial Port</h1>
<p>Turns out there is a microcontroller on the board that is responsible for setup of the DRAM and other components. It has a read only 6 pin FTDI serial header next to the PCI Express port, and it outputs lots of useful debugging information at 38400bps. I don&rsquo;t have an 6 pin FTDI cable lying around, but I have a lot of serial adapters and some jumpers, so I find the port and connect:</p>
<p><img src="https://edolnx-public.objects-us-east-1.dream.io/cello/IMG_20180207_125551_r.jpg" alt="6 pin FTDI header next to PCI Express connected to serial adapter" /></p>
<p>The green cable is ground and connected to the labeled pin 1. The yellow cable is connected to the RX pin on my serial adapter. I plug this into a Linux host, fire up <code>picocom</code>, and get the following output upon power on of the Cello board:</p>
<pre><code>SCP Bootrom version : 1.1004
SCP Bootloader version : 2.1001
BL2 POST code : 0x00000020
BL2 POST code : 0x00000021
BL2 POST code : 0x00000022
BL2 POST code : 0x00000025
BL2 POST code : 0x00000026
BL2 POST code : 0x00000023
BL2 POST code : 0x00000024
BL2 POST code : 0x00000027
........
System Control Processor Firmware
Version: 1.0.0.2 (ROD1002C_0493 R)
Advanced Micro Devices, Inc.
All rights reserved.
[INFO] Seattle Model B SOC detected. Now initializing.
[INFO] Interpreting EEPROM settings...
[INFO] EEPROM root table - ID: 0000000000000000, Major: 0, Minor: 152.
[INFO] EEPROM area - Sign: PFDT, Rev: 0, Offset: 0x00000070, Len: 19.
[INFO] EEPROM area - Sign: PMAT, Rev: 0, Offset: 0x00000000, Len: 0.
[INFO] EEPROM area - Sign: USAT, Rev: 0, Offset: 0x00000000, Len: 0.
[INFO] EEPROM area - Sign: EATL, Rev: 0, Offset: 0x00000000, Len: 0.
[INFO] EEPROM area tables found - 4.
[INFO] Area table: Platform and feature definition signature.
[INFO] Clock generator device set to I2C0/0x68, SpreadSpectrum=1.
[INFO] VRM device set to I2C0/0x40.
[INFO] EEPROM device set to I2C0/0x54, 64kB, Page size 128 bytes.
[INFO] Flash device set to SPI0, 16MB, Page size 256 bytes.
[INFO] MAC address 0 added: AA:BB:CC:DD:EE:01
[INFO] MAC address 1 added: AA:BB:CC:DD:EE:02
[INFO] RTC device set to I2C0/0x6f, format 1.
[WARN] Unrecognized table entry type: 0, sub type: 0.
[INFO] Area table: Platform management signature.
[INFO] Area table: UEFI/SCP shared area signature.
[INFO] Area table: End of area tables signature.
[INFO] Management device unconfigured, set to listen on I2C2/0x70.
[INFO] SOC executed from cold reset.
[INFO] ISCP Task started
[INFO] Default value of ACLOCK is 1700MHz
</code></pre>
<p>I cannot tell if this is good or bad, but it&rsquo;s the first sign of life I have seen on this board. Ard informs me that now all I need to do is press the &ldquo;power button&rdquo;. Which makes me say &ldquo;wait, what?&rdquo;</p>
<p>Turns out, these two buttons on the board are &ldquo;reset&rdquo; and (for lack of a better label) &ldquo;boot&rdquo;:</p>
<p><img src="https://edolnx-public.objects-us-east-1.dream.io/cello/IMG_20180207_125608_r.jpg" alt="Power button is closest to the USB connector, Reset closer towards the power connectors" /></p>
<p>The reset button will cold boot the board and cause LED7 to blink. The other button seemed to do nothing, but that starts the boot sequence. The SCP serial port starts showing me lots of secrets, and informs me that I cannot count:</p>
<pre><code>Memory Training Firmware
Version: 1.0.0.2
Memory Training for DDR3, one DIMM/channel
Memory Training ... start
[INFO] UEFI memory Setup fail safe counter = 10
********** Load default memory setup settings **********
SPD Channel 0 Dimm 0 SmbusAddr A0, DIMM absent
SPD Channel 1 Dimm 0 SmbusAddr A4, 18KSF1G72HZ-1G4D1 UDIMM ECC 2Rx8
92 11 0B 08 04 21 02 09 0B 11 01 08 0C 00 7E 00
69 78 69 30 69 11 20 89 20 08 3C 3C 00 F0 83 05
80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 0F 11 03 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 80 2C 01 12 26 B1 F7 CD B3 B1 93
31 38 4B 53 46 31 47 37 32 48 5A 2D 31 47 34 44
31 20 44 31 80 2C 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
********** Both channels must have the same memory size **********
******************************************
* *
* Memory Training Failed *
* *
******************************************
</code></pre>
<p>It turns out that you need 2 identical SO-DIMM modules installed on the board, of the same size, to boot. Not the end of the world, as I had two but it means I will need to order two more for the other board. So, I drop in the other module, re-apply power, press the boot button and wait&hellip;</p>
<p>You see, it takes about 3min for these boards to boot. The whole time it&rsquo;s spewing lots of interesting data over the SCP serial port, but it&rsquo;s obvious that microcontroller is running debug code and taking it&rsquo;s time. Eventually, I get a UEFI boot prompt on the console&rsquo;s USB serial port at 115200bps, and can start the GRUB application from the USB CDROM:</p>
<pre><code>UEFI Interactive Shell v2.1
EDK II
UEFI v2.60 (AMD Seattle, 0x00010000)
Mapping table
FS0: Alias(s):CD0d0a:;BLK1:
PcieRoot(0x0)/Pci(0x2,0x2)/Pci(0x0,0x0)/USB(0x3,0x0)/CDROM(0x0)
BLK2: Alias(s):
VenHw(0D51905B-B77E-452A-A2C0-ECA0CC8D514A,000030E00000000000)/Sata(0x
1,0xFFFF,0x0)
BLK0: Alias(s):
PcieRoot(0x0)/Pci(0x2,0x2)/Pci(0x0,0x0)/USB(0x3,0x0)
Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell&gt; FS0:
FS0:\&gt; cd efi
FS0:\efi\&gt; cd boot
FS0:\efi\boot\&gt; dir
Directory of: FS0:\efi\boot\
12/09/2017 13:56 &lt;DIR&gt; 2,048 .
12/09/2017 13:56 &lt;DIR&gt; 2,048 ..
12/09/2017 13:56 401,408 bootaa64.efi
1 File(s) 401,408 bytes
2 Dir(s)
FS0:\efi\boot\&gt; bootaa64.efi
</code></pre>
<h1 id="installing-an-os">Installing an OS</h1>
<p>After I stop jumping for joy, I realize I need an installer. I have this lovely Zalman ZM-VE200 (which is similar to <a href="http://zalman.com/contents/products/view.html?no=20">this device</a>), which lets me load an ISO on an SSD/HDD and present it as an USB CDROM. It&rsquo;s a lifesaver. So, I drop the stock <a href="https://cdimage.debian.org/debian-cd/current/arm64/iso-dvd/">Debian Stretch ARM64 DVD</a> on and plug it in. Then I tell the UEFI shell to reboot and&hellip; it works. The Debian installer comes up no problem. So, I power off the board, attach Ethernet and a brand new SATA SSD, and boot again.</p>
<p>This time, things are not as smooth. I use <code>picocom</code> as my default serial console of choice, and UEFI doesn&rsquo;t seem to like it. Ironically, <code>minicom</code> lets you get through the UEFI menus and prompts without issue, but then the Debian installer is terrible. A little more testing, and I figure out that the UEFI really wants an ANSI serial terminal, and GRUB and Debian prefers a linux/xterm/vt102 terminal. This is a simple fix for picocom where I can start with <code>TERM=ANSI picocom -b 115200 /dev/ttyUSB0</code> and later restart without the <code>TERM=ANSI</code> and all is well. I had very strange issues with <code>minicom</code>, which should come as a surprise to no one.</p>
<p>The Debian installer goes along without issue, finds the NIC, but can&rsquo;t DHCP. My instinct to grab the DVD instead of a netboot ISO pays off! I tell it to ignore the ethernet and figure I&rsquo;ll look at it later. SATA is detected without issue, install completes without issue. Rebooting however&hellip;.</p>
<h1 id="lack-of-console">Lack of console</h1>
<p>The Debian installer and/or GRUB2 figure out that the console is a serial port, and you can see the GRUB2 menu without issue. However, the Linux kernel seems to ignore the serial console. Ard tells me that the console should be <code>ttyAMA0</code>, so I edit the kernel command line in GRUB to remove the stupid <code>quiet</code> flag and add <code>console=ttyAMA0,115200n8</code>. This is when I learned that GRUB, too, wants an ANSI terminal. Now I can see the kernel messages, and then Debian starts up and gives me a login prompt! Holy cow! I have a fully installed and working Debian environment on the Cello. To make those changes permanent, you will need to modify the <code>/etc/defaults/grub</code> file and edit the line that starts with <code>GRUB_CMDLINE_LINUX_DEFAULT</code></p>
<h1 id="lack-of-ethernet">Lack of ethernet</h1>
<p>On to that pesky ethernet. The ethernet device is detected as enp2s0, but a keen reader may discover an issue here:</p>
<pre><code>cello2:~$ ip link list
1: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp2s0: &lt;BROADCAST,MULTICAST&gt; mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
</code></pre>
<p>Turns out the Ethernet MAC is missing it&rsquo;s address. This explains why the installer couldn&rsquo;t use DHCP. No the end of the world, an address can be specified in the <code>/etc/network/interfaces</code> file like so:</p>
<pre><code>auto enp2s0
iface enp2s0 inet dhcp
hwaddress ether 00:00:1A:00:1A:01
iface enp2s0 inet6 auto
</code></pre>
<p>Note that if you have multiple boards, those MAC addresses must be unique on each system. I used the AMD vendor code, and then made up the rest. You can use whatever you like.</p>
<p>I reached out to ask Ard about this, and he informed me that the tool Realtek (who makes the RTL8169 PHY device used on this board) only provides an x86 UEFI application to set these addresses. At the time LeMaker had no way of doing this, but now there is apparently an emulation layer that can be loaded into UEFI which allows running x86 PE/COFF code for tools like this. This isn&rsquo;t a big deal once the OS is installed, but can be a problem if you want to netboot or do a netinstall. In those cases, you would need a workaround:</p>
<pre><code>So we don't have the means to set the MAC address persistently,
unfortunately. What we do have is a firmware based driver that sets it
from the firmware at boot.
Please try the attached driver. You need to drop it into the ESP (the
FAT partition that has /efi/debian/grubaa64.efi in it), and enter the
following commands from the shell:
setvar -guid 8d97e056-777c-4850-ab61-8166b1777f2d MacOverride -nv -bs
=112233445566
where 112233445566 is the MAC address you want to assign (note that
copy paste may not work very well in the UEFI shell so if it doesn't
seem to work, please double check whether you got all the GUID digits
right). Then you can load the driver
load fs0:Realtek8169MacOverride.efi
and check whether it prints something like 'using MAC override value
11:22:33:44:55:66'
If that works, you can register the driver to be loaded automatically at boot
bcfg driver add 0 fs0:Realtek8169MacOverride.efi &quot;R8169MacOverride&quot;
and it should set the MAC address in the PCI config space at each boot.
</code></pre>
<p>Totally not ideal, but here is <a href="https://edolnx-public.objects-us-east-1.dream.io/cello/Realtek8169MacOverride.efi">the tool if anyone is interested</a>.</p>
<h1 id="next-steps">Next Steps</h1>
<p>So, at this point, everything is working. I had planned to deploy these to a data center to be database servers, but it&rsquo;s obvious that would be a terrible idea. These are very much development boards, and when something goes wrong will need button presses to fix. So, they are going to stay at my home lab instead. I&rsquo;m working on building an Open Source ARM CI pipeline where community members can get various software built on various armel, armhf, and arm64 platforms and distributions. Since these boards have Cortex-A57 cores, they can run ARMv6, ARMv7, and ARMv8 code. So my goal is to make them compile boxes, and then I have a suite of other machines to test on. But, before any of this some changes are going to be needed.</p>
<p>First, I need a case. While there are many 96boards CE cases around, I have yet to find an EE one. So, I&rsquo;m probably going to design a one that borrows heavily from <a href="http://wiki.t-firefly.com/index.php/Firefly-RK3399/Peripherals_%E5%A4%96%E5%A3%B3/en">The Firefly-RK3399 case</a> which is just some acrylic and stand-offs. It gets the job done. I&rsquo;m probably going to make one edge open so that if you have something like the <a href="https://www.96boards.org/product/poplar/">Poplar</a> board, you can use the PCI Express slot. Not so important for this board since the PCI Express slot is non-functional.</p>
<p>The second, the CPU fan on this board is small and very loud. My goal is with this case, I&rsquo;m going to remove that small fan and instead place a nice big 120mm standard PC case fan on top and have it cool the CPU and other components. The CPU cooler uses a standard PC Case 3pin fan connector, so this is a simple change once there is a place to mount it on the case.</p>
<p>Once I have the case design done, I will post here for others. It will be CC0 licensed.</p>
<h1 id="summary">Summary</h1>
<p>If you skipped all the text above, here is what you need to know:</p>
<ul>
<li>You will need an IDENTICAL PAIR of ECCx72 SO-DIMM modules, I&rsquo;m using <a href="http://www.memory4less.com/micron-8gb-sodimm-pc10600-mt18ksf1g72hz-1g4d1ze">8GB modules from memory4less.com</a></li>
<li>You will need a USB-TTL serial adapter to connect to the SCP FTDI header next to the PCI Express slot in order to see if the SCP manages to configure the memory and other hardware correctly. That serial port operates at 38400bps.</li>
<li>If you don&rsquo;t get output from the SCP, you may need to apply firmware to the board. See above, good luck, and let me know if you succeed.</li>
<li>If you do get output, then press the &ldquo;Boot&rdquo; button. You should get lots more output on the SCP and if memory configuration succeeds, it will boot and in several minutes you can interact with the UEFI console on the integrated USB serial port at 115200bps.</li>
</ul>
<p>Hope that helps others, feel free to leave comments below if you have questions or concerns. Thanks a bunch to Ard, Robert, and Ricardo from 96boards for their help getting my Cello boards up and running!</p>Quick update: Sub $200 Stratum 1 NTP Server with ODROID and Adafruit partshttps://www.gigofham.com/archives/73
Sat, 11 Jun 2016 22:19:34 +0000https://www.gigofham.com/archives/73<p>I was building another one of these, and found some bugs in my previous post. I&rsquo;ve corrected the post, the biggest being that the kernels in Armbian 5.10 and later include a 1-Wire interface in the DTB which conflicts with the 1PPS pin used by the GPS module. The solution is to either comment out the <code>w1</code> stanza in the device tree, or move it to another pin. I moved it to <code>GPIOY_7</code> which is two pins down on the same row of the connector. The DTS and DTB links in the previous post have been corrected, along with the new SHA256 checksum of the DTB itself. I also tried to build on an ODROID-C2 platform, but it&rsquo;s missing the GPIO 1PPS client in the kernel, so I&rsquo;ve gone back to just using ODROID-C1+ for now. I also found that if you do not have the I2C display on top of the GPS module, but the module is near a window and pointing up the GPS will lock. Pretty cool. Onward to deployment!</p>
Sub $200 Stratum 1 NTP Server with ODROID and Adafruit partshttps://www.gigofham.com/archives/70
Thu, 02 Jun 2016 01:23:20 +0000https://www.gigofham.com/archives/70
<p>Time is important. In modern computing, I would say time is second in importance only to good entropy. Unfortunately, getting accurate, reliable time is getting harder. The global NTP pool is under attack, and worse it&rsquo;s also being used as an amplification vector for DDOS attacks. Primarily because of those two reasons, I&rsquo;ve decided to block NTP traffic at the border of all Data Center. But this leave a problem: how to get time. The easy answer is an off-the-shelf NTP server that uses CDMA or GPS. The easiest is CDMA because it requires no external antenna. The problem is the major CDMA carriers have committed to turning off the CDMA networks in the next few years. So that leaves GPS. No big deal, until you try to get pricing information for a standalone GPS backed NTP server. Cheapest I could find was only rated at about 10 clients and costs well over three thousand dollars. So I kept digging.</p>
<p>The commercial products have some fancy features: Oven controlled oscillators, 10Mpps outputs, sometimes even rubidium oscillators. I don&rsquo;t really need any of that, I just need a descent NTP source that supports the 1PPS signal for synchronization. The 1PPS signal is important because you get messages from the GPS receiver once every second, including time. That message is long, and it&rsquo;s over a 9600bps serial link. That means it can have a variance well in the hundreds of miliseconds. That&rsquo;s too much. The 1PPS system in the Linux kernel, tied to the 1PPS signal from a GPS receiver gets you down into the 5 microsecond range for variance. Much better.</p>
<p>It turns out, gpsd does almost everything you need to be a stratum 1 NTP server. Pair that with NTPd or chrony and you&rsquo;re in great shape. Several people have done this with Raspberry Pi systems, but networking on the Pi is miserable since it&rsquo;s over a USB interface and now most of that work you did to eliminate variance is gone. Some people have also tried to do this with a BeagleBone based system, which has worked but is a bit hacky because the BeagleBone was never really as popular as the RPi. But, I was going in the right direction. Cheap board, simple GPS module, done and done.</p>
<p>Adafruit stocks and ships a number of lovely modules with a 1PPS out, and I started with their <a href="https://www.adafruit.com/products/746">Ultimate GPS breakout board</a> because it&rsquo;s the cheapest and most flexible. I then tried to tie that to various systems. First I got the module working and talking to gpsd over a USB Serial interface. Worked great. The serial interface devices I have lying around are not spectacular so I had no way to test the 1PPS. No big deal. I did have several <a href="http://www.linksprite.com/linksprite-pcduino3-nano/">pcDuino 3 Nano</a> boards, and Adafruit also ships a <a href="https://www.adafruit.com/products/1272">GPS module for the arduino family</a>. Cool. The levels are wrong, because an Arduino is 5V logic, and the pcDuino is 3.3 but you can <a href="http://store.linksprite.com/t-board-to-bridge-arduino-shield-to-pcduino-with-level-shifter/">get adapter boards</a>. I also wanted a <a href="https://www.adafruit.com/products/772">simple display</a> to show status so it could run headless (and also because the serial port would be consumed by the GPS module). So get all those parts in, assemble, and&hellip;nothing. Turns out the serial port on the pcDuino 3 Nano is a non-starter. Not wired correctly or something. OK, back to the drawing board.</p>
<p>While the RPi is out, there are a lot of boards that are &ldquo;RPi compatible&rdquo; in terms of form factor and the expansion ports. One such is the ODROID series from <a href="http://www.hardkernel.com">HardKernel</a>. I had a couple <a href="http://odroid.com/dokuwiki/doku.php?id=en:odroid-c2">ODROID-C2</a> boards, so I grabbed the Adafruit <a href="https://www.adafruit.com/products/2324">Ultimate GPS Hat</a> and a <a href="https://www.adafruit.com/products/1115">simple display hat</a> and tied them to the C2. Initially, there was much better success than with the pcDuino 3 nano. The C2 also has two serial ports, one for a consle and a second on the RPi connector. It could talk to the GPS reciever no problem, and gpsd got a lock and time. Next was 1PPS, and that&rsquo;s where I hit a brick wall. At the time, the C2 kernel did not support an IRQ for the GPIO pins (I&rsquo;m told it does now) so there was no way to use the pps-gpio driver in the kernel. Drat. However, the <a href="http://odroid.com/dokuwiki/doku.php?id=en:odroid-c1">ODROID-C1+</a> didn&rsquo;t have that problem so I ordered a pair of those and waited.</p>
<p>Finally got the boards in, tied everything together and had&hellip;minimal success. It took weeks to get the boards, so this project started as a October timeframe &ldquo;Hey! I have an idea!&rdquo; and I just got it working last week the way I want. The first issue was the image I was using, odrobian, didn&rsquo;t have the gpio-pps module and had problems booting on every MicroSD card I tried. Turns out, this is mostly a kernel and u-boot issue on this board. Switching to <a href="http://www.armbian.com">armbian</a> fixed all those (and it ships with the needed module).</p>
<p>The serial port on the C1+ has an interesting bug, in that it won&rsquo;t work unless you run a specific command:</p>
<p><code>stty -F /dev/ttyS2 9600</code></p>
<p>This forces the port to 9600bps and then everything is fine. Other tools can later change the baud rate, but for whatever reason gpsd cannot. Adding that line to /etc/rc.local solved the problem permanently.</p>
<p>The next issue is the 1PPS signal. The pps-gpio module uses the device tree to find which port the 1PPS signal should monitor. After a lot of research I found that the correct port for the Adafruit hat is GPIOY_3. This means you need to add the following stanza to the dts:</p>
<pre style="padding-left: 30px;">pps {
compatible = "pps-gpio";
gpios = "GPIOY_3";
};</pre>
<p>This conflicts with the 1-Wire interface defined earlier in the file (search for <code>w1</code> and comment out the whole stanza, or change the pin used to a different value like <code>GPIOY_8</code>). Or.. you can just download my compiled <a href="http://edolnx-public.objects-us-east-1.dream.io/ODROID-C1/3.10.101-odroidc1-meson8b_odroidc.dtb">dtb</a> that&rsquo;s a drop in replacement. (SHA256 sum: 21a16397fcd1248f2fa79ce94f567cd1b230bae4926c1b83bdaaad12bde82b6c ) I also have the source <a href="http://edolnx-public.objects-us-east-1.dream.io/ODROID-C1/3.10.101-odroidc1.dts">dts</a> available if you want to compare.</p>
<p>A few software tweaks, and it all works. I haven&rsquo;t finished the software for the display yet, but you can at least get time. Here&rsquo;s the HOWTO:</p>
<h1 id="building-a-odroid-c1-based-ntp-server-with-adafruit-ultimate-gps-hat-v3">Building a ODROID-C1+ based NTP server with Adafruit Ultimate GPS Hat v3</h1>
<h2 id="parts-list">Parts List:</h2>
<ul>
<li><a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G143703355573">ODROID-C1+</a> (an ODROID-C2 may work now, I haven&rsquo;t tested)</li>
<li>MicroSD card (I&rsquo;m using a Samsung EVO PRO 16GB, you don&rsquo;t need a lot of space) and USB adapter for it</li>
<li><a href="https://www.adafruit.com/products/2324">Adafruit Ultimate GPS Hat</a></li>
<li>CR1220 Battery for GPS Hat (allows GPS to retain state data for faster startup, buy locally)</li>
<li><a href="http://www.adafruit.com/products/851">Adafruit uFL GPS cable</a> (optional for external antenna)</li>
<li><a href="http://www.adafruit.com/products/960">Adafruit external GPS antenna</a> (optional, but highly recommended)</li>
<li><a href="https://www.adafruit.com/products/1115">Adafruit Blue&amp;White 16x2 LCD+Keypad Kit for Raspberry Pi</a> (optional)</li>
<li><a href="http://www.adafruit.com/products/2336">Adafruit brass standoffs for the hats</a> (optional, <a href="https://www.adafruit.com/products/2337">16mm variants</a> are great for GPS hat)</li>
<li><a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G134111972476">ODROID Power Adapter/Cable</a> (min 2A, don&rsquo;t use MicroUSB for power)</li>
<li><a href="https://www.adafruit.com/products/1979">Adafruit Raspberry Pi Extra Tall Stacking Header</a></li>
<li>Case (You can use the <a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G143730080763">ODROID one</a>, or modify <a href="https://www.adafruit.com/products/2256">this one from Adafruit</a> slightly for the power connector to work, or 3D print your own)</li>
<li><a href="http://armbian.com/">Armbian</a> Image</li>
<li><a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G134111883934">ODROID Serial Header and USB module</a> (you can&rsquo;t debug boot error messages on anything but the serial console)</li>
</ul>
<h2 id="software-used">Software used:</h2>
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/ODROID-C1/Armbian_5.10_Odroidc1_Debian_jessie_3.10.101.7z">Armbian 5.10</a> (beware, 5.11 has a busted kernel, but 5.12 seems to be fine)</li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/ODROID-C1/3.10.101-odroidc1-meson8b_odroidc.dtb">dtb</a> with the pps stanza for the 1PPS line from the GPS module</li>
<li>chrony (it&rsquo;s easier to integrate with gpsd than ntpd, more secure, and will keep things moving along during bad weather when you may not have GPS lock. It&rsquo;s also compatible with regular ntp clients.)</li>
<li>gpsd from Debian Testing (the version in Jessie has broken 1PPS support)</li>
</ul>
<h2 id="hardware-notes">Hardware notes:</h2>
<ul>
<li>The GPS antenna needs a good view of the sky. I&rsquo;ve had good luck just placing the external antenna on a window sill.</li>
<li>If you are going to add the I2C Display module, the external GPS antenna is a must because the onboard antenna will be obstructed</li>
<li>You may not want to put the whole device on a window, it could overheat with the sun shining on it directly. Use the external antenna instead and keep the rest of it cool and safe.</li>
<li>If your antenna is going to be outdoors, don&rsquo;t forget GPS compatible lightning arrestors!</li>
<li>The ODROID boards have a lovely heatsink on them, so the header bundled with the GPS hat will not work. Use the extra tall or stacking header and make sure there is clearance between the board and the heatsink. I placed the stacking header on the ODROID, and some business cards between the GPS Hat and the heatsink, then tacked both sides of the header with my soldering iron to ensure a gap. YMMV.</li>
</ul>
<h2 id="configuration">Configuration:</h2>
<ul>
<li>Flash the Armbian image onto the MicroSD card from a Linux box using a USB adapter (using dd)</li>
<li>Boot the device. Note the messages about a reboot, wait 5min for the reboot to occur.
<ul>
<li>If the reboot does not occur:
<ul>
<li>Login with the default root password of &ldquo;1234&rdquo; and follow the prompts to change the password and add a user account. Then, shutdown the system.</li>
<li>Place the MicroSD card some other Linux box over a USB device and run e2fsck and then resize2fs on the 2nd partition (ex: <code>sudo e2fsck -f /dev/sdb2 &amp;&amp; sudo resize2fs /dev/sdb2</code> )</li>
<li>Remove the card from the USB adapter, and place back into the ODROID and power up.</li>
<li>The system should boot and the root filesystem should be almost the same size of the MicroSD card used.</li>
</ul></li>
</ul></li>
<li>Login with the default root password of &ldquo;1234&rdquo; and follow the prompts to change the password and add a user account (if not already done)</li>
<li>Backup the existing dtb file: <code>sudo cp /boot/dtb/meson8b_odroidc.dtb /boot/dtb/meson8b_odroidc.dtb.orig</code></li>
<li>Download the custom dtb file above, and then place on the system as <code>/boot/dtb/meson8b_odroidc.dtb</code></li>
<li>Edit the file <code>/etc/rc.local</code> and add the line <code>stty -F /dev/ttyS2 9600</code> before the line <code>exit 0</code></li>
<li>Reboot</li>
<li>Login with the new user account from now on (it&rsquo;s good practice). You can also login over SSH for easier cut and paste.</li>
<li>Create a new file <code>/etc/apt/preferences.d/testing_unstable</code> with the following contents:</li>
</ul>
<pre style="padding-left: 30px;">Package: *
Pin: release a=stable
Pin-Priority: 700
Package: *
Pin: release a=testing
Pin-Priority: 650
Package: *
Pin: release a=unstable
Pin-Priority: 600</pre>
<ul>
<li>Create a new file <code>/etc/apt/sources.list.d/testing_unstable.list</code> with the following contents:</li>
</ul>
<pre style="padding-left: 30px;">deb http://httpredir.debian.org/debian testing main contrib non-free
deb-src http://httpredir.debian.org/debian testing main contrib non-free
deb http://httpredir.debian.org/debian/ unstable main contrib non-free
deb-src http://httpredir.debian.org/debian/ unstable main contrib non-free</pre>
<ul>
<li>Pin the current kernel image (it works, and the one currently available from armbian did not for me at all, causing a kernel panic at boot. You can try now, if it fails just repeat up to this step) with the following command: <code>sudo apt-mark hold linux-image-odroidc1</code></li>
<li>Get the new package list and perform a security update: `sudo apt-get update &amp;&amp; sudo apt-get dist-upgrade -y`</li>
<li>Install chrony and the prerequisites for gpsd (we have to do it this way to not confuse the package manager when we pull the newer gpsd): <code>sudo apt-get install chrony python-gtk2 python-cairo pps-tools</code></li>
<li>Install gpsd from Debian Testing: <code>apt-get install gpsd gpsd-clients -t testing</code></li>
<li>Stop the gpsd service: <code>sudo service gpsd stop</code></li>
<li><p>Edit the <code>/etc/default/gpsd</code> file changing the following values:</p>
<ul>
<li><pre>USBAUTO="false"</pre></li>
<li><pre>DEVICES="/dev/ttyS2 /dev/pps0"</pre></li>
<li><pre>GPSD_OPTIONS="-n"</pre></li>
</ul></li>
<li><p>Test the 1PPS link from the gps reciever using the command <code>sudo ppstest /dev/pps0</code></p></li>
<li><p>Modify the <code>/etc/chrony/chrony.conf</code> file to contain the following:</p></li>
</ul>
<pre style="padding-left: 30px;">refclock SHM 0 refid GPS precision 1e-1 offset 0.280 delay 0.2
refclock SOCK /var/run/chrony.pps0.sock refid PPS
keyfile /etc/chrony/chrony.keys
commandkey 1
driftfile /var/lib/chrony/chrony.drift
log tracking measurements statistics
logdir /var/log/chrony
maxupdateskew 100.0
dumponexit
dumpdir /var/lib/chrony
local stratum 10
allow 10/8
allow 192.168/16
allow 172.16/12
logchange 0.5
rtconutc</pre>
<ul>
<li>Restart chrony: <code>sudo service chrony restart</code></li>
<li>Next, we are going to pre-load the latest almanac into the GPS module so it will lock on MUCH quicker (for details, see the <a href="https://github.com/f5eng/mt3339-utils">github repo for the tools</a>). Run the following commands:</li>
</ul>
<pre style="padding-left: 30px;">wget https://github.com/f5eng/mt3339-utils/archive/gps.zip
unzip gps.zip
cd mt3339-utils-gps
eporetrieve MTK14.EPO /tmp/MTK14.EPO
epoinfo /tmp/MTK14.EPO
(You should get some formatted output and not an error)
epoloader @epoloader.conf /tmp/MTK14.EPO /dev/ttyS2</pre>
<ul>
<li>Edit the gpsd init script located at <code>/etc/init.d/gpsd</code>, and add the word &#8216;chrony&rsquo; to the Should-Start line.</li>
<li>Start the gpsd service</li>
<li>Check the gps status using gpsmon, it should lock within a few moments and show a time and the PPS block should have numbers. Type <code>quit</code> to exit.</li>
<li>Now you can check the chrony service to see if it&rsquo;s getting GPS information using <code>chronyc sources</code></li>
<li>That&rsquo;s it! You can adjust the allow lines in chrony.conf and deploy. Don&rsquo;t forget to give the device a static IP as well.</li>
</ul>
My new ARM goal: hostinghttps://www.gigofham.com/archives/64
Thu, 24 Mar 2016 22:19:48 +0000https://www.gigofham.com/archives/64
<p>I&rsquo;ve been doing a lot of work on various ARM things. A lot of it has been in support of making community builds of Chef for other folks, but I have a new goal: providing ARM nodes to the Internet at large as a hosting option.</p>
<p>I&rsquo;m not going to go into a lot of details here yet, as I&rsquo;m saving a lot of that content for my hosting company&rsquo;s blog. I&rsquo;ll announce it here when it goes live. There are a few things in the way, but A lot of the deep dive technical stuff will be posted here. So, now that I&rsquo;ve gotten that out of the way let&rsquo;s get into the usual ARM update:</p>
<h2 id="rk3368-8211-it-s-been-fun">RK3368 &#8211; It&rsquo;s been fun</h2>
<p>First the bad news: I&rsquo;m giving up on the RK3368 as an ARMv8 platform for now. Part of this is the kernel mess (but things are looking better for 4.6), but most of it is availability of hardware (low), the (relative) high cost of the hardware, the focus of Android on this SOC, and the common implementation of the serial port connected to the MMC/SD/TF slot. Finding RK3368 hardware is somewhat difficult, and when I do it&rsquo;s always an Android Media Player/STB. Getting these to run anything else is..really hard. Rockchip doesn&rsquo;t support upstream das u-boot, and while the #linux-rockchip community on freenode is making strides, they are focused on the much more available ARMv7 variants instead of the ARMv8 RK3368. It&rsquo;s understandable, and I&rsquo;m not at all upset. The reality of the situation is that the RK3368 based systems I can find are much more expensive than other ARMv8 solutions. That then just leaves the Serial Port on MMC/SD/TF slot issue, which is a real deal breaker for me. In a 100% remote environment, OOB console is required and for most things that means IPMI or Serial. IPMI is out on most ARM devices so that means serial. USB isn&rsquo;t great for primary storage (for a lot of reasons), and having the storage card share pins with the serial console just means it&rsquo;s a disaster waiting to happen. I had high hopes for the <a href="http://www.geekbox.tv/">Geekbox</a>, but even that turned out to be a real disappointment. The focus for this SOC is obviously Android, and that&rsquo;s fine, but it doesn&rsquo;t suit my needs at all.</p>
<h3 id="on-the-geekbox-http-www-geekbox-tv">On the <a href="http://www.geekbox.tv/">GeekBox</a></h3>
<p>I don&rsquo;t want to rag on them too hard, it&rsquo;s a lovely piece of kit. But it suffers from some shortcomings. Stock and availability of components being one of them. I&rsquo;ve placed several orders for them, and they never ship on the advertised time frame. The folks at Geek Buying are very nice, and very transparent about supply issues, but it&rsquo;s still been rather frustrating. Then there is the odd selection of accessories. The docking station is mostly a USB hub and easier exposure of the edge connector functions, but there is no case for it. The CPU fan accessory does nothing and doesn&rsquo;t have a reasonable mount even if you want to use it (you do, I&rsquo;ve melted the case by accident on one of mine doing compiles). The lack of case when the board is in the docking station caused me to loose another board due to static discharge. Then there is the software: android runs fine. No major issues there. The Linux side however, is a bit of a mess. It&rsquo;s a customized version of Ubuntu 14.04&rsquo;s unsupported arm64 port. Most vendors seem to be taking this route, and I don&rsquo;t understand why. The real problem: the video is not stable. The signal is solid (it&rsquo;s digital after all), but the picture has waves going up and down the screen on every monitor I&rsquo;ve tried. Android is fine, it has something to do with the Linux kernel they provide. Makes it very hard to even get the DHCP IP address and then drop out to a console. In fact, a FB console is not even supported. X or nothing. I chose nothing.</p>
<p>Again, this isn&rsquo;t a bad device, just doesn&rsquo;t suit my needs.</p>
<h2 id="the-odroid-c2-http-www-hardkernel-com-main-products-prdt-info-php-g-code-g145457216438">The <a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G145457216438">ODROID-C2</a></h2>
<p>This device has been a breath of fresh air. Also ARMv8 (based on an amlogic 905S SoC) but only 4 cores (compared to the RK3368&rsquo;s 8 cores). No onboard storage, you have to add a MicroSD/TF card. The best part: broad community support. This SoC just landed in 4.6 and the makers of the board (<a href="http://www.hardkernel.com/main/main.php">HardKernel</a> out of Republic of Korea) are working with Amlogic to get full support upstream in mainline. Right now, mine are still running their heavily patched 3.14 kernel &#8211; but the source and configs are readily available and adding support for things like LXC was very simple. I&rsquo;m running an &ldquo;odrobian&rdquo; install, which is a minimal Debain Jessie root on an ext4 filesystem and some kernel config tweaks for Docker support. I&rsquo;ve rolled my own kernel to add the rest of the configuration options needed for LXC and I&rsquo;m running that for my build and test nodes.</p>
<p>I also just deployed four of these into production: two for recursive name servers, and two for authoritative name servers. They are cheap ($40 from <a href="http://www.hardkernel.com/main/main.php">HardKernel</a>, and there are worldwide resellers) and they are case compatible with RPi B rev 2. They also appear to sip power. I picked up a <a href="https://www.adafruit.com/products/2690">USB Meter from Adafruit</a>, and I can&rsquo;t get them above 0.3A at 5V for most of my testing. Now, I don&rsquo;t entirely believe those numbers (this device has a very low sample rate), so I&rsquo;m going to do some more testing with a NI VirtualBench at TechShop this weekend to see if I can learn more. I&rsquo;ll post more details on that once I have them. For now, I&rsquo;m being ultra conservative and deploying with a <a href="http://smile.amazon.com/dp/B011LSGMT2">PSU that can support six devices up to 2A@5v each</a>.</p>
<p>So far, these boards are great. Biggest thing to be aware of: when you buy one, get their power cord. It&rsquo;s a 2.5mm barrel connector (HardKernel makes and sells <a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G138960965859">pigtails</a> and <a href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G141637559827">USB adapters</a>). Those connectors are apparently common everywhere but here.</p>
<h3 id="on-power">On Power</h3>
<p>The crappy cell phone chargers we use for MicroUSB power everywhere are usually not enough for most of these devices (I&rsquo;ve killed more than a couple RPi boards this way), so get a good solid power supply for more than $3. The ones sold by <a href="https://www.sparkfun.com/categories/28">SparkFun</a> and <a href="https://www.adafruit.com/category/44">Adafruit</a> are well tested and good if you need basic MicroUSB power, they also offer good all around 5V supplies for general use. If you have several devices, <a href="http://www.ikea.com/us/en/catalog/products/10291881/">IKEA makes an inexpensive 3 port USB charge</a>r that I&rsquo;ve been using without issue, or the <a href="http://smile.amazon.com/dp/B011LSGMT2">Anear power adapter</a> I&rsquo;m using in a few places with up to six devices pulling 2.4A@5v each seems to be working very well also.</p>
<h2 id="pcdunio3-nano-http-www-linksprite-com-linksprite-pcduino3-nano"><a href="http://www.linksprite.com/linksprite-pcduino3-nano/">PcDunio3 Nano</a></h2>
<p>I&rsquo;ve recently pulled mine out of mothballs for a few reasons:</p>
<ol>
<li>They run a mainline kernel now, so they are great for a serial console server which cheap USB-Serial adapters</li>
<li>I need an inexpensive but reasonably accurate NTP server fed from GPS</li>
</ol>
<p>These are descent little boards, with a Dual Core ARMv7 Allwinner A20 proc and 1GB of RAM. They also have onboard eMMC, but the TF/MicroSD is better supported. They also have native 1Gb Ethernet (instead of a USB interface like the RPi products).</p>
<p><a href="http://armbian.com/">The Armbian project</a> supports these (along with most commonly available Allwinner based boards), with a descent Jessie rootfs and either a recompiled vendor kernel or a vanilla upstream kernel. I&rsquo;m using the vanilla for my console server, and the vendor kernel for the GPS stuff for now. I want to move to the vanilla kernel for that as well, but there are Device Tree issues to work out.</p>
<p>These are also neat because they have Arduino compatible headers, but at a different voltage (3.3v vs the normal Ardunio of 5v). The good news is that there is a <a href="http://store.cutedigi.com/t-board-to-bridge-arduino-shield-to-pcduino-v1-with-level-shifter/">level conversion shield</a>, the bad news is that it doesn&rsquo;t quite fit on the nano (the Ethernet jack is in the way), so I&rsquo;ve needed to get some header extensions to plug it in completely. Once I get the headers extensions, I plan on stacking an <a href="http://adafru.it/1272">Adafruit GPS Logger Shield</a> w/ Module and then a <a href="http://adafru.it/715">Adafruit I2C Controlled Keypad and display</a> shield on top. I&rsquo;ve already tested that the GPS module is compatible with gpsd, and with a vanilla kernel I should be able to get the 1PPS off the shield and into gpsd and ntpd for an inexpensive Stratum 1 time source. I&rsquo;ll update here will the full build out and configuration once I&rsquo;ve got it all/mostly working.</p>
<h2 id="pine64">Pine64</h2>
<p>I don&rsquo;t have any yet, but I&rsquo;m excited. Quad Core ARMv8, 2GB RAM, under $30. I ordered 22 of these through their Kickstarter. I hope to have them by the end of the month, and they will be the basis for Gen 1 of my hosting product. I&rsquo;ll be talking a lot more about these once they arrive and I start testing and design for support hardware.</p>
<h2 id="huskyboard-lemaker-cello">Huskyboard / LeMaker Cello</h2>
<p>I&rsquo;m really hoping that 2016 is the year of AMD&rsquo;s comeback for many reasons. Either way, the A series Opteron has always caught my eye. The <a href="http://www.96boards.org/products/ee/huskyboard/">Huskyboard</a> is still &ldquo;Soon&rdquo; but the <a href="http://www.lenovator.com/product/103.html">LeMaker Cello</a> is shipping in the next couple months, looks to be nearly identical specs wise, and comes in at $299. I&rsquo;ve ordered two for testing and might use one to be my new mail server since I can add 2 SSDs over SATA to them. Expect more on these in the future as well. I just wish either of them shipped with headers for the second onboard gigabit NIC.</p>
<h2 id="on-sd-cards">On SD Cards</h2>
<p>Typically, I&rsquo;ve been buying the cheapest 16G SD cards I can get my hands on from Fry&rsquo;s. These are usually name brand (PNY, Toshiba, Patriot, etc) and they have been &ldquo;reasonable&rdquo;. With the goal of making a commercial hosting product with these, &ldquo;reasonable&rdquo; probably won&rsquo;t fly with paying customers. So, I did some research and everything was pointing in one direction: Samsung EVO. So, I went back to Fry&rsquo;s and run some tests with iozone3. I&rsquo;m including the results below:</p>
<ul>
<li>Toshiba 16GB on PcDuino 3 Nano Armbian 5.0
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/a20-toshiba16.xls">iozone3 xls</a> (was done over SSH, may need to redo)</li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/a20-toshiba16.log">iozone3 log</a> (was done over SSH, may need to redo)</li>
</ul></li>
<li>Samsung EVO 64GB on PcDuino 3 Nano Armbian 5.0
<ul>
<li>missing files, will re-run and update once I get another EVO 64GB</li>
</ul></li>
<li>Samsung EVO+ 32GB on PcDuino 3 Nano Armbian 5.0
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/a20-evoplus32.xls">iozone3 xls</a></li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/a20-evoplus32.log">iozone3 log</a></li>
</ul></li>
<li>Toshiba 16GB on ODROIC-C2 odrobian Jessie (kernel3.14.29-14-odrobian+)
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-toshiba16.xls">iozone3 xls</a></li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-toshiba16.log">iozone3 log</a></li>
</ul></li>
<li>Samsung EVO 64GB on ODROIC-C2 odrobian Jessie (kernel3.14.29-14-odrobian+)
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-evo64.xls">iozone3 xls</a></li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-evo64.log">iozone3 log</a></li>
</ul></li>
<li>Samsung EVO+ 32GB on ODROIC-C2 odrobian Jessie (kernel3.14.29-14-odrobian+)
<ul>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-evoplus32.xls">iozone3 xls</a></li>
<li><a href="http://edolnx-public.objects-us-east-1.dream.io/arm-sd-performance/905s-evoplus32.log">iozone3 log</a></li>
</ul></li>
</ul>
<p>All of the tests were run on the device while connected over a serial console, using the default EXT4 filesystem on the SD card directly. No other activity was being run on these devices at the time, but they were connected to a local LAN with (no WiFi). Here is the exact iozone3 command used:</p>
<pre style="padding-left: 30px;">iozone -az -i 0 -i 1 -e -R -+T -b ~/output.xls | tee ~/output.log</pre>
<p>Power wise, it looks like the Samsung EVO is the big winner. It appeared to consume less power than the other cards. Again, my power measurements are inaccurate and I hope to address that this weekend. Beyond that, the Toshiba is the clear looser but there doesn&rsquo;t seem to be much difference between the EVO and EVO+ on these boards (probably due to limitations of the MMC interface of the SOC). So, I&rsquo;m sticking with the EVO cards when I care about performance. I also don&rsquo;t have a 32GB EVO card or a 32GB Toshiba to do a direct 1:1 comparison. I hope to fix that before I get the Pine64 boards. The price/performance curve definitely has the EVO in it&rsquo;s favor, where the bulk of the 64GB MicroSD cards on Amazon are (as of 24MAR2016) around $15, the <a href="http://smile.amazon.com/dp/B00IVPU7AO">EVO is $20</a> and the <a href="http://smile.amazon.com/dp/B01273JZMG">EVO+ is $25</a>. These tests show that the EVO+ doesn&rsquo;t appear to be worth the extra 20% price increase for these devices.</p>
<h2 id="phew">Phew</h2>
<p>I think that&rsquo;s a good update for now, I&rsquo;ll post more as I have it!</p>
Mainline Linux 4.4 kernel on a Jetson TK1https://www.gigofham.com/archives/62
Fri, 12 Feb 2016 18:07:01 +0000https://www.gigofham.com/archives/62
<p>I was really hoping this would be easier than it was. Most (all?) of the pieces needed are upstream, with a minor patch for das U-boot needed to make it all work with L4T. I&rsquo;m going to provide links to my u-boot images and kernel packages, but first I wanted to go over what I&rsquo;ve done and why.</p>
<p>My Jetson TK1 still has L4T installed on the internal eMMC device. I used that to debug and build a few minor tools. My target was Debian 8 on an SSD attached to the SATA port. In order to make life easier, I just copy the boot files (kernel, initramfs, dtb) to the eMMC:/boot directory and then add an entry to the extlinux.conf to boot them.</p>
<p>Known issues:</p>
<ul>
<li>It takes about 30s for the kernel, SATA interface, and my SSD to get their act together on boot. I had the same problem when trying to mount the SSD from L4T. I had hoped a new kernel would help this, it did not. Everything works, and it&rsquo;s stable but you get a LOT of SATA error messages on boot once it tries to mount the filesystem from the SATA SSD.</li>
<li>I didn&rsquo;t bother to try and bring the binary nvidia drivers over, I&rsquo;m just using the open source stuff (this is a compile box for me and runs headless anyway)</li>
<li>No network boot support in das U-boot (this is a mystery to me as to how to fix this, if anyone knows please reach out)</li>
</ul>
<p>Why the heck did I even build my own kernel? An excellent question, it was mostly to satisfy these requirements:</p>
<ul>
<li>XFS filesystem support</li>
<li>systemd support (the L4T kernel will fail to boot Debian 8 properly)</li>
<li>LXC support (running build and test targets in containers, and many of the required features were not in 3.10)</li>
<li>LVM support</li>
<li>More advanced networking options (nftables, etc)</li>
</ul>
<p>Getting a kernel build was interesting, and learning what features were required for LXC was also more involved than I would have liked. I also learned from the #tegra channel on freenode that I was also going to need to update das U-boot as well. So, let&rsquo;s get started.</p>
<h2 id="das-u-boot">Das U-Boot</h2>
<p>I built against the 2016.01 release. Pretty much everything is built in, and you can generate a mostly working build with the following command from the official source tree:</p>
<pre style="padding-left: 30px;">make defconfig_jetson-tk1</pre>
<p>However, this build will fail to boot L4T or any of the default nvidia 3.10 kernels because the command line length is too short (512B allowed, the default command line is 518B). So, this is the change I made (this patch will not apply with a cut and paste due to formatting issues, I&rsquo;ve linked it below in the files section):</p>
<pre style="padding-left: 30px;">diff --git a/include/configs/tegra-common.h b/include/configs/tegra-common.h
index ba819c4..b448b4b 100644
--- a/include/configs/tegra-common.h
+++ b/include/configs/tegra-common.h
@@ -76,7 +76,7 @@
* Increasing the size of the IO buffer as default nfsargs size is more
* than 256 and so it is not possible to edit it
*/
-#define CONFIG_SYS_CBSIZE (256 * 2) /* Console I/O Buffer Size */
+#define CONFIG_SYS_CBSIZE (256 * 4) /* Console I/O Buffer Size */
/* Print Buffer Size */
#define CONFIG_SYS_PBSIZE (CONFIG_SYS_CBSIZE + \
sizeof(CONFIG_SYS_PROMPT) + 16</pre>
<p>At that point, everything started to work. I was able to generate and flash the image to the Jetson TK1 board using the tegra-uboot-flasher-scripts provided by nvidia on a separate host. Follow the directions there, and know that you will need the Das U-Boot source, and the cboot-image-configs from nvidia, a device-tree-compiler, and a cross-compiler toolchain if the other host is also not an ARM. On a Debain 8 amd64 host, I installed these packages and used these commands:</p>
<pre style="padding-left: 30px;">sudo apt-get install gcc-arm-none-eabi binutils-arm-none-eabi device-tree-compiler
export CROSS_COMPILE="arm-none-eabi-"
git clone git://git.denx.de/u-boot.git
git clone https://github.com/NVIDIA/tegra-uboot-flasher-scripts.git
git clone https://github.com/NVIDIA/cbootimage-configs.git
git clone https://github.com/NVIDIA/tegrarcm.git
cd u-boot
git checkout tag/v2016.01
patch -p1 &lt; ~/u-boot-tegra-commandline.patch
cd ../tegra-uboot-flasher-scripts
./build --socs tegra124 --boards jetson-tk1 build
sudo ./tegra-uboot-flasher flash jetson-tk1</pre>
<p>The resulting image is linked below if you want to skip all that crap and just flash it directly.</p>
<h2 id="linux-kernel-4-4">Linux Kernel 4.4</h2>
<p>At the time I built it, the 4.4 kernel was the latest stable. Getting started was similar to Das U-Boot:</p>
<pre style="padding-left: 30px;">make defconfig_tegra</pre>
<p>Gets you to a state where most of the options you need to just get the darned thing to boot are turned on. I also used the <a href="http://lxc.sourceforge.net/man/lxc.html">LXC Kernel Requirements</a> list and the lxc-checkconfig tool to ensure all the options I needed for LXC were turned on. There were many more than I expected. It also turns out you need to pass a command line argument of &ldquo;cgroup_enable=memory&rdquo; in order for everything to work correctly. Along with a few other tweaks, this was my final kernel config, which generated the following DEB using make-kpkg. You also need to &ldquo;make dtb&rdquo; in the kernel directory to get the required dtb for booting out. Then I copied the kernel, generated initramfs, and dtb to the eMMC partition and added the following kernel config to eMMC:/boot/extlinux/extlinux.conf</p>
<pre style="padding-left: 30px;">LABEL debian
MENU LABEL debian 8 on sda1 with 4.4 kernel and ramdisk
LINUX /boot/vmlinuz-4.4.0
FDT /boot/dtb/tegra124-jetson-tk1.dtb
APPEND initrd=/boot/initrd.img-4.4.0 console=ttyS0,115200n8 console=tty1 ignore_loglevel earlyprintk cgroup_enable=memory root=/dev/sda1</pre>
<h2 id="the-files">The files</h2>
<p>Here are all the files needed and their SHA256 checksums:</p>
<pre style="padding-left: 30px;">f7ffbfe4f3a10b6c91f41963957360190c0102daafc731f1d805950c5fdd9cda <a href="http://edolnx-public.objects-us-east-1.dream.io/Jetson-TK1/tegra124-jetson-tk1.dtb">tegra124-jetson-tk1.dtb</a>
a96b9a0e779f0f1abe4e5f85e146ef9a71c409ebfbb937a00eb8a2c303d0ab16 <a href="http://edolnx-public.objects-us-east-1.dream.io/Jetson-TK1/linux-image-4.4.0_5_armhf.deb">linux-image-4.4.0_5_armhf.deb</a>
843ae0cce792966395083fbd7590993410d94f8bd34e07953ce236239a0e5323 <a href="http://edolnx-public.objects-us-east-1.dream.io/Jetson-TK1/config-4.4.0">config-4.4.0
</a>e29210c3fef917f8c89a83fb00b43f80a9c8b9ede46284a3b5c8421805f9a57a <a href="http://edolnx-public.objects-us-east-1.dream.io/Jetson-TK1/jetson-tk1-emmc.img">jetson-tk1-emmc.img</a>
75bfde768022e83e5995000291ff97217d11cd6e59673a4124d881992d2edd47 <a href="http://edolnx-public.objects-us-east-1.dream.io/Jetson-TK1/u-boot-tegra-commandline.patch">u-boot-tegra-commandline.patch</a>
</pre>
<p>Hope this helps others trying to build a mainline kernel for the Jetson TK1. Please feel free to contact me if you have questions!</p>
The very overdue ARM updatehttps://www.gigofham.com/archives/60
Thu, 28 Jan 2016 01:04:42 +0000https://www.gigofham.com/archives/60
<p>Lots to get to, so I&rsquo;ll just jump right in:</p>
<h2 id="uppel-cx-r8-http-edolnx-public-objects-us-east-1-dream-io-cx-r8-index-html"><a href="http://edolnx-public.objects-us-east-1.dream.io/CX-R8/index.html">Uppel CX-R8</a></h2>
<p>I gave up on this device a bit too soon it seems. Someone found these blog posts and reached out about rooting a CX-R8. While I wasn&rsquo;t able to help him much, he helped me greatly by providing links to a TWRP recovery image which makes it super simple to install a SuperSU update.zip file to the system allowing root access. Very cool. I&rsquo;ve linked the images to my files page (available at the link above). Downloading it this way is a lot easier than the forums and EXE wrapped downloaders I found.</p>
<p>That&rsquo;s about all I have for the CX-R8. I have mine in use as a media player at a friend&rsquo;s house with Android at this point because the bootloader was such a pain in the butt to use.</p>
<h2 id="rikomagic-mk68">Rikomagic MK68</h2>
<p>No major updates here either. I&rsquo;ve also been using this as a media player locally, but since retired it because it&rsquo;s having all sorts of stuttering problems even with just MP3 playback through Kodi. I&rsquo;m betting it&rsquo;s got something to do with the 3 billion plugins they install at the factory for me. Haven&rsquo;t cleared it out yet to see if that helps.</p>
<h2 id="cubieboard-4">Cubieboard 4</h2>
<p>Supposedly shipped months ago. I&rsquo;ve not seen it, never got a tracking number (shipping notification, but no tracking) and the seller is not responding to email. So much for that $200.</p>
<h2 id="geekbox">Geekbox</h2>
<p>I was pretty excited about this, and have it partially working. The major problem is the Android kernel is TERRIBLE for everything except Android and piece of hardware seems to ship with it. Some but not all of the drivers for the RK3368 are in the mainline 4.4 kernel, maybe enough to do what I want (ethernet and serial console, don&rsquo;t care about video at all) but I&rsquo;m not sure. The other issue is the U-boot image everyone ships with is garbage as well, supporting only the Android kernel format of a signed image written directly to a mmc partition. The good news is that unlike all the other RK3368 boards and images I have found so far, the Geekbox folks post EVERYTHING to <a href="https://github.com/geekboxzone">Github</a>. So, I&rsquo;m going to try and forward port the U-boot code for the RK3368 to the current mainline and see what happens. Then I might be able to use the PXELINUX/EXTLINUX/ISOLINUX style config file to move between kernels which would be a complete joy. For now, I&rsquo;m trying to build Chef images in a chroot just to get something out the door. To make matters worse, I bought two geekbox systems in order to build on one while hacking around with the other. Having only a single CX-R8 taught me that having a single box you can login to is non-delightful. Well, the second geekbox was the victim of an unfortunate static discharge accident and is now toast. So I&rsquo;m waiting for a replacement to ship from China, and hoping this one won&rsquo;t take two months to arrive.</p>
<h2 id="jetson-tk1">Jetson TK1</h2>
<p>I&rsquo;m happy to report that I have finally achieved what I wanted to build on an ARM board. It took a lot of work but the TK1 is finally doing what I want: a Linux 4.4 kernel on Debian 8 armhf providing LXC containers for various distros to build and test on. Getting here was hard. Like every other board, this too ships with an Android configuration highly modified 3.10 kernel and an ancient U-boot. Building a new kernel worked, but it would not boot. Folks on the #tegra IRC channel suggested upgrading U-boot. Again, only have the single TK1 (assumed the Cubieboard would have arrived by now) and was compiling on the TK1 for the TK1 (which avoided cross-compiles but was not smart).</p>
<p>I thought U-boot was hard to screw up, and I was very wrong. The biggest issue was the upstream code has a single configuration entry that is incompatible with the default kernel: the length of the command line is limited to 512B and the default command line is well longer than that. I wound up running around looking at definitions of CONFIG_SYS_CBSIZE and finally found you had to change one in particular to solve the problem:</p>
<pre style="padding-left: 30px;">diff --git a/include/configs/tegra-common.h b/include/configs/tegra-common.h
index ba819c4..b448b4b 100644
--- a/include/configs/tegra-common.h
+++ b/include/configs/tegra-common.h
@@ -76,7 +76,7 @@
* Increasing the size of the IO buffer as default nfsargs size is more
* than 256 and so it is not possible to edit it
*/
-#define CONFIG_SYS_CBSIZE (256 * 2) /* Console I/O Buffer Size */
+#define CONFIG_SYS_CBSIZE (256 * 4) /* Console I/O Buffer Size */
/* Print Buffer Size */
#define CONFIG_SYS_PBSIZE (CONFIG_SYS_CBSIZE + \
sizeof(CONFIG_SYS_PROMPT) + 16)</pre>
<p>After that, it was much easier. Having a command line of 1024 bytes meant the stock L4T install would boot and I could debug correctly. I also didn&rsquo;t realize that LXC required about 20 kernel configuration options, but was able to get that all straighted out. I&rsquo;ll post my kernel config, U-boot config, kernel package, and extlinux.conf in a few days. With that all configured, that box is humming along pretty well. Booting off of SATA is strage with the Patriot BLAZE SSD I have installed, it takes about 20sec of SATA errors on boot and then works just fine. Not sure if that is a Tegra bug, SSD bug, Kernel bug, or what. The 3.10 kernel exhibited the same behavior. I&rsquo;ve been able to build Chef clients for Debian 7 and 8 for the armhf and armel architectures. Next is to add Ubuntu 14.04 ARMv7 support, and then wrap it all up in a CI pipeline.</p>
<h2 id="more-to-come">More to come</h2>
<p>That&rsquo;s about it for now. The TK1 is not responding to SSH and the RPi I was going to use as a term server for it is segfaulting all over the place. I guess I&rsquo;ll replace that with a RPi2 shortly to make that issue go away. Once I can get back into the base OS, I&rsquo;ll post all of the packages and configs for general consumption. I still don&rsquo;t have PXE booting working with my U-boot but it&rsquo;s not the end of the world. USB, SATA, and MMC all work so there are options. On top of all of that GitHub is down at the moment, so I can&rsquo;t do much until that comes back up. I&rsquo;m at PAX South this weekend, but expect to post another update during next week sometime.</p>
Hacking the Uppel/Sunchip CX-R8, Rikomagic MK68, and Tegra Jetson TK1https://www.gigofham.com/archives/51
Sat, 05 Dec 2015 21:48:36 +0000https://www.gigofham.com/archives/51
<p>Long overdue follow-up from my previous post. Let me dive in by topic:</p>
<h2 id="uppel-sunchip-cx-r8-http-edolnx-public-objects-us-east-1-dream-io-cx-r8-index-html"><a href="http://edolnx-public.objects-us-east-1.dream.io/CX-R8/index.html">Uppel/Sunchip CX-R8</a></h2>
<p>After a lot of roadblocks, I&rsquo;ve given up on this particular platform. The primary reason for this was getting the MK68 boxes and being able to compare the two. The CX-R8 is pretty hacker un-friendly. The bootloader does some form of signing the boot, recovery, and kernel images. All of the existing Rockchip tools I could find to make these images say the ones from the device have bad checksums, and the images it creates the device reports as having bad checksums. I wound up putting the stock image back on and giving this device to a friend as a media player.</p>
<h2 id="rikomagic-mk68-http-edolnx-public-objects-us-east-1-dream-io-rkm-mk68-index-html"><a href="http://edolnx-public.objects-us-east-1.dream.io/RKM-MK68/index.html">Rikomagic MK68</a></h2>
<p>Out of the box, this device is much more hacker friendly. To start with SuperSU is pre-installed in the factory image, so the device is pre-rooted. Very handy for getting files on and off. On the hardware front, there is some bad news. Looking at the top of the board there is a set of holes that look like the serial port on the CX-R8. Sadly, they are not. Looking at the &ldquo;<a href="http://edolnx-public.objects-us-east-1.dream.io/RKM-MK68/DSC_9582.JPG">Top of board&rdquo; image I posted</a>, the port is on the far left hand side next to the processor heatsink. The pin on the left edge of the board is Ground, and the pin closest to the heatsink is +3.3v. The two in the middle? No idea, and I don&rsquo;t have a scope to probe them properly. However, the UART2 serial port is shared with the SD card, so I was able to find that the RX pin is the 5th pad from the bottom and the TX pin is the 3rd pad from the top on this image. Also interesting: unlike the CX-R8, the OTG port is the USB port on the back of the device instead of the one next to the MicroSD slot. Also unlike the CX-R8, this is clearly stated in the manual (!!), and it came with a USB A-A cable for flashing (!!). The rkflashtool programs worked as expected, and I was able to put a stock Debian 8 initrd with no kernel modules in it, adjust the params file to my USB root filesystem, and watch it boot into Debian 8. I was able to make a USB drive with a arm64 root filesystem using the <a href="https://wiki.ubuntu.com/ARM/RootfsFromScratch/QemuDebootstrap">new debootstrap with qemu-static support</a>. Very handy that.</p>
<p>However, now the bad news. Since I was using the kernel that ships with the device (in order to ensure the hardware is correctly supported) and that doesn&rsquo;t have the correct hooks for f**king systemd. So, systemd fails to find the Ethernet device or the serial ports and thus no way to login to the device. I can think of two ways of working around this: build a new kernel or move off of systemd. Moving off of systemd on Debian 8 doesn&rsquo;t look like an option, and Debian 7 didn&rsquo;t ship with arm64 packages. There may be an arm64 Debian 7 repo around somewhere, but I haven&rsquo;t found it yet (I also haven&rsquo;t looked very hard). The second option is to build a new kernel. This leads to a couple of interesting issues. I haven&rsquo;t found the RK3386 patched 3.10 kernel to just make some quick changes to the existing config, and it doesn&rsquo;t look like they left the /proc/config.gz option on. The latest 4.3 kernel does look like it has enough support for the SoC and the Ethernet in order to get that all working, which is all I really need (no video is expected and that&rsquo;s fine). However, building a kernel is not something I have done in a very long time and cross-compiling a kernel is something that I haven&rsquo;t done in over a decade. Needless to say, that hasn&rsquo;t gone anywhere very fast. Cross-compiling an arm64 kernel has been a complete disaster for me.</p>
<p>In brighter news: one of the nice folks in the #linux-rockchip channel (naobsd as I recall) pointed me at the <a href="http://www.geekbuying.com/item/GeekBox-Open-Source-Cross-TV-BOX-Android-Ubuntu-Dual-Boot-4K-RK3368-Octa-Core-2G-16G-AC-WIFI-1000M-LAN-BT4-1-HDMI2-0-OTG-358067.html">GeekBox</a>. It&rsquo;s another RK3368 based system, but it comes with Ubuntu on it as a boot option and has a docking board with a SATA port! One of those, with the docking board, the UART adapter, and a real CPU heatsink with fan I was able to get for about $150. So I ordered two. They are currently very backordered, but the nice folks who will be shipping it to me from China assure me they will fulfill my order as soon as they are back in stock. I&rsquo;m going to wait for one of those to show up, and build a new kernel on that for the MK68 and possibly just use that as-is with some containers or chroots on it for build targets on Debian 8 and Ubuntu. I&rsquo;ll update again once those arrive for testing &amp; compiling.</p>
<h2 id="nvidia-jetson-tk1">nVidia Jetson TK1</h2>
<p>In the mean time, the <a href="http://cubietruck.com/collections/frontpage/products/cubieboard4-cc-a80-high-performance-mini-pc-development-board">Cubieboard 4</a> I ordered seems to be lost in the great black hole that is international shipping and customs without a tracking number. I was hoping to use this for all the ARMv6 and ARMv7 builds, but I&rsquo;ve not seen it and the shipper is non-responsive to my inquires of a tracking or customs document number. Another kind soul on the #linux-rockchip channel suggested getting a Jetson TK1 board since it has a native SATA port (turns out very few ARM boards have native SATA, and that Geekbox docking board is just a SATA-USB interface as well). I figured this would be much easier to get something working with. So, I grabbed one via a <a href="https://developer.nvidia.com/embedded/makejtk1">Make: promotion</a> for only $99. It arrived a few days ago.</p>
<p>This device comes running Ubuntu 14.04, which is very nice. The downside is that all the current gen ARM boards seem to target a heavily patched 3.10 kernel because that&rsquo;s what Android requires. But, major MAJOR bonus: nVidia has found a great workaround for the bootloader issue that I&rsquo;m going to steal for other boards that don&rsquo;t implement this. They also use u-boot to bring up the board, but then embed EXTLINUX (from the SYSLINUX) project as a second stage bootloader. This means there is a text file on the system that tells it what kernel to use, and can even present a menu to choose what you want to boot from over the serial port. This is so much nicer than having to re-flash the parameters &ldquo;file&rdquo; from a host device while in recovery mode to adjust kernel command line options or try a new kernel!</p>
<p>With the Jetson having a SATA port and the Geekboards also having a SATA port I went and grabbed three 240GB SSDs from Fry&rsquo;s post black friday for under $80/ea. The Patriot Blaze SSD I grabbed for the Tegra is giving lots of SCSI errors, but it also turns out that the board has an older software loadout on it&rsquo;s internal flash. So, I went and got the latest build of their JetPack tool to install, and it&rsquo;s been a parade of failures ever since.</p>
<p>The JetPack installer refuses to function on any host machine that is not x86_64 running Ubuntu 14.04. So, I put a LiveCD of 14.04 in a system with a 32GB ext4 formatted USB drive in to put the stupid installer on. It then goes and downloads a bunch of junk from the Internet, builds a new image, and slowly re-flashes all 16GB of flash on the TK1 over USB2. Then, it fails to boot. No idea why. The process is very unclear of what is supposed to happen, it may need to make a TFTP boot but while it can configure a DHCP server correctly, it&rsquo;s not running it&rsquo;s own TFTP server or putting the TFTP files in the right place, so I have no idea what it&rsquo;s trying to do. I&rsquo;m sure I&rsquo;m missing some package it wants, but it&rsquo;s not telling me that either. Grumble. Going to have to troll the nVidia developer forums next week to figure out what&rsquo;s wrong with that. But soon, I hope I can get that working again and at least get the 32bit ARM build environments setup there.</p>
<hr />
<p>So that&rsquo;s where all the ARM stuff stands for now. Not a lot of successes to report, but some forward progress. More updates next weekend, I&rsquo;m sure.</p>
Hacking the Uppel/Sunchip CX-R8https://www.gigofham.com/archives/49
Tue, 10 Nov 2015 02:48:42 +0000https://www.gigofham.com/archives/49<p>As part of my job at Chef, I&rsquo;m responsible for getting Chef on strange platforms. At the 2015 Chef Seattle Community Summit, I discovered that the community has a lot of interest in getting Chef on strange platforms as well. I was helping a community member get Chef running on an ARMv8 (also known as AArch64 or ARM64 depending on your platform or compiler), but I needed an inexpensive platform to put in the forthcoming community builds CI. All of the options from the usual suspects (MSI, APM, etc) were in excess of $1200 at the minimum. So, I decided to get creative. The requirements are simply an ARMv8 processor with a minimum of 2GB of RAM, and USB for storage. Some well crafted DuckDuckGo searches later, and I found the <a href="http://www.rikomagic.com/en/product/showpro_id_73_pid_20.html">Rikomagic MK68</a> which seems to fit the bill. They even have a Linux version (the MK68 LE) in the works, but the less waiting the better.</p>
<p>A couple of searches at AliExpress later, and two are on their way from China. Then, I started researching the chip used in this product: the Rockchip RK3368, and find (totally by accident) the <a href="http://smile.amazon.com/gp/product/B013OUDFYK">Uppel CX-R8 on Amazon</a>, and it&rsquo;s available with 2-day delivery (where as the order from AliExpress is going to take 5-30 days). The Uppel arrived this afternoon.</p>
<p>I&rsquo;ve been documenting the process of putting Debian on this device with copious photos and screenshots up on my <a href="http://edolnx-public.objects-us-east-1.dream.io/CX-R8/index.html">DreamObjects public bucket</a>. I&rsquo;ve also been getting a lot of assistance from the fine folks in the #linux-rockchip channel on Freenode IRC. It turns out this is a pretty standard board made by Sunchip. No surprise there, I&rsquo;m guessing it&rsquo;s the same board in the MK68 also. We&rsquo;ll find out in 10-20 days.</p>
<p>In the mean time, we&rsquo;ve discovered that the USB port adjacent to the SD card reader is an OTG port. Using a USB3 A-A cable, I was able to get the firmware loader to drop back into slave mode for programming, and the <a href="http://linux-rockchip.info/mw/index.php?title=Main_Page">Linux Rockchip folks</a> have the tools to talk to that <a href="https://github.com/linux-rockchip/rkflashtool">available via GitHub</a>. The <a href="http://chinagadgetsreviews.com/download-latest-android-lollipop-5-1-1-stock-firmware-for-sunchip-cx-r8-tv-box.html">stock Android image is available</a>, and it&rsquo;s downloading via MEGA now so I can disassemble it and make a Debian 8 version. Then I should be able to flash the onboard storage with a Debian image and boot off the USB as a root device. I can install that using debootstrap from any system.</p>
<p>After I get that figured out, I&rsquo;ll post more details on what hardware is used (ala iFixit style tear-down) and the images required to get Debian 8 on the device. Then I&rsquo;ve got a nice ARMv8 &ldquo;server&rdquo; for compiles, and more coming for testing and even use as an Android media player.</p>
<p>Next will be getting this <a href="http://cubietruck.com/collections/frontpage/products/cubieboard4-cc-a80-high-performance-mini-pc-development-board">Cubieboard4</a> system up and running for ARMv6 and ARMv7 compiles. More on that once it arrives. In the mean time, the CX-R8 hacking continues..</p>