Sunday, April 19. 2020

I've spent the last couple of days trying to deploy Fedora CoreOS to some physical hardware/bare metal for a colleague using the official PXE installer from Fedora CoreOS. It wasn't very pleasant, and just wouldn't work reliably.

Maybe my expectations were to high, in that I thought I could use Ignition to prepare more of the system for me, as my colleague has been able to bare metal installs correctly. I just tried to use Ignition as documented.

A few interesting aspects I encountered:

The PXE installer for it has a 618MB initrd file. This takes quite a while to transfer via tftp!

It can't build software RAID for the main install device (and the developers have no intention of adding this), and it seems very finicky to build other RAID sets for other partitions.

And, well, I just kept having problems where the built systems would hang during boot for no obvious reason.

The time to do an installation was incredibly long.

The initrd image is really just running coreos-installer against the nominated device.

During the night I got feed up with that process and wrote a Fully Automatic Installer (FAI) profile that'd install CoreOS instead. I can now use setup-storage from FAI using it's standard disk_config files. This allows me to build complicated disk configurations with software RAID and LVM easily.

A big bonus is that a rebuild is a lot faster, timed from typing reboot to a fresh login prompt is 10 minutes - and this is on physical hardware so includes BIOS POST and RAID controller set up, twice each.

FAI was initially developed to deploy Debian systems, it has since been extended to be able to install a number of other operating systems, however I think this is a good example of how easy it is to deploy non-Debian derived operating systems using FAI without having to modify FAI itself.

At the start of July, the LCA2019 team announced that the Call for Proposals for linux.conf.au 2019 were open! This Call for Proposals will close on July 30. If you want to submit a proposal, you don't have much time!

linux.conf.au is one of the best-known community driven Free and Open Source Software conferences in the world. In 2019 we welcome you to join us in Christchurch, New Zealand on Monday 21 January through to
Friday 25 January.

Sunday, September 17. 2017

I tried to install LEDE on my home router which is running LEDE, only to be told that libc wasn't installed. Huh? What's going on?! It looked to all intents as purposes as though libc wasn't installed. And it looked like nothing was installed.

What to do if opkg list-installed is returning nothing?

I finally tracked down the status file it uses as being /usr/lib/opkg/status. And it was empty. Oh dear.

Fortunately the info directory had content. This means we can rebuild the status file. How? This is what I did:

Saturday, September 2. 2017

I found to make all this work I had to piece together a bunch of information from different locations. This fills in some of the blanks from the official Raspberry Pi documentation. See here, here, and here.

You need to be able to mount both the boot and the root partitions. Do this by tracking the offset of each one and multiplying it by the sector size, which is given on the line saying "Sector size" (typically 512 bytes), for example with the 2017-04-01 image, boot has an offset of 8192, so I mount it like this (it is VFAT):

When I first set this up, I used OpenWRT on my router, and I had to patch /etc/init/dnsmasq to support setting DHCP option 43. As of the writting of this article, a similar patch has been merged, but isn't in a release yet, and, well, there may never be another release of OpenWRT. I'm now running LEDE, and the the good news is it already has the patch merged (hurrah!). If you're still on OpenWRT, then here's the patch you'll need:

This lets you put the following in /etc/config/dnsmasq, this says that any device that uses DHCP and has a MAC issued by the Raspberry PI Foundation, should have option 66 (boot server) and option 43 set as specified. Set the IP address on option 66 to the device that should be used for tftp on your network, if it's the same device that provides DHCP then it isn't required. I had to set the boot server, as my other network boot devices are using a different server (with an older tftpd-hpa, I explain the problem further down).

Initially I used a version of tftpd that was too old and didn't support how the RPi tried to discover if it should use the serial number based naming scheme. The version of tftpd-hpa Debian Jessie works just fine. To find out the serial number you'll probably need to increase the logging of tftpd-hpa, do so by editing /etc/default/tftpd-hpa and adding "-v" to the TFTP_OPTIONS option. It can also be useful to watch tcpdump to see the requests and responses, for example (10.1.0.203 is the IP of the RPi I'm working with):

tcpdump -n -i eth0 host 10.1.0.203 and dst port 69

This was able to tell me the serial number of my RPi, so I made a directory in my tftpboot directory with the same serial number and copied all the boot files into there. I then found that I had to remove the init= portion from the cmdline.txt file I'm using. To ease debugging I also removed quiet. So, my current cmdline.txt contains (newlines entered for clarity, but the file has it all on one line):

Now you can hopefully boot. Unless you into this bug, as I did. Where the RPi will sometimes fail to boot. Turns out the fix, which is mentioned on the bug report, is to put bootcode.bin (and only bootcode.bin) onto an SD card. That'll then load the fixed bootcode, and which will then boot reliably.

Sunday, August 21. 2016

I'm in the process of building a new MythTV front end using a Raspberry Pi 3 to replace our aging VIA EPIA M10000, which has been in use since about 2003.

For MythTV, I'm using MythTV Light from Peter Bennett. I have a dedicated back end that lives in the garage, so the front end is nice and easy. With the VIA front end, I built an IR receiver that plugs into the serial port. For the new box, I decided to try using a Sapphire Remote using Mark Lord's excellent looking driver. However, since his driver uses a Makefile which just install the module into the right place, I decided to use the Debian way of doing things. Below is my approach.

apt-get install raspberrypi-kernel-headers dkms

Download the tar ball from http://rtr.ca/sapphire_remote/. Extract it in /usr/src/modules and then rename the directory to sapphire-remote-6.6 (the version may differ!). Put the following into a file called dkms.conf in that directory:

Something I've been wanting to do with our Asterisk PBX at Catalyst for a while is to allow having callers that hit VoiceMail to be forwarded the callee's cellphone if allowed. As part of an Asterisk migration we're currently carrying out I finally decided to investigate what is involved. One of the nice things about the VoiceMail application in Asterisk is that callers can hit 0 for the operator, or * for some other purpose. I decided to use * for this purpose.

I'm going to assume a working knowledge of Asterisk dial plans, and I'm not going to try and explain how it works. Sorry.

When a caller hits * the VoiceMail application exits and looks for a rule that matches a. Now, the simple approach looks like this within our macro for handling standard extensions:

[macro-stdexten]
...
exten => a,1,Goto(pstn,027xxx,1)
...

(Where I have a context called pstn for placing calls out to the PSTN).

This'll work, but anyone who hits * will be forwarded to my cellphone. Not what I want. Instead we need to get the dialled extension into a place where we can perform extension matching on it. So instead we'll have this (the extension is passed into macro-stdexten as the first variable - ARG1):

[macro-stdexten]
...
exten => a,1,Goto(vmfwd,${ARG1},1)
...

Then we can create a new context called vmfwd with extension matching (my extension is 7231):

[vmfwd]
exten => 7231,1,Goto(pstn,027xxx,1)

I actually have a bit more in there to do some logging and set the caller ID to something our SIP provider will accept, but you get the gist of it. All I need to do is to arrange for a rule per extension that is allowed to have their VoiceMail callers be forwarded to voicemail. Fortunately I have that part automated.

The only catch is for extensions that aren't allowed to be forwarded to a cellphone. If someone calling their VoiceMail hits * their call will be hung up and I get nasty log messages about no rule for them. How do we handle them? Well, we send them back to VoiceMail. In the vmfwd context we add a rule like this:

Already attending linux.conf.au? Come a couple of days earlier and attend the mini-DebConf too! There will be a day of talks with a strong focus on the Debian project and a bug squashing day.

Debian Miniconf

After 5 years, the Debian Miniconf is back! Run as part of linux.conf.au 2015, this event will attract speakers talking on topics that suit the broader audience attending LCA. The Debian Miniconf has been one of the largest miniconfs in the history of linux.conf.au.

Friday, July 11. 2014

I've spent a reasonable chunk of the past year working on a project we launched last month, Catalyst Cloud! It is using OpenStack with Ceph as the object store. It has taken a lot of work, and it is now very exciting seeing the level of interest there we're receiving about this new service!

The great part of this is that we can now offer private cloud services to our customers which provides all the flexibility that we've come to expect with the "cloud", but hosted in New Zealand by a New Zealand owned company so no concerns about jurisdiction of your data! Not only are we able to offer private cloud services on our OpenStack cluster(s), but we can also deploy OpenStack onto our customers own hardware using our ProdStack solution (I get to look directly at the Dashboard shown on that page, which is pretty cool).

Next up is deploying another OpenStack cluster in our new data centre (which is another project I'm working on). In the near future we also hope to start using Open Compute Project hardware for our clusters.

Tuesday, January 28. 2014

Back in the old days, we had workstations. And only workstations. They lived on a network, and having them work in that network was simple. Printers just worked (thank you printcap), network shares just worked (thank you NFS) and life was good.

Then along came laptops. We wanted to be more mobile, using our laptops on different networks or even without a network! No one wanted hardcoded printers anymore, or network shares defined in /etc/fstab. Using an Automounter was an option, but if you were on a different network then having the Automounter around would stall tools like nautilus and file indexers etc.

So we need something which can start up relevant services when you connect to a network, and then stop them when you leave that network.

To support this, a few years ago I wrote a NetworkManager dispatcher.d script to do just that. When you connect to a specific network (using the NetworkManager UUID or a specific gateway MAC) or a VPN connection then autofs is started, users GTK bookmarks have any bookmarks for their Network shares added and CUPS is restarted.

When the connection goes away, then autofs is stopped, any GTK bookmarks for the Network shares are removed and any mounts for the Network shares are lazily unmounted.

The linux.conf.au 2014 papers committee is looking for a broad range of proposals, and will consider submissions on anything from programming and software, to desktop, mobile, gaming, userspace, community, government, space and education. There is only one rule:

Your proposal must be related to open source

This year, the papers committee is going to be focused on linux on the frontier and deep technical content-- that might range from cybernetics and mobile operating environments to large astronomy projects and big data projects.

However, the conference is to a large extent what the speakers make it -- if we receive many excellent submissions on a topic, then it’s sure to be represented at the conference. Here’s a few ideas to get you started:

The Cloud - What is it, how can we use it and why is it running on my toaster?

LCA is known for presentations and tutorials that are strongly technical in nature, but proposals for presentations on other aspects of free software and open culture, such as educational and cultural applications of open source, are welcome.

The conference will showcase the best of open source and community-driven software and hardware. It will be held in Canberra at the Australian National University from Monday 28 January to Saturday 2 February, 2013, and provides a great opportunity for open source developers, users, hackers, and makers to share their ideas and further improve their projects.

Information on Proposals

The linux.conf.au 2013 papers committee is looking for a broad range of proposals, and will consider submissions on anything from programming and software, to desktop, userspace, community, government, and education. There is only one rule:

Your proposal must be related to open source.

This year, the papers committee is going to be focused on deep technical content, and things we think are going to really matter in the future -- that might range from freedom and privacy to open source cloud systems or to energy efficient server farms of the future.

However, the conference is to a large extent what the speakers make it -- if we receive many excellent submissions on a topic, then it’s sure to be represented at the conference.

These services used to have IPv6 enabled, but when I moved them from my home server to one hosted in a data centre we lost IPv6 support. However in the last few months, our hosting company has deployed IPv6 support to their hosting facility, and I finally found time to finish setting it up on the server.

Awesome. I've just registered for lca2011, it looks to be an awesome conference! They have a great line up of speakers, some amazing venues (I know, I've seen some of them already), and a fantastic group of people in one place.

The reason that I've already seen some of the venues is that Susanne and I attended the ghosts weekend that the lca2011 Ninjas held earlier in the year. During the ghosts weekend the current organising team bring a number of previous organisers, some people from Linux Australia and their team into one place, lock them in a room and absorb as much information as they possibly can. They bombard us with questions, they tease out tidbits of knowledge and in exchange for that they let us into a few of their secrets and show us around. Admittedly they show us around and then interrogate us on what we think of them.

That was my first time in Brisbane, and an overnight stay is no where near long enough. I'm looking forward to spending a week in Brisbane, and hopefully getting a chance to take in a bit more of the city. As much of a chance as you get when attending a conference that is though...