Tag Archives: fedora

virt-builder can throw out new virtual machines with existing operating systems in a few seconds, and you can also write these directly to a USB key or hard disk:

# virt-builder fedora-20 -o /dev/sdX

What you’ve not been able to do is create a bootable CD-ROM or ISO image.

For that I was using the awful livecd-creator program. This needs root and is incredibly fragile. You can have a kickstart that works one day, but not the next, and requires massive hacks to get working … which is the exact reason why I set off to find out how to make virt-builder create ISOs.

Read-only

The background as to why this is difficult: CDs are not writable.

You can take all the files from a Fedora guest built by virt-builder and turn them into an ISO, and put ISOLINUX on it but such a guest would not be able to boot, or at least, it would fail the first time it tried to write to the disk. One day overlayfs (which just went upstream a few days ago) will solve this, but until that is widely available in upstream kernels, we’re going to need something that creates a writable overlay at boot time.

Boot Time

I have chosen dracut (another tool I have a love/hate, mainly hate, relationship with), which has a useful module called dmsquash-live. This implements the boot side of making a live CD writable, for Fedora and RHEL. It’s what livecd-creator uses.

It requires that we have a filesystem containing a squashfs in a particular location on the CD:

/LiveOS/squashfs.img

That squashfs has to contain inside it a disk image with this precise name:

/LiveOS/rootfs.img

and the disk image is the root filesystem.

The Script

The script below creates all of this, and effectively replaces livecd-creator with something manageable that doesn’t require root, and is only 100 lines of shell (take that OO/Python!)

Update:Kashyap notes that the script will fail if you’re using tmp-on-tmpfs, so you might need to disable that or modify the script to use /var/tmp instead.

Once you’ve run the script you can try booting the image using:

$ qemu-kvm -m 2048 -cdrom boot.iso -boot d

The Future

One improvement to this script would be to remove the dependency on dmsquash-live. We don’t need the baroque complexity of this script, and could write a custom dracut module (perhaps even, a tiny self-contained initramfs) which would do what we need. It could even use overlayfs to simplify things greatly.

I have scratch-built a Fedora RPM here which is just a 235K download but contains (by magic!) a fully bootable Fedora appliance. After installing the RPM in Fedora 20, do the following to boot the virtual machine:

One way would be for all users [however that is defined] to have a vote. The problem with that is that a feature which few people use, but which really matters for those that use it would probably get dropped by a simple majority vote.

A better idea would be to use an economic system called a revealed preference. The idea is by asking people to risk their own money on the outcome of a vote, you hope to get a truer picture of their feelings. This technique also excludes moaners with lots of time on their hands to argue on mailing lists.

Let’s try this with the systemd / TCP wrappers example. I’m going to have six voters. Four are mostly apathetic about the feature. But two of them use it, and one of those is going to have to change his whole infrastructure around if TCP wrappers goes away.

But first I have to assign a cost to this feature1. Unlike Chris’s Trident example, removing TCP wrappers from systemd is cheap. But it’s not completely free, assuming that Lennart is going to have to write some code, communicate the change, update documentation and so on. I’ll say it costs £12, which is £2/voter.

Alice and Bob perceive some small benefit to the change because they think it’ll make systemd cleaner. Fred is the one who is going to have to make significant changes to his company network, and he’s not happy. Charlie and Diane are completely neutral.

The net benefits are calculated by subtracting the benefit from the cost:

One thing you should notice from the “TOTAL” row is that there is no (expressed) net benefit to the change. Fred’s large negative vote has soured the whole thing. It sounds unfair that Fred is able to block this, but read on …

All we’ve done so far is asked people to guess numbers. To make it a revealed preference, we have to get people to pay real money. In this case, we’re going to ask some people to pay what is called a Clarke tax.

Eleanor and Fred have to pay £11 in tax. (I’m unclear if this is split equally or pro-rata according to their vote). They pay this real money to Alice and Bob. Even after paying the tax, Eleanor and Fred and still better off (according to their claim). Alice and Bob have been compensated for their lost benefit.

The bids in the auction are sealed — ie. people shouldn’t be able to collude. Let’s imagine however that Alice estimated Fred’s £1000 cost and tried to neutralize it by claiming a £2000 benefit to the change. Alice would win (total net benefit becomes positive £952), but she and the other winners would have to pay a tax of £1046. This is a costly victory, but the money goes some way to compensating Fred for the changes he has to make to his company network.

1One issue with this is the estimate of the cost of the feature. I’m sure systemd developers will claim that although dropping TCP wrappers costs a bit of money in the short term, it pays dividends in the long term because of reduced code maintenance and bug reports. In other words that the cost is negative. You have to be able to provide credible costs for this to work, but I think you can express that by having the feature developers join in the voting process, in other words, revealing their true preferences as well.

In libguestfs ≥ 1.26 we are going to start splitting the package up into smaller dependencies. Since the full libguestfs package has lots of dependencies because it has to be able to process lots of obscure filesystems, the question is how best to split up the dependencies? We could split off, say, XFS support into a subpackage, but how do we know if that will save any space?

Given the set of dependencies, we want to know the incremental cost of adding another dependency.

We can get an exact measure of this by using supermin to build a chroot containing the set of dependencies, and a second chroot containing the set of dependencies + the additional package. Then we simply compare the sizes of the two chroots. The advantage of using supermin is that the exact same script [see end of posting] will work for Fedora and Debian/Ubuntu since supermin hides the complexity of dealing with the different package managers through its package manager abstraction.

The results of this, using the libguestfs appliance dependencies, on Fedora 20, sorted by dependency size, with my comments added:

gdisk adds 25420 KB

This is a surprising result in first place, since gdisk is a fairly small, unassuming C++ program (only ~11KLoC). My initial thought was it must be something to do with being written in C++, but I tested that and it’s not true. The real problem is that gdisk depends on libicu (a Unicode library) which adds 24.6 MB to the appliance. [Note: this issue has been fixed in Rawhide.]

lvm2 adds 19432 KB

The default disk layout of many Linux distros uses LVM so this and similar dependencies have to stay in base libguestfs.

binutils adds 16604 KB

This is a sorry tale. The one file we use from binutils is /usr/bin/strings (33KB). Unfortunately this single binary pulls in a huge dependency (even worse, it’s a development package, and this causes problems on production systems). I don’t really understand why strings is included in binutils.

gfs2-utils adds 9648 KB

zfs-fuse adds 5208 KB

Split off in the proposed reorganization.

ntfsprogs adds 4572 KB

e2fsprogs adds 4312 KB

Most Linux distros use ext4, and we want to support Windows out of the box, so these are included in base libguestfs.

xfsprogs adds 3532 KB

Split off in the proposed reorganization.

iproute adds 3180 KB

We use /sbin/ip to set up the network card inside the appliance. It’s a shame this “better” replacement for ifconfig is so large.

About the author

I am Richard W.M. Jones, a computer programmer. I have strong opinions on how we write software, about Reason and the scientific method. Consequently I am an atheist [To nutcases: Please stop emailing me about this, I'm not interested in your views on it] By day I work for Red Hat on all things to do with virtualization.

My motto is "often wrong". I don't mind being wrong (I'm often wrong), and I don't mind changing my mind.

This blog is not affiliated or endorsed by Red Hat and all views are entirely my own.