tools

The EC2 image build process is public, but the AMI registration portion is
not. Here are the necessary tools to create an image file, but see the
2013-05-26 news post for information on how to register the images in EC2.

ami-build-backend
- These files are held on the PXE server, and fetched when the guest
boots.

ami-builder-image
- This is a fork of archiso
with some changes to automatically pull down my install script and do a few
other things.

ec2-packages
- These are the sources for all the packages contained in the 'ec2' Pacman
repository.

recent changes and news

2014-07-26

I've added AMIs for the Beijing, China
(cn-north-1) region.

2014-06-27

We're now up to Linux 3.15.2. I've removed
xen-fbfront from the initramfs, because the module was causing 30-second boot
delays:

The module is not required for an instance to boot correctly, so it can be
removed from the initramfs. If you are running an AMI older than the 2014.06.27
release and would like to improve your instance's boot time, you can prune the
module yourself:

The AMI now uses systemd's networkd,
timesyncd, and resolved services. This makes the AMI have a significantly
smaller footprint. Right now our biggest non-core packages are CUDA (in the GPU
AMI) and cloud-init, which has a large dependency chain. I'd like to slim
things even further, but I'll need to investigate how to do so.

2014-03-24

We're up to Linux 3.13.7 for the ec2
kernel and 3.10.34 for the ec2-lts kernel. I didn't make a news post
earlier, but kernels are now built with 'debug' and 'strip'
options, which will create split-out debug information packages (i.e.
linux-ec2-debug, linux-ec2-lts-debug). This is useful for tools like perf,
oprofile, and systemtap. Note that the -debug packages are compressed with
'lrzip'. New AMI builds have lrzip preinstalled, but if you're
running an instance based on one of the older AMIs, you will need to install
lrzip before you can make use of the -debug packages.

2013-11-28

New AMIs are being built right now and
contain a couple changes:

EBS root volumes are now automatically resized to fill the block
device. You can take advantage of this feature by launching an instance
with a root volume size larger than the snapshot.

The resolv.conf file permissions are now 0644, allowing non-root
users to resolve hostnames.

2013-11-26

Geoff H. and David B. both reported an issue
with the current AMI release. The /etc/resolv.conf permissions are set to 0600
rather than 0644, which means that non-root users cannot resolve hostnames to
IP addresses. This is an unintentional regression, most likely caused by
a default 'umask' change in some package. dhclient will create a new
resolv.conf and copy it over any existing file, which preserves the target
file's permissions. But if no such file exists, then the permissions of the
source file are copied. Previously, this worked fine because the file was
generated with 0644 permissions, but now it's being generated with 0600. I've
implemented a fix
for future AMI builds. In the meantime, if non-root users need to perform
DNS requests in your instances, be sure to do 'chmod 0644
/etc/resolv.conf'.

2013-11-06

A new ec2-pacman-mirrors package is
available, and will provide your instances with optimal Arch Linux mirrors for
your EC2 region. The upgrade path is as follows:

New AMIs will be published very shortly which use the new mirror list and
point to the new EC2 package repository.

2013-05-26

I've added some links to this page, which are
the complete set of files needed to do an EC2 image build. This does not
include the AMI registration process, however. The tools Amazon provides for
HVM AMI registration are still under NDA at the moment, and the bits necessary
to do that are included in my AMI registration tools. So I can't make those
public right now. The process itself can be replicated relatively easily,
though:

Build your VM image using the build-backend and builder-image repos above.
PXE is what I use, but you could just as easily make it into an ISO or
something. If you intend to do an S3-backed AMI, you will need to make the
image no larger than 10GB (I use 8GB).

Trim the image down (I do a 'mount -o loop,discard' on the image, then
'fstrim' the mount point, making the image into a sparse file).

Tarball the image (tar cSzf, S to preserve the sparseness).

Upload the tarball to S3.

In each region, launch an instance and attach an empty 8GB EBS volume to
them.

On each of those instances, download the tarball and extract with 'tar
xSf'.

Use 'ddpt', an enhanced dd which pays attention to the sparseness of the
image, to copy the raw image file into the EBS device. I use "ddpt
if=<imagefile> of=/dev/xvdf bs=512 conv=sparse oflag=sparse,fsync". The
sparseness aspect is important, because otherwise you're copying empty blocks
onto the EBS device, which makes the snapshot take much longer, and is really
just a waste of time. EBS volumes already read-as-zero, so there's no sense
copying zero blocks.

I've started creating AMIs which have CUDA
preinstalled. These are for the cg1.4xlarge instance type.

2013-02-05

Nothing too exciting lately. Today's release
has Linux 3.7.6.

2012-11-22

New AMI releases, now with cloud-init. Thanks
to Jeremy D for contributing his
time and effort to making cloud-init work well on Arch Linux.

2012-11-12

Released new AMIs, primarily for the new AWS
region in Sydney, Australia (ap-southeast-2).

2012-11-08

Today's AMIs are released. Nothing too fancy
in this build: just updated packages, including linux-ec2 3.6.6-1.

2012-10-21

I've added a new linux-ec2 package which
contains a patched v3.6.2 kernel. There are a few major differences between
this kernel and the Arch Linux stock kernel:

Hangs on Xen fixed (patches from 3.6.3 stable-queue).

CONFIG_PREEMPT_VOLUNTARY instead of CONFIG_PREEMPT, this will allow
for better scheduling as a domU.

CONFIG_HZ=100 instead of CONFIG_HZ=300, this allows for better
performance on many-CPU instances, as there are fewer timer interrupts to
preempt other tasks.

Many drivers removed, particularly those that didn't make sense for
running in an EC2 instance. I've left drivers for my own hardware so I can
experiment with it as a dom0 kernel as well. The kernel size is roughly half
the stock Arch Linux kernel due to the stripped drivers.

I am also building new AMIs right now, and am beating the i386 AMIs into
working order. Once done I'll publish the next release (which should be
2012.10.21). Once it's available, it will show in the tables above.

2012-10-16

Do not upgrade HVM instance
kernels to anything between 3.6.0 and 3.6.2 inclusive. You must wait for 3.6.3
or else your instance will not boot. We're currently waiting
on this
patch to be integrated into the mainline stable tree. This is also why I
am probably not doing an AMI release this week, as the HVM AMIs would be
totally broken.

I've also taken a look at building i386 (well, i686) AMIs. I'm not really
sure that it's worth the effort. Nobody really uses 32-bit AMIs anymore, and
we'd need to fork the
kernel just to make it happen. For now, i686 is on ice.

release notes

These AMIs are as close to a "vanilla" install as I can make them
without making them functionally impaired on EC2. But here's the complete
list of differences between the EC2 builds and a stock install:

High performance kernel specifically for EC2, including
paravirtualization support on i386 and x86_64 AMIs, and more Xen-friendly
process scheduling.

Kernel modules included in initrd, some of which are relevant outside of
EC2 contexts (e.g. if you want to run the image in a non-EC2 environment such
as KVM or Hyper-V):

pacman loads (and automatically lsigns) the 'archlinux' and
'ec2' keyrings on the first boot (the latter keyring contains my public
key used for package signing in the ec2 repo).

pacman mirror list is automatically selected at boot based on a list
I created (based on rankmirrors run on instances in each region). These lists
are provided by the package ec2-pacman-mirrors, which is in the ec2
repo.

dhclient is used instead of dhcpcd for robustness reasons. I found that
dhcpcd gave up too quickly if it tried to do a DHCPREQUEST when the vif
wasn't completely up, making the EC2 instance inaccessible.