Where did my space go?

The Amazon EBS disk (/dev/xvda) presented to the OS has the right size: 33G

However the partition where the OS resides (/dev/xvda1) is stuck at 8G.

Linux AMIs come preinstalled with a customized version of the cloud-init package. This includes Python modules that can resize partitions and filesystems, if CLI tools like growpart and gpartare present. On standard Debian AMIs, they aren’t there.

CentOS AMIs/EPEL in comparison have the cloud-utils-growpart rpm package available providing the growpart tool.

cloud-init tools will also detect if the root file system (/) size differs from the partition size and utilize the appropriate filesystem expansion tool (e.g. resize2fs for ext4) to match the partition size.

Resizing an online partition on a linux AMI

Naturally we would like our AMIs to resize the root partition by themselves, to use all available space.

Since growpart is not available in Debian AMIs, we can accomplish this with parted and an init script that runs on the first boot (when the instance gets launched).

Making parted work non-interactively

One challenge here is that most tools will complain about resizing the partition containing a mounted [root] file system.

The version of parted shipped with Debian jessie allows us to grow a partition interactively and -- albeit not so well documented -- non interactively as well.

Normally, a command like this would suffice, but because the partition is in use, parted prompts for confirmation which would interrupt the flow in a script:

root@ip-172-31-2-65:~# /sbin/parted ---pretend-input-tty /dev/xvda resizepart 1 yes 100%
Warning: Partition /dev/xvda1 is being used. Are you sure you want to continue?
Information: You may need to update /etc/fstab.

The only thing remaining is to resize the filesystem. This will be done automatically by the cloud-init tools after a reboot or by running resize2fs /dev/xvda1

Putting it all together

If you are building your own custom AMI, it makes sense to add an init script that does the above steps immediately after a new instance gets launched and ensures it won’t rerun itself again.

An important requirement is to have parted installed; you can either do that inside the script or ensure parted gets preinstalled as part of the scripts building your custom AMI.

For Debian AMIs running the following script will create an init script linked to /etc/rc5.d/S05ami-resizerootpart.sh, that will take care of all the resizing. As the last step it will deactivate itself and report activities under /var/log/resizerrootfs.log

Are other AMIs affected by this problem?

Yes! CentOS6 AMIs are affected. Instead of parted though -- which is an old version that can’t resize online partitions -- make sure growpart is installed from the package cloud-utils-growpart. This package is part of the EPEL6 repo. The resize command is:

/usr/bin/growpart /dev/xdva 1

Finally SLES11.x AMIs seem to suffer from the same issue.

Conclusion

Some Linux AMIs will fail to utilize the whole EBS volume size you've chosen when you launch them!

If you have been bitten by this issue, I'd love to hear about it and whether the ideas presented here were of help.