Unfortunately, there is no process written to monitor the overlay consumption level and provide graceful warning of its imminent exhaustion and subsequent failure of the root filesystem. So where long-term usage of a LiveOS image is anticipated, special attention to overlay consumption is advised.

Unfortunately, there is no process written to monitor the overlay consumption level and provide graceful warning of its imminent exhaustion and subsequent failure of the root filesystem. So where long-term usage of a LiveOS image is anticipated, special attention to overlay consumption is advised.

−

: (Last-ditch [[#Overlay recover|recovery of a persistent overlay]] for a failed root filesystem may be attempted from a separate boot of a working system.)

+

: (Last-ditch [[#Overlay recovery|recovery of a persistent overlay]] for a failed root filesystem may be attempted from a separate boot of a working system.)

Several developments in the LiveOS design may be considered to maximize the endurance of the LiveOS image.

Several developments in the LiveOS design may be considered to maximize the endurance of the LiveOS image.

Line 523:

Line 523:

Once you have a working filesystem, you should proceed to merge your overlay into the original filesystem as described [[#Merge overlay into new image|above]].

Once you have a working filesystem, you should proceed to merge your overlay into the original filesystem as described [[#Merge overlay into new image|above]].

+

==References==

==References==

<references />

<references />

:: See this [http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/14644 dm-devel thread].

:: See this [http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/14644 dm-devel thread].

* The method of [[LiveOS_image#Merge_overlay_into_new_image|mirroring the LiveOS]] was adapted from Douglas McClendon's [http://cloudsession.com/dawg/projects/zyx-liveinstaller/ ZyX-LiveInstaller].

* The method of [[LiveOS_image#Merge_overlay_into_new_image|mirroring the LiveOS]] was adapted from Douglas McClendon's [http://cloudsession.com/dawg/projects/zyx-liveinstaller/ ZyX-LiveInstaller].

Introduction

Fedora has developed Live CDUSB DVD images for their GNU/Linux operating system. Since the image file systems are stored in the /LiveOS folder of the image, this is the name we'll use to reference the product.

This page shares some critical information about the LiveOS design that helps users take better advantage of their more-limited-than-usual disc resources.

References

Storage

When a Live CD or Live DVD (a LiveOS image on read-only disc media) is booted, temporary storage is prepared for the system in RAM on each boot by /sbin/dmsquash-live-root in initrd0, the initial ram disk filesystem. An in-memory, copy-on-write, system overlay is used (see File Systems below). The boot script prepares up to 0.5 GiB of RAM space in a sparse file by default.

Critical limitationShould this temporary overlay storage be totally consumed, the system will likely crash with Input/output or Bus errors.

Operating system file systems

Live Operating System, LiveOS, file systems are found within disk image .img files.

If one mounts a Fedora-Live-Desktop.iso file or Live CD, the resulting block device file system will list 3 folders:

When the LiveOS image is loaded onto a USB or SD disk, the isolinux folder is copied into a syslinux folder.

If persistence is requested with the --overlay-size-mb NNN option, a Device-mapper overlay file for the root file system is created in LiveOS.

If a separate home file system is requested with the --home-size-mb NNN option, an ext4 formatted home.img file system is created.

The structure is then as follows:

Fedora-Live-Desktop USB/SD device!(mounted on /run/initramfs/live) (Before Fedora 17 the mount point is /mnt/live in the root file system.)
/syslinux
/LiveOS
|- home.img!(mounted on /home) (This occurs during boot up by the /etc/rc.d/init.d/livesys script.)
/liveuser
/Desktop
/Documents
/Downloads
...
/lost+found
|- livecd-iso-to-disk
|- overlay-NAME-XXXX-XXXX (Where NAME is the device partition name and XXXX are hex numerals.)
|- osmin.img
|- squashfs.img!(mount) (At boot up by the /usr/sbin/dmsquash-live-root.sh script in the initramfs.)
/LiveOS
|- ext3fs.img!(mounted on '/') (This occurs during boot up by the dmsquash-live-root.sh script.)
/bin -> usr/bin
/boot
/dev
/etc
/home (If there is a home.img, then this is its mount point directory.)

If the --skipcompress option is used during loading with livecd-iso-to-disk, the squashfs.img compression layer will be skipped. The structure is then as follows:

Persistent storage

The Fedora LiveOS system allows for persistent storage in 3 locations:

An all-purpose, persistent overlay-based file space that saves all updates and changes to the root LiveOS filesystem. But, this storage space is limited by its write-once, fixed-sized design, and must be used with caution (see File Systems below).

Persistent home folder - a re-writable, re-sizable (with difficulty), uncompressed, optionally-encryptable, file space for anything that goes in the user's /home/ folder. A persistent home folder is an option that may be selected at the time of installation of the LiveOS image (although with some effort, one could manually create a home.img filesystem in /LiveOS/ and move the /home/ folder to it). This installation option is only available through the script, 'livecd-iso-to-disk'. The Windows and Fedora 'Live USB Creator' installers do not provide this option at present.

Host device file space - this is the USB stick or SD card file system that is outside of the LiveOS file tree, but which is accessible through the /run/initramfs/live or /mnt/live (before Fedora 17) mount point of a booted LiveOS installation. There, one finds the boot configuration files and anything else one had on the device before loading the LiveOS image. One may save files here without consuming the other, limited file spaces. (This file space is limited only by the device capacity).

Home filesystem

One may find many advantages to installing the LiveOS, with a persistent, home folder (using the --home-size-mb NN --delete-home options), which will hold all the user files and documents one wishes and, perhaps later, throw away—all without consuming the write-once overlay, which can be consumed very quickly (and overlay file space is not normally reusable).

Device filesystem

Additionally, keeping some storage space on the device disc outside of the LiveOS system will let you copy, carry, and delete large resource files, such as image.iso files, or anything you might want to use or share.

Installation options

Fedora 17 Live Desktop may be installed on a 1 GB USB device using the following options with livecd-iso-to-disk (on a single, terminal or console command line, even though the wiki may wrap the following text to accommodate your browser window size):

where '?' in the final parameter represents the target bootable device node, such as sdb1 or sdc1, etc.

The above configuration would allow space for the home folder, the operating system, and a little on the device root.

But with a larger capacity device, one may allocate the resources to suit the anticipated use, as described above.

File Systems

The Fedora LiveOS uses the Device-mapper service of the Linux kernel to manage the file stores on the device. This is the same service that is used by Logical Volume Manager to provide disc partition services.

One critical limitation, mentioned above, is that the LiveOS persistent overlay is a write-once file space. This is related to its use of device mapper snapshots to combine a read-only file system image (copied from the compressed SquashFS.img on the read-only LiveCD or installation .iso file) with a Copy-on-write service that tracks only changed blocks of data in the snapshot (overlay) file and then re-referencing file pointers to the updated blocks.[1][2] Any changes to the operating system files are stored as differences from the base. As such, "deletions" of files are saved as additional difference references, and the originals are hidden. With this mechanism, physical storage space is consumed in the write-once file space rather than recovered.

Consumption of the space allocated for persistent storage in the snapshot overlay file may be tracked with the device mapperdmsetup status report.

For example, dmsetup status live-rw may return

live-rw: 0 8388608 snapshot 42296/1048576 176

where the fraction after snapshot is the number of 512-byte sectors consumed in the overlay over the total number assigned to the overlay. (The final number is the metadata sectors consumed.)

Unfortunately, there is no process written to monitor the overlay consumption level and provide graceful warning of its imminent exhaustion and subsequent failure of the root filesystem. So where long-term usage of a LiveOS image is anticipated, special attention to overlay consumption is advised.

/var/lib/NetworkManager (holds a timestamps file that is deleted and refreshed quite often, every 5 minutes)

/var/log/audit (holds the SELinux audit.log that records a great many file accesses.)

/var/spool/abrt (holds often large, error reports and core dumps). Users could be advised to act on any abrt reports in the current session or copy the reports to other permanent storage, such as in /home/ or external storage.

there may be other good candidates...

Merge overlay into new image

A new root filesystem image can be constructed by using some Device-mapper tools.

If one has sufficient free disk space available, typically on an attached hard drive, one can copy (and uncompress) the current filesystem to a working folder. (This may require, for example, 4.3 GiB of free space for Fedora 17 Live Desktop.)

Device-mapper's mirror target allows one to create a 'mirror' image of the LiveOS root filesystem (the snapshot based on the SquashFS and overlay device). The mirror image can be then be recompressed, and if one has an additional 660 MiB or more of free space (depending on how much software or other root filesystem files have been added) on your LiveOS device filesystem (the USB/SD card filesystem), one can delete the old SquashFS image and replace it with the recompressed version.

The snapshot overlay can then be reset before rebooting the LiveOS device.

The following Bash script demonstrates the process, where the $1 parameter is a mount point directory for a disk partition with sufficient storage capacity (a LiveOS folder will be created there for holding the mirrored filesystem), and $2 is the LiveOS device node name (such as, /run/initramfs/livedev, for the currently booted LiveOS image, or /dev/sdc1, for example, for an attached LiveOS device). Options xtrace and verbose have been set to aid debugging.

Test and practice the following script!This procedure will delete the LiveOS filesystem before replacing it with a recompressed merged version. Be sure to verify that you will have sufficient space for the replacement before running the full script. (You could comment out the deletion, and squash into a directory on a large working drive to determine the new filesystem size.)

The mirror method copies and merges in one pass, while the snapshot-merge method copies the original filesystem first and then merges in the changes. I've found the mirror method to be about 15% faster.

Overlay recovery

Test and practice the followingThis procedure is not thoroughly validated, and may destroy your data.

If one 'exhausts' the limited storage capacity of a LiveOS overlay, Device-mapper will mark the filesystem as 'Invalid', as shown by the dmsetup status command executed in Terminal or a console (if you haven't crashed):