If you have ever used Unetbootin to create a bootable USB, you may have noticed the option -

“Space used to preserve files across reboots (Ubuntu only)”

What is different about Ubuntu? Actually, it is not so limited and can be used for most distributions derived from Debian or Ubuntu.

On the other hand, if you use Fedora LiveUSB Creator, you would have noticed the option:

Persistent Storage

Both Ubuntu and Fedora offer persistent storage but using different mechanisms. I have found persistent storage immensely useful, especially for booting from diskless machines. It is interesting to learn about the differences between them and how they work.

OverlayFS

If you try Unetbootin, it will create a file casper-rw on the USB. In addition, an additional kernel boot option – persistent – will be added in the syslinux configuration file.

The persistent storage need not be a file. It can be a partition instead. The label of the partition has to be 'casper-rw'. Persistence seems to work on a Mint live usb provided a partition is used. For Mint,

• Create two partitions a a usb

• Install the live iso on one partition

• Remove the casper-rw file

• Format the second partition as ext2

• Use e2label to change the label of the partition to 'casper-rw'

The root system will be mounted as 'overlayfs' type, which is a module included in the Ubuntu/Debian derived distributions.

The system mounts the squashfs root as a readonly filesystem(/rofs). It then mounts the casper-rw file/partition as a read-write filesystem(/cow). Both are mounted as a composite filesystem with /rofs as the lowerdir and /cow as the upperdir. Overlayfs will search for a file on upperdir first and then on lowerdir. If a file is modified, it will be stored in the upperdir.

Snapshots and Device Mapper

A layered filesystem is reminiscent of the description in Snapshot(computer_storage) in Wikipedia. Volume Manager supports snapshots. Fedora does not include the overlayfs module and uses the concept of a snapshot to achieve the same goal.

As before, you will need a read only filesystem. However, the overlay is no longer a filesystem. It, instead, stores information about the blocks which have been changed and the modified data of the block. So, the data read from the basic filesystem will be replaced by data in the modified blocks.

This has some interesting consequences. A Fedora live usb also contains a file containing a squash filesystem. When you loop mount this file, you do not get a root filesystem. Instead, you will find that it contains an image of an ext4 filesystem 'LiveOS/ext3fs'! The reason for this convoluted setup becomes clear when you realise that squashfs is a readonly filesystem. Hence, you cannot use it as a read only snapshot and modify its data.

Fedora LiveUSB Creator will create a file with a name LiveOS/overlay-LIVE-<UUID>. A kernel option – rd.live.overlay=LABEL=LIVE - would be added in the syslinux configuration file. A script in the initramfs will loop mount the read only ext4 filesystem. It will create a volume manager snapshot from it and then mount the snapshot with the overlay file as a read-write filesystem on /dev/mapper/live-rw. The essentials of the script will be as follows (adapted from dmsquash-live-root.sh in /usr/lib/dracut/modules.d/90dmsquash-live/):

BASE=$( losetup -f )

losetup -r $BASE /run/initramfs/squashfs/LiveOS/ext3fs.img

OVERLAY=$( losetup -f )

losetup $OVERLAY /run/initramfs/<overlay-file-path>

sz=$(blockdev --getsz $BASE)

echo 0 $sz snapshot $BASE $OVERLAY p 8 | dmsetup create live-rw

An obvious drawback of this approach is that you may use only a filesystem supported by Volume Manager. This, in particular, rules out network filesystems which can be useful in the context of LTSP and similar diskless environments.

Unionfs/aufs

Unionfs and, later, aufs were designed to offer a union of multiple filesystems. Each filesystem is stacked on top of the preceding ones. The search for a file starts with the top stack and goes down the stack. Each filesystem may be of a different type. There are user space implementations of unionfs which use the fuse kernel module.

Ubuntu switched from aufs to overlayfs. Overlayfs is simpler in that it restricts the union to just two filesystems – an upper layer and a lower layer. This module may be included in the 3.10 linux kernel.

Overlayfs mechanism is superb for packing a large set of applications on a limited space by taking advantage of the compression-enabled filesystem, like squashfs. You can go back to the original state anytime by just clearing the upperdir. In addition to being useful in a live usb, it is an excellent option for small devices where the distribution of updates is controlled.

June 2013