How to install Fedora on top of ZFS

This time, we will help you set up a newly installed Fedora system on your computer, which will boot from a ZFS file system.  In a later guide, we'll show how to migrate an existing Fedora system to ZFS.

What's this ZFS thing?  Why would I want it on my computer?

ZFS is, bar none, the best file system ever invented.  It's better than ext4, better than XFS, and better than btrfs (which is, to date, the most advanced file system included in the Linux kernel).

If you want high reliability, data integrity and ease of administration for your computer, then you want to install Fedora atop ZFS.

I'll help you do exactly that.

Gather what you'll need

This guide should take about 25 to 30 minutes of your time, unless you're making a speed run.  You will need:

1. A computer with a hard disk (or SSD, same thing).  Conceivably, you could install on an USB drive too.  We'll stick to the hard disk in this case.
2. A USB drive or CD-ROM with Fedora 16 Live booting from it.  You can record a CD or use liveusb-creator to afford your USB the Live CD.

Install Fedora

We are going to install a Fedora copy on your computer.

Boot Fedora Live

Tell your computer that you want to boot from the USB drive or CD-ROM containing the Fedora Live installation you have at hand.  For this, the instructions will vary from computer to computer, so we won't say much about that.

Start the installation program

Now run the Install to Hard Disk program in the System tools menu.

You can use the installation program as usual.  We'll just have to make a quick stop on the partitioning part, where we'll hold your hands.  Select Create custom layout from the options:

Partition your disk correctly

OK, now that you're you're here, continue and create the following partition layout:

1. A BIOS boot partition, 2 MB in size
2. A /boot partition (ext4 or ext2, whatever you prefer) around 500 MB in size
3. A swap partition, probably twice as big as the RAM memory in your system.
4. An ext4 partition, mounted nowhere.  You must select Fill to maximum available size
5. Another ext4 partition, mounted nowhere.  You must select the same Fill option as before.
6. Once you have done this, select the very last partition, and change it so it mounts on /.

In sum: it's very important that the last partition on disk is the / partition, and that the swap partition be at the beginning of the disk.  This is how it will look like in the end:

Now hit Next.

Continue the installation after rebooting

You can continue the installation as normal.  When the installation is done, eject the CD or USB drive, and reboot.  After rebooting, complete the Fedora post-install wizard.  Here's the product of your work:

Disable SELinux

As root, edit /etc/selinux/config and disable SELinux by changing SELINUX to disabled.  Save the file.

Then disable enforcement for the duration of your current session by running the command:

setenforce 0

Note: yes, SELinux is important.  We know.  For the time being, however, your system will require that you disable SELinux until all issues preventing the use of SELinux with ZFS are ironed out.  Maybe you can help us fix them.

Install ZFS

Since the installation of ZFS is a bit involved, we'll give you a few steps to run by hand and then a script that will take care of ZFS just fine.  The following steps you will need to run as root, so open a terminal window and run the following commands as root.

Install dependencies first

You'll need to a series of dependencies to compile ZFS, so install them first:

yum install -y git patch kernel-devel gcc zlib-devel libuuid-devel libtool automake autoconf

Clone ZFS

OK, after doing this, clone the ZFS repositories:

cd /usr/src
git clone https://github.com/zfsonlinux/spl
git clone https://github.com/Rudd-O/zfs

This should net you two directories in /usr/src containing these sources.

Add the compile script

You will need to run the following script soon.  Save the contents of that file as /usr/src/deploy-zfs:  Don't forget to make it executable.

Patch GRUB to accept ZFS root file systems

OK, now it's time to patch GRUB.  Do the following steps:

cp /sbin/grub2-mkconfig /usr/src/grub-mkconfig.in
patch -p2 < /usr/src/zfs/grub2/grub-mkconfig-zfs-support.diff
mv /usr/src/grub-mkconfig.in /usr/src/grub2-mkconfig

Tell the system to run deploy-zfs on every kernel installation

To get this step done, do the following:

mkdir -p /etc/kernel/postinst.d
ln -s /usr/src/deploy-zfs /etc/kernel/postinst.d/

Run deploy-zfs

At this point, you're ready to install ZFS.  Run it:

/usr/src/deploy-zfs

Reboot

Since the kernel-devel package is only available for the latest kernel, the ZFS modules are only going to be available on the latest kernel.  So you are now going to reboot using the latest kernel.

Reboot your machine.  When it reboots, select the latest kernel.  After boot is done, confirm that you're using the latest kernel using the commands rpm -q kernel and uname -r.

Prepare the​ ZFS pool and file systems

OK.  You have rebooted.  Good.

Remember when we created a partition that was empty initially during the installation?  Excellent.  We're going to create a pool right there.  In the following example, we assume that your root file system is mounted in /dev/sda5 and the empty space is mounted in /dev/sda4.  It would be a good idea to check this fact with the mount and parted commands before you proceed.

Create the storage pool

Run the following commands:

zpool create pool -m none -o ashift=12 /dev/sda4

The -m none parameter tells the pool to not mount itself anywhere.  The -o ashift=12 parameter is optional, but it will make your pool perform better on advanced-format 4K sector disks (like SSDs and big hard disks).

If you did it right, you should be able to do the following and see a screen similar to this:

Create the root file system, along with other file systems

Strictly speaking, this step is optional.  You could use your pool exactly as it was when it was created.  However, it's a good idea to follow the next step because it allows you to carry multiple operating systems and system directories in your pool more easily, so it's recommended.

zfs create pool/ROOT
zfs create pool/ROOT/fedora

You should see something like this, after listing them:

pool/ROOT/fedora will be your root file system.  Feel free to create as many file systems underneath it as you would like, now.  pool/ROOT/fedora/home, pool/ROOT/fedora/var, go wild like this:

zfs create pool/ROOT/fedora/home
zfs create pool/ROOT/fedora/var

Note: there is an exception to this.  Remember that /usr cannot be an independent file system for now -- you can create it and set its canmount property to no, then create child file systems with canmount=yes within /usr, but you cannot directly mount any file system on /usr without your system breaking completely.  This issue will be addressed in the future by the ZFS team.  In the meantime. /usr must be available as part of the root file system.

Set optional properties on the pool

This is an optional step.  Now is the best time to set properties like compression or no access time updates on your pool.  If you have a decent CPU, compression will give you higher performance, especially with slower disks.  Disabling access time recording gives an extra boost.  This is the moment to set them:

zfs set compression=on pool
zfs set atime=off pool

OK.  Now every byte hitting your pool will be compressed.  Nice space savings!

Export the pool and reimport it with an alternate root

Run the following commands:

zpool export pool
zpool import -o altroot=/sysroot pool

This will make the ZFS system mount any file systems within the pool in /sysroot.

Mount the file system that will be the destination for your operating system

Set the mountpoint property on the root file system for your operating system:

zfs set mountpoint=/ root/POOL/fedora

Your file systems will now look like this:

Migrate and configure the operating system

Copy the operating system to /sysroot

The next step is straightforward and it only requires waiting a bit:

rsync -axv / /sysroot/

Note the slash in the end of /sysroot/!

Populate /dev

You'll need the device files too:

rsync -axv /dev/ /sysroot/dev/

Chroot into the /sysroot file system

That's it.  Use chroot to enter that directory as if it was the root.

Mount /boot, /sys and /proc

mount -t proc proc /proc
mount -t sysfs sys /sys
mount /boot

This is how your system should look like after mounting these file systems.

Edit fstab to fix the root file system

Now you'll be editng the root file system entry in fstab.  Edit the file /etc/fstab, and change the line that mounts / as follows:

pool/ROOT/fedora / zfs defaults 0 0

You may want to replace defaults with noatime for higher performance.  This is how it should look like:

Redo GRUB configuration

We will be using the special grub2-mkconfig that we patched earlier for this.  Run the following command:

/usr/src/grub2-mkconfig | tee /boot/grub2/grub.cfg

Your screen should look a bit like this.  Note the root=pool/ROOT/fedora parameter!

Regenerate the initial RAM disk

OK, now it's time to regenerate the initial RAM disk.  The following command accomplishes that:

dracut -f /boot/initramfs-`uname -r`.img `uname -r`

Ignore the errors it gives you about no such dataset.  These are temporary error messages because you are in a chroot.

Unmount mounted file systems inside the chroot, and exit the chroot

Clean up your chroot, and exit it.

umount /boot
umount /sys
umount /proc
exit

Export the pool

To ensure a clean system, export the pool now.

zpool export pool

If this process hangs, just wait thirty seconds, and then hard power off your machine.  ZFS won't lose a bit because of this.

Reboot

Reboot your operating system.  The system should boot normally.

Confirm that your system is now using ZFS as the root file system

Run the mount command.  It should show you the following, confirming that ZFS is now your root file system, and everything works normally:

Enlarge the pool to take the unused space up

The last and final step is short and sweet.  It involves nuking the ext4 partition and enlarging the ZFS partition to take up the empty space.  We'll show you how to do that right now.

Nuke the ext4 partition

Open parted on a terminal as root.  Follow these steps:

• Remove partition 5: rm 4
  This is the ext4 partition we created earlier that was mounted on /.
• Resize partition 4 to take up the newly freed space:
  • Change to sectors unit: unit s
  • Show the list of partitions: p
  • Make a note of the starting size of partition 4, and the total size of the disk.
  • Remove partition 4: rm 4
    Ignore any errors.
  • Create a new partition: mkpart.
    • Give it no name.
    • Set its type to ext4 (doesn't matter, really).
    • Set its starting point to the old partition 4's starting point.
    • Set its ending point to the size of the disk in sectors minus 1.
    • Ignore all errors.

After all the changes, parted should look like this:

Reboot your machine now.  As long as the starting sector of the ZFS partition remains unchanged, ZFS will be unfazed by this change in partitions.

Enlarge the pool

After the reboot, run the command:

zpool online -e pool /dev/sda4

Once you have done that, the pool will have gobbled up the new space in the bigger partition.  Look:

That's it!

And that is it.  That is really the end.  You have a system that runs ZFS and boots from it.  Great job!

From now on, every time you update the kernel, the deploy-zfs script will be run right after the new kernel is deployed into place, allowing you to boot updated kernels with ZFS just fine.  In the future, the ZFS on Linux project will release more enhancements to get your system up and running with the latest version of ZFS without compilation.  For the moment, your system is going to do fine.  Make sure you update with git pull the spl and zfs source trees from time to time!

Good luck, and happy hacking!

Huge thanks to @shiinee for helping me with test-driving the entire process and documenting each step!