[[!meta title="FreeBSD root on ZFS"]]
[[!meta description="How to install FreeBSD on a ZFS system 'root' partition"]]
[[!meta copyright="Copyright &copy; 2009 Yarema <<yds@CoolRat.org>>"]]
[[!meta date="2007-12-20"]]

# How to install FreeBSD on a ZFS system 'root' partition

## Problem

[ZFS][] &mdash; the breakthrough file system in FreeBSD 7 (ported from
Sun's [Solaris 10 Operating System][Solaris]) delivers virtually unlimited
capacity, provable data integrity, and near-zero administration.  However,
FreeBSD's [sysinstall(8)][] does not yet support installing the system
onto anything more exotic than a commonly used UFS partition scheme.
Furthermore, FreeBSD's boot [loader(8)][] cannot yet load the kernel and
modules from ZFS.

## Solution

Install FreeBSD onto a bootable USB flash drive along with a copy of the
[zfsboot][] script on it.  Boot from the USB flash drive and run the
[zfsboot][] script.  This [usbinstall][] script can be used to prepare a
bootable USB drive.

[[!toggle id="usbinstall.sh" text="Show usbinstall source"]]
[[!toggleable id="usbinstall.sh" text="""
[[!inline pages="zfsboot/usbinstall.sh" feeds=no]]
"""]]

## Implementation

The zfsboot script presented here will create the regular bootable 'a'
partition, but configure it as RAID-1 using [gmirror(8)][].  This approach
provides the possibility to directly boot from any of the drives by
configuring the BIOS boot order. This works even if only one drive is used,
although [gmirror(8)][] will not be doing any mirroring.
See Ralf S. Engelschall's <<rse@FreeBSD.org>> excellent
[System Disk Mirroring HOWTO](http://People.FreeBSD.org/~rse/mirror/ "Ralf S. Engelschall's System Disk Mirroring HOWTO")
and note the implementation here is using *GEOM mirror on partitions of a
slice on a disk* for the boot partition.

Since we're forced to use a regular 'a' boot partition there is some advntage
in using a regular 'b' partion for swap. Due to the way ZFS is currently
implemented there is a known issue (in FreeBSD and OpenSolaris) where ZFS
needs to allocate memory to send an I/O request.  When there is no memory,
ZFS cannot allocate any and thus cannot swap a process out and free it.
Such low memory conditions can cause the machine to effectively freeze &mdash;
unable to swap out to disk right when it is most crucial to do so. See
<http://Lists.FreeBSD.org/pipermail/freebsd-current/2007-September/076831.html>

The script assumes the FreeBSD 8 disk1 ISO to be mounted on /dist on another
host and /root/.ssh/authorized_keys on that host should allow the target machine
booted via the usb drive to connect without requiring a password.

[[!toggle id="zfsboot.sh" text="Show zfsboot source"]]
[[!toggleable id="zfsboot.sh" text="""
[[!inline pages="zfsboot/zfsboot.sh" feeds=no]]
"""]]


ZFS filesystem definition file should be a variation of the [zfsboot.fs][]
modified to suit your needs.

## Files

All of the zfsboot scripts and config files can be found
[here](/zfsboot/ "FreeBSD root on ZFS: files").

## References

See the following links for further details:

* <http://Wiki.FreeBSD.org/ZFSOnRoot>
* <http://Wiki.FreeBSD.org/ZFS>
* <http://Wiki.FreeBSD.org/ZFSTuningGuide>
* <http://People.FreeBSD.org/~rse/mirror/>
* [gmirror(8)][]
* [geom(4)][]
* [zpool(8)][]
* [zfs(8)][]

[[!tag tags/FreeBSD]]
[[!tag tags/tech]]

[usbinstall]:		/zfsboot/usbinstall	"Script to create a bootable FreeBSD USB drive"
[zfsboot]:		/zfsboot/zfsboot	"Script to install a bootable FreeBSD root on ZFS"
[zfsboot.fs]:		/zfsboot/zfsboot.fs	"ZFS filesystem definition file"

[ZFS]:			http://Hub.OpenSolaris.org/bin/view/Community+Group+zfs/whatis	"What is ZFS?"
[Solaris]:		http://www.Sun.com/software/solaris				"Solaris Operating System"

[geom(4)]:		http://www.FreeBSD.org/cgi/man.cgi?manpath=FreeBSD+8.0-stable&sektion=4&query=geom	"modular disk I/O request transformation framework"
[gmirror(8)]:		http://www.FreeBSD.org/cgi/man.cgi?manpath=FreeBSD+8.0-stable&sektion=8&query=gmirror	"control utility for mirrored devices"
[loader(8)]:		http://www.FreeBSD.org/cgi/man.cgi?manpath=FreeBSD+8.0-stable&sektion=8&query=loader	"kernel bootstrapping final stage"
[sysinstall(8)]:	http://www.FreeBSD.org/cgi/man.cgi?manpath=FreeBSD+8.0-stable&sektion=8&query=sysinstall "system installation and configuration tool"
[zfs(8)]:		http://www.FreeBSD.org/cgi/man.cgi?manpath=FreeBSD+8.0-stable&sektion=8&query=zfs	"configures ZFS file systems"
[zpool(8)]:		http://www.FreeBSD.org/cgi/man.cgi?manpath=FreeBSD+8.0-stable&sektion=8&query=zpool	"configures ZFS storage pools"