Part 4 of experiments in FreeBSD and Kubernetes: UEFI Booting + Installing CBSD
At the end of the previous post, I hit an issue while trying to create a Debian guest backed by a ZFS volume on the FreeBSD hypervisor. Installation from a virtual CD ISO onto the ZFS volume went fine, but when trying to boot from the ZFS virtual disk using
grub-bhyve, the FreeBSD kernel panicked.
Booting with UEFI
After spending half a day waiting for the
sysutils/bhyve-firmware port and its missing dependencies, which include the past-its-expiration-date Python 2.7, to compile, I try to boot the Debian guest using UEFI, but after a few failed attempts and more reading, it looks like converting an existing BIOS disk is iffy, especially as UEFI requires disk space of its own, which I didn’t preserve.
So I boot from the installer image again, but adding UEFI bootrom. (Note that UEFI boot does not require running
And install. And then run the same command, but without the ISO virtual device. And success!
Ok, the terminal is messed up because UEFI assumes a graphical output, so I effectively have a 3-line display. (Not joking.) So I get the IP address with
ip addr show | grep inet (after cursing the lack of
ifconfig) and ssh in.
So, yes, using a ZFS volume as the root disk for a Linux VM works (and should provide some disk I/O performance benefits, which I am not benchmarking), but you have to use UEFI instead of BIOS.
Grub Booting ZFS Redux
Meanwhile, on FreeBSD Twitter, I was connected with Chuck Tuffli, who gave me access to an update development version of
grub-hyve to test with the grub-based Debian ZFS volume. Before I had overwritten my original Debian ZFS volume for UEFI booting, I had created a clone:
grub-bhyve read the ZFS volume without making FreeBSD panic, and I was able to load and boot the kernel for Debian.
Hopefully this updated
grub-bhyve version, which also adds XFS support, will get pushed to the ports tree soon.
Getting Ready for CBSD
The last two posts and change have demonstrated some of the permutations of possible bhyve guests and the amount of effort even simple proofs-of-concept involve.
The CBSD project exists to help manage that complexity. While it also supports FreeBSD jails and Xen VMs, I’m going to focus on bhyve. I’m building it from the
sysutil/cbsd port, so now, among other packages, I now have
After the port finishes compiling, it reminds the user to finish installation by running
env workdir="/usr/cbsd" /usr/local/cbsd/sudoexec/initenv and interactively fed it the following values:
With that installed, I reboot the host to clear the network interfaces and VMs I had created and then try to start bhyve.
root@nucklehead:~ # cbsd bconstruct-tui No kldloaded module: vmm Please add vmm_load="YES" to /boot/loader.conf and put kld_list="vmm if_tuntap if_bridge nmdm" into your /etc/rc.conf then reboot the host. Press any key...
Given then the CBSD had added entries to both files, I’m assuming it skipped these because they had been loaded in the running kernel when I ran
initenv, but it did not or could not test for cross-boot persistence.
After adding the entries and rebooting, I try again.
root@nucklehead:~ # cbsd bconstruct-tui The current version requires tmux Please run pkg install tmux or make -C /usr/ports/sysutils/tmux install it.
That did not require installing half the ports tree this time, so when I ran
cbsd bconstruct-tui again, it opened the text UI. Third time’s a charm.
The next post will focus on experiments with CBSD.