Adventures in Freebernetes: Bespoke VMs in CBSD

Part 7 of experiments in FreeBSD and Kubernetes: Custom Linux VM Installs in CBSD

See all posts in this series


Custom Linux VM

Finally, as promised, I’m going to make a bhyve VM for a Linux distribution for which CBSD currently does not have an existing configuration. (I’m using CBSD version 12.1.16.) I chose Alpine Linux, in part because they have offer an ISO image for installation. (I wanted to use Google’s Container-Optimized OS, but they don’t distribute a public anything. The same for AWS’s Bottlerocket distribution, which is also optimized as a server for container runtimes. Both are open-source, but both require compiling from source if you’re not deploying in their respective clouds, and that is a completely different experiment.)

To start, I cd into the directory with the profile configuration files. Then I copy a random Linux configuration and edit to change the name, version, and download site for the ISO image. I’m not sure about some of the options, so I just leave them as-is.

root@nucklehead:~ # cd ~cbsd/etc/defaults/
root@nucklehead:/usr/cbsd/etc/defaults # cp vm-linux-ArchLinux-x86-2020.conf vm-linux-AlpineLinux-3.12.1-x86_64-standard.conf
root@nucklehead:/usr/cbsd/etc/defaults # vi vm-linux-AlpineLinux-3.12.1-x86_64-standard.conf
view raw gistfile1.txt hosted with ❤ by GitHub
# don't remove this line:
vm_profile="AlpineLinux-3.12.1-x86_64-standard"
vm_os_type="linux"
# this is one-string additional info strings in dialogue menu
long_description="Linux Alpine 3.12.1"
# custom settings:
fetch=1
# Official resources to fetch ISO's
iso_site="http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/"
# Official CBSD project mirrors
# mirrors maintainer:
# cbsd.lifec0re.net: brahmann at lifec0re dot net
# electro.bsdstore.ru: olevole at olevole dot ru
# mirror.bsdstore.ru: olevole at olevole dot ru
# clonos.ca.ircdriven.net clonos.us.ircdriven.net: onecoldworld at gmail dot com
cbsd_iso_mirrors="http://cbsd.lifec0re.net/iso/ http://electro.bsdstore.ru/iso/ https://mirror.bsdstore.ru/iso/ https://clonos.ca.ircdriven.net/iso/ https://clonos.us.ircdriven.net/iso/ https://electrode.bsdstore.ru/iso/"
iso_img="alpine-standard-3.12.1-x86_64.iso"
# register_iso as:
register_iso_name="cbsd-iso-${iso_img}"
register_iso_as="iso-${vm_profile}"
default_jailname="alpine"
imgsize="4g"
# on virtio, installer staled/freezed on Detecting HW stage
virtio_type="ahci-hd"
# VNC
vm_vnc_port="0"
vm_efi="uefi"
# disable profile?
xen_active=1
bhyve_active=1
# Available in ClonOS?
clonos_active=0
sha256sum="17df5aaf6fad99e57767b51f6934ebb799b7e72e8bed8a5f1d4a0639899b7e9b"
iso_img_dist_size="0"
# enable birtio RNG interface?
virtio_rnd="1"
# firmware settings
cd_boot_firmware="bhyve"
hdd_boot_firmware="refind"

When I run cbsd bconstruct-tui and navigate to the OS profile menu, my new entry for Alpine Linux is there.

Screenshot of CBSD Linux VM menu with new Alpine Linux entry at the top
Screenshot of CBSD VM creation menu configured to create a VM from the new Alpine Linux configuration

After exiting, I run cbsd bstart alpine and it boots up.

Global VM ZFS guid: 12522299721660299493
To edit VM properties use: cbsd bconfig jname=alpine1
To start VM use: cbsd bstart alpine1
To stop VM use: cbsd bstop alpine1
To remove VM use: cbsd bremove alpine1
For attach VM console use: cbsd blogin alpine1
Creating alpine1 complete: Enjoy!
root@nucklehead:/usr/cbsd/etc/defaults # cbsd bstart alpine1
Looks like /usr/cbsd/vm/alpine1/dsk1.vhd is empty.
May be you want to boot from CD?
[yes(1) or no(0)]
yes
Temporary boot device: cd
vm_iso_path: iso-AlpineLinux-3.12.1-x86_64-standard
No such media: /usr/cbsd/src/iso/cbsd-iso-alpine-standard-3.12.1-x86_64.iso in /usr/cbsd/src/iso
Shall i download it from: http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/?
[yes(1) or no(0)]
yes
Download to: /usr/cbsd/src/iso/cbsd-iso-alpine-standard-3.12.1-x86_64.iso
Scanning for fastest mirror…
Mirror source: Bytes per 3sec:
* [ 1/7 ] http://cbsd.lifec0re.net/iso/: failed
* [ 2/7 ] http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/: 339968
* [ 3/7 ] http://electro.bsdstore.ru/iso/: 0
* [ 4/7 ] https://clonos.ca.ircdriven.net/iso/: 0
* [ 5/7 ] https://clonos.us.ircdriven.net/iso/: failed
* [ 6/7 ] https://electrode.bsdstore.ru/iso/: 0
* [ 7/7 ] https://mirror.bsdstore.ru/iso/: 0
Winner: http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/
Processing: http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/alpine-standard-3.12.1-x86_64.iso
/usr/cbsd/src/iso/cbsd-iso-alpine-standard-3.1 125 MB 196 kBps 10m53s
Checking CRC sum: 17df5aaf6fad99e57767b51f6934ebb799b7e72e8bed8a5f1d4a0639899b7e9b…Passed
Automatically register iso as: cbsd-iso-alpine-standard-3.12.1-x86_64.iso
Path already exist for: iso-AlpineLinux-3.12.1-x86_64-standard
VRDP is enabled. VNC bind/port: 192.168.0.11:5904
For attach VM console, use: vncviewer 192.168.0.11:5904
Resolution: 1024×768.
em0
bhyve renice: 1
Waiting for PID.
PID: 71631
view raw gistfile1.txt hosted with ❤ by GitHub
Screenshot of the VNC console of the newly booted alpine installer disk

I’ve used the Alpine container image as a base in Dockerfiles, but I had never installed it on a server, virtual or real. Fortunately, the ISO image comes with an installer command, setup-alpine. I walk through the options, then reboot. It does look like I don’t need the CBSD mirrors, because those seem to be alternate hosts for the supported ISO images.

Screenshot of the alpine console showing the system installer
Yes, I messed up on the root password the first time
Screenshot of the alpine console showing more installer steps
Screenshot of the final steps of the alpine installer

That was a success and pretty straightforward, once I figured out where the configuration files lived.

Customizing with cloud-init

What if I wanted to automate the installation using CBSD’s support for populating cloud-init files?

With cloud-init, you generally use a disk image, rather than installing from an ISO image. And, in fact, CBSD seems to be using custom-made raw disk images in place of ISO images for its cloud VM profiles. (I suppose you could still make starting with an ISO image work, if booting the ISO triggered a totally automated installation, including installing the cloud-init package; it would still need to be a two-step process to populate the resulting installation’s cloud-init files with the VM’s custom configuration, but CBSD doesn’t support that, so nevermind.)

I could try to build a raw disk image for Alpine to use for cloud installs.

BUT WAIT! It looks like cloud-init support isn’t in the Alpine release I used, 3.12.1, but it has been added as a supported package in the unreleased edge branch. Ok, let’s see if we can install the unnumbered edge branch because using a tag recycled for rolling versions never broke anything. (Actually, it has. Don’t do this in production, kids.)

It took a little digging to find where Alpine hides its edge development build artifacts. (Here.) Unfortunately, it looks like they stopped building ISO images or other artifacts for edge several years ago, so that’s out. (As I noted at the beginning of this post, if I wanted to compile an entire distribution from source, I would have been using another distro.)


OK FINE. In the next part, I will actually create an image to create and configure an Alpine Linux VM using cloud-init.

Sources / References

One thought on “Adventures in Freebernetes: Bespoke VMs in CBSD

Add yours

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: