See all posts in this series

Table of Contents

Intro

This how-to will show you the basics of running a Linux container directly on FreeBSD, no virtual machine required!

Thanks to FreeBSD’s Linux emulation and lots of hard work by many people, including the contributions of everyone involved in this containerd pull request and especially Samuel Karp’s runj, an Open Container Initiative (OCI)-compliant way to launch FreeBSD jails, we can now run Linux containers via containerd.

This compatibility is all pretty early-stage and is not guaranteed to work for all containerd use cases (yet!), but all of this work is a huge step toward using FreeBSD hosts natively as Kubernetes nodes

Please note that currently networking doesn’t seem to be implemented!

Prerequisitites

Audience

This post assumes a basic working knowing of Linux containers, compiling source code, and basic FreeBSD system administration skills. (You can find a cheat sheet here)

Platform

I’m running on FreeBSD 13.1-RELEASE. I haven’t tested on other versions.

You also need a working ZFS pool.

Setup

All these commands need to be run as root.

Packages/ports

Install the following packages or ports:

  • bash (shells)
  • git (devel)
  • gmake (devel)
  • go (lang)

Set up Go

# Check /usr/local to find the name of the installed go dir and make a        
# symbolic link to /usr/local/go. My version of `go` is 1.19                  
ln -s /usr/local/go119 /usr/local/go                                          
export GOPATH=/usr/local/go

Build containerd

# Check out containerd source
git clone https://github.com/containerd/containerd.git
cd containerd
# Minimum required containerd version is v1.6.7
# Find the latest with `git tag -l | tail`
git checkout v1.6.8
# Use gnu make!
gmake install
# Start containerd
service containerd onestart
# For persistence across reboots, add containerd to /etc/rc.conf run at boot
# echo 'containerd_enable="YES"' >> /etc/rc.conf
#
# This also seems to be necessary
mkdir /var/lib/containerd/io.containerd.snapshotter.v1.zfs

Build runj

git clone https://github.com/samuelkarp/runj.git
cd runj
gmake install

Enable Linux emulation

kldload linux
# To load the Linux module at boot, run
# echo 'linux_load="YES"' >> /boot/loader.conf
service linux onestart
# To enable Linux emulation at boot, run
# echo 'linux_enable="YES"' >> /etc/rc.conf

Run a Linux container!

# Pull the image
ctr image pull --platform=linux docker.io/library/alpine:latest
#
# And ....
ctr run --rm --tty --runtime wtf.sbk.runj.v1 --snapshotter zfs --platform linux docker.io/library/alpine:latest mylinuxcontainer uname -a

[root@nucklehead ~]# ctr image pull --platform=linux docker.io/library/alpine:latest
docker.io/library/alpine:latest:                                                  resolved       |++++++++++++++++++++++++++++++++++++++| 
index-sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad:    done           |++++++++++++++++++++++++++++++++++++++| 
manifest-sha256:1304f174557314a7ed9eddb4eab12fed12cb0cd9809e4c28f29af86979a3c870: done           |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:213ec9aee27d8be045c6a92b7eac22c9a64b44558193775a1a7f626352392b49:    done           |++++++++++++++++++++++++++++++++++++++| 
config-sha256:9c6f0724472873bb50a2ae67a9e7adcb57673a183cea8b06eb778dca859181b5:   done           |++++++++++++++++++++++++++++++++++++++| 
elapsed: 2.5 s                                                                    total:  2.7 Mi (1.1 MiB/s)                                       
unpacking linux/amd64 sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad...
done: 227.895869ms
[root@nucklehead ~]# ctr run --rm --tty --runtime wtf.sbk.runj.v1 --snapshotter zfs --platform linux \
> docker.io/library/alpine:latest mylinuxcontainer uname -a
 
Linux 3.17.0 FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC x86_64 Linux
[root@nucklehead ~]# uname -a
FreeBSD nucklehead 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64
[root@nucklehead ~]#

The command line

Breaking down the command ctr run --rm --tty --runtime wtf.sbk.runj.v1 --snapshotter zfs --platform linux docker.io/library/alpine:latest mylinuxcontainer uname -a

  • ctr run – containerd client and its subcommand to run a container
  • --rm – remove the container upon completion
  • --tty – attach terminal for STDIN/STDOUT to your terminal
  • --runtime wtf.sbk.runj.v1 – use the runj runtime for FreeBSD
  • --snapshotter zfs – use ZFS as the storage backend
  • --platform linux – use a container image built for Linux
  • docker.io/library/alpine:latest – the container image name, in this case the one we pulled earlier
  • mylinuxcontainer – the name I gave my container. You can use any string
  • uname -a – the command to run in the container

And that’s it! Wow!

If you have questions or comments, you can @ me on Twitter.

References