Step-by-step tutorial for manually deploying a Kubernetes cluster on FreeBSD bhyve VMs
This tutorial will take you step-by-step through setting up a fully-functional Kubernetes cluster installed on
bhyve virtual machines (VMs) on a single FreeBSD host/hypervisor. We’ll be following Kelsey Hightower’s Kubernetes the Hard Way tutorial based on Kubernetes version 1.18.6, adapting it for the FreeBSD environment. While the VM guests that make up the Kubernetes cluster run Ubuntu Linux, we will use native FreeBSD functionality and tools as much as possible for the virtual infrastructure underneath, such as the virtualization platform, but also the cluster’s virtual network and support services.
- Setting up
- Creating a custom CBSD configuration for creating our cluster’s VMs
- Configuring the FreeBSD firewall, DNS, and routing for cluster networking
You can find custom files and examples in my
This tutorial is based on a series of meandering posts I wrote about my original experiments working through the tutorial on top of FreeBSD. You can read them starting here.
Note that Kubernetes the Hard Way is one of the most manual ways to create a Kubernetes cluster, so while it’s great for understanding what all the piece are in a cluster and how they fit together, it’s not the most practical method for most users.
This tutorial covers installing the cluster on a single FreeBSD host. You will end up with a fully-functional cluster, suitable for learning how to use Kubernetes, testing applications, running your containers, etc. However, it’s not suitable for most production uses, because of a lack of redundancy and security hardening.
This tutorial is not a Kubernetes user tutorial. It won’t spend time defining terms or providing deep explanations of concepts. For that, you should start with the official Kubernetes documentation.
For this tutorial, you don’t need to know anything about Kubernetes. You do need to have a host with FreeBSD installed; an understanding of basic FreeBSD system administration tasks, such as installing software from FreeBSD ports or packages, and loading kernel modules; and familiarity with
sh. Experience with FreeBSD
bhyve virtual machines and the CBSD interface is useful but not required.
Host (FreeBSD Hypervisor) Requirements
- Hardware, physical or virtual
- The guest VMs have a shared total of 12 “CPUs” between them, but these do not have to map to actual host CPUs. My system has a total of 8 cores, for example
- CPUs must support FreeBSD bhyve virtualization (see the FreeBSD Handbook page on bhyve for compatible CPUs)
- RAM: at least 18Gb; 30Gb+ preferred
- Free disk space: at least 100Gb
- Operating system
- FreeBSD: 13.0-CURRENT. It may work with 12.0-RELEASE, but it haas not been tested
- File system: ZFS. It could work with UFS with user modifications
- CPUs: Intel(R) Core(TM) i5-6260U: 4 CPUs, 2 cores each
- RAM: 32Gb
- Operating system
- FreeBSD 13.0-HEAD-f659bf1d31c
You can skip this section in Kubernetes the Hard Way.