Step-by-step tutorial for deploying a Kubernetes cluster with k3s on FreeBSD bhyve VMs
In the last tutorial in this series, we went through the steps of creating a Kubernetes cluster on FreeBSD
bhyve VMs, adapting the completely manual process in Kelsey Hightower’s Kubernetes the Hard Way. This tutorial will also build a Kubernetes cluster on
bhyve, but it will install a lightweight
k3s control plane using the
k3sup tool, which automates much of the process.
- Configuring CBSD to create and manage
k3supto bring up a
- Configuring the FreeBSD firewall, DNS, and routing for cluster networking
While this tutorial builds a cluster with a redundant control plane, all the VMs are on a single hypervisor, making it suitable for testing and experimentation, but it is not production grade.
This tutorial only covers creating a cluster. For Kubernetes basics and terminology, you should start with the official Kubernetes documentation.
For this tutorial, you don’t need to know anything about Kubernetes. You do need to have a host with FreeBSD installed; an understanding of basic FreeBSD system administration tasks, such as installing software from FreeBSD ports or packages, and loading kernel modules; and familiarity with
sh. Experience with FreeBSD
bhyve virtual machines and the CBSD interface is useful but not required.
Kubernetes Cluster Specs
We will use these targets for the Kubernetes cluster we’re building. Most of these settings can be adjusted.
- Control plane (“K3s servers”)
- 3 VMs: 2 CPU cores, 2Gb RAM, 20Gb virtual disk each
- If you plan on creating more than 10 worker nodes, increase the control plane VM sizes
- Worker nodes (“K3s agents”)
- 3 VMs: 2 CPU cores, 2Gb RAM, 10Gb virtual disk each
- VM OS: Ubuntu Server 20.04
- Kubernetes version: 1.19
- Control plane backing database: embedded
- K3s also supports using an external datastore, such as MySQL
Host (FreeBSD Hypervisor) Requirements
- Hardware, physical or virtual
- CPUs must support FreeBSD bhyve virtualization (see the FreeBSD Handbook page on bhyve for compatible CPUs)
- CPU core allocations for
bhyveVMs are completely virtual, so you can have VMs running with a total virtual core count greater than your host system’s. You should use the suggested core count for VMs, as Kubernetes will use those for scheduling. You can increase the cores if you have a larger host.
- RAM: Minimum 2Gb per VM. You can use less for the agent VMs if necessary.
- Free disk space: 100Gb
- Operating system
- FreeBSD: 13.0-CURRENT. It may work with 12.0-RELEASE, but it has not been tested
- File system: ZFS
My Test System
- CPUs: Intel(R) Core(TM) i5-6260U: 4 CPUs, 2 cores each
- RAM: 32Gb
- Operating system
- FreeBSD 13.0-HEAD-f659bf1d31c