Step-by-step tutorial for deploying a Kubernetes cluster with k3s on FreeBSD bhyve VMs
See all posts in the FreeBSD Virtualization Series
- Overview
- Technical Specs
- Part 1: Required Tools
- Part 2: Building k3sup
- Part 3: Creating VMs
- Part 4: Configure Networking
- Part 5: Create the Cluster
- Part 6: Test the Cluster
- Part 7: Clean Up
- Sources / References
Overview
In the last tutorial in this series, we went through the steps of creating a Kubernetes cluster on FreeBSD bhyve
VMs, adapting the completely manual process in Kelsey Hightower’s Kubernetes the Hard Way. This tutorial will also build a Kubernetes cluster on bhyve
, but it will install a lightweight k3s
control plane using the k3sup
tool, which automates much of the process.
Topics covered:
- Compiling
k3sup
on FreeBSD - Configuring CBSD to create and manage
bhyve
VMs - Running
k3sup
to bring up ak3s
cluster onbhyve
VMs - Configuring the FreeBSD firewall, DNS, and routing for cluster networking
While this tutorial builds a cluster with a redundant control plane, all the VMs are on a single hypervisor, making it suitable for testing and experimentation, but it is not production grade.
This tutorial only covers creating a cluster. For Kubernetes basics and terminology, you should start with the official Kubernetes documentation.
Intended Audience
For this tutorial, you don’t need to know anything about Kubernetes. You do need to have a host with FreeBSD installed; an understanding of basic FreeBSD system administration tasks, such as installing software from FreeBSD ports or packages, and loading kernel modules; and familiarity with csh
or sh
. Experience with FreeBSD bhyve
virtual machines and the CBSD interface is useful but not required.
Technical Specs
Kubernetes Cluster Specs
We will use these targets for the Kubernetes cluster we’re building. Most of these settings can be adjusted.
- Control plane (“K3s servers”)
- 3 VMs: 2 CPU cores, 2Gb RAM, 20Gb virtual disk each
- If you plan on creating more than 10 worker nodes, increase the control plane VM sizes
- Worker nodes (“K3s agents”)
- 3 VMs: 2 CPU cores, 2Gb RAM, 10Gb virtual disk each
- VM OS: Ubuntu Server 20.04
- Kubernetes version: 1.19
- Control plane backing database: embedded
etcd
- K3s also supports using an external datastore, such as MySQL
Host (FreeBSD Hypervisor) Requirements
- Hardware, physical or virtual
- CPU
- CPUs must support FreeBSD bhyve virtualization (see the FreeBSD Handbook page on bhyve for compatible CPUs)
- CPU core allocations for
bhyve
VMs are completely virtual, so you can have VMs running with a total virtual core count greater than your host system’s. You should use the suggested core count for VMs, as Kubernetes will use those for scheduling. You can increase the cores if you have a larger host.
- RAM: Minimum 2Gb per VM. You can use less for the agent VMs if necessary.
- Free disk space: 100Gb
- CPU
- Operating system
- FreeBSD: 13.0-CURRENT. It may work with 12.0-RELEASE, but it has not been tested
- File system: ZFS
My Test System
- Hardware
- CPUs: Intel(R) Core(TM) i5-6260U: 4 CPUs, 2 cores each
- RAM: 32Gb
- Operating system
- FreeBSD 13.0-HEAD-f659bf1d31c
Leave a Reply