Adventures in Freebernetes Tutorial: Build Your Own Bare-VM k3s Cluster

Part 3: Create VMs

3.1 Choose Your Network Layout

3.2 Create the Linux VMs

3.1 Choose Your Network Layout

3.1.1 Select Subnets

My FreeBSD hypervisor has a 192.168.0.0/24 address on its physical network interface. I’m going to use a VLAN in 10.0.0.0/8 for the cluster and its pods and services. You can use another block, but you will have to adjust commands throughout the tutorial.

• 10.0.0.1/32 – VLAN gateway on bridge interface
• 10.0.0.2/32 – Virtual IP for Kubernetes API endpoint
• 10.0.10.0/24 – VM block
  • 10.0.10.1[1-3] – K3s servers
  • 10.0.10.2[1-3] – K3s agents (nodes)
• 10.1.0.0/16 – pod network
• 10.2.0.0/16 – service network

3.1.2 Pick a .local Zone for DNS

This zone just needs to resolve locally on the FreeBSD host. I’m going with k3s.local because I’m too lazy to think of a clever pun right now.

3.2 Create the Linux VMs

3.2.1 Initialize CBSD

If you haven’t run CBSD on your FreeBSD host before, you will need to set it up. You can use this seed file. Edit it first to set node_name to your FreeBSD host’s name and to change jnameserver and nodeippool if you are using a private network other than 10.0.0.0/8.

	~ # sysrc cbsd_workdir="/usr/cbsd"
	cbsd_workdir: -> /usr/cbsd
	~ # wget https://raw.githubusercontent.com/kbruner/freebernetes/main/k3s/cbsd/initenv.conf
	[ edit initenv.conf ]
	~ # /usr/local/cbsd/sudoexec/initenv inter=0 `pwd`/initenv.conf # need full path for initenv.conf ¯\_(ツ)_/¯
	[ lots of output ]
	~ # grep cbsd /etc/rc.conf
	cbsd_workdir="/usr/cbsd"
	cbsdrsyncd_enable="YES"
	cbsdrsyncd_flags="–config=/usr/cbsd/etc/rsyncd.conf"
	cbsdd_enable="YES"
	~ # service cbsdrsyncd stop
	~ # sysrc -x cbsdrsyncd_enable
	~ # sysrc -x cbsdrsyncd_flags
	~ # grep cbsd /etc/rc.conf
	cbsd_workdir="/usr/cbsd"
	cbsdd_enable="YES"

view raw commands + shell hosted with ❤ by GitHub

	sysrc cbsd_workdir="/usr/cbsd"
	wget https://raw.githubusercontent.com/kbruner/freebernetes/main/k3s/cbsd/initenv.conf
	vi initenv.conf
	/usr/local/cbsd/sudoexec/initenv inter=0 `pwd`/initenv.conf
	service cbsdrsyncd stop
	sysrc -x cbsdrsyncd_enable
	sysrc -x cbsdrsyncd_flags

view raw commands.sh hosted with ❤ by GitHub

3.2.2 Create VMs

Copy this instance.jconf VM template file and update ci_gw4, ci_nameserver_search, and ci_nameserver_address fields as needed. If you want to set a password for the ubuntu user in case you want to log in on the console via VNC, you can assign it to cw_user_pw_user, but note this is a plain-text field.

When you run cbsd bcreate, if CBSD does not have a copy of the installation ISO image, it will prompt you asking to download it. After the first time, it will re-use the local image.

	~ # for i in 0 1 2; do
	cbsd bcreate jconf=/root/instance.jconf jname="server-$i" \
	ci_ip4_addr="10.0.10.1${i}/24" ci_jname="server-$i" \
	ci_fqdn="server-${i}.k3s.local" ip_addr="10.0.10.1${i}" \
	imgsize="20g" vm_cpus="2" vm_ram="2g"
	done
	Global VM ZFS guid: 414491023960324541
	To edit VM properties use: cbsd bconfig jname=server-0
	To start VM use: cbsd bstart server-0
	To stop VM use: cbsd bstop server-0
	To remove VM use: cbsd bremove server-0
	For attach VM console use: cbsd blogin server-0
	Creating server-0 complete: Enjoy!
	auto-generate cloud-init settings: /usr/cbsd/jails-system/server-0/cloud-init
	[ similar output for server-1 and server-2 ]
	~ # for i in 0 1 2; do cbsd bstart jname="server-$i"; done
	cloud-init: enabled
	vm_iso_path: cloud-ubuntu-x86-20.04.1
	cloud init image initialization..
	Clone cloud image into first/system vm disk (zfs clone method)
	/sbin/zfs get -Ht snapshot userrefs zroot/ROOT/default/cbsd-cloud-cloud-Ubuntu-x86-20.04.1.raw@boot-server-0
	Eject cloud source: media mode=detach name=cloud-ubuntu-x86-20.04.1 path=/usr/cbsd/src/iso/cbsd-cloud-cloud-Ubuntu-x86-20.04.1.raw type=iso jname=server-0
	DELETE FROM media WHERE name="cloud-ubuntu-x86-20.04.1" AND path="/usr/cbsd/src/iso/cbsd-cloud-cloud-Ubuntu-x86-20.04.1.raw" AND jname="server-0"
	vm_iso_path: changed
	Detach to: server-0
	All CD/ISO ejected: server-0
	VRDP is enabled. VNC bind/port: 127.0.0.1:5901
	For attach VM console, use: vncviewer 127.0.0.1:5901
	Resolution: 1024×768.
	bhyve renice: 1
	Execute master script: cloud_init_set_netname.sh
	:: /usr/cbsd/jails-system/server-0/master_prestart.d/cloud_init_set_netname.sh
	Waiting for PID.
	PID: 15858
	CBSD setup: bhyve ipfw counters num: 99/100
	[ similar output for server-1 and server-2 ]
	[ repeat for agent VMs ]

view raw commands + output hosted with ❤ by GitHub

	# create server VMs
	for i in 0 1 2; do
	cbsd bcreate jconf=/root/instance.jconf jname="server-$i" \
	ci_ip4_addr="10.0.10.1${i}/24" ci_jname="server-$i" \
	ci_fqdn="server-${i}.k3s.local" ip_addr="10.0.10.1${i}" \
	imgsize="20g" vm_cpus="2" vm_ram="2g"
	done
	# start server VMs
	for i in 0 1 2; do cbsd bstart jname="server-$i"; done
	# create agent VMs
	for i in 0 1 2; do
	cbsd bcreate jconf=/root/instance.jconf jname="agent-$i" \
	ci_ip4_addr="10.0.10.2${i}/24" ci_jname="agent-$i" \
	ci_fqdn="agent-${i}.k3s.local" ip_addr="10.0.10.2${i}" \
	imgsize="10g" vm_cpus="2" vm_ram="2g"
	done
	# start agent VMs
	for i in 0 1 2; do cbsd bstart jname="agent-$i"; done

view raw commands.sh hosted with ❤ by GitHub

Pages: 1 2 3 4 5 6 7 8