K3s Notes - Part 1
Experimenting with Docker Swarm, I got to the task of building something with the Raspberry Pi (RPI v3, RPI v4) that I have in my lab, after googling a lot and chatting with my friend Sergio, he recommended using K3s which is a Kubernetes distribution with backend sqlite3 based lightweight storage system compatible with ARM architecture.
I reviewed the official K3s documentation and I was able to set up my laboratory, the type of exercise that will be developed will be a Server node and four Worker nodes with the sqlite3 database.
Architecture
There are two types of nodes:
- Server node, it is the node that runs K3s server.
- Worker node, is the node that runs the K3s agent.
There are also two ways of implementation:
- Single-server Setup with an Embedded DB, in this configuration, each agent node is registered on the same Server node.
- High-Availability K3s Server with an External DB, Which consists of:
- Two or more Server nodes that will serve the Kubernetes API and run other control plane services.
- An external data store (as opposed to the built-in SQLite data store used in single server configurations).
2.1 Fixed Registration Address for Agent Nodes
In the high availability server configuration, each node must also register with the Kubernetes API using a fixed registration address, after registration, the agent nodes establish a connection directly with one of the Server nodes.
Previous Configurations
-
Necessary RPI configurations: Before installing the K3s binaries it is necessary to make some extra configurations in the Raspberry Pi operating system, we will use Raspbian Buster for this laboratory.
-
Enabling legacy iptables on Raspbian Buster
Raspbian Buster defaults to using nftables instead of iptables. K3s networking features require iptables and do not work with nftables. Follow the steps below to switch configure Buster to use legacy iptables:
1$ sudo iptables -F
2$ sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
3$ sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
4$ sudo reboot
- Enabling cgroups for Raspbian Buster
Standard Raspbian Buster installations do not start with cgroups enabled. K3s needs cgroups to start the systemd service. cgroups can be enabled by appending cgroup_memory=1 and cgroup_enable=memory to /boot/cmdline.txt.
1# Inside the file cmdline.txt
2console=serial0,115200 console=tty1 root=PARTUUID=58b06195-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_memory=1 cgroup_enable=memory
With these changes applied, the operating system must be restarted on the Raspberry Pi.
Install Server node
According to the official documentation we can use the official scripts to install the binaries for the K3s Server node:
1$ curl -sfL https://get.k3s.io | sh -
This script will install all the necessary tools such as:
- kubectl
- crictl
- ctr
- k3s-killall.sh
- k3s-uninstall.sh
It will also create the configuration file /etc/rancher/k3s/k3s.yaml
Install Worker Nodes
After the installation of the K3s server we can add nodes:
1$ curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh -
where:
- K3S_URL: - K3S_URL: This is the IP address or domain of the K3s server.
- K3S_TOKEN: It is a token that is stored in the K3s server, /var/lib/rancher/k3s/server/node-token
- The hostname of the new nodes must be different.
Basic Operations
We can execute the common Kubernetes commands or use the K3s command, to these two commands we must prepend the sudo command so that the orders are executed without problems.
1$ sudo k3s kubectl get nodes
2# or also
3$ sudo kubectl get nodes
4$ sudo kubectl get pods --all-namespaces
Daemon control:
1$ sudo systemctl status k3s
2$ sudo systemctl stop k3s