Infrastructure & Engineering

Building Home K8s [kubeadm + CRI-O + Cilium]

Otama
3 min read
Modified
Progress 1 / 18
Table of Contents

Recently, I’ve been enjoying tweaking my home lab environment, so I decided to put some effort into building a Kubernetes cluster. I thought “Building with kubeadm seems pretty tedious…”, but when I tried it, it went surprisingly smoothly, so I’m summarizing the record as a memorandum.

This time, I stood up Proxmox VE on a mini PC equipped with N150 and created VMs there to work on. Hardware-wise, the specs are quite modest, but it’s necessary and sufficient for verification purposes.

The content basically follows the procedures in the documentation, so there isn’t much deep technical diving. However, there were a few points where I thought “I’d get stuck if I forgot this” while actually running it, so it might be helpful if viewed from that perspective.

Execution Environment

I run it on VMs on Proxmox so that I can easily destroy and create environments.

  • CPU: Intel N150 (Cores assigned to VM = 2)
  • RAM: 8GB
  • Disk: 256GB
  • OS: ubuntu server 24.04.2

Procedure

I will perform the following tasks referring to the Official Documentation.

  • Install container runtime
  • Install kubeadm
  • Launch cluster
  • Install CNI

Install Container Runtime

Ref: https://kubernetes.io/docs/setup/production-environment/container-runtimes/

Enable IPv4 Forwarding

Terminal window
# As per documentation
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF


sudo modprobe overlay
sudo modprobe br_netfilter


# Kernel parameters required for this configuration, values persist after reboot
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF


# Apply kernel parameters without reboot
sudo sysctl --system

Install cri-o (Container Runtime)

Install referring to cri-o’s Official Documentation.

Terminal window
CRIO_VERSION=v1.32


# Install dependencies required for work
sudo apt-get update
sudo apt-get install -y software-properties-common curl


# Add cri-o repository
curl -fsSL https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/deb/Release.key |
    sudo gpg --dearmor -o /etc/apt/keyrings/cri-o-apt-keyring.gpg


echo "deb [signed-by=/etc/apt/keyrings/cri-o-apt-keyring.gpg] https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/deb/ /" |
    sudo tee /etc/apt/sources.list.d/cri-o.list


# Install cri-o
sudo apt-get update
sudo apt-get install -y cri-o

Once installed, start cri-o. If status is active, startup is successful.

Terminal window
systemctl start crio.service
systemctl status crio.service

Install kubeadm

Ref: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm

Disable swap

If there is no swap, you can ignore this section.

Terminal window
# Check if swap exists
free -h


# Temporarily disable swap (returns on reboot)
sudo swapoff -a


# To persist, comment out or delete swap line in /etc/fstab
sudo sed -i '/ swap / s/^$.*$$/#\1/g' /etc/fstab

Install kubeadm

Terminal window
KUBERNETES_VERSION=v1.33


# Add Kubernetes repository
curl -fsSL https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/deb/Release.key |
    sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg


echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/deb/ /" |
    sudo tee /etc/apt/sources.list.d/kubernetes.list


# Install kubelet, kubeadm, kubectl
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Create Cluster

Ref: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm

Initialize kubeadm

Run the following command while praying no errors occur. If trying cilium’s kube-proxy replacement feature, skip kube-proxy setup here.

Terminal window
# Usually this
sudo kubeadm init


# If replacing kube-proxy with cilium, this
kubeadm init --skip-phases=addon/kube-proxy

If displayed as follows, execute commands as instructed to complete initialization.

Terminal window
To start using your cluster, you need to run the following as a regular user:


  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config


Alternatively, if you are the root user, you can run:


  export KUBECONFIG=/etc/kubernetes/admin.conf

Run the following command here. You can see control-plane is created, but it’s still in NotReady state. This is because CNI is not installed, so we will install it in the next section.

Terminal window
kubectl get nodes


# NAME      STATUS     ROLES           AGE     VERSION
# test-01   NotReady   control-plane   6m21s   v1.33.0

Install CNI

Install helm

Since cilium is introduced via helm, first install helm.

Note: helm is a package manager for kubernetes (it has other uses like managing deployment definitions, but omitting details).

Follow the procedure in Official Documentation as usual.

Terminal window
# Working dependencies
sudo apt-get install apt-transport-https --yes


# Add helm repository
curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list


# Install helm
sudo apt-get update
sudo apt-get install helm

Install Cilium

Install referring to this documentation. If trying kube-proxy replacement feature, use kubeProxyReplacement=true.

Terminal window
# Add cilium repository
helm repo add cilium https://helm.cilium.io/


# Install cilium (Normal)
helm install cilium cilium/cilium --version 1.17.3 \
  --namespace kube-system


# Install cilium (If replacing kube-proxy with cilium)
API_SERVER_IP=<your_api_server_ip>
# Kubeadm default is 6443
API_SERVER_PORT=<your_api_server_port>
helm install cilium cilium/cilium --version 1.17.4 \
    --namespace kube-system \
    --set kubeProxyReplacement=true \
    --set k8sServiceHost=${API_SERVER_IP} \
    --set k8sServicePort=${API_SERVER_PORT}

Install Cilium CLI

Install CLI as it’s mandatory for checking CNI status or customizing.

Terminal window
# Verify variables
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi


# Download package
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}


# Check hash
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum


# Unzip and place in /usr/local/bin
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

You can check if it works by hitting the following command.

Terminal window
cilium status

Confirm node

Hit the following command again and confirm that kubelet is in a working state. If status becomes Ready, it is successful. If NotReady, probably missed something somewhere.

Terminal window
kubectl get node


# Success if status is ready
# NAME      STATUS   ROLES           AGE   VERSION
# test-01   Ready    control-plane   28m   v1.33.0

Conclusion

So, I completed up to launching a Kubernetes cluster with a single-node configuration! Since I did everything including introducing Cilium via CLI base, I’m slightly hyped about the “I built it myself” feeling.

Next, I’d like to try GitOps-like operation using this environment, or expanding to multiple nodes. I’ll write again if interesting topics come up while playing around bit by bit.

Next Article ↓

【初心者向け】自宅Kubernetesクラスタにワーカーノードを追加する方法

>-

blog.otama-playground.com

Related Posts