On-Prem & kubeadm Clusters
TL;DR
For kubeadm clusters, prepare Linux hosts, disable swap or configure kubelet appropriately, install a CRI runtime, initialize the control plane, install a CNI, join workers, back up etcd, and manage upgrades one minor version at a time.
Underlying Infrastructure Checklist
- Time sync, DNS, hostnames, static/control-plane IPs, and reliable network between nodes.
- Container runtime installed and CRI socket available, usually containerd.
- Required ports open between control-plane and worker nodes.
- Load balancer or virtual IP for HA API endpoint.
- Disk, CPU, memory, OS kernel, and cgroup settings aligned with kubelet/runtime.
containerd Prep
bashcontainerd-prep.sh
# Host-level commands; adapt for OS and client standards.
sudo modprobe overlay
sudo modprobe br_netfilter
cat <kubeadm Init
yamlkubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: v1.30.0 # Match your approved target version.
controlPlaneEndpoint: "k8s-api.example.com:6443" # HA LB/VIP endpoint.
networking:
podSubnet: "10.244.0.0/16" # Must match CNI requirements.
serviceSubnet: "10.96.0.0/12"
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sockbashkubeadm-init.sh
sudo kubeadm init --config kubeadm-config.yaml --upload-certs
mkdir -p $HOME/.kube
sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# Install CNI before expecting worker Pods to run.
kubectl get nodesExtension Interfaces
| Interface | Purpose | Examples |
|---|---|---|
| CRI | Container runtime API used by kubelet. | containerd, CRI-O. |
| CNI | Pod networking and network policy. | Calico, Cilium, Flannel. |
| CSI | Storage provisioning/attach/mount. | EBS CSI, vSphere CSI, Ceph CSI. |
| CRD/API extensions | Custom Kubernetes APIs. | cert-manager, Prometheus Operator. |
Cluster Lifecycle And Upgrades
bashkubeadm-upgrade-shape.sh
# Shape only: follow version-specific docs and client change process.
kubectl drain <control-plane-node> --ignore-daemonsets
sudo kubeadm upgrade plan
sudo kubeadm upgrade apply v1.30.x
sudo apt-mark unhold kubelet kubectl kubeadm
sudo apt-get install -y kubelet=1.30.x-* kubectl=1.30.x-* kubeadm=1.30.x-*
sudo apt-mark hold kubelet kubectl kubeadm
sudo systemctl daemon-reload
sudo systemctl restart kubelet
kubectl uncordon <control-plane-node>HA Control Plane Notes
- Use 3 or 5 control-plane nodes for etcd quorum.
- Put API servers behind a stable load balancer or VIP.
- Back up etcd before upgrades or risky control-plane work.
- Monitor apiserver, scheduler, controller-manager, etcd, and kubelet.