174 lines
12 KiB
Markdown
174 lines
12 KiB
Markdown
---
|
|
tags:
|
|
- Documentation
|
|
- Bookstack
|
|
- Notes
|
|
---
|
|
# Kubernetes
|
|
|
|
# ☸️ Production-Ready Kubernetes Setup Guide (Ubuntu)
|
|
|
|
This guide walks you through setting up a Kubernetes cluster using `kubeadm` on Ubuntu Server (20.04, 22.04, or 24.04).
|
|
|
|
## 1. Prerequisites & Resource Allocation
|
|
|
|
Ensure all machines in your planned cluster meet the following baseline requirements:
|
|
|
|
- **Operating System:** Clean installation of Ubuntu Server on all nodes.
|
|
- **Master Node:** Minimum 2 vCPUs, 4GB RAM, and 50GB storage.
|
|
- **Worker Nodes:** Minimum 1 vCPU (2 recommended), 2GB RAM (4GB recommended).
|
|
- **Network:** Unique hostnames, MAC addresses, and product\_uuids for every node. Fully static IP addresses are highly recommended.
|
|
|
|
## 2. Preparation (Execute on ALL Nodes)
|
|
|
|
Run these steps across every machine (Master and Workers) to prepare the operating system.
|
|
|
|
### Step 2.1: System Update & Disable Swap
|
|
|
|
Kubernetes requires swap memory to be completely disabled to ensure kubelet resource scheduling works deterministically.
|
|
|
|
<div class="code-block ng-tns-c1605810258-279 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk-bash"><div class="formatted-code-block-internal-container ng-tns-c1605810258-279"><div class="animated-opacity ng-tns-c1605810258-279"><div class="code-block-decoration header-formatted gds-emphasized-body-m ng-tns-c1605810258-279 ng-star-inserted"><span class="ng-tns-c1605810258-279">Bash</span><div class="buttons ng-tns-c1605810258-279 ng-star-inserted"></div></div></div></div></div>```
|
|
# Update local package lists and upgrade existing software
|
|
sudo apt update && sudo apt upgrade -y
|
|
|
|
# Disable swap immediately
|
|
sudo swapoff -a
|
|
|
|
# Persist the change across reboots by commenting out the swap line in /etc/fstab
|
|
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
|
|
|
|
```
|
|
|
|
<div class="code-block ng-tns-c1605810258-279 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk--1"><div class="formatted-code-block-internal-container ng-tns-c1605810258-279"><div class="animated-opacity ng-tns-c1605810258-279"></div></div></div>### Step 2.2: Configure Kernel Modules and Networking
|
|
|
|
Load the necessary kernel modules for container isolation and networking, and configure required `sysctl` network parameters.
|
|
|
|
<div class="code-block ng-tns-c1605810258-280 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk-bash-1"><div class="formatted-code-block-internal-container ng-tns-c1605810258-280"><div class="animated-opacity ng-tns-c1605810258-280"><div class="code-block-decoration header-formatted gds-emphasized-body-m ng-tns-c1605810258-280 ng-star-inserted"><span class="ng-tns-c1605810258-280">Bash</span><div class="buttons ng-tns-c1605810258-280 ng-star-inserted"></div></div></div></div></div>```
|
|
# Configure modules to load automatically on boot
|
|
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
|
|
overlay
|
|
br_netfilter
|
|
EOF
|
|
|
|
# Load the modules into the current kernel session
|
|
sudo modprobe overlay
|
|
sudo modprobe br_netfilter
|
|
|
|
# Enable IPv4 forwarding and iptables bridging rules
|
|
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
|
|
net.bridge.bridge-nf-call-iptables = 1
|
|
net.bridge.bridge-nf-call-ip6tables = 1
|
|
net.ipv4.ip_forward = 1
|
|
EOF
|
|
|
|
# Apply sysctl parameters immediately without a reboot
|
|
sudo sysctl --system
|
|
|
|
```
|
|
|
|
<div class="code-block ng-tns-c1605810258-280 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk--4"><div class="formatted-code-block-internal-container ng-tns-c1605810258-280"><div class="animated-opacity ng-tns-c1605810258-280"></div></div></div>### Step 2.3: Install and Configure Containerd (CRI)
|
|
|
|
We will install `containerd`, generate its default configurations, and explicitly enable the `SystemdCgroup` driver so it aligns properly with Kubernetes' cgroup management.
|
|
|
|
<div class="code-block ng-tns-c1605810258-281 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk-bash-2"><div class="formatted-code-block-internal-container ng-tns-c1605810258-281"><div class="animated-opacity ng-tns-c1605810258-281"><div class="code-block-decoration header-formatted gds-emphasized-body-m ng-tns-c1605810258-281 ng-star-inserted"><span class="ng-tns-c1605810258-281">Bash</span><div class="buttons ng-tns-c1605810258-281 ng-star-inserted"></div></div></div></div></div>```
|
|
# Install the containerd package
|
|
sudo apt update && sudo apt install -y containerd
|
|
|
|
# Generate default configuration directory and file
|
|
sudo mkdir -p /etc/containerd
|
|
containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
|
|
|
|
# Configure containerd to use SystemdCgroup
|
|
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/etc/containerd/config.toml
|
|
|
|
# Restart and enable containerd service
|
|
sudo systemctl restart containerd
|
|
sudo systemctl enable containerd
|
|
|
|
```
|
|
|
|
<div class="code-block ng-tns-c1605810258-281 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk--7"><div class="formatted-code-block-internal-container ng-tns-c1605810258-281"><div class="animated-opacity ng-tns-c1605810258-281"></div></div></div>### Step 2.4: Install Kubernetes CLI/Server Tools (`pkgs.k8s.io`)
|
|
|
|
This pulls from the modernized community-hosted repositories. *(Note: Adjust `v1.30` in the path if you intend to pin your cluster to a specific minor edition).*
|
|
|
|
<div class="code-block ng-tns-c1605810258-282 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk-bash-3"><div class="formatted-code-block-internal-container ng-tns-c1605810258-282"><div class="animated-opacity ng-tns-c1605810258-282"><div class="code-block-decoration header-formatted gds-emphasized-body-m ng-tns-c1605810258-282 ng-star-inserted"><span class="ng-tns-c1605810258-282">Bash</span><div class="buttons ng-tns-c1605810258-282 ng-star-inserted"></div></div></div></div></div>```
|
|
# Download dependencies for secure repository signatures
|
|
sudo apt-get update && sudo apt-get install -y apt-transport-https ca-certificates curl gpg
|
|
|
|
# Download the public signing key for the modern Kubernetes package repository
|
|
sudo mkdir -p /etc/apt/keyrings
|
|
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
|
|
|
# Add the correct apt repository reference string
|
|
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
|
|
|
|
# Update your lists and install the core binaries
|
|
sudo apt-get update
|
|
sudo apt-get install -y kubelet kubeadm kubectl
|
|
|
|
# Prevent accidental background package upgrades from breaking cluster state
|
|
sudo apt-mark hold kubelet kubeadm kubectl
|
|
|
|
```
|
|
|
|
<div class="code-block ng-tns-c1605810258-282 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk--10"><div class="formatted-code-block-internal-container ng-tns-c1605810258-282"><div class="animated-opacity ng-tns-c1605810258-282"></div></div></div>## 3. Control Plane Initialization (Execute on MASTER Node Only)
|
|
|
|
### Step 3.1: Initialize via Kubeadm
|
|
|
|
Choose your Pod network CIDR block. If using **Calico**, the default block is `192.168.0.0/16`. If using **Flannel**, the default block is `10.244.0.0/16`.
|
|
|
|
<div class="code-block ng-tns-c1605810258-283 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk-bash-4"><div class="formatted-code-block-internal-container ng-tns-c1605810258-283"><div class="animated-opacity ng-tns-c1605810258-283"><div class="code-block-decoration header-formatted gds-emphasized-body-m ng-tns-c1605810258-283 ng-star-inserted"><span class="ng-tns-c1605810258-283">Bash</span><div class="buttons ng-tns-c1605810258-283 ng-star-inserted"></div></div></div></div></div>```
|
|
# Initialize the master plane node (Using Calico default networking block)
|
|
sudo kubeadm init --pod-network-cidr=192.168.0.0/16
|
|
|
|
```
|
|
|
|
<div class="code-block ng-tns-c1605810258-283 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk--13"><div class="formatted-code-block-internal-container ng-tns-c1605810258-283"><div class="animated-opacity ng-tns-c1605810258-283"></div></div></div>*(Once complete, look at the end of your terminal output. Copy and save the unique `kubeadm join` block generated—you will need it for your workers).*
|
|
|
|
### Step 3.2: Configure local `kubectl` access
|
|
|
|
To allow your non-root system user account to issue instructions to the API server:
|
|
|
|
<div class="code-block ng-tns-c1605810258-284 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk-bash-5"><div class="formatted-code-block-internal-container ng-tns-c1605810258-284"><div class="animated-opacity ng-tns-c1605810258-284"><div class="code-block-decoration header-formatted gds-emphasized-body-m ng-tns-c1605810258-284 ng-star-inserted"><span class="ng-tns-c1605810258-284">Bash</span><div class="buttons ng-tns-c1605810258-284 ng-star-inserted"></div></div></div></div></div>```
|
|
mkdir -p $HOME/.kube
|
|
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
|
sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
|
|
|
```
|
|
|
|
<div class="code-block ng-tns-c1605810258-284 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk--16"><div class="formatted-code-block-internal-container ng-tns-c1605810258-284"><div class="animated-opacity ng-tns-c1605810258-284"></div></div></div>### Step 3.3: Deploy the Pod CNI (Calico Deployment)
|
|
|
|
Deploying a single clear network operator ensures network tracking stays clean.
|
|
|
|
<div class="code-block ng-tns-c1605810258-285 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk-bash-6"><div class="formatted-code-block-internal-container ng-tns-c1605810258-285"><div class="animated-opacity ng-tns-c1605810258-285"><div class="code-block-decoration header-formatted gds-emphasized-body-m ng-tns-c1605810258-285 ng-star-inserted"><span class="ng-tns-c1605810258-285">Bash</span><div class="buttons ng-tns-c1605810258-285 ng-star-inserted"></div></div></div></div></div>```
|
|
# Install the Tigera Calico Operator
|
|
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.0/manifests/tigera-operator.yaml
|
|
|
|
# Download the custom resource manifest definition
|
|
wget https://raw.githubusercontent.com/projectcalico/calico/v3.26.0/manifests/custom-resources.yaml
|
|
|
|
# (Optional modification) If you used a pod-network-cidr other than 192.168.0.0/16,
|
|
# edit custom-resources.yaml using nano to alter the "cidr:" line inside to match it.
|
|
|
|
# Apply the custom network layout manifest
|
|
kubectl create -f custom-resources.yaml
|
|
|
|
```
|
|
|
|
<div class="code-block ng-tns-c1605810258-285 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk--19"><div class="formatted-code-block-internal-container ng-tns-c1605810258-285"><div class="animated-opacity ng-tns-c1605810258-285"></div></div></div>## 4. Join Worker Nodes (Execute on WORKER Nodes Only)
|
|
|
|
Move over to your designated worker VMs. Take the output you preserved from Step 3.1 and execute it with administrative flags:
|
|
|
|
<div class="code-block ng-tns-c1605810258-286 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk-bash-7"><div class="formatted-code-block-internal-container ng-tns-c1605810258-286"><div class="animated-opacity ng-tns-c1605810258-286"><div class="code-block-decoration header-formatted gds-emphasized-body-m ng-tns-c1605810258-286 ng-star-inserted"><span class="ng-tns-c1605810258-286">Bash</span><div class="buttons ng-tns-c1605810258-286 ng-star-inserted"></div></div></div></div></div>```
|
|
sudo kubeadm join <master-node-ip>:6443 --token <token-string> \
|
|
--discovery-token-ca-cert-hash sha256:<hash-string>
|
|
|
|
```
|
|
|
|
<div class="code-block ng-tns-c1605810258-286 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk--22"><div class="formatted-code-block-internal-container ng-tns-c1605810258-286"><div class="animated-opacity ng-tns-c1605810258-286"></div></div></div>### Missing your join token string?
|
|
|
|
If you ever lose the string or it expires (tokens automatically timeout after 24 hours), generate a brand-new execution block by running this command back on the **Master node**:
|
|
|
|
<div class="code-block ng-tns-c1605810258-287 ng-animate-disabled ng-trigger ng-trigger-codeBlockRevealAnimation" id="bkmrk-bash-8"><div class="formatted-code-block-internal-container ng-tns-c1605810258-287"><div class="animated-opacity ng-tns-c1605810258-287"><div class="code-block-decoration header-formatted gds-emphasized-body-m ng-tns-c1605810258-287 ng-star-inserted"><span class="ng-tns-c1605810258-287">Bash</span><div class="buttons ng-tns-c1605810258-287 ng-star-inserted"></div></div></div></div></div>```
|
|
kubeadm token create --print-join-command
|
|
``` |