Kubernetes is a powerful container orchestration platform, widely used to manage containerized applications in production environments. While cloud providers offer managed Kubernetes services, there are scenarios where you might need to set up Kubernetes on bare metal servers. Two popular tools for setting up Kubernetes on bare metal are Kubeadm and Kubespray. This article will explore both tools, their use cases, and a step-by-step guide on how to use them to deploy Kubernetes on bare metal.

Why Set Up Kubernetes on Bare Metal?

Setting up Kubernetes on bare metal servers is often preferred in the following situations:

1. Full Control: You have complete control over the underlying infrastructure, including hardware configurations, networking, and security policies.
2. Cost Efficiency: For organizations with existing physical infrastructure, using bare metal can be more cost-effective than renting cloud-based resources.
3. Performance: Bare metal deployments eliminate the overhead of virtualization, providing direct access to hardware and potentially better performance.
4. Compliance and Security: Certain industries require data to be stored on-premises to meet regulatory or compliance requirements. Bare metal setups ensure that data never leaves your physical infrastructure.

Overview of Kubeadm and Kubespray

Kubeadm and Kubespray are both tools that simplify the process of deploying a Kubernetes cluster on bare metal, but they serve different purposes and have different levels of complexity.

• Kubeadm: A lightweight tool provided by the Kubernetes project, Kubeadm initializes a Kubernetes cluster on a single node or a set of nodes. It’s designed for simplicity and ease of use, making it ideal for setting up small clusters or learning Kubernetes.
• Kubespray: An open-source project that automates the deployment of Kubernetes clusters across multiple nodes, including bare metal, using Ansible. Kubespray supports advanced configurations, such as high availability, network plugins, and persistent storage, making it suitable for production environments.

Setting Up Kubernetes on Bare Metal Using Kubeadm

Kubeadm is a straightforward tool for setting up Kubernetes clusters. Below is a step-by-step guide to deploying Kubernetes on bare metal using Kubeadm.

Prerequisites

• Multiple Bare Metal Servers: At least one master node and one or more worker nodes.
• Linux OS: Ubuntu or CentOS is commonly used.
• Root Access: Ensure you have root or sudo privileges on all nodes.
• Network Access: Nodes should be able to communicate with each other over the network.

Step 1: Install Docker

Kubeadm requires a container runtime, and Docker is the most commonly used one. Install Docker on all nodes:

sudo apt-get update
sudo apt-get install -y docker.io
sudo systemctl enable docker
sudo systemctl start docker

Step 2: Install Kubeadm, Kubelet, and Kubectl

Install the Kubernetes components on all nodes:

sudo apt-get update
sudo apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Step 3: Disable Swap

Kubernetes requires that swap be disabled. Run the following on all nodes:

sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab

Step 4: Initialize the Master Node

On the master node, initialize the Kubernetes cluster:

sudo kubeadm init --pod-network-cidr=192.168.0.0/16

After the initialization, you will see a command with a token that you can use to join worker nodes to the cluster. Keep this command for later use.

Step 5: Set Up kubectl for the Master Node

Configure kubectl on the master node:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Step 6: Deploy a Network Add-on

To enable communication between pods, you need to install a network plugin. Calico is a popular choice:

kubectl apply -f https://docs.projectcalico.org/v3.14/manifests/calico.yaml

Step 7: Join Worker Nodes to the Cluster

On each worker node, use the kubeadm join command from Step 4 to join the cluster:

sudo kubeadm join <master-ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

Step 8: Verify the Cluster

Check the status of your nodes to ensure they are all connected:

kubectl get nodes

All nodes should be listed as Ready.

Setting Up Kubernetes on Bare Metal Using Kubespray

Kubespray is more advanced than Kubeadm and is suited for setting up production-grade Kubernetes clusters on bare metal.

Prerequisites

• Multiple Bare Metal Servers: Ensure you have SSH access to all servers.
• Ansible Installed: Kubespray uses Ansible for automation. Install Ansible on your control machine.

Step 1: Prepare the Environment

Clone the Kubespray repository and install dependencies:

git clone https://github.com/kubernetes-sigs/kubespray.git
cd kubespray
pip install -r requirements.txt

Step 2: Configure Inventory

Kubespray requires an inventory file that lists all nodes in the cluster. You can generate a sample inventory from a predefined script:

cp -rfp inventory/sample inventory/mycluster
declare -a IPS=(192.168.1.1 192.168.1.2 192.168.1.3)
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

Replace the IP addresses with those of your servers.

Step 3: Customize Configuration (Optional)

You can customize various aspects of the Kubernetes cluster by editing the inventory/mycluster/group_vars files. For instance, you can enable specific network plugins, configure the Kubernetes version, and set up persistent storage options.

Step 4: Deploy the Cluster

Run the Ansible playbook to deploy the cluster:

ansible-playbook -i inventory/mycluster/hosts.yaml --become --become-user=root cluster.yml

This process may take a while as Ansible sets up the Kubernetes cluster on all nodes.

Step 5: Access the Cluster

Once the installation is complete, configure kubectl to access your cluster from the control node:

mkdir -p $HOME/.kube
sudo cp -i inventory/mycluster/artifacts/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Verify that all nodes are part of the cluster:

kubectl get nodes

Kubeadm vs. Kubespray: When to Use Each

• Kubeadm:
• Use Case: Ideal for smaller, simpler setups, or when you need a quick way to set up a Kubernetes cluster for development or testing.
• Complexity: Simpler and easier to get started with, but requires more manual setup for networking and multi-node clusters.
• Flexibility: Limited customization and automation compared to Kubespray.
• Kubespray:
• Use Case: Best suited for production environments where you need advanced features like high availability, custom networking, and complex configurations.
• Complexity: More complex to set up, but offers greater flexibility and automation through Ansible.
• Flexibility: Highly customizable, with support for various plugins, networking options, and deployment strategies.

Conclusion

Setting up Kubernetes on bare metal provides full control over your infrastructure and can be optimized for specific workloads or compliance requirements. Kubeadm is a great choice for simple or development environments, offering a quick and easy way to get started with Kubernetes. On the other hand, Kubespray is designed for more complex, production-grade deployments, providing automation and customization through Ansible. By choosing the right tool based on your needs, you can efficiently deploy and manage a Kubernetes cluster on bare metal servers.