Monaf Horany

Menu

All runbooks

Runbooks

A highly available control plane behind HAProxy

Three control-plane nodes behind one TCP load balancer, so losing a node loses nothing. The whole trick is that the API server is fronted at layer 4, not layer 7.

HAProxyHAkube-apiserverTCP4 steps · 20–30 min

Addresses in this guide are placeholders in 10.0.0.0/24 and the domain is example.dev. Everything else is verbatim from the working notes.

  1. 01

    Why layer 4

    The Kubernetes API server speaks TLS end to end and authenticates clients by certificate. Terminating HTTP at the balancer would break mutual TLS, so the frontend runs in TCP mode and passes bytes through untouched. Every node then joins against the virtual address rather than any single control plane.

  2. 02

    Install HAProxy

    bash
    sudo apt update && sudo apt install -y haproxy
  3. 03

    Configure the frontend and backend

    sudo nano /etc/haproxy/haproxy.cfg
    ini
    defaults
  mode                    http
  log                     global
  option                  httplog
  option                  dontlognull
  option                  http-server-close
  option                  forwardfor except 127.0.0.0/8
  option                  redispatch
  retries                 1
  timeout http-request    10s
  timeout queue           20s
  timeout connect         5s
  timeout client          35s
  timeout server          35s
  timeout http-keep-alive 10s
  timeout check           10s

frontend kubernetes-frontend
  bind 10.0.0.199:6443
  option tcplog
  mode tcp
  default_backend kubernetes-backend

backend kubernetes-backend
  option tcp-check
  mode tcp
  balance roundrobin
  server cp-1 10.0.0.197:6443 check fall 3 rise 2
  server cp-2 10.0.0.198:6443 check fall 3 rise 2
  server cp-3 10.0.0.196:6443 check fall 3 rise 2
  4. 04

    Start it

    bash
    sudo systemctl restart haproxy
sudo systemctl enable haproxy
sudo systemctl status haproxy